infer

Documentation

Overview

Package infer contains inference algorithms: maximum likelihood estimation by gradient descent and approximation of the posterior by Markov Chain Monte Carlo methods (notably Hamiltonian Monte Carlo family of algorithms).

Index

• func FuncGrad(m model.Model) (Func func(x []float64) float64, Grad func(grad, x []float64))
• func Optimize(opt Grad, m model.Model, x []float64, niter, nplateau int, eps float64) (iter int, ll0, ll float64)
• type Adam
  • func (opt *Adam) Step(m model.Model, x []float64) (ll float64, grad []float64)
• type DepthAdapter
  • func (da *DepthAdapter) Adapt(nuts *NUTS, samples <-chan []float64, nIter int)
• type DualAveraging
  • func (da *DualAveraging) Step(t, x, gradSum float64) float64
• type Grad
• type HMC
  • func (hmc *HMC) Sample(m model.Model, x []float64, samples chan []float64)
• type MCMC
• type Momentum
  • func (opt *Momentum) Step(m model.Model, x []float64) (ll float64, grad []float64)
• type NUTS
  • func (nuts *NUTS) MeanDepth() float64
  • func (nuts *NUTS) Sample(m model.Model, x []float64, samples chan []float64)
• type Sampler
  • func (s *Sampler) Stop()
• type SgHMC
  • func (sghmc *SgHMC) Sample(m model.Model, x []float64, samples chan []float64)

Functions

func FuncGrad

func FuncGrad(m model.Model) (
	Func func(x []float64) float64,
	Grad func(grad, x []float64),
)

FuncGrad returns the function to minimize and the gradient, suitable as fields for gonum optimize.Problem, corresponding to maximization of the model's log-likelihood.

func Optimize

func Optimize(
	opt Grad,
	m model.Model, x []float64,
	niter, nplateau int,
	eps float64,
) (
	iter int,
	ll0, ll float64,
)

Optimize wraps a gradient-based optimizer into an optimization loop with early stopping if a plateau is reached.

Types

type Adam

type Adam struct {
	Rate  float64 // learning rate
	Beta1 float64 // first momentum factor
	Beta2 float64 // second momentum factor
	Eps   float64 // stabilizer
	// contains filtered or unexported fields
}

Adam (https://arxiv.org/abs/1412.6980).

func (*Adam) Step

func (opt *Adam) Step(
	m model.Model,
	x []float64,
) (
	ll float64,
	grad []float64,
)

Step implements the Optimizer interface.

type DepthAdapter

type DepthAdapter struct {
	DualAveraging
	Depth   float64
	NAdpt   int
	MinGrad float64
}

Parameters of adaptation to the target depth

func (*DepthAdapter) Adapt

func (da *DepthAdapter) Adapt(
	nuts *NUTS,
	samples <-chan []float64,
	nIter int,
)

Adapt adapts NUTS sampler to the target depth. At most nIter iterations are run.

type DualAveraging

type DualAveraging struct {
	Rate float64
}

Parameters of dual averaging.

func (*DualAveraging) Step

func (da *DualAveraging) Step(t, x, gradSum float64) float64

Step implements Nesterov's primal-dual averaging, oversimplified.

chi = -gradSum/math.Sqrt(t)
eta = Rate/t
x = eta*chi + (1-eta)*x

type Grad

type Grad interface {
	Step(m model.Model, x []float64) (ll float64, grad []float64)
}

Grad is the interface of gradient-based optimizers. Step makes a single step over parameters in the gradient direction.

type HMC

type HMC struct {
	Sampler
	// Parameters
	L   int     // number of leapfrog steps
	Eps float64 // leapfrog step size
}

Vanilla Hamiltonian Monte Carlo Sampler.

func (*HMC) Sample

func (hmc *HMC) Sample(
	m model.Model,
	x []float64,
	samples chan []float64,
)

type MCMC

type MCMC interface {
	Sample(
		m model.Model,
		x []float64,
		samples chan []float64,
	)
	Stop()
}

MCMC is the interface of MCMC samplers.

type Momentum

type Momentum struct {
	Rate  float64 //learning rate
	Decay float64 // rate decay
	Gamma float64 // gradient momentum factor
	// contains filtered or unexported fields
}

Gradient ascent with momentum (https://www.nature.com/articles/323533a0). If the momentum factor is not set, and thus 0, reduces to vanilla gradient ascent.

func (*Momentum) Step

func (opt *Momentum) Step(
	m model.Model,
	x []float64,
) (
	ll float64,
	grad []float64,
)

Step implements the Optimizer interface.

type NUTS

type NUTS struct {
	Sampler
	// Parameters
	Eps      float64 // step size
	Delta    float64 // lower bound on energy for doubling
	MaxDepth int     // maximum depth
	// Statistics
	// Depth belief is encoded as a vector of beta-bernoulli
	// distributions. If the depth is greater than the element's
	// index i, Depth[i][0] is incremented; for index depth,
	// Depth[depth][1] is incremented.
	Depth [][2]float64 // depth belief
	// contains filtered or unexported fields
}

No U-Turn Sampler (https://arxiv.org/abs/1111.4246).

func (*NUTS) MeanDepth

func (nuts *NUTS) MeanDepth() float64

MeanDepth returns the average observed depth.

func (*NUTS) Sample

func (nuts *NUTS) Sample(
	m model.Model,
	x []float64,
	samples chan []float64,
)

type Sampler

type Sampler struct {
	Stopped bool
	Samples chan []float64
	// Statistics
	NAcc, NRej int // the number of accepted and rejected samples
}

Sampler is the structure for embedding into concrete samplers.

func (*Sampler) Stop

func (s *Sampler) Stop()

Stop stops a sampler gracefully, using the samples channel for synchronization. Stop must be called before further calls to differentiated code. A part of the MCMC interface.

type SgHMC

type SgHMC struct {
	Sampler
	// Parameters
	L int // number of steps
	// The parameterization follows Equation (15) in
	// https://arxiv.org/abs/1402.4102
	Eta   float64 // learning rate
	Alpha float64 // friction (1 - momentum)
	V     float64 // diffusion
}

Stochastic gradient Hamiltonian Monte Carlo

func (*SgHMC) Sample

func (sghmc *SgHMC) Sample(
	m model.Model,
	x []float64,
	samples chan []float64,
)