Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Function ¶
There are many different ways one can optimize a function, but the most effective ones need gradient. The interface is designed so that one can compose function easily. The semantics is simply add gradient from this function to corresponding dimensions on the output paramenter gradient.
type IndexIterator ¶
For more information read: http://ewencp.org/blog/golang-iterators/ We need an reasonably efficient way to enumerate all indexes.
For example, we can do: sum := 0.0
for it := param.IndexIterator(); it.Next(); {
sum += param.Get(it.Index());
}
func MakeRangeIndexIterator ¶
func MakeRangeIndexIterator(psize int) IndexIterator
type Minimizer ¶
type Minimizer interface {
Minimize(function Function, stopCriteria StopCriteria, param Parameter) (float32, error)
}
High level interface for minimization. This assume that we start with one point in the parameter space, and end with an optimal point. Return true if we find optimal point.
type Parameter ¶
type Parameter interface {
Get(index int) float32
Set(index int, value float32)
Add(index int, value float32)
// This allow us to generate parameter with same width.
CloneWithoutCopy() Parameter
// This allow one to enumerate through all parameters
IndexIterator() IndexIterator
// Return raw data pointer
Data() []float32
}
We need some interface to define function and how we optimize them.
func NewAllTheSameParameter ¶
This creates a parameter that has the same value on all dimensions
func NewVecParameter ¶
This creates a new Vector based parameter
func NewVecParameterWithData ¶
type ProjectedGradient ¶
type ProjectedGradient struct {
// contains filtered or unexported fields
}
This defines how we do projected gradient.
func NewProjectedGradient ¶
func NewProjectedGradient(projector *Projection, beta, sigma, alpha float32) *ProjectedGradient
func (*ProjectedGradient) Minimize ¶
func (pg *ProjectedGradient) Minimize(loss Function, stop StopCriteria, vec Parameter) (float32, error)
This implementation is based on "Projected Gradient Methods for Non-negative Matrix Factorization" by Chih-Jen Lin. Particularly it is based on the discription of an improved projected gradient method in page 10 of that paper.
type Projection ¶
type Projection struct {
// contains filtered or unexported fields
}
Project is used to clip the parameter and gradient.
func NewProjection ¶
func NewProjection(ub, lb Parameter) *Projection
this creates a Project with specified upper and lower bound. NOTE(baigang): to fix scope visibility of `upper_bound` and `lower_bound` inside `Projection`.
func (*Projection) ClipGradient ¶
func (p *Projection) ClipGradient(base, gradient Parameter)
We assume the base and gradient are in the same dimensions. In another words, the IndexIterator will return the same from base and gradient.
func (*Projection) ClipPoint ¶
func (p *Projection) ClipPoint(vec Parameter)
type Regularization ¶
type Regularization struct {
// contains filtered or unexported fields
}
This implements l1 l2 regularization.
type Rosenbrock ¶
type Rosenbrock struct {
// contains filtered or unexported fields
}
Rosenbrock is used as standard test function for optimization.
f(x, y) = (1-x)^2 + 100(y-x^2)^2, it has a global minimum of 0
We can have multiple copies of this function, and different scaling too to make it harder to solve.
func (*Rosenbrock) Evaluate ¶
func (r *Rosenbrock) Evaluate(in, out Parameter) float32
This implementation should only work with parameter that have indexes range from [0, 2*numOfCopies).
type StopCriteria ¶
This is used to figure out where one can stop the optimization. Implementation can be count based, or gradient norm based.
func MakeComposedCriterion ¶
func MakeComposedCriterion(criteria ...StopCriteria) StopCriteria
func MakeFixCountStopCriteria ¶
func MakeFixCountStopCriteria(iter int) StopCriteria
func MakeGradientNormStopCriteria ¶
func MakeGradientNormStopCriteria(thres float32) StopCriteria
func MakeTimeoutCriterion ¶
func MakeTimeoutCriterion(limit time.Duration) StopCriteria
type SumFunction ¶
type SumFunction struct {
// contains filtered or unexported fields
}
This defines an additive function. One can compose the function this way easily.
Source Files