Documentation
¶
Index ¶
- Variables
- type Abs
- type Add
- type AddScalar
- type At
- type AtVec
- type CELU
- type ColView
- type Concat
- type Cos
- type Div
- type DivScalar
- type Dot
- type Dropout
- type ELU
- type Exp
- type Function
- type GELU
- type HardSigmoid
- type HardTanh
- type Identity
- type LeakyReLU
- type Log
- type Max
- type MaxPooling
- type Min
- type Mish
- type Mul
- type Neg
- type Operand
- type Pow
- type Prod
- type ProdScalar
- type ReLU
- type Reciprocal
- type ReduceMean
- type ReduceSum
- type Reshape
- type ReverseSubScalar
- type RotateR
- type RowView
- type SELU
- type Sigmoid
- type Sin
- type SoftPlus
- type SoftShrink
- type Softmax
- type Softsign
- type SparseMax
- type SparseMaxLoss
- type Sqrt
- type Square
- type Stack
- type Sub
- type SubScalar
- type Swish
- type SwishB
- type Tan
- type Tanh
- type Threshold
- type Transpose
- type UnaryElementwise
- type Vec
- type View
Constants ¶
This section is empty.
Variables ¶
var NewSiLU = NewSwish
NewSiLU (Sigmoid Linear Unit) returns a new function of the form f(x) = x * sigmoid(x). The function in an alias of NewSwish.
Functions ¶
This section is empty.
Types ¶
type Abs ¶ added in v0.6.0
type Abs struct {
*UnaryElementwise
}
Abs is an operator to perform element-wise absolute value function.
type Add ¶
type Add struct {
// contains filtered or unexported fields
}
Add is an operator to perform element-wise sum over two values. y = x1 + x2
type AddScalar ¶
type AddScalar struct {
// contains filtered or unexported fields
}
AddScalar is an operator to perform element-wise addition over two values.
func NewAddScalar ¶
NewAddScalar returns a new AddScalar Function.
type At ¶
type At struct {
// contains filtered or unexported fields
}
At is an operator to obtain the i,j-th value of a matrix.
type AtVec ¶
type AtVec struct {
// contains filtered or unexported fields
}
AtVec is an operator to obtain the i-th value of a vector.
type CELU ¶ added in v0.2.0
type CELU struct {
// contains filtered or unexported fields
}
CELU is an operator to perform the CELU activation. CELU(x) = max(0,x) + min(0,α ∗ (exp(x/α) − 1))
type ColView ¶
type ColView struct {
// contains filtered or unexported fields
}
ColView is an operator to extract the i-th column from a matrix.
func NewColView ¶
NewColView extracts the i-th column from the input matrix.
type Concat ¶
type Concat struct {
// contains filtered or unexported fields
}
Concat is an operator to perform vector concatenation.
type Cos ¶ added in v0.6.0
type Cos struct {
*UnaryElementwise
}
Cos is an operator to perform element-wise cos.
type Div ¶
type Div struct {
// contains filtered or unexported fields
}
Div is an operator to perform element-wise division over two values.
type DivScalar ¶
type DivScalar struct {
// contains filtered or unexported fields
}
DivScalar is an operator to perform element-wise division with a scalar value.
func NewDivScalar ¶
NewDivScalar returns a new DivScalar Function.
type Dot ¶
type Dot struct {
// contains filtered or unexported fields
}
Dot is an operator to perform the dot product over two matrices. y = x1 dot x2
type Dropout ¶
type Dropout struct {
// contains filtered or unexported fields
}
Dropout is an operator to perform elements dropout with a probability.
func NewDropout ¶
NewDropout returns a new Dropout Function.
type ELU ¶
type ELU struct {
// contains filtered or unexported fields
}
ELU is an operator to perform the ELU activation function. ELU(x) = max(0,x) + min(0,α ∗ (exp(x) − 1))
type Exp ¶ added in v0.6.0
type Exp struct {
*UnaryElementwise
}
Exp is an operator to perform element-wise base-e exponential.
type Function ¶
type Function interface {
// Forward computes the output of the function.
Forward() mat.Matrix
// Backward computes the backward pass.
Backward(gy mat.Matrix)
}
Function represents a function with automatic differentiation features.
type GELU ¶ added in v0.6.0
type GELU struct {
*UnaryElementwise
}
GELU is an operator to perform element-wise GELU.
type HardSigmoid ¶ added in v0.6.0
type HardSigmoid struct {
*UnaryElementwise
}
HardSigmoid is an operator to perform element-wise hard sigmoid.
func NewHardSigmoid ¶
func NewHardSigmoid(x Operand) *HardSigmoid
NewHardSigmoid returns a new UnaryElementwise hard sigmoid function.
type HardTanh ¶ added in v0.6.0
type HardTanh struct {
*UnaryElementwise
}
HardTanh is an operator to perform element-wise hard hyperbolic tangent.
func NewHardTanh ¶
NewHardTanh returns a new UnaryElementwise hard hyperbolic tangent function.
type Identity ¶
type Identity struct {
// contains filtered or unexported fields
}
Identity is an operator to perform identity function. y = x
func NewIdentity ¶
NewIdentity returns a new Identity Function.
type LeakyReLU ¶
type LeakyReLU struct {
// contains filtered or unexported fields
}
LeakyReLU is an operator to perform the LeakyReLU activation function. LeakyReLU(x) = max(0,x) + slope ° min(0,x)
func NewLeakyReLU ¶
NewLeakyReLU returns a new LeakyReLU Function.
type Log ¶ added in v0.6.0
type Log struct {
*UnaryElementwise
}
Log is an operator to perform element-wise natural logarithm.
type Max ¶
type Max struct {
// contains filtered or unexported fields
}
Max is an operator to perform element-wise max. y = max(x1, x2)
type MaxPooling ¶
type MaxPooling struct {
// contains filtered or unexported fields
}
MaxPooling is an operator to perform max pooling.
func NewMaxPooling ¶
func NewMaxPooling(x Operand, r, c int) *MaxPooling
NewMaxPooling returns a new MaxPooling Function.
func (*MaxPooling) Backward ¶
func (r *MaxPooling) Backward(gy mat.Matrix)
Backward computes the backward pass.
func (*MaxPooling) Forward ¶
func (r *MaxPooling) Forward() mat.Matrix
Forward computes the output of the function.
type Min ¶
type Min struct {
// contains filtered or unexported fields
}
Min is an operator to perform element-wise min. y = min(x1, x2)
type Mish ¶ added in v0.6.0
type Mish struct {
*UnaryElementwise
}
Mish is an operator to perform element-wise mish.
func NewMish ¶
NewMish returns a new UnaryElementwise Mish function.
Mish is a self-regularized non-monotonic activation function which can be mathematically defined as f(x) = x * tanh(softplus(x)).
Reference: "Mish: A Self Regularized Non-Monotonic Neural Activation Function" by Diganta Misra, 2019 (https://arxiv.org/pdf/1908.08681.pdf)
type Mul ¶
type Mul struct {
// contains filtered or unexported fields
}
Mul is an operator to perform matrix-vector multiplication.
type Neg ¶ added in v0.6.0
type Neg struct {
*UnaryElementwise
}
Neg is an operator to perform element-wise f(x) = -x
type Operand ¶
type Operand interface {
// Value returns the value of the operand.
Value() mat.Matrix
// PropagateGrad propagates the gradients gx to the operands.
PropagateGrad(gx mat.Matrix)
// RequiresGrad returns true if the operand requires gradients.
RequiresGrad() bool
}
Operand is implemented by any value that implements automatic differentiation features.
type Pow ¶
type Pow struct {
// contains filtered or unexported fields
}
Pow is an operator to perform element-wise pow function.
type Prod ¶
type Prod struct {
// contains filtered or unexported fields
}
Prod is an operator to perform element-wise product over two values.
type ProdScalar ¶
type ProdScalar struct {
// contains filtered or unexported fields
}
ProdScalar is an operator to perform element-wise product with a scalar value.
func NewProdScalar ¶
func NewProdScalar(x1, x2 Operand) *ProdScalar
NewProdScalar returns a new ProdScalar Function.
func (*ProdScalar) Backward ¶
func (r *ProdScalar) Backward(gy mat.Matrix)
Backward computes the backward pass.
func (*ProdScalar) Forward ¶
func (r *ProdScalar) Forward() mat.Matrix
Forward computes the output of the node.
type ReLU ¶ added in v0.6.0
type ReLU struct {
*UnaryElementwise
}
ReLU is an operator to perform element-wise Rectified Linear Unit (ReLU)
type Reciprocal ¶ added in v0.6.0
type Reciprocal struct {
*UnaryElementwise
}
Reciprocal is an operator to perform element-wise reciprocal.
func NewReciprocal ¶
func NewReciprocal(x Operand) *Reciprocal
NewReciprocal returns a new UnaryElementwise reciprocal function.
type ReduceMean ¶
type ReduceMean struct {
// contains filtered or unexported fields
}
ReduceMean is an operator to perform reduce-mean function.
func NewReduceMean ¶
func NewReduceMean(x Operand) *ReduceMean
NewReduceMean returns a new ReduceMean Function.
func (*ReduceMean) Backward ¶
func (r *ReduceMean) Backward(gy mat.Matrix)
Backward computes the backward pass.
func (*ReduceMean) Forward ¶
func (r *ReduceMean) Forward() mat.Matrix
Forward computes the output of this node.
type ReduceSum ¶
type ReduceSum struct {
// contains filtered or unexported fields
}
ReduceSum is an operator to perform reduce-sum function.
func NewReduceSum ¶
NewReduceSum returns a new ReduceSum Function.
type Reshape ¶
type Reshape struct {
// contains filtered or unexported fields
}
Reshape is a Function which reshapes an operand into a new matrix of given rows × columns size.
func NewReshape ¶
NewReshape returns a new Reshape Function.
type ReverseSubScalar ¶
type ReverseSubScalar struct {
// contains filtered or unexported fields
}
ReverseSubScalar is the element-wise subtraction function over two values.
func NewReverseSubScalar ¶
func NewReverseSubScalar(x1, x2 Operand) *ReverseSubScalar
NewReverseSubScalar returns a new ReverseSubScalar Function.
func (*ReverseSubScalar) Backward ¶
func (r *ReverseSubScalar) Backward(gy mat.Matrix)
Backward computes the backward pass.
func (*ReverseSubScalar) Forward ¶
func (r *ReverseSubScalar) Forward() mat.Matrix
Forward computes the output of the function.
type RotateR ¶ added in v0.2.0
type RotateR struct {
// contains filtered or unexported fields
}
RotateR is a function to perform a right circular shift of a vector.
func NewRotateR ¶ added in v0.2.0
NewRotateR returns a new RotateR Function. `i` is the number of places by which the elements are shifted.
type RowView ¶
type RowView struct {
// contains filtered or unexported fields
}
RowView is a function to extract the i-th row from the input matrix.
func NewRowView ¶
NewRowView returns a new RowView Function.
type SELU ¶ added in v0.2.0
type SELU struct {
// contains filtered or unexported fields
}
SELU function: f(x) = scale ∗ (max(0,x) + min(0, α ∗ (exp(x) − 1)))
type Sigmoid ¶ added in v0.6.0
type Sigmoid struct {
*UnaryElementwise
}
Sigmoid is an operator to perform element-wise sigmoid.
func NewSigmoid ¶
NewSigmoid returns a new UnaryElementwise sigmoid function.
type Sin ¶ added in v0.6.0
type Sin struct {
*UnaryElementwise
}
Sin is an operator to perform element-wise sin.
type SoftPlus ¶
type SoftPlus struct {
// contains filtered or unexported fields
}
SoftPlus function: f(x) = 1 / β ∗ log(1 + exp(β ∗ x))
func NewSoftPlus ¶
NewSoftPlus returns a new SoftPlus Function.
type SoftShrink ¶
type SoftShrink struct {
// contains filtered or unexported fields
}
SoftShrink function: f(x) = x − λ if x > λ; x + λ if x < −λ; 0 otherwise.
func NewSoftShrink ¶
func NewSoftShrink(x, lambda Operand) *SoftShrink
NewSoftShrink returns a new SoftShrink Function.
func (*SoftShrink) Backward ¶
func (r *SoftShrink) Backward(gy mat.Matrix)
Backward computes the backward pass.
func (*SoftShrink) Forward ¶
func (r *SoftShrink) Forward() mat.Matrix
Forward computes the output of the function.
type Softmax ¶
type Softmax struct {
// contains filtered or unexported fields
}
Softmax is a single-input softmax function.
type Softsign ¶ added in v0.6.0
type Softsign struct {
*UnaryElementwise
}
Softsign is an operator to perform element-wise softsign.
func NewSoftsign ¶
NewSoftsign returns a new UnaryElementwise softsign function.
type SparseMax ¶
type SparseMax struct {
// contains filtered or unexported fields
}
SparseMax function implementation, based on https://github.com/gokceneraslan/SparseMax.torch
func NewSparseMax ¶
NewSparseMax returns a new SparseMax Function.
type SparseMaxLoss ¶
type SparseMaxLoss struct {
// contains filtered or unexported fields
}
SparseMaxLoss function implementation, based on https://github.com/gokceneraslan/SparseMax.torch
func NewSparseMaxLoss ¶
func NewSparseMaxLoss(x Operand) *SparseMaxLoss
NewSparseMaxLoss returns a new SparseMaxLoss Function.
func (*SparseMaxLoss) Backward ¶
func (s *SparseMaxLoss) Backward(gy mat.Matrix)
Backward computes the backward pass.
func (*SparseMaxLoss) Forward ¶
func (s *SparseMaxLoss) Forward() mat.Matrix
Forward computes the output of the function.
type Sqrt ¶ added in v0.6.0
type Sqrt struct {
*UnaryElementwise
}
Sqrt is an operator to perform element-wise square root.
type Square ¶ added in v0.6.0
type Square struct {
*Prod
}
Square is an operator to perform element-wise square.
type Stack ¶
type Stack struct {
// contains filtered or unexported fields
}
Stack is a Function which stacks together all given operand matrices, producing a single bigger matrix as result.
type Sub ¶
type Sub struct {
// contains filtered or unexported fields
}
Sub is an element-wise subtraction function over two values.
type SubScalar ¶
type SubScalar struct {
// contains filtered or unexported fields
}
SubScalar is an element-wise subtraction function with a scalar value.
func NewSubScalar ¶
NewSubScalar returns a new SubScalar Function.
type Swish ¶
type Swish struct {
*UnaryElementwise
}
Swish is an operator to perform element-wise x * sigmoid(x).
type SwishB ¶ added in v0.5.0
type SwishB struct {
// contains filtered or unexported fields
}
SwishB function: f(x) = x * sigmoid.
Reference: "Searching for Activation Functions" by Ramachandran et al, 2017. (https://arxiv.org/pdf/1710.05941.pdf)
type Tan ¶ added in v0.6.0
type Tan struct {
*UnaryElementwise
}
Tan is an operator to perform element-wise tangent.
type Tanh ¶ added in v0.6.0
type Tanh struct {
*UnaryElementwise
}
Tanh is an operator to perform element-wise hyperbolic tangent.
type Threshold ¶
type Threshold struct {
// contains filtered or unexported fields
}
Threshold function: f(x) = x if x > threshold; k otherwise.
func NewThreshold ¶
NewThreshold returns a new Threshold Function.
type Transpose ¶
type Transpose struct {
// contains filtered or unexported fields
}
Transpose is a Function to calculate the transpose of the matrix-operand.
func NewTranspose ¶
NewTranspose returns a new Transpose Function.
type UnaryElementwise ¶
type UnaryElementwise struct {
// contains filtered or unexported fields
}
UnaryElementwise is a single-input element-wise function.
func (*UnaryElementwise) Backward ¶
func (r *UnaryElementwise) Backward(gy mat.Matrix)
Backward computes the backward pass.
func (*UnaryElementwise) Forward ¶
func (r *UnaryElementwise) Forward() mat.Matrix
Forward computes the output of this node.
type Vec ¶
type Vec struct {
// contains filtered or unexported fields
}
Vec is a Function to reshape an matrix-operand into a column vector.
Source Files
¶
- add.go
- addscalar.go
- at.go
- atvec.go
- celu.go
- colview.go
- concat.go
- div.go
- divscalar.go
- dot.go
- dropout.go
- elu.go
- fn.go
- identity.go
- leakyrelu.go
- max.go
- maxpooling.go
- min.go
- misc.go
- mul.go
- pow.go
- prod.go
- prodscalar.go
- reducemean.go
- reducesum.go
- reshape.go
- reversesubscalar.go
- rotater.go
- rowview.go
- selu.go
- softmax.go
- softplus.go
- softshrink.go
- sparsemax.go
- stack.go
- sub.go
- subscalar.go
- swishb.go
- test_utils.go
- threshold.go
- transpose.go
- unaryelementwise.go
- vec.go
- view.go