ag

package
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 24, 2021 License: BSD-2-Clause Imports: 10 Imported by: 0

README

Automatic Differentiation

The ag package (a.k.a. auto-grad) is the centerpiece of the spaGO machine learning framework.

Neural models optimized by back-propagation require gradients to be available during training. The set of expressions characterizing the forward-step of such models must be defined within the ag.Graph to take advantage of automatic differentiation.

Let's see if spaGO can tell us what two plus five is. Then, let's go one step further now and ask spaGO to give us the gradients on a and b, starting with arbitrary output gradients.

Write some code:

package main

import (
	"fmt"

	mat "github.com/nlpodyssey/spago/pkg/mat32"
	"github.com/nlpodyssey/spago/pkg/ml/ag"
)

func main() {
	// create a new node of type variable with a scalar
	a := ag.NewVariable(mat.NewScalar(2.0), true)
	// create another node of type variable with a scalar
	b := ag.NewVariable(mat.NewScalar(5.0), true)
	// create an addition operator (the calculation is actually performed here)
	c := ag.Add(a, b)
	// print the result
	fmt.Printf("c = %v\n", c.Value())

	ag.Backward(c, ag.OutputGrad(mat.NewScalar(0.5)))
	fmt.Printf("ga = %v\n", a.Grad())
	fmt.Printf("gb = %v\n", b.Grad())
}

It should print:

c = [7]
ga = [0.5]
gb = [0.5]

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Backward 

func Backward(node Node, opts ...BackwardOption)

Backward performs the back-propagation. See Graph.Backward() for more information.

func BackwardAll 

func BackwardAll()

BackwardAll performs full back-propagation from the last node of the graph. It requires the root nodes to have assigned gradients already.

func ClearGlobalGraph 

func ClearGlobalGraph()

ClearGlobalGraph clears the global graph. This is a destructive operation. See Graph.Clear() for more information.

func ClearGlobalGraphForReuse 

func ClearGlobalGraphForReuse()

ClearGlobalGraphForReuse does the same thing as ClearGlobalGraph(), with the difference that the graph structure is maintained. See Graph.ClearForReuse() for more information.

func Forward 

func Forward(opts ...ForwardOption)

Forward computes the results of the entire global raph.

func IncTimeStep 

func IncTimeStep()

IncTimeStep increments the value of the global graph's TimeStep by one.

func Operands 

func Operands(xs []Node) []fn.Operand

Operands cast a slice of nodes into a slice of operands.

func ReplaceValue 

func ReplaceValue(node Node, value mat.Matrix)

ReplaceValue replaces the current value of a variable Node with the given value, on the global graph. It panics if node is not a variable.

func TimeStep 

func TimeStep() int

TimeStep is an integer value associated with the global graph, which can be useful to perform truncated back propagation.

func ZeroGrad 

func ZeroGrad()

ZeroGrad sets the gradients of all nodes of the global graph to zero.

Types

type BackwardOption 

type BackwardOption func(*backwardHandler)

BackwardOption allows to adapt the Backward() to your specific needs.

func OutputGrad 

func OutputGrad(grad mat.Matrix) BackwardOption

OutputGrad is an option that sets the output gradients which are the starting point for the back-propagation (Backward).

func Truncate 

func Truncate(backSteps int) BackwardOption

Truncate is an option that sets the number of back steps for the Truncated Back-Propagation.

type ForwardOption 

type ForwardOption func(*forwardHandler)

ForwardOption allows to adapt the Forward() to your specific needs.

func Range 

func Range(fromTimeStep, toTimeStep int) ForwardOption

Range allows you to limit the forward computation within a time-step range. By default, the forward computes from the first node at time-step 0 to the last node at the current time-step.

type GradValue 

type GradValue interface {
	// Value returns the value of the node.
	// If the node is a variable it returns its value, otherwise returns the cached result of the forward pass.
	Value() mat.Matrix
	// ScalarValue returns the scalar value of the node. It panics if the value is not a scalar.
	ScalarValue() mat.Float
	// Grad returns the gradients accumulated during the backward pass.
	Grad() mat.Matrix
	// HasGrad returns true if there are accumulated gradients.
	HasGrad() bool
	// RequiresGrad returns true if the node requires gradients.
	RequiresGrad() bool
	// PropagateGrad propagates the gradients to the node.
	PropagateGrad(gx mat.Matrix)
	// ZeroGrad set the gradients to zeros.
	ZeroGrad()
}

GradValue extends the fn.Operand interface providing more convenient methods to handle gradients in the context of automatic differentiation.

type Graph 

type Graph struct {
	// contains filtered or unexported fields
}

The Graph a.k.a. expression graph or computational graph is the centerpiece of the spaGO machine learning framework. It takes the form of a directed graph with no directed cycles (DAG).

func GetGlobalGraph 

func GetGlobalGraph() *Graph

GetGlobalGraph returns the global graph. Although technically you could reassign the returned graph, please do not do so; imagine that its reference is immutable. Otherwise you are likely to generate inconsistent computations. To clean the global graph, you can use ClearGlobalGraph() or ClearGlobalGraphForReuse().

func NewGraph 

func NewGraph(opts ...GraphOption) *Graph

NewGraph returns a new initialized graph. It can take an optional random generator of type rand.Rand.

func (*Graph) Abs 

func (g *Graph) Abs(x Node) Node

Abs returns a new operator node as a result of the `Abs` function.

func (*Graph) Add 

func (g *Graph) Add(x1 Node, x2 Node) Node

Add returns a new operator node as a result of the fn.Add function. The first node may be null. This help to keep the code as concise as possible e.g. during accumulation.

func (*Graph) AddScalar 

func (g *Graph) AddScalar(x1 Node, x2 Node) Node

AddScalar returns a new operator node as a result of the fn.AddScalar function.

func (*Graph) At 

func (g *Graph) At(x Node, i int, j int) Node

At returns a new operator node as a result of the fn.At function.

func (*Graph) AtVec 

func (g *Graph) AtVec(x Node, i int) Node

AtVec returns a new operator node as a result of the fn.AtVec function.

func (*Graph) Backward 

func (g *Graph) Backward(node Node, opts ...BackwardOption)

Backward performs the back-propagation. It visits each node in reverse topological order, to propagate the gradients from the given node all the way back to the leaf. Note that the gradients are summed to the existing ones. Unless that's what you want, make sure all nodes have zero gradients.

The back-propagation starts from the node's output gradients, following these mutually exclusive rules: 

a) the node has gradients (probably assigned externally via node.PropagateGrads()), use those;
b) the output gradients are passed through the backward options, use those;
c) the output gradients are automatically assigned by finding the derivative of the node with respect
   to the node itself (dy/dy = 1).

If the optional back steps are set, a Truncated Back-Propagation Through Time is carried out, that is: the visit ends as soon as it is encountered a node with time-step less or equal to the number of back steps. The TBTT can perform without the need to recalculate the values of previous nodes (Williams and Peng, 1990).

func (*Graph) BackwardAll 

func (g *Graph) BackwardAll()

BackwardAll performs full back-propagation from the last node of the graph. It requires the root nodes to have assigned gradients already.

func (*Graph) CELU added in v0.2.0 

func (g *Graph) CELU(x Node, alpha Node) Node

CELU returns a new operator node as a result of the fn.CELU function.

func (*Graph) Clear 

func (g *Graph) Clear()

Clear cleans the graph. This is a destructive operation. It is not mandatory to call this method, but it is strongly recommended to do so when you finish using the graph. The cleaning of the graph improves the memory management and therefore the efficiency of execution. Clear releases the matrices underlying the nodes so to reduce the need of future new time-consuming allocations. It is important to stress that calling g.Clean(), the "value" and "grad" of the operators nodes are freed (set to nil). Whoever is using the Value() or Grad() properties of a node, does so at his own risk. It is therefore recommended to make always a copy of the return value of Value() or Grad(). Alternatively, you can use the convenient graph's methods g.GetCopiedValue(node) and g.GetCopiedGrad(node).

func (*Graph) ClearForReuse 

func (g *Graph) ClearForReuse()

ClearForReuse does the same thing as Clear(), with the difference that the graph structure (i.e. how nodes are connected to each other) is maintained. This allows you to efficiently use the graph as if it were "pre-computed" (see the ForwardAll() method for this usage).

func (*Graph) ColView 

func (g *Graph) ColView(x Node, column int) Node

ColView returns a new operator node as a result of the fn.ColView function.

func (*Graph) Concat 

func (g *Graph) Concat(xs ...Node) Node

Concat returns a new operator node as a result of the fn.Concat function.

func (*Graph) ConcurrentComputations added in v0.3.0 

func (g *Graph) ConcurrentComputations() int

ConcurrentComputations returns the maximum number of concurrent computations handled by the Graph for heavy tasks such as forward and backward steps.

func (*Graph) Constant 

func (g *Graph) Constant(value mat.Float) Node

Constant returns a scalar Node that that doesn't require gradients. For the same value, a previously created Node is returned without creating a new one. Useful for example in the case of epsilon and number like 0.0 or 1.0.

func (*Graph) Cos 

func (g *Graph) Cos(x Node) Node

Cos returns a new operator node as a result of the `Cos` function.

func (*Graph) Div 

func (g *Graph) Div(x1 Node, x2 Node) Node

Div returns a new operator node as a result of the fn.Div function.

func (*Graph) DivScalar 

func (g *Graph) DivScalar(x1 Node, x2 Node) Node

DivScalar returns a new operator node as a result of the fn.DivScalar function.

func (*Graph) Dot 

func (g *Graph) Dot(x1 Node, x2 Node) Node

Dot returns a new operator node as a result of the fn.Dot function.

func (*Graph) Dropout 

func (g *Graph) Dropout(x Node, p mat.Float) Node

Dropout returns a new operator node as a result of the fn.Dropout function.

func (*Graph) ELU 

func (g *Graph) ELU(x Node, alpha Node) Node

ELU returns a new operator node as a result of the fn.ELU function.

func (*Graph) Exp 

func (g *Graph) Exp(x Node) Node

Exp returns a new operator node as a result of the `Exp` function.

func (*Graph) Forward 

func (g *Graph) Forward(opts ...ForwardOption)

Forward computes the results of the entire Graph. Usually you don't need to execute Forward() manually in the define-by-run configuration (default). If you do, all values will be recalculated. You can also choose through the Range option to recalculate only a portion of nodes. Instead, it is required to obtain the value of the nodes in case the Graph has been created with IncrementalForward(false).

func (*Graph) GELU added in v0.2.0 

func (g *Graph) GELU(x Node) Node

GELU returns a new operator node as a result of the fn.GELU function.

func (*Graph) GetCopiedGrad 

func (g *Graph) GetCopiedGrad(node Node) mat.Matrix

GetCopiedGrad returns a copy of the gradients of a Node. If the gradients are nil, GetCopiedGrad returns nil as well. The returned value is a copy, so it is safe to use even after the graph has been cleared calling Graph.Clear(). It is important to remember that the Grad() property of a Node is a weak access, as the matrix derived from graph's operations can be freed.

func (*Graph) GetCopiedValue 

func (g *Graph) GetCopiedValue(node Node) mat.Matrix

GetCopiedValue returns a copy of the value of a Node. If the value is nil, GetCopiedValue returns nil as well. The returned value is a copy, so it is safe to use even after the graph has been cleared calling Graph.Clear(). It is important to remember that the Value() property of a Node is a weak access, as the matrix derived from graph's operations can be freed.

func (*Graph) HardSigmoid 

func (g *Graph) HardSigmoid(x Node) Node

HardSigmoid returns a new operator node as a result of the `HardSigmoid` function.

func (*Graph) HardTanh 

func (g *Graph) HardTanh(x Node) Node

HardTanh returns a new operator node as a result of the `HardTanh` function.

func (*Graph) Identity 

func (g *Graph) Identity(x Node) Node

Identity returns a new operator node as a result of the fn.Identity function.

func (*Graph) IncTimeStep 

func (g *Graph) IncTimeStep()

IncTimeStep increments the value of the graph's TimeStep by one.

func (*Graph) IncrementalForwardEnabled added in v0.5.0 

func (g *Graph) IncrementalForwardEnabled() bool

IncrementalForwardEnabled returns whether the computation happens during the graph definition. See ag.IncrementalForward() option.

func (*Graph) Invoke 

func (g *Graph) Invoke(operator OpName, xs ...Node) Node

Invoke returns a new node as a result of the application of the input operator.

func (*Graph) LeakyReLU 

func (g *Graph) LeakyReLU(x Node, alpha Node) Node

LeakyReLU returns a new operator node as a result of the fn.LeakyReLU function.

func (*Graph) Log 

func (g *Graph) Log(x Node) Node

Log returns a new operator node as a result of the `Log` function.

func (*Graph) LogSoftmax added in v0.5.0 

func (g *Graph) LogSoftmax(x Node) Node

LogSoftmax returns a new operator node as a result of Log(Softmax(x)).

func (*Graph) MarshalBinary added in v0.6.0 

func (g *Graph) MarshalBinary() ([]byte, error)

MarshalBinary satisfies encoding.BinaryMarshaler interface and prevents a Graph to be encoded to binary representation. This is relevant in the context of a Graph being part of a nn.Model: when serializing a model to binary, we want to skip the Graph, since it is part of the runtime context only.

func (*Graph) Max 

func (g *Graph) Max(x1 Node, x2 Node) Node

Max returns a new operator node as a result of the fn.Max function.

func (*Graph) MaxPooling 

func (g *Graph) MaxPooling(x Node, rows, columns int) Node

MaxPooling returns a new operator node as a result of the fn.MaxPooling function.

func (*Graph) Mean 

func (g *Graph) Mean(xs []Node) Node

Mean returns the value that describes the average of the sample.

func (*Graph) Min 

func (g *Graph) Min(x1 Node, x2 Node) Node

Min returns a new operator node as a result of the fn.Min function.

func (*Graph) Mish 

func (g *Graph) Mish(x Node) Node

Mish returns a new operator node as a result of the `Mish` function.

func (*Graph) Mul 

func (g *Graph) Mul(x1 Node, x2 Node) Node

Mul returns a new operator node as a result of the fn.Mul function.

func (*Graph) Neg 

func (g *Graph) Neg(x Node) Node

Neg returns a new operator node as a result of the `Neg` function.

func (*Graph) NewOperator 

func (g *Graph) NewOperator(f fn.Function, operands ...Node) Node

NewOperator creates a new operator along with its forward pass. Please note that operations must be performed among nodes belonging to the same graph; it panics otherwise.

func (*Graph) NewScalar 

func (g *Graph) NewScalar(value mat.Float) Node

NewScalar creates a variable node that doesn't require gradients. TODO: Why shouldn't gradient be required by default?

func (*Graph) NewScalarWithName added in v0.6.0 

func (g *Graph) NewScalarWithName(value mat.Float, name string) Node

NewScalarWithName creates a variable node that doesn't require gradients. TODO: Why shouldn't gradient be required by default?

func (*Graph) NewVariable 

func (g *Graph) NewVariable(value mat.Matrix, requiresGrad bool) Node

NewVariable creates and returns a new node.

func (*Graph) NewVariableWithName added in v0.6.0 

func (g *Graph) NewVariableWithName(value mat.Matrix, requiresGrad bool, name string) Node

NewVariableWithName creates and returns a new node.

func (*Graph) NewWrap 

func (g *Graph) NewWrap(value GradValue) Node

NewWrap creates a new wrapper Node for the given value, attaching it to the graph.

func (*Graph) NewWrapNoGrad 

func (g *Graph) NewWrapNoGrad(value GradValue) Node

NewWrapNoGrad is similar to NewWrap, but it disables automatic differentiation on the new node.

func (*Graph) Nodes added in v0.6.0 

func (g *Graph) Nodes() []Node

Nodes returns the nodes of the graph.

func (*Graph) PositiveELU 

func (g *Graph) PositiveELU(x Node) Node

PositiveELU returns a new operator node as a result of ELU(x) + 1.

func (*Graph) Pow 

func (g *Graph) Pow(x Node, power mat.Float) Node

Pow returns a new operator node as a result of the fn.Pow function.

func (*Graph) Prod 

func (g *Graph) Prod(x1 Node, x2 Node) Node

Prod returns a new operator node as a result of the fn.Prod function.

func (*Graph) ProdScalar 

func (g *Graph) ProdScalar(x1 Node, x2 Node) Node

ProdScalar returns a new operator node as a result of the fn.ProdScalar function.

func (*Graph) ReLU 

func (g *Graph) ReLU(x Node) Node

ReLU returns a new operator node as a result of the `ReLU` function.

func (*Graph) Reciprocal 

func (g *Graph) Reciprocal(x Node) Node

Reciprocal returns a new operator node as a result of the `Reciprocal` function.

func (*Graph) ReduceMean 

func (g *Graph) ReduceMean(x Node) Node

ReduceMean returns a new operator node as a result of the fn.ReduceMean function.

func (*Graph) ReduceSum 

func (g *Graph) ReduceSum(x Node) Node

ReduceSum returns a new operator node as a result of the fn.ReduceSum function.

func (*Graph) ReplaceValue 

func (g *Graph) ReplaceValue(node Node, value mat.Matrix)

ReplaceValue replaces the current value of a variable Node with the given value. It panics if node is not a variable.

func (*Graph) Reshape 

func (g *Graph) Reshape(x Node, rows, columns int) Node

Reshape returns a new operator node as a result of the fn.Reshape function.

func (*Graph) ReverseSub 

func (g *Graph) ReverseSub(x1 Node, x2 Node) Node

ReverseSub returns a new operator node as a result of the fn.ReverseSub function.

func (*Graph) RotateR 

func (g *Graph) RotateR(x Node, i int) Node

RotateR performs the right circular shift. `i` is the number of places by which the elements are shifted.

func (*Graph) RowView 

func (g *Graph) RowView(x Node, row int) Node

RowView returns a new operator node as a result of the fn.RowView function.

func (*Graph) SELU added in v0.2.0 

func (g *Graph) SELU(x Node, alpha Node, scale Node) Node

SELU returns a new operator node as a result of the fn.SELU function.

func (*Graph) SiLU added in v0.5.0 

func (g *Graph) SiLU(x Node) Node

SiLU returns a new operator node as a result of the fn.SiLU function.

func (*Graph) Sigmoid 

func (g *Graph) Sigmoid(x Node) Node

Sigmoid returns a new operator node as a result of the `Sigmoid` function.

func (*Graph) Sin 

func (g *Graph) Sin(x Node) Node

Sin returns a new operator node as a result of the `Sin` function.

func (*Graph) SoftPlus 

func (g *Graph) SoftPlus(x Node, beta Node, threshold Node) Node

SoftPlus returns a new operator node as a result of the fn.SoftPlus function.

func (*Graph) SoftShrink 

func (g *Graph) SoftShrink(x Node, lambda Node) Node

SoftShrink returns a new operator node as a result of the fn.SoftShrink function.

func (*Graph) Softmax 

func (g *Graph) Softmax(x Node) Node

Softmax returns a new operator node as a result of the fn.Softmax function.

func (*Graph) Softsign 

func (g *Graph) Softsign(x Node) Node

Softsign returns a new operator node as a result of the `SoftSign` function.

func (*Graph) SparseMax 

func (g *Graph) SparseMax(x Node) Node

SparseMax returns a new operator node as a result of the fn.SparseMax function.

func (*Graph) SparseMaxLoss 

func (g *Graph) SparseMaxLoss(x Node) Node

SparseMaxLoss returns a new operator node as a result of the fn.SparseMaxLoss function.

func (*Graph) Sqrt 

func (g *Graph) Sqrt(x Node) Node

Sqrt returns a new operator node as a result of the `Sqrt` function.

func (*Graph) Square 

func (g *Graph) Square(x Node) Node

Square returns a new operator node as a result of the fn.Prod(x, x) function.

func (*Graph) Stack 

func (g *Graph) Stack(xs ...Node) Node

Stack returns a new operator node as a result of the fn.Stack function.

func (*Graph) Sub 

func (g *Graph) Sub(x1 Node, x2 Node) Node

Sub returns a new operator node as a result of the fn.Sub function.

func (*Graph) SubScalar 

func (g *Graph) SubScalar(x1 Node, x2 Node) Node

SubScalar returns a new operator node as a result of the fn.SubScalar function.

func (*Graph) Sum 

func (g *Graph) Sum(xs ...Node) Node

Sum returns the value that describes the sum of the sample. It panics if the input is empty.

func (*Graph) Swish 

func (g *Graph) Swish(x Node) Node

Swish returns a new operator node as a result of the fn.Swish function.

func (*Graph) SwishB added in v0.5.0 

func (g *Graph) SwishB(x Node, beta Node) Node

SwishB returns a new operator node as a result of the fn.SwishB function.

func (*Graph) T 

func (g *Graph) T(x Node) Node

T returns a new operator node as a result of the fn.T function.

func (*Graph) Tan 

func (g *Graph) Tan(x Node) Node

Tan returns a new operator node as a result of the `Tan` function.

func (*Graph) Tanh 

func (g *Graph) Tanh(x Node) Node

Tanh returns a new operator node as a result of the `Tanh` function.

func (*Graph) Threshold 

func (g *Graph) Threshold(x Node, threshold Node, k Node) Node

Threshold returns a new operator node as a result of the fn.Threshold function.

func (*Graph) TimeStep 

func (g *Graph) TimeStep() int

TimeStep is an integer value associated with the graph, which can be useful to perform truncated back propagation. This value is 0 for a new Graph, and can be incremented calling IncTimeStep.

func (*Graph) UnmarshalBinary added in v0.6.0 

func (g *Graph) UnmarshalBinary(data []byte) error

UnmarshalBinary satisfies encoding.BinaryUnmarshaler interface.

func (*Graph) Vec 

func (g *Graph) Vec(x Node) Node

Vec returns a new operator node as a result of the fn.Vec function.

func (*Graph) View 

func (g *Graph) View(x Node, row, column, xStride, yStride int) Node

View returns a new operator node as a result of the fn.View function.

func (*Graph) ZeroGrad 

func (g *Graph) ZeroGrad()

ZeroGrad sets the gradients of all nodes to zero.

type GraphOption 

type GraphOption func(*Graph)

GraphOption allows to configure a new Graph with your specific needs.

func ConcurrentComputations 

func ConcurrentComputations(value int) GraphOption

ConcurrentComputations sets the maximum number of concurrent computations handled by the Graph for heavy tasks such as forward and backward steps. The value 1 corresponds to sequential execution.

func IncrementalForward 

func IncrementalForward(value bool) GraphOption

IncrementalForward sets whether to compute the forward during the graph definition (default true). When enabled it lets you access to the Value() resulting from the computation. There are particular cases where you don't need intermediate values and computing the forward after the graph definition can be more efficient though.

func Rand 

func Rand(rand *rand.LockedRand) GraphOption

Rand sets the generator of random numbers.

func RandSeed 

func RandSeed(seed uint64) GraphOption

RandSeed set a new generator of random numbers with the given seed.

type Node 

type Node interface {
	GradValue
	// Graph returns the graph this node belongs to.
	Graph() *Graph
	// ID returns the ID of the node in the graph.
	ID() int
	// TimeStep returns the time-step associated to this node.
	TimeStep() int
}

Node is implemented by any value that can represent a node of a Graph.

func Abs 

func Abs(x Node) Node

Abs returns a new operator node as a result of the `Abs` function.

func Add 

func Add(x1 Node, x2 Node) Node

Add returns a new operator node as a result of the fn.Add function. The first node may be null. This help to keep the code as concise as possible e.g. during accumulation.

func AddScalar 

func AddScalar(x1 Node, x2 Node) Node

AddScalar returns a new operator node as a result of the fn.AddScalar function.

func At 

func At(x Node, i int, j int) Node

At returns a new operator node as a result of the fn.At function.

func AtVec 

func AtVec(x Node, i int) Node

AtVec returns a new operator node as a result of the fn.AtVec function.

func CELU added in v0.2.0 

func CELU(x Node, alpha Node) Node

CELU returns a new operator node as a result of the fn.CELU function.

func ColView 

func ColView(x Node, column int) Node

ColView returns a new operator node as a result of the fn.ColView function.

func Concat 

func Concat(xs ...Node) Node

Concat returns a new operator node as a result of the fn.Concat function.

func Cos 

func Cos(x Node) Node

Cos returns a new operator node as a result of the `Cos` function.

func Div 

func Div(x1 Node, x2 Node) Node

Div returns a new operator node as a result of the fn.Div function.

func DivScalar 

func DivScalar(x1 Node, x2 Node) Node

DivScalar returns a new operator node as a result of the fn.DivScalar function.

func Dot 

func Dot(x1 Node, x2 Node) Node

Dot returns a new operator node as a result of the fn.Dot function.

func Dropout 

func Dropout(x Node, p mat.Float) Node

Dropout returns a new operator node as a result of the fn.Dropout function.

func ELU 

func ELU(x Node, alpha Node) Node

ELU returns a new operator node as a result of the fn.ELU function.

func Exp 

func Exp(x Node) Node

Exp returns a new operator node as a result of the `Exp` function.

func GELU added in v0.2.0 

func GELU(x Node) Node

GELU returns a new operator node as a result of the fn.GELU function.

func HardSigmoid 

func HardSigmoid(x Node) Node

HardSigmoid returns a new operator node as a result of the `HardSigmoid` function.

func HardTanh 

func HardTanh(x Node) Node

HardTanh returns a new operator node as a result of the `HardTanh` function.

func Identity 

func Identity(x Node) Node

Identity returns a new operator node as a result of the fn.Identity function.

func Invoke 

func Invoke(operator OpName, xs ...Node) Node

Invoke returns a new node as a result of the application of the input operator.

func LeakyReLU 

func LeakyReLU(x Node, alpha Node) Node

LeakyReLU returns a new operator node as a result of the fn.LeakyReLU function.

func Log 

func Log(x Node) Node

Log returns a new operator node as a result of the `Log` function.

func LogSoftmax added in v0.5.0 

func LogSoftmax(x Node) Node

LogSoftmax returns a new operator node as a result of Log(Softmax(x)).

func Map added in v0.2.0 

func Map(mapping func(Node) Node, xs []Node) []Node

Map returns a transformed version of xs with all its components modified according to the mapping function. It is useful for applying an operator to a sequence of nodes. Keep in mind that using this function has an overhead because of the callback, however insignificant compared to mathematical computations.

func Max 

func Max(x1 Node, x2 Node) Node

Max returns a new operator node as a result of the fn.Max function.

func MaxPooling 

func MaxPooling(x Node, rows, columns int) Node

MaxPooling returns a new operator node as a result of the fn.MaxPooling function.

func Mean 

func Mean(xs []Node) Node

Mean returns the value that describes the average of the sample.

func Min 

func Min(x1 Node, x2 Node) Node

Min returns a new operator node as a result of the fn.Min function.

func Mish 

func Mish(x Node) Node

Mish returns a new operator node as a result of the `Mish` function.

func Mul 

func Mul(x1 Node, x2 Node) Node

Mul returns a new operator node as a result of the fn.Mul function.

func Neg 

func Neg(x Node) Node

Neg returns a new operator node as a result of the `Neg` function.

func NewOperator 

func NewOperator(f fn.Function, operands ...Node) Node

NewOperator creates a new operator along with its forward pass.

func NewScalar 

func NewScalar(value mat.Float) Node

NewScalar creates a variable node that doesn't require gradients.

func NewVariable 

func NewVariable(value mat.Matrix, requiresGrad bool) Node

NewVariable creates and returns a new node.

func NewWrap 

func NewWrap(value GradValue) Node

NewWrap creates a new wrapper Node for the given value, attaching it to the global graph.

func NewWrapNoGrad 

func NewWrapNoGrad(value GradValue) Node

NewWrapNoGrad is similar to NewWrap, but it disables automatic differentiation on the new node.

func Nodes added in v0.6.0 

func Nodes() []Node

Nodes returns the nodes of the graph.

func PositiveELU 

func PositiveELU(x Node) Node

PositiveELU returns a new operator node as a result of ELU(x, 1.0) + 1.

func Pow 

func Pow(x Node, power mat.Float) Node

Pow returns a new operator node as a result of the fn.Pow function.

func Prod 

func Prod(x1 Node, x2 Node) Node

Prod returns a new operator node as a result of the fn.Prod function.

func ProdScalar 

func ProdScalar(x1 Node, x2 Node) Node

ProdScalar returns a new operator node as a result of the fn.ProdScalar function.

func ReLU 

func ReLU(x Node) Node

ReLU returns a new operator node as a result of the `ReLU` function.

func Reciprocal 

func Reciprocal(x Node) Node

Reciprocal returns a new operator node as a result of the `Reciprocal` function.

func ReduceMean 

func ReduceMean(x Node) Node

ReduceMean returns a new operator node as a result of the fn.ReduceMean function.

func ReduceSum 

func ReduceSum(x Node) Node

ReduceSum returns a new operator node as a result of the fn.ReduceSum function.

func Reshape 

func Reshape(x Node, rows, columns int) Node

Reshape returns a new operator node as a result of the fn.Reshape function.

func ReverseSub 

func ReverseSub(x1 Node, x2 Node) Node

ReverseSub returns a new operator node as a result of the fn.ReverseSub function.

func RotateR added in v0.2.0 

func RotateR(x Node, i int) Node

RotateR performs the right circular shift. `i` is the number of places by which the elements are shifted.

func RowView 

func RowView(x Node, row int) Node

RowView returns a new operator node as a result of the fn.RowView function.

func SELU added in v0.2.0 

func SELU(x Node, alpha Node, scale Node) Node

SELU returns a new operator node as a result of the fn.SELU function.

func SiLU added in v0.5.0 

func SiLU(x Node) Node

SiLU returns a new operator node as a result of the fn.SiLU function.

func Sigmoid 

func Sigmoid(x Node) Node

Sigmoid returns a new operator node as a result of the `Sigmoid` function.

func Sin 

func Sin(x Node) Node

Sin returns a new operator node as a result of the `Sin` function.

func SoftPlus 

func SoftPlus(x Node, beta Node, threshold Node) Node

SoftPlus returns a new operator node as a result of the fn.SoftPlus function.

func SoftShrink 

func SoftShrink(x Node, lambda Node) Node

SoftShrink returns a new operator node as a result of the fn.SoftShrink function.

func Softmax 

func Softmax(x Node) Node

Softmax returns a new operator node as a result of the fn.Softmax function.

func Softsign 

func Softsign(x Node) Node

Softsign returns a new operator node as a result of the `SoftSign` function.

func SparseMax 

func SparseMax(x Node) Node

SparseMax returns a new operator node as a result of the fn.SparseMax function.

func SparseMaxLoss 

func SparseMaxLoss(x Node) Node

SparseMaxLoss returns a new operator node as a result of the fn.SparseMaxLoss function.

func Sqrt 

func Sqrt(x Node) Node

Sqrt returns a new operator node as a result of the `Sqrt` function.

func Square 

func Square(x Node) Node

Square returns a new operator node as a result of the fn.Prod(x, x) function.

func Stack 

func Stack(xs ...Node) Node

Stack returns a new operator node as a result of the fn.Stack function.

func Sub 

func Sub(x1 Node, x2 Node) Node

Sub returns a new operator node as a result of the fn.Sub function.

func SubScalar 

func SubScalar(x1 Node, x2 Node) Node

SubScalar returns a new operator node as a result of the fn.SubScalar function.

func Sum 

func Sum(xs ...Node) Node

Sum returns the value that describes the sum of the sample.

func Swish 

func Swish(x Node) Node

Swish returns a new operator node as a result of the fn.Swish function.

func SwishB added in v0.5.0 

func SwishB(x Node, beta Node) Node

SwishB returns a new operator node as a result of the fn.SwishB function.

func T 

func T(x Node) Node

T returns a new operator node as a result of the fn.T function.

func Tan 

func Tan(x Node) Node

Tan returns a new operator node as a result of the `Tan` function.

func Tanh 

func Tanh(x Node) Node

Tanh returns a new operator node as a result of the `Tanh` function.

func Threshold 

func Threshold(x Node, threshold Node, k Node) Node

Threshold returns a new operator node as a result of the fn.Threshold function.

func Vec 

func Vec(x Node) Node

Vec returns a new operator node as a result of the fn.Vec function.

func View 

func View(x Node, row, column, xStride, yStride int) Node

View returns a new operator node as a result of the fn.View function.

type OpName 

type OpName int

OpName is the enumeration-like type used for the set of operators supported by spaGO. 

const (
	// OpIdentity identifies the Graph.Identity operator.
	OpIdentity OpName = iota
	// OpDropout identifies the Graph.Dropout operator.
	OpDropout
	// OpAtVec identifies the Graph.AtVec operator.
	OpAtVec
	// OpAt identifies the Graph.At operator.
	OpAt
	// OpAdd identifies the Graph.Add operator.
	OpAdd
	// OpSub identifies the Graph.Sub operator.
	OpSub
	// OpSubScalar identifies the Graph.SubScalar operator.
	OpSubScalar
	// OpAddScalar identifies the Graph.AddScalar operator.
	OpAddScalar
	// OpReverseSub identifies the Graph.ReverseSub operator.
	OpReverseSub
	// OpProd identifies the Graph.Prod operator.
	OpProd
	// OpDiv identifies the Graph.Div operator.
	OpDiv
	// OpProdScalar identifies the Graph.ProdScalar operator.
	OpProdScalar
	// OpDivScalar identifies the Graph.DivScalar operator.
	OpDivScalar
	// OpMul identifies the Graph.Mul operator.
	OpMul
	// OpDot identifies the Graph.Dot operator.
	OpDot
	// OpReshape identifies the Graph.Reshape operator.
	OpReshape
	// OpMaxPooling identifies the Graph.MaxPooling operator.
	OpMaxPooling
	// OpView identifies the Graph.View operator.
	OpView
	// OpRowView identifies the Graph.RowView operator.
	OpRowView
	// OpColView identifies the Graph.ColView operator.
	OpColView
	// OpVec identifies the Graph.Vec operator.
	OpVec
	// OpRotateR identifies the Graph.RotateR operator.
	OpRotateR
	// OpT identifies the Graph.T operator.
	OpT
	// OpSquare identifies the Graph.Square operator.
	OpSquare
	// OpPow identifies the Graph.Pow operator.
	OpPow
	// OpSqrt identifies the Graph.Sqrt operator.
	OpSqrt
	// OpTan identifies the Graph.Tan operator.
	OpTan
	// OpTanh identifies the Graph.Tanh operator.
	OpTanh
	// OpSigmoid identifies the Graph.Sigmoid operator.
	OpSigmoid
	// OpHardSigmoid identifies the Graph.HardSigmoid operator.
	OpHardSigmoid
	// OpHardTanh identifies the Graph.HardTanh operator.
	OpHardTanh
	// OpSoftsign identifies the Graph.Softsign operator.
	OpSoftsign
	// OpReLU identifies the Graph.ReLU operator.
	OpReLU
	// OpCELU identifies the Graph.CELU operator.
	OpCELU
	// OpGELU identifies the Graph.GELU operator.
	OpGELU
	// OpELU identifies the Graph.ELU operator.
	OpELU
	// OpPositiveELU identifies the Graph.PositiveELU operator.
	OpPositiveELU
	// OpSwishB identifies the Graph.SwishB operator.
	OpSwishB
	// OpSwish identifies the Graph.Swish operator.
	OpSwish
	// OpSiLU identifies the Graph.SiLU operator.
	OpSiLU
	// OpMish identifies the Graph.Mish operator.
	OpMish
	// OpLeakyReLU identifies the Graph.LeakyReLU operator.
	OpLeakyReLU
	// OpSELU identifies the Graph.SELU operator.
	OpSELU
	// OpSoftPlus identifies the Graph.SoftPlus operator.
	OpSoftPlus
	// OpSoftShrink identifies the Graph.SoftShrink operator.
	OpSoftShrink
	// OpThreshold identifies the Graph.Threshold operator.
	OpThreshold
	// OpSoftmax identifies the Graph.Softmax operator.
	OpSoftmax
	// OpLogSoftmax identifies the Graph.LogSoftmax operator.
	OpLogSoftmax
	// OpSparseMax identifies the Graph.SparseMax operator.
	OpSparseMax
	// OpSparseMaxLoss identifies the Graph.SparseMaxLoss operator.
	OpSparseMaxLoss
	// OpSin identifies the Graph.Sin operator.
	OpSin
	// OpCos identifies the Graph.Cos operator.
	OpCos
	// OpExp identifies the Graph.Exp operator.
	OpExp
	// OpLog identifies the Graph.Log operator.
	OpLog
	// OpAbs identifies the Graph.Abs operator.
	OpAbs
	// OpNeg identifies the Graph.Neg operator.
	OpNeg
	// OpReciprocal identifies the Graph.Reciprocal operator.
	OpReciprocal
	// OpMax identifies the Graph.Max operator.
	OpMax
	// OpMin identifies the Graph.Min operator.
	OpMin
	// OpReduceSum identifies the Graph.ReduceSum operator.
	OpReduceSum
	// OpReduceMean identifies the Graph.ReduceMean operator.
	OpReduceMean
	// OpMean identifies the Graph.Mean operator.
	OpMean
	// OpSum identifies the Graph.Sum operator.
	OpSum
	// OpConcat identifies the Graph.Concat operator.
	OpConcat
	// OpStack identifies the Graph.Stack operator.
	OpStack
)

func GetOpName 

func GetOpName(str string) (OpName, error)

GetOpName maps a string to an operator. It panics if the string does not match any operator (not even using lowercase).

type Operator added in v0.6.0 

type Operator struct {
	// contains filtered or unexported fields
}

Operator is a type of node.

func (*Operator) Grad added in v0.6.0 

func (r *Operator) Grad() mat.Matrix

Grad returns the gradients accumulated during the backward pass.

func (*Operator) Graph added in v0.6.0 

func (r *Operator) Graph() *Graph

Graph returns the graph this node belongs to.

func (*Operator) HasGrad added in v0.6.0 

func (r *Operator) HasGrad() bool

HasGrad returns true if there are accumulated gradients.

func (*Operator) ID added in v0.6.0 

func (r *Operator) ID() int

ID returns the ID of the node in the graph.

func (*Operator) Name added in v0.6.0 

func (r *Operator) Name() string

Name returns the Name of the operator. The name is taken from the name of r.function via reflection.

func (*Operator) Operands added in v0.6.0 

func (r *Operator) Operands() []Node

Operands returns the operands of the operator.

func (*Operator) PropagateGrad added in v0.6.0 

func (r *Operator) PropagateGrad(grad mat.Matrix)

PropagateGrad accumulates the gradients to the node itself.

func (*Operator) RequiresGrad added in v0.6.0 

func (r *Operator) RequiresGrad() bool

RequiresGrad returns true if the node requires gradients.

func (*Operator) ScalarValue added in v0.6.0 

func (r *Operator) ScalarValue() mat.Float

ScalarValue returns the the scalar value of the node. It panics if the value is not a scalar. Note that it is not possible to start the backward step from a scalar value.

func (*Operator) TimeStep added in v0.6.0 

func (r *Operator) TimeStep() int

TimeStep returns the time-step of the node.

func (*Operator) Value added in v0.6.0 

func (r *Operator) Value() mat.Matrix

Value returns the cached result of the function.

func (*Operator) ZeroGrad added in v0.6.0 

func (r *Operator) ZeroGrad()

ZeroGrad clears the gradients.

type Variable added in v0.6.0 

type Variable struct {
	// contains filtered or unexported fields
}

Variable is a type of node.

func (*Variable) Grad added in v0.6.0 

func (r *Variable) Grad() mat.Matrix

Grad returns the gradients accumulated during the backward pass.

func (*Variable) Graph added in v0.6.0 

func (r *Variable) Graph() *Graph

Graph returns the graph this node belongs to.

func (*Variable) HasGrad added in v0.6.0 

func (r *Variable) HasGrad() bool

HasGrad returns true if there are accumulated gradients.

func (*Variable) ID added in v0.6.0 

func (r *Variable) ID() int

ID returns the ID of the node in the graph.

func (*Variable) Name added in v0.6.0 

func (r *Variable) Name() string

Name returns the Name of the variable (it can be empty). Never refer to a variable by its name and use it only for debugging purposes. The name is set by g.NewVariableWithName().

func (*Variable) PropagateGrad added in v0.6.0 

func (r *Variable) PropagateGrad(grad mat.Matrix)

PropagateGrad accumulates the gradients to the node itself.

func (*Variable) RequiresGrad added in v0.6.0 

func (r *Variable) RequiresGrad() bool

RequiresGrad returns true if the node requires gradients.

func (*Variable) ScalarValue added in v0.6.0 

func (r *Variable) ScalarValue() mat.Float

ScalarValue returns the the scalar value of the node. It panics if the value is not a scalar. Note that it is not possible to start the backward step from a scalar value.

func (*Variable) TimeStep added in v0.6.0 

func (r *Variable) TimeStep() int

TimeStep returns the time-step of the node.

func (*Variable) Value added in v0.6.0 

func (r *Variable) Value() mat.Matrix

Value returns the value of the variable itself.

func (*Variable) ZeroGrad added in v0.6.0 

func (r *Variable) ZeroGrad()

ZeroGrad clears the gradients.

type Wrapper added in v0.6.0 

type Wrapper struct {
	GradValue
	// contains filtered or unexported fields
}

Wrapper is a type of node.

func (*Wrapper) Grad added in v0.6.0 

func (r *Wrapper) Grad() mat.Matrix

Grad returns the gradients accumulated during the backward pass.

func (*Wrapper) Graph added in v0.6.0 

func (r *Wrapper) Graph() *Graph

Graph returns the graph this node belongs to.

func (*Wrapper) HasGrad added in v0.6.0 

func (r *Wrapper) HasGrad() bool

HasGrad returns true if there are accumulated gradients.

func (*Wrapper) ID added in v0.6.0 

func (r *Wrapper) ID() int

ID returns the ID of the node in the graph.

func (*Wrapper) PropagateGrad added in v0.6.0 

func (r *Wrapper) PropagateGrad(gx mat.Matrix)

PropagateGrad propagates the gradients to the node.

func (*Wrapper) RequiresGrad added in v0.6.0 

func (r *Wrapper) RequiresGrad() bool

RequiresGrad returns true if the node requires gradients.

func (*Wrapper) TimeStep added in v0.6.0 

func (r *Wrapper) TimeStep() int

TimeStep returns the time-step of the node.

func (*Wrapper) ZeroGrad added in v0.6.0 

func (r *Wrapper) ZeroGrad()

ZeroGrad set the gradients to zeros.

Source Files

View all Source files

Directories

Path Synopsis
encoding
dot
Package dot creates a graphviz compatible version of the ag.Graph.
 Package dot creates a graphviz compatible version of the ag.Graph.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.