context

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 18, 2023 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package context defines the Context and Variable types: Context organizes variablesMap and variablesMap manages the storage of values typically used as variablesMap.

Index

Constants

View Source 
const ParameterPrefix = "var:"

ParameterPrefix is used to prefix parameter names for variablesMap.

View Source 
const ScopeSeparator = "/"

ScopeSeparator is used between levels of scope. Scope names cannot use this character.

View Source 
const TrainingGraphParamKey = "training"

Variables

This section is empty.

Functions

func EscapeScopeName 

func EscapeScopeName(scopeName string) string

EscapeScopeName replaces ScopeSeparator in the string and replaces them by "_".

func GetGraphParam 

func GetGraphParam[T any](ctx *Context, graph *Graph, name string, defaultValue T) T

GetGraphParam from Context object and cast to the give type. If parameter name is not defined, or if it cannot be cast to the given type, return defaultValue instead.

It's a typed wrapper to Context.GetGraphParam()

func GetParam 

func GetParam[T any](ctx *Context, name string, defaultValue T) T

GetParam from Context object and cast to the give type. If parameter name is not defined, or if it cannot be cast to the given type, return defaultValue instead.

It's a typed wrapper to Context.GetParam()

Types

type Context 

type Context struct {
	// contains filtered or unexported fields
}

Context organizes information shared in a model (or anything else). A model can spawn multiple computation graphs, e.g: one computation for a training step, one for an evaluation step running on batch, and one for a prediction computation on exactly one example. All these computation graphs should share the same variable (weight) values (and other information). These variables and (hyper-) parameters are organized here.

The Context organizes 3 types of information used by a model, and its graphs:

  1. Variables: model variables or weights.
  2. Parameters (normal): hyperparameters and also any arbitrary information that needs sharing among the graph building functions using the Context.
  3. Per-graph parameters: each graph will have it's own value. E.g: the parameter "training" indicates if the model is being used for training or inference, and this value will be different for a training or evaluation graph.

All 3 types of information are organized in "scopes". The Context object is actually a thin wrapper that contains the current scope (similar to a current directory) and a link to the actual data. One can change scopes by using Context.In("new_scope"): it returns a new Context with the new scope set, but still pointing (sharing) all the data with the previous Context. E.g:

One could create a new context with:

``` 

	func main() {
		ctx := context.NewContext(manager)
		ctx.SetParam("dropout_rate") = 0.2  // Set default dropout to 0.2
		...
	}

	func ModelGraph(ctx *context.Context, inputs []*Node) (logits *Node) {
		...
		{
			ctx := ctx.In("output_layer)  // Enter "output_layer" scope in temporary new context (same data, different scope)
        	ctx.SetParam("dropout_rate", 0.6  // Let's say we want the "output_layer" only to have dropout=0.6
         	logits = Dense(ctx, logits, output_dim)
		}  // Exiting "output_later" scope, ctx is back to it's original scope.
	}

Finally, Context also allows one to checkpoint the variable values (save and load). See checkpoint package.

TODO: Handling of devices with multiple instances (e.g.: multiple GPUs/TPUs).

func NewContext 

func NewContext(manager *ml.Manager) *Context

NewContext constructs a new and empty context.

func (*Context) CalculateGradientsGraph 

func (ctx *Context) CalculateGradientsGraph(loss *Node) []*Node

CalculateGradientsGraph returns the gradient of the loss with respect to each trainable variable in the context that was used in the current graph. It returns a tuple Node. Non-trainable variables (Variable.Trainable == false) are not touched.

Typically, this is used by an optimizer.

Note, if during the computation graph the value of the variable is changed with Variable.SetValueGraph, this will calculate the gradient with respect to the new value (*Node) set.

func (*Context) Checked 

func (ctx *Context) Checked(checked bool) *Context

Checked returns a new context with the checked flag set accordingly. If checked is true checks for reuse/uniqueness are checked according to IsReuse(). If checked is false Variables are dynamically reused or created when needed, without any checks. Usually it is set to true when building models -- to prevent layers to overstepping on each other -- and set to false for supporting variables (like optimizers, metrics and preprocessing).

Notice that re-usability is part of the "reference" component of a Context.

func (*Context) EnumerateGraphParams 

func (ctx *Context) EnumerateGraphParams(graph *Graph, fn func(scope, key string, value any))

EnumerateGraphParams enumerates all parameters for the graph, for all scopes calls fn with their values.

func (*Context) EnumerateParams 

func (ctx *Context) EnumerateParams(fn func(scope, key string, value any))

EnumerateParams enumerates all parameters for all scopes calls fn with their values.

func (*Context) EnumerateVariables 

func (ctx *Context) EnumerateVariables(fn func(v *Variable))

EnumerateVariables will call fn for each variable in the context. Notice the order of visitation is deterministic.

Notice that variables information is stored in the "data" component of Context objects, and is shared among all connected context references.

func (*Context) Error 

func (ctx *Context) Error() error

Error holds an error when creating this Context reference. If it is set, all variables are also created in error, and everything fails (gracefully). This allows one to defer error handling. Only the first error is stored. After an error is set all operations in Context are no-ops.

Notice that Error is part of the "data" component, so it's shared among all the connected Context references.

func (*Context) ExecPopulateGraphParamsMap 

func (ctx *Context) ExecPopulateGraphParamsMap(graph *Graph, params ml.ParamsMap)

ExecPopulateGraphParamsMap will enter the parameter values for every variable used in the given graph.

`Exec*` methods are used by those implementing an executor (context.Exec) or related tests, not normally needed by end users.

func (*Context) ExecSetVariablesInParams 

func (ctx *Context) ExecSetVariablesInParams(params ml.ParamsMap, graph *ml.Graph)

ExecSetVariablesInParams adds all variables (all scopes) used by the graph to the ParamsMap objects.

`Exec*` methods are used by those implementing an executor (context.Exec) or related tests, not normally needed by end users.

func (*Context) GetGraphParam 

func (ctx *Context) GetGraphParam(graph *Graph, key string) (value any, found bool)

GetGraphParam returns, for the givne graph, the value for the given param key, searching successively from the current scope back to the root scope ("/"), in case the key is not found.

E.g: if current scope is "/a/b", it will search for the key in "/a/b" scope, then in "/a" and finally in "/", and return the first result found.

This is very similar to GetParam, but used for parameters that are graph specific. For example Context.IsTraining and Context.SetTraining uses a Graph parameter to set this state, as the same Context is used for evaluation/inference graphs and training graphs, and they should have different values.

func (*Context) GetParam 

func (ctx *Context) GetParam(key string) (value any, found bool)

GetParam returns the value for the given param key, searching successively from the current scope back to the root scope ("/"), in case the key is not found.

E.g: if current scope is "/a/b", it will search for the key in "/a/b" scope, then in "/a" and finally in "/", and return the first result found.

See also GetGraphParam for parameters that are graph specific.

func (*Context) In 

func (ctx *Context) In(scope string) *Context

In returns a new reference to the Context with the extra given scope. No ScopeSeparator ("/") is allowed in scope.

Notice that Scope is part of the "reference" component of a Context.

func (*Context) InAbsPath 

func (ctx *Context) InAbsPath(scopePath string) *Context

InAbsPath returns a new reference to the Context with the extra given scope. It should start and have each element separated by ScopeSeparator.

Notice that Scope is part of the "reference" component of a Context.

func (*Context) InitializeVariables 

func (ctx *Context) InitializeVariables()

InitializeVariables initializes all variables in the Context that don't yet have a value. Variables create with VariableWithValue or for which values were preloaded are not initialized. Errors are returned in Context.Error().

Notice that variables information is stored in the "data" component of Context objects, and is shared among all connected context references.

func (*Context) InspectVariable 

func (ctx *Context) InspectVariable(scope, name string) *Variable

InspectVariable returns the variable with the given name for inspection. This shouldn't be used during building of models, since this bypass the Reuse checks. It returns nil if a variable with the given name hasn't been created.

Notice that variables information is stored in the "data" component of Context objects, and is shared among all connected context references.

func (*Context) IsChecked 

func (ctx *Context) IsChecked() bool

IsChecked returns whether context is checking reuse rules.

Notice that re-usability is part of the "reference" component of a Context.

func (*Context) IsReuse 

func (ctx *Context) IsReuse() bool

IsReuse returns whether Context is marked for reuse. This is irrelevant if IsChecked is false.

Notice that re-usability is part of the "reference" component of a Context.

func (*Context) IsTraining 

func (ctx *Context) IsTraining(graph *Graph) bool

IsTraining returns whether context is being used for training. This is only a convention and is defined by having Globals["training"] == true. See SetTraining to change this value.

Notice that global parameters is part of the "reference" component of a Context, so this change won't affect other connected context references.

func (*Context) Loader 

func (ctx *Context) Loader() Loader

Loader returns the current configured Loader for this context. See SetLoader for details on how the Loader is used.

Notice that loader configuration is stored in the "data" component of Context objects, and is shared among all connected context references.

func (*Context) NeedsInitialization 

func (ctx *Context) NeedsInitialization() bool

NeedsInitialization returns whether there are variables that needs initialization.

Notice that variables information is stored in the "data" component of Context objects, and is shared among all connected context references.

func (*Context) NumVariables 

func (ctx *Context) NumVariables() int

NumVariables return the number of variables in this Context.

func (*Context) Ok 

func (ctx *Context) Ok() bool

Ok returns whether Context is still valid, the same as saying it has no error.

Notice that Error is part of the "data" component, so it's shared among all the connected Context references.

func (*Context) ResetError 

func (ctx *Context) ResetError()

ResetError clears the error in a Context, and allows it to function normally.

Notice that Error is part of the "data" component, so it's shared among all the connected Context references.

func (*Context) Reuse 

func (ctx *Context) Reuse() *Context

Reuse returns a new reference to the Context set to reuse of variables, if it is not already in reuse mode. Otherwise, returns itself. If checked is false, this setting is irrelevant.

Notice that re-usability is part of the "reference" component of a Context.

func (*Context) Scope 

func (ctx *Context) Scope() string

Scope returns the full scope path.

Notice that Scope is part of the "reference" component of a Context.

func (*Context) SetError 

func (ctx *Context) SetError(err error)

SetError sets the Context into error. After this all operations become no-ops. Only the first error is stored, the following errors are discarded. If stackTrace is set to true, includes a stack trace of the error.

Notice that Error is part of the "data" component, so it's shared among all the connected Context references.

func (*Context) SetErrorf 

func (ctx *Context) SetErrorf(format string, args ...any)

SetErrorf is similar to SetError, but allows formatting in place. It also automatically adds stack trace.

Notice that Error is part of the "data" component, so it's shared among all the connected Context references.

func (*Context) SetGraphParam 

func (ctx *Context) SetGraphParam(graph *Graph, key string, value any)

SetGraphParam sets the given Graph param in the current scope. It will be visible (by GetGraphParam) for this Graph within this scope and descendant scopes (but not by other scopes).

This is very similar to SetParam, but used for parameters that are graph specific. For example Context.IsTraining and Context.SetTraining uses a Graph parameter to set this state, as the same Context is used for evaluation/inference graphs and training graphs, and they should have different values.

func (*Context) SetLoader 

func (ctx *Context) SetLoader(loader Loader)

SetLoader configures given loader to be used as the default Loader for this Context.

Loader is used just after any new variable is created, either with VariableWithValue or VariableWithShape. If the Loader has a value of the variable created, it will override the value given in VariableWithValue, or skip the initializer for VariableWithShape.

An example of a loader in gomlx/context/checkpoints.

Notice that loader configuration is stored in the "data" component of Context objects, and is shared among all connected context references.

func (*Context) SetParam 

func (ctx *Context) SetParam(key string, value any)

SetParam sets the given param in the current scope. It will be visible (by GetParam) within this scope and descendant scopes (but not by other scopes).

See also SetGraphParam for parameters that are graph specific.

func (*Context) SetTraining 

func (ctx *Context) SetTraining(graph *Graph, value bool)

SetTraining marks the context for the given graph as training. This is a convention adopted by the library components, and it simply sets Context.Globals["training"] to the given value. See IsTraining to check for this value.

Notice that global parameters is part of the "reference" component of a Context, so this change won't affect other connected context references.

func (*Context) Unique 

func (ctx *Context) Unique() *Context

Unique returns a new reference to the Context, set to only allow new variables, if it is not already in unique mode. If checked is false, this setting is irrelevant.

Notice that re-usability is part of the "reference" component of a Context.

func (*Context) VariableWithShape 

func (ctx *Context) VariableWithShape(name string, shape shapes.Shape) *Variable

VariableWithShape creates or returns an existing variable with the given shape in the current scope. It is initialized with the current variable initializer set for the context. By default, variables are marked as trainable.

If a Loader is configured (see SetLoader), and the value is available to load, it will override the value given here -- e.g.: the value could be actually loaded from the last checkpoint.

Notice that variables information is stored in the "data" component of Context objects, and is shared among all connected context references.

func (*Context) VariableWithValue 

func (ctx *Context) VariableWithValue(name string, value any) *Variable

VariableWithValue creates a variable that is initialized with the given value in the current scope. By default, variables are marked as trainable. The value given must be concrete, that is a tensor or a normal Go value, that can be converted to a tensor -- a graph *Node does not work here, this is assumed to be a constant.

If a Loader is configured (see SetLoader), and the value is available to load, it will override the value given here -- e.g.: the value could be actually loaded from the last checkpoint.

Notice that variables information is stored in the "data" component of Context objects, and is shared among all connected context references.

func (*Context) WithInitializer 

func (ctx *Context) WithInitializer(initializer VariableInitializer) *Context

WithInitializer returns a new reference to the Context, with the initializer set.

Notice that default variable initialization is part of the "reference" component of a Context, so this change won't affect other context references.

type ContextExecGraphFn 

type ContextExecGraphFn interface {
	func(*Context, *Node) |
		func(*Context, *Node, *Node) |
		func(*Context, *Node, *Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node, *Node, *Node) |

		func(*Context, *Node) *Node | func(*Context, *Node, *Node) *Node |
		func(*Context, *Node, *Node, *Node) *Node |
		func(*Context, *Node, *Node, *Node, *Node) *Node |
		func(*Context, *Node, *Node, *Node, *Node, *Node) *Node |
		func(*Context, *Node, *Node, *Node, *Node, *Node, *Node) *Node |

		func(*Context, *Node) (*Node, *Node) | func(*Context, *Node, *Node) (*Node, *Node) |
		func(*Context, *Node, *Node, *Node) (*Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node) (*Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node, *Node) (*Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node, *Node, *Node) (*Node, *Node) |

		func(*Context, *Node) (*Node, *Node, *Node) |
		func(*Context, *Node, *Node) (*Node, *Node, *Node) |
		func(*Context, *Node, *Node, *Node) (*Node, *Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node) (*Node, *Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node, *Node) (*Node, *Node, *Node) |
		func(*Context, *Node, *Node, *Node, *Node, *Node, *Node) (*Node, *Node, *Node) |

		func(*Context, *Node) []*Node | func(*Context, *Node, *Node) []*Node |
		func(*Context, *Node, *Node, *Node) []*Node |
		func(*Context, *Node, *Node, *Node, *Node) []*Node |
		func(*Context, *Node, *Node, *Node, *Node, *Node) []*Node |
		func(*Context, *Node, *Node, *Node, *Node, *Node, *Node) []*Node |

		func(*Context, []*Node) |
		func(*Context, []*Node) *Node |
		func(*Context, []*Node) (*Node, *Node) |
		func(*Context, []*Node) (*Node, *Node, *Node) |
		func(*Context, []*Node) []*Node |

		func(*Context, *Graph) |
		func(*Context, *Graph) *Node |
		func(*Context, *Graph) (*Node, *Node) |
		func(*Context, *Graph) (*Node, *Node, *Node) |
		func(*Context, *Graph) []*Node
}

ContextExecGraphFn is a type parameter for accepted function types for NewExec constructor.

type Exec 

type Exec struct {
	// contains filtered or unexported fields
}

Exec creates and executes computation graphs that take as input a Context as needed based on the inputs shapes, to allow the function to access (both read and set) variables and everything in the Context. Otherwise very similar to graph.Exec.

It simplifies the process of executing a graph building function with real values.

For example, assume you wrote: 

def LogitsGraph(ctx *context.Context, inputs *Node) *Node {
    logits := layers.Dense(ctx.In("dense0", inputs, 5))
	logits = layers.Dense(ctx.In("dense1", logits, 5))
	logits = Sigmoid(logits)
	return logits
}

And then with Exec one can do: 

 ctx := context.NewContext(manager)
	var logitsFn = context.NewExec(manager, ctx, LogitsGraph)
	batch := [][]float32{ {1, 2, 3}, {4, 5, 6} } // 2 examples with 3 features (shape=[2,3])
	fmt.Printf("Logits(%v) = %v\n", batch, logitsFn.Call(batch)[0].Value())

Call method always outputs a slice with all the outputs, even when there are no outputs (in which case it returns nil). Call takes as input the materialized values (tensors) you want to execute the graph with -- you don't need to path the *Context of *Graph to Call, those are filled automatically by Exec.

Notice ctxGraphFn, the function that builds the computation graph, is only called the first time Call is used -- so it's slower the first time, since it has to compile the graph. After that the execution is much faster. But that means that changes to Context.SetParams() or Context.SetGraphParams() after the frist execution will not have any effect.

Exec also manages updates to variables automatically. Example: we implement a counter, which takes no input values (just the *Graph object), it just creates a variable if not there yet, increments it and returns its value.

``` 

 ctx := context.NewContext(manager)
 counter := context.NewExec(manager, ctx, func(ctx *context.Context, g *Graph) *Node {
	  dtype := types.Int64
	  counterVar := ctx.WithInitializer(initializers.Zeros).VariableWithShape("counter", types.MakeShape(dtype))
	  counterNode := counterVar.ValueGraph(graph)
	  counterNode = Add(counterNode, OnesLike(counterNode))
	  counterVar.SetValueGraph(counterNode)
	  return counterNode
	})
 fmt.Printf("%s\n", counter.Call()[0])  // == 1
 fmt.Printf("%s\n", counter.Call()[0])  // == 2
 fmt.Printf("%s\n", ctx.InspectVariable(ctx.Scope(), "counter").Value())  // == 2

```

Behind the scenes it automatically adds the used variables (Variable.ValueGraph) as side inputs, and the updated variables (Variable.SetValueGraph) as extra outputs of the graph, and during the graph execution it automatically use these values to update the materialized variable values in Context variables. It will also automatically initialize Context variables when needed.

Notice, like with graph.Exec, the need to build different graphs for different shapes can be expensive when sizes of the inputs varies a lot. The usual solution is to use shapes with size in a power scale (for instance powers of 2) and masking of tensors for unused slices. For safety concerns there are a maximum number of different instantiations of the graph. It can be set or disabled with SetMaxCache.

Errors in Call are returned inside the returned tensors.

There is concurrency safety with the cache of Graphs, but XLA concurrency is not documented. TODO: figure it out.

func NewExec 

func NewExec[F ContextExecGraphFn](manager *Manager, ctx *Context, ctxGraphFn F) *Exec

NewExec constructs an Exec object that uses the given ctxGraphFn to build computation graphs with a Context. ctxGraphFn must take a *Context input parameter followed by one or more *Node parameters as input and return one or more *Node.

The Context ctx passed will be passed to all computation graph construction calls (ctxGraphFn), as well as during the graph execution later. If set to nil, it automatically creates a new one.

Before the execution of a graph, if needed, it initializes the variables in the context.

This is a generic wrapper around NewExecAny that check that types are correct (but doesn't support all possible types of ctxGraphFn).

func NewExecAny 

func NewExecAny(manager *Manager, ctx *Context, ctxGraphFn any) (*Exec, error)

NewExecAny constructs an Exec object that uses the given ctxGraphFn to build computation graphs with a Context. ctxGraphFn must take a *Context input parameter followed by one or more *Node parameters as input and return one or more *Node. Alternatively it can, instead of *Node inputs, take a *Graph object -- if effectively no tensors will be used as input.

The Context ctx passed will be passed to all computation graph construction calls (ctxGraphFn), as well as during the graph execution later. If set to nil, it automatically creates a new one.

Before the execution of a graph, if needed, it initializes the variables in the context. And updated variables in the graph are updated also during execution. More details see Exec.

If any input or output parameter of ctxGraphFn is not a *Node (except the *Context and optionally a *Graph), or if there are no inputs or outputs, it returns an error.

func (*Exec) Call 

func (e *Exec) Call(args ...any) ([]tensor.Tensor, error)

Call parses the arguments into tensors (if they are not yet) and executes the graph corresponding to the shapes of the arguments.

Notice Context shouldn't be passed by Call, it will use automatically the context stored in context.Exec -- you can change it with SetContext.

If a graph does not yet exist one is created, compiled and cached for the shapes. It passes the context to the registered ctxGraphFn. After the very first invocation of Call the context is marked as Context.Reuse().

It returns the outputs in a slice, even if there is only one output, or an error.

func (*Exec) CallWithGraph 

func (e *Exec) CallWithGraph(args ...any) (outputs []tensor.Tensor, g *Graph, err error)

CallWithGraph is similar to Call, but it also returns the computation graph used in the call. Since Exec creates different computation graphs for different set of parameters, this can help disambiguate in case the user needs to use the Graph for something else.

It returns the outputs in a slice, even if there is only one output, and the graph used to execute the computation or an error.

func (*Exec) Context 

func (e *Exec) Context() *Context

Context returns the associated Context object, usually created during the creation of the Exec object. It can be set to something different with SetContext().

func (*Exec) Finalize 

func (e *Exec) Finalize()

Finalize clears the cache, finalizing the graphs. The Exec object shouldn't be used after that.

func (*Exec) GetNodeLogger 

func (e *Exec) GetNodeLogger() graph.LoggerFn

GetNodeLogger returns the currently registered LoggerFn.

func (*Exec) InDevice 

func (e *Exec) InDevice(deviceNum int) *Exec

InDevice sets the device num to be used by graphs constructed by Exec. This should be called before any invocations of Call(). It returns a reference to itself so calls can be cascaded.

func (*Exec) Name 

func (e *Exec) Name() string

Name returns the Exec name, a string used as prefix for Graph construction.

func (*Exec) SetContext 

func (e *Exec) SetContext(context *Context) *Exec

SetContext associates the given Context to the Exec object. Should be called before the first Call is made. Notice that only after the first time context is used to build a graph, it is set to Reuse. If the Context variables were already created, it should be marked with Context.Reuse. It returns a reference to itself so calls can be cascaded.

func (*Exec) SetMaxCache 

func (e *Exec) SetMaxCache(maxCacheSize int) *Exec

SetMaxCache sets the maximum size of the cache. Set it to -1 to have unlimited cache size. It returns a reference to itself so calls can be cascaded.

func (*Exec) SetNodeLogger 

func (e *Exec) SetNodeLogger(loggerFn graph.LoggerFn)

SetNodeLogger with the function to be called for the nodes marked for logging during execution. If set to nil nothing will be logged.

func (*Exec) WithName 

func (e *Exec) WithName(name string) *Exec

WithName sets the name of Exec, used to provide the name to graphs created. This should be called before any invocations of Call(). It returns a reference to itself so calls can be cascaded.

type Graph 

type Graph = graph.Graph

type Loader 

type Loader interface {
	// LoadVariable tries to load the variable v, usually specified by its scope and name.
	// If it's not found, returns false, and initialization continues as usual.
	// Errors can be reported with Context.SetError.
	LoadVariable(ctx *Context, v *Variable) (value tensor.Tensor, found bool)
}

Loader can be implemented by any library providing loading of variables for Context. Loader implementations need to provide values on demand -- as a variables are used, even if they load everything up-front.

An example of a loader in gomlx/context/checkpoints. An example for testing can be found in context_test.go:ConstantLoader.

type Manager 

type Manager = graph.Manager

type Node 

type Node = graph.Node

type ScopedParams 

type ScopedParams struct {
	// contains filtered or unexported fields
}

ScopedParams provides a mapping from string to any data type that is "scoped":

  • For every scope there is a map of string to data.
  • Accessing a key triggers a search from the current scope up to the root scope, the first result found is returned.

Example: let's say the current ScopedParams hold: 

Scope: "/": { "x":10, "y": 20, "z": 40 }
Scope: "/a": { "y": 30 }
Scope: "/a/b": { "x": 100 }

ScopedParams.Get("/a/b", "x") -> 100
ScopedParams.Get("/a/b", "y") -> 30
ScopedParams.Get("/a/b", "z") -> 40
ScopedParams.Get("/a/b", "w") -> Not found.

Notice that "/" (== ScopeSeparator constant) separates parts of the scope path, and the root scope is referred to as "/". There is no "empty" scope, and every scope name must start with a ScopeSeparator.

func NewScopedParams 

func NewScopedParams() *ScopedParams

NewScopedParams create an empy ScopedParams.

func (*ScopedParams) Enumerate 

func (p *ScopedParams) Enumerate(fn func(scope, key string, value any))

Enumerate enumerates all parameters stored in the ScopedParams structure and calls the given closure with them.

func (*ScopedParams) Get 

func (p *ScopedParams) Get(scope, key string) (value any, found bool)

Get retrieves the value for the given key in the given scope or any parent scope. E.g: Get("/a/b", "myKey") will search for "myKey" in scopes "/a/b", "/a" and "/" consecutively until "myKey" is found.

It returns the first value found if any, and whether some value was found.

func (*ScopedParams) Set 

func (p *ScopedParams) Set(scope, key string, value any)

Set sets the value for the given key, in the given scope.

type Shape 

type Shape = types.Shape

type Variable 

type Variable struct {

	// Trainable indicates whether variable is trainable. If set to false it won't be
	// touched by trainers of the model.
	Trainable bool
	// contains filtered or unexported fields
}

Variable is a value shared among computation graphs, or across multiple executions of the same graph. It's commonly used to store the weights (aka. parameters) of an ML model. It's defined in a scope in a Context.

The materialized value can be accessed in between graph executions by Value and SetValue methods.

During the computation graph building, for a particular graph, one can access the graph value (Node) of a variable with the methods

They are only initialized when Context.InitializeVariables. That is, they are created and used in graph building possibly before they are actually initialized -- when building a graph they are passed as parameters (the corresponding graph node is called ParameterNode), and have their values passed only during execution.

func (*Variable) ChangedInGraph 

func (v *Variable) ChangedInGraph(g *Graph) bool

ChangedInGraph returns whether the variable is in use and was changed in the computation graph g.

func (*Variable) InUseByGraph 

func (v *Variable) InUseByGraph(g *Graph) bool

InUseByGraph returns whether the variable is currently in use by the given graph.

func (*Variable) Name 

func (v *Variable) Name() string

Name of the variable within the scope.

func (*Variable) Ok 

func (v *Variable) Ok() bool

Ok returns whether the variable is valid. If something went wrong during creation, the variable will not be Ok. The error can be read from Context.Error().

func (*Variable) ParamNode 

func (v *Variable) ParamNode(g *Graph) *Node

ParamNode returns the given Graph g's Node that corresponds to the parameter that will be fed with the current variable value when the graph is executed. It's the initial value of the variable in the computation Graph.

If parameter Node hasn't been created for the Graph g yet, one is created.

Since the value of a variable can change in the middle of the graph (e.g: something that uses the variable after a gradient descent is applied) consider using ValueGraph to read the current associated value of a variable in a graph.

func (*Variable) ParameterName 

func (v *Variable) ParameterName() string

ParameterName used when creating a parameter node in a Graph to access the variable.

func (*Variable) Scope 

func (v *Variable) Scope() string

Scope where the variable was created.

func (*Variable) SetTrainable 

func (v *Variable) SetTrainable(trainable bool) *Variable

SetTrainable sets the variable trainable status. Returns itself, so calls can be cascated.

func (*Variable) SetValue 

func (v *Variable) SetValue(value tensor.Tensor)

SetValue updates the tensor holding the variable value. NOTE: Because often variables are large in size, the previous value is immediately freed (as opposed to wait for garbage collection). If the previous value is used somewhere else, use SetValuePreservingOld.

func (*Variable) SetValueGraph 

func (v *Variable) SetValueGraph(value *Node)

SetValueGraph sets the Node associated with the current value of the variable for the computation graph where value is defined.

This is used to "communicate" among different parts of the graph building that this value Node should be used as the variable value.

train.Trainer will use the last value set here during graph building and use it as output of the graph execution and then update the variables (with SetValue) accordingly after each graph execution, for example, after each Trainer.TrainStep call.

func (*Variable) SetValuePreservingOld 

func (v *Variable) SetValuePreservingOld(value tensor.Tensor)

SetValuePreservingOld updates the tensor holding the variable value, and dont' free old value. If previous value is not used, use SetValue instead that will free it immediately.

func (*Variable) Shape 

func (v *Variable) Shape() shapes.Shape

Shape returns the variable shape.

func (*Variable) String 

func (v *Variable) String() string

String implements stringer.

func (*Variable) Value 

func (v *Variable) Value() tensor.Tensor

Value returns the tensor holding the variable value. Use this to manipulate the value in Go. If building a computation Graph use Node().

WARNING: memory management here is tricky: a call to SetValue will trigger the current value to be deallocated, and what is returned by a previous call to Value to become invalid. The recommendation is not to use this is a concurrent set up -- or to create proper locking mechanisms.

func (*Variable) ValueGraph 

func (v *Variable) ValueGraph(g *Graph) *Node

ValueGraph returns the Node of the Graph that holds the current value of the variable. It can be changed for the graph (for instance when applying a gradient descent) by SetGraph.

type VariableInitializer 

type VariableInitializer = initializers.VariableInitializer

VariableInitializer builds a valueNode that returns a value to initialize a variable of the given shape. It is defined in the Context.

Source Files

View all Source files

Directories

Path Synopsis
Package checkpoints implements checkpoint management: saving and loading of checkpoints.
 Package checkpoints implements checkpoint management: saving and loading of checkpoints.
Package ctxtest holds test utilities for packages that depend on context package.
 Package ctxtest holds test utilities for packages that depend on context package.
Package initializers include several weight initializers, to be used with context.
 Package initializers include several weight initializers, to be used with context.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.