envlp

package
v1.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 2, 2022 License: BSD-3-Clause Imports: 8 Imported by: 0

README

Docs: GoDoc

See Wiki Env page for detailed docs.

Package envlp defines the Env interface for environments, using the looper control framework to manage the incrementing of counters on the Env, instead of the Env automatically incrementing counters on its own, which is the behavior of the original env.Env environments.

The Env determines the nature and sequence of States that can be used as inputs to a model and it can also accept Action responses from the model that affect how the environment evolves in the future.

By adhering to this interface, it is then easier to mix-and-match environments with models.

Env / Agent

Env uses a post increment logic for stepping the state, consistent with the looper framework: current State of the Env should be ready to use after the Init() call, and Step() is called after the current State is used, to advance so its ready to be used for the next iteration, while also incrementing the lowest-level counter that tracks Step updates (e.g., the Trial level). No other counters should be updated.

The Counters are associated with etime Times, as used in looper, which manages the counter updating based on Max values specified in the Env, or other relevant stopping criteria in the Stop() functions on Loops. Each looper.Stack can be associated with an Env, which it manages, and both the Stack and the Env have an associated etime.Mode evaluation mode (Train, Test, etc) that should be consistent.

Thus, multiple different environments will typically be used in a model, for each different evaluation mode. Nevertheless, these different Envs can share a common database of patterns, e.g., using the etable.IdxView to present different indexed views into a shared common etable.Table (e.g., train / test splits). The basic FixedTable env implementation uses this.

Particular paradigms of environments must establish naming conventions for these state elements which then allow the model to use the information appropriately -- e.g., "Input", "Output" are widely used, but more realistic and multimodal models have many different types of state. The Env interface only provides the most basic framework for establishing these paradigms, and ultimately a given model will only work within a particular paradigm of environments following specific conventions.

Typically each specific implementation of this Env interface will have a Config() function that at least takes the eval mode for the env as an arg, and multiple parameters etc that can be modified to control env behavior -- all of this is paradigm-specific and outside the scope of this basic interface.

The States32 type is convenient for managing named env states.

Standard boilerplate code

Here's some standard boilerplate code used in most Env implementations:

// MyEnv is a skeleton environment to use as a starting point for custom envs
type MyEnv struct {
	Nm    string     `desc:"name of this environment"`
	Dsc   string     `desc:"description of this environment"`
	EMode string     `desc:"eval mode for this env"`
	Ctrs  envlp.Ctrs `desc:"counters for this environment"`
}

func (ev *MyEnv) Name() string          { return ev.Nm }
func (ev *MyEnv) Desc() string          { return ev.Dsc }
func (ev *MyEnv) Mode() string          { return ev.EMode }
func (ev *MyEnv) Counters() *envlp.Ctrs { return &ev.Ctrs }
func (ev *MyEnv) Counter(time etime.Times) *envlp.Ctr {
	return ev.Ctrs.ByScope(etime.ScopeStr(ev.EMode, time.String()))
}
func (ev *MyEnv) String() string { return "" }
func (ev *MyEnv) CtrsToStats(stats *estats.Stats) {
	ev.Ctrs.CtrsToStats(stats)
}

func (ev *MyEnv) Config(mode string) {
	ev.EMode = mode
	ev.Ctrs.SetTimes(mode, etime.Run, etime.Epoch, etime.Trial)
}

func (ev *MyEnv) Validate() error {
	return nil
}

func (ev *MyEnv) Init() {
	run := 0
	rc := ev.Counter(etime.Run)
	if rc != nil {
		run = rc.Cur
	}
	ev.Ctrs.Init()
	if rc != nil {
		rc.Set(run)
	}
}

func (ev *MyEnv) Step() {
	tc := ev.Counter(etime.Trial)
	tc.Incr()
}

func (ev *MyEnv) State(element string) etensor.Tensor {
	return nil
}

func (ev *MyEnv) Action(element string, input etensor.Tensor) {
	// nop
}

// Compile-time check that implements Env interface
var _ envlp.Env = (*MyEnv)(nil)

Documentation

Overview

Package envlp defines an interface for environments, which determine the nature and sequence of States that can be used as inputs to a model and it can also accept Action responses from the model that affect how the enviroment evolves in the future.

This version uses the looper control framework to manage the incrementing of counters on the Env, instead of the Env automatically incrementing counters on its own, which is the behavior of the original `env.Env` environment.

By adhering to this interface, it is then easier to mix-and-match environments with models.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Ctr 

type Ctr struct {
	Cur   int            `desc:"current counter value"`
	Prv   int            `view:"-" desc:"previous counter value, prior to last Incr() call (init to -1)"`
	Max   int            `desc:"maximum counter value -- only used if > 0"`
	Scope etime.ScopeKey `view:"-" desc:"the scope of this counter"`
}

Ctr is a counter that counts increments at a given time scale. It keeps track of the previous value.

func (*Ctr) Incr 

func (ct *Ctr) Incr()

Incr increments the counter by 1.

func (*Ctr) Init 

func (ct *Ctr) Init()

Init initializes counter -- Cur = 0, Prv = -1

func (*Ctr) IsOverMax 

func (ct *Ctr) IsOverMax() bool

IsOverMax returns true if counter is at or over Max (only if Max > 0)

func (*Ctr) ResetIfOverMax 

func (ct *Ctr) ResetIfOverMax() bool

ResetIfOverMax resets the current counter value to 0, if counter is at or over Max (only if Max > 0). returns true if reset

func (*Ctr) Set 

func (ct *Ctr) Set(cur int) bool

Set sets the Cur value if different from Cur, while preserving previous value. Returns true if changed

type Ctrs 

type Ctrs struct {
	Order []etime.ScopeKey        `desc:"ordered list of the counter scopes, from outer-most (highest) to inner-most (lowest)"`
	Ctrs  map[etime.ScopeKey]*Ctr `desc:"map of the counters by scope"`
}

Ctrs contains an ordered slice of scopes, and a lookup map of counters by scope, used to manage counters in the Env.

func (*Ctrs) ByScope 

func (cs *Ctrs) ByScope(scope etime.ScopeKey) *Ctr

ByScope returns counter by scope key -- nil if not found

func (*Ctrs) ByScopeTry 

func (cs *Ctrs) ByScopeTry(scope etime.ScopeKey) (*Ctr, error)

ByScopeTry returns counter by scope key -- returns nil, error if not found

func (*Ctrs) CtrsToStats 

func (cs *Ctrs) CtrsToStats(stats *estats.Stats)

CtrsToStats sets the current counter values to estats Int values by their time names only (no eval Mode).

func (*Ctrs) Init 

func (cs *Ctrs) Init()

Init does Init on all the counters

func (*Ctrs) SetScopes 

func (cs *Ctrs) SetScopes(scopes ...etime.ScopeKey)

SetScopes initializes returns a new Ctrs based on scopes ordered from highest to lowest

func (*Ctrs) SetTimes 

func (cs *Ctrs) SetTimes(mode string, times ...etime.Times)

SetTimes initializes Ctrs for given mode and list of times ordered from highest to lowest

type CurPrvF32 

type CurPrvF32 struct {
	Cur float32 `desc:"current value"`
	Prv float32 `desc:"previous value"`
}

CurPrvF32 is basic state management for current and previous values, float32 values

func (*CurPrvF32) Diff 

func (cv *CurPrvF32) Diff() float32

Diff returns the difference between current and previous values

func (*CurPrvF32) Incr 

func (cv *CurPrvF32) Incr()

Incr increments Cur by 1

func (*CurPrvF32) Set 

func (cv *CurPrvF32) Set(cur float32)

Set sets the new current value, copying Cur to Prv

type CurPrvInt 

type CurPrvInt struct {
	Cur int `desc:"current value"`
	Prv int `desc:"previous value"`
}

CurPrvInt is basic state management for current and previous values, int values

func (*CurPrvInt) Diff 

func (cv *CurPrvInt) Diff() int

Diff returns the difference between current and previous values

func (*CurPrvInt) Incr 

func (cv *CurPrvInt) Incr()

Incr increments Cur by 1

func (*CurPrvInt) Set 

func (cv *CurPrvInt) Set(cur int)

Set sets the new current value, copying Cur to Prv

type CurPrvString 

type CurPrvString struct {
	Cur string `desc:"current value"`
	Prv string `desc:"previous value"`
}

CurPrvString is basic state management for current and previous values, string values

func (*CurPrvString) Set 

func (cv *CurPrvString) Set(cur string)

Set sets the new current value, copying Cur to Prv

type Element 

type Element struct {
	Name     string   `desc:"name of this element -- must be unique"`
	Shape    []int    `` /* 160-byte string literal not displayed */
	DimNames []string `desc:"names of the dimensions within the Shape -- optional but useful for ensuring correct usage"`
}

Element specifies one element of State or Action in an environment

func (*Element) FromColumn 

func (ch *Element) FromColumn(sc *etable.Column)

FromColumn copies element data from etable Column that describes an etable.Table

type Elements 

type Elements []Element

Elements is a list of Element info

func (*Elements) FromSchema 

func (ch *Elements) FromSchema(sc etable.Schema)

FromSchema copies element data from a etable Schema that describes an etable.Table

type Env 

type Env interface {
	// Name returns a name for this environment, which can be useful
	// for selecting from a list of options etc.
	Name() string

	// Desc returns an (optional) brief description of this environment.
	Desc() string

	// Mode returns the evaluation mode (etime.Modes) for this environment
	// (Train, Test, etc). This is used for the Scope of the counters.
	Mode() string

	// Validate checks if the various specific parameters for this
	// Env have been properly set -- if not, error message(s) will
	// be returned.  If everything is OK, nil is returned, in which
	// case calls to Counters(), States(), and Actions() should all
	// return valid data.  It is essential that a model *always* check
	// this as a first step, because the Env will not generally check
	// for errors on any subsequent calls (for greater efficiency
	// and simplicity).
	Validate() error

	// Init initializes the environment at start of a new Run, preserving
	// the current Run level counter value, if that counter is present, but
	// resetting all other counters to 0.
	// In general the Env can expect that the Sim will have established a
	// different random seed per run, prior to calling this method,
	// sufficient to enable different run-level behavior.
	// The current State() must be updated to reflect the first step of
	// the environment, consistent with the post-increment model, where
	// Step is called *after* the current state is used.
	Init()

	// Step advances to the next step of environment state,
	// rendering any new State values as needed, and incrementing the
	// counter associated with stepping (e.g., Trial).
	// This is called *after* using the current State, making it ready
	// for the next iteration.
	// The looper control system will detect when the Trial is over Max
	// and reset that back to 0, while updating other higher counters as needed.
	// The Env should expect this and prepare a next state consistent with
	// the Trial (stepping level) counter reset back to 0.
	Step()

	// Counters returns the full set of counters used in the Env.
	// A specific scope counter can be accessed as Counters().ByScope(scope)
	Counters() *Ctrs

	// Counter returns counter for given standard etime.Times value, using
	// the Mode set for this environment to generate a ScopeKey string.
	Counter(time etime.Times) *Ctr

	// String returns a string representation of the current step contents.
	// This is typically saved as a TrialName in the CtrsToStats function,
	// and satisfies the standard Stringer interface.
	String() string

	// State returns the given element's worth of tensor data from the environment
	// based on the current state of the env (prepared by Init or the last Step).
	// If no output is available on that element, then nil is returned.
	// The returned tensor must be treated as read-only as it likely points to original
	// source data -- please make a copy before modifying (e.g., Clone() method)
	State(element string) etensor.Tensor

	// Action sends tensor data about e.g., responses from model back to act
	// on the environment and influence its subsequent evolution.
	// The nature and timing of this input is paradigm dependent, but
	// in general it should happen prior to the Step() call, so that
	// Action sets values on the Env that are then used in the Step call
	// to generate the appropriate next state values.
	Action(element string, input etensor.Tensor)

	// CtrsToStats sets the current counter values to estats Int values
	// by their time names only (no eval Mode).  These values can then
	// be read by elog LogItems to record the counters in logs.
	// Typically, a TrialName string is also expected to be set,
	// to describe the current trial (Step) contents in a useful way,
	// and other relevant info (e.g., group / category info) can also be set.
	CtrsToStats(stats *estats.Stats)
}

Env defines an interface for environments, which determine the nature and sequence of States that can be used as inputs to a model, and the Env also can accept Action responses from the model that affect state evolution.

The Env holds a set of Ctr counters for different time scales according to the etime scoping system, which are assumed to be managed by the looper.Stack looping control system. The Env should only increment the inner-most counter that tracks the Stepping of the environment.

State is comprised of one or more elements, each of which consists of an etensor.Tensor chunk of values that can be obtained by the model. Likewise, Actions provide tensor elements as input to the Env.

type Envs 

type Envs map[string]Env

Envs is a map of environments organized according to the evaluation mode string (recommended key value)

func (*Envs) Add 

func (es *Envs) Add(evs ...Env)

Add adds Env(s), using its Mode as the key

func (*Envs) ByMode added in v1.2.2 

func (es *Envs) ByMode(mode etime.Modes) Env

ByMode returns env by etime.Modes evaluation mode as the map key. returns nil if not found

func (*Envs) Init 

func (es *Envs) Init()

Init initializes the map if not yet

type FixedTable 

type FixedTable struct {
	Nm         string          `desc:"name of this environment"`
	Dsc        string          `desc:"description of this environment"`
	EMode      string          `desc:"eval mode for this env"`
	Table      *etable.IdxView `` /* 285-byte string literal not displayed */
	Sequential bool            `` /* 140-byte string literal not displayed */
	Order      []int           `desc:"permuted order of items to present if not sequential -- updated every time through the list"`
	Ctrs       Ctrs            `desc:"counters for this environment"`
	TrialName  CurPrvString    `desc:"if Table has a Name column, this is the contents of that"`
	GroupName  CurPrvString    `desc:"if Table has a Group column, this is contents of that"`
	NameCol    string          `desc:"name of the Name column -- defaults to 'Name'"`
	GroupCol   string          `desc:"name of the Group column -- defaults to 'Group'"`
}

FixedTable is a basic Env that manages patterns from an etable.Table, with either sequential or permuted random ordering, and uses standard Run, Epoch, Trial etime.Times counters to record progress and iterations through the table. It uses an IdxView indexed view of the Table, so a single shared table can be used across different environments, with each having its own unique view. The State calls directly access column names in the associated table, and the Action method has no effect.

func (*FixedTable) Action 

func (ft *FixedTable) Action(element string, input etensor.Tensor)

func (*FixedTable) Config 

func (ft *FixedTable) Config(tbl *etable.IdxView, mode string)

Config configures the environment to use given table IndexView and evaluation mode (e.g., etime.Train.String()). If mode is Train then a Run counter is added, otherwise just Epoch and Trial. NameCol and GroupCol are initialized to "Name" and "Group" so set these to something else after this if needed.

func (*FixedTable) Counter 

func (ft *FixedTable) Counter(time etime.Times) *Ctr

func (*FixedTable) Counters 

func (ft *FixedTable) Counters() *Ctrs

func (*FixedTable) CtrsToStats 

func (ft *FixedTable) CtrsToStats(stats *estats.Stats)

func (*FixedTable) Desc 

func (ft *FixedTable) Desc() string

func (*FixedTable) Init 

func (ft *FixedTable) Init()

func (*FixedTable) Mode 

func (ft *FixedTable) Mode() string

func (*FixedTable) Name 

func (ft *FixedTable) Name() string

func (*FixedTable) NewOrder 

func (ft *FixedTable) NewOrder()

NewOrder sets a new random Order based on number of rows in the table.

func (*FixedTable) PermuteOrder 

func (ft *FixedTable) PermuteOrder()

PermuteOrder permutes the existing order table to get a new random sequence of inputs just calls: erand.PermuteInts(ft.Order)

func (*FixedTable) Row 

func (ft *FixedTable) Row() int

Row returns the current row number in table, based on Sequential / perumuted Order and already de-referenced through the IdxView's indexes to get the actual row in the table.

func (*FixedTable) SetTrialNames 

func (ft *FixedTable) SetTrialNames()

SetTrialNames sets the TrialName and GroupName

func (*FixedTable) State 

func (ft *FixedTable) State(element string) etensor.Tensor

func (*FixedTable) Step 

func (ft *FixedTable) Step()

func (*FixedTable) String 

func (ft *FixedTable) String() string

func (*FixedTable) Validate 

func (ft *FixedTable) Validate() error

type States32 

type States32 map[string]*etensor.Float32

States32 is a map of named *etensor.Float32 tensors, used by Envs to manage states.

func NewStates32 

func NewStates32(names ...string) States32

NewStates32 returns a new States32 map for given names

func (*States32) ByNameTry 

func (st *States32) ByNameTry(nm string) (*etensor.Float32, error)

ByNameTry accesses the tensor map by name, generating an error if the state is not found.

func (*States32) SetShape 

func (st *States32) SetShape(shape, strides []int, names []string, nm ...string)

SetShape sets the Shape of given tensor(s) by name If strides is nil, row-major strides will be inferred. If names (dimension names) is nil, a slice of empty strings will be created.

func (*States32) SetZeros 

func (st *States32) SetZeros()

SetZeros sets all states to zeros -- call prior to rendering new

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.