paramstores

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 21, 2021 License: BSD-3-Clause Imports: 15 Imported by: 0

Documentation

Overview

Package paramstores contains the implementation of all the supported storage adapters for the network parameters

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type FileAdapter

type FileAdapter struct {
	Path string
}

FileAdapter is a neural net param store implementation that uses the filesystem, its main purpose is to facilitate testing. Given its low performance it is strongly discouraged for production use

func NewFileAdapter

func NewFileAdapter(conf map[string]interface{}) (*FileAdapter, error)

NewFileAdapter returns an initialized file net param store object

func (FileAdapter) Delete

func (fa FileAdapter) Delete(id string) error

Delete can be used to delete the state of a specific neural net from a file on disk

func (FileAdapter) List

func (fa FileAdapter) List(offset, limit int, pattern string) ([]string, int, error)

List can be used to get the IDs of the stored nets

func (FileAdapter) Load

func (fa FileAdapter) Load(id string, np NetParams) (bool, error)

Load can be used to retrieve the state of a specific neural net from a file on disk

func (FileAdapter) Save

func (fa FileAdapter) Save(id string, np NetParams) error

Save can be used to upsert the state of a specific neural net to a file on disk

type MLPParams

type MLPParams struct {
	Accuracy       float32
	ActivationFunc string
	Averages       map[string]float32
	Deviations     map[string]float32
	Epoch          int
	ErrMargin      float32
	Inputs         []string
	LearningRate   float32
	Topology       []int
	Outputs        []string
	Weights        [][]float32
}

MLPParams holds the minimum information required to rebuild the net from scratch plus some metadata that is required for other parts of the system

func (MLPParams) Brief

func (np MLPParams) Brief() *types.BriefNet

Brief returns a standard summarized version of the net's params (not enough to rebuild it but enough to compare it)

func (*MLPParams) Marshal

func (np *MLPParams) Marshal() ([]byte, error)

Marshal is used to tell the param store how to write a NetParams object for an MLP net

func (MLPParams) String

func (np MLPParams) String() string

func (*MLPParams) Unmarshal

func (np *MLPParams) Unmarshal(b []byte) error

Unmarshal is used to tell the param store how to read a NetParams object for an MLP net

type NetParamStore

type NetParamStore interface {
	Delete(id string) error
	// Returns an array of net IDs matching a glob pattern, use * to retrieve all
	List(offset, limit int, pattern string) ([]string, int, error)
	// Load retrieves the NetParams for a net from a store, will return true and a nil error if found
	Load(id string, np NetParams) (bool, error)
	Save(id string, np NetParams) error
}

NetParamStore is an abstraction over the storage service that will be used to share the nets between instances, it should allow queries by ID and be fairly performant (Redis is a good option here but it's good to have flexibility)

func New

func New(conf config.Config) (NetParamStore, error)

New creates and returns the corresponding type of param store for the given configuration

type NetParams

type NetParams interface {
	Brief() *types.BriefNet
	Marshal() ([]byte, error)
	Unmarshal(b []byte) error
}

NetParams serves to ensure that any new net implementation will come with instructions for writing and reading it to and from the store as well as a way to get a standard representation of it that the API can use

type RedisAdapter

type RedisAdapter struct {
	// contains filtered or unexported fields
}

RedisAdapter is the neural net param store implementation for Redis

func NewRedisAdapter

func NewRedisAdapter(conf map[string]interface{}) (*RedisAdapter, error)

NewRedisAdapter returns an initialized Redis param store object

func (*RedisAdapter) Delete

func (ra *RedisAdapter) Delete(id string) error

Delete can be used to delete the state of a specific neural net from Redis

func (*RedisAdapter) List

func (ra *RedisAdapter) List(offset, limit int, pattern string) ([]string, int, error)

List can be used to page through the IDs of the nets that are stored in Redis by starting with offset 0 and then continuing to call the method with the updated cursor value it returns until it becomes 0

func (*RedisAdapter) Load

func (ra *RedisAdapter) Load(id string, np NetParams) (bool, error)

Load can be used to retrieve the state of a specific neural net from Redis

func (*RedisAdapter) Save

func (ra *RedisAdapter) Save(id string, np NetParams) error

Save can be used to upsert the state of a specific neural net to Redis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL