neural

Documentation

Index

• func ValidateTrainConfig(c *config.TrainConfig) error
• type ActivFunc
• type Cost
• type CrossEntropy
  • func (c CrossEntropy) CostFunc(inMx, outMx, labelsMx mat64.Matrix) float64
  • func (c CrossEntropy) Delta(outMx, expMx mat64.Matrix) mat64.Matrix
• type Layer
  • func NewLayer(c *config.LayerConfig, layerIn int) (*Layer, error)
  • func (l Layer) ActFn() func(int, int, float64) float64
  • func (l Layer) ActGrad() func(int, int, float64) float64
  • func (l *Layer) Deltas() *mat64.Dense
  • func (l *Layer) FwdOut(inputMx mat64.Matrix) (mat64.Matrix, error)
  • func (l Layer) ID() string
  • func (l Layer) Kind() LayerKind
  • func (l *Layer) SetWeights(w *mat64.Dense) error
  • func (l *Layer) Weights() *mat64.Dense
• type LayerKind
  • func (l LayerKind) String() string
• type LogLikelihood
  • func (c LogLikelihood) CostFunc(inMx, outMx, labelsMx mat64.Matrix) float64
  • func (c LogLikelihood) Delta(outMx, expMx mat64.Matrix) mat64.Matrix
• type Network
  • func NewNetwork(c *config.NetConfig) (*Network, error)
  • func (n *Network) AddLayer(layer *Layer) error
  • func (n *Network) BackProp(inMx, errMx mat64.Matrix, fromLayer int) error
  • func (n *Network) Classify(inMx mat64.Matrix) (mat64.Matrix, error)
  • func (n *Network) ForwardProp(inMx mat64.Matrix, toLayer int) (mat64.Matrix, error)
  • func (n Network) ID() string
  • func (n Network) Kind() NetworkKind
  • func (n Network) Layers() []*Layer
  • func (n *Network) Train(c *config.TrainConfig, inMx *mat64.Dense, labelsVec *mat64.Vector) error
  • func (n *Network) Validate(valInMx *mat64.Dense, valOut *mat64.Vector) (float64, error)
• type NetworkKind
  • func (n NetworkKind) String() string

Functions

func ValidateTrainConfig

func ValidateTrainConfig(c *config.TrainConfig) error

ValidateTrainConfig validates training configuration. It returns error if any of the supplied configuration parameters are invalid.

Types

type ActivFunc

type ActivFunc func(int, int, float64) float64

ActivFunc defines a neuron activation function

type Cost

type Cost interface {
	// CostFunc defines neural network cost function for given input, output and labels.
	// It returns a single number: cost for given input and output
	CostFunc(mat64.Matrix, mat64.Matrix, mat64.Matrix) float64
	// Delta implements function that calculates error in the last network layer
	// It returns the output error matrix
	Delta(mat64.Matrix, mat64.Matrix) mat64.Matrix
}

Cost is neural network training cost

type CrossEntropy

type CrossEntropy struct{}

CrossEntropy implements Cost interface

func (CrossEntropy) CostFunc

func (c CrossEntropy) CostFunc(inMx, outMx, labelsMx mat64.Matrix) float64

CostFunc implements cross entropy cost function. C = -(sum(sum((out_k .* log(out) + (1 - out_k) .* log(1 - out)), 2)))/samples

func (CrossEntropy) Delta

func (c CrossEntropy) Delta(outMx, expMx mat64.Matrix) mat64.Matrix

Delta calculates the error of the last layer and returns it D = (out_k - out)

type Layer

type Layer struct {
	// contains filtered or unexported fields
}

Layer represents a Neural Network layer.

func NewLayer

func NewLayer(c *config.LayerConfig, layerIn int) (*Layer, error)

NewLayer creates a new neural network layer and returns it. Layer weights are initialized to uniformly distributed random values (-1,1) NewLayer fails with error if the neural network supplied as a parameter does not exist.

func (Layer) ActFn

func (l Layer) ActFn() func(int, int, float64) float64

ActFn returns layer activation function

func (Layer) ActGrad

func (l Layer) ActGrad() func(int, int, float64) float64

ActGrad returns layer gradient activation function

func (*Layer) Deltas

func (l *Layer) Deltas() *mat64.Dense

Deltas returns layer's output deltas matrix Deltas matrix is initialized to zeros and is only non-zero if the back propagation algorithm has been run.

func (*Layer) FwdOut

func (l *Layer) FwdOut(inputMx mat64.Matrix) (mat64.Matrix, error)

FwdOut calculates forward output of the network layer for given input. If the layer is an INPUT layer, it returns the matrix supplied as an argument.

func (Layer) ID

func (l Layer) ID() string

ID returns layer id

func (Layer) Kind

func (l Layer) Kind() LayerKind

Kind returns layer kind

func (*Layer) SetWeights

func (l *Layer) SetWeights(w *mat64.Dense) error

SetWeights allows to set neural network layer weights. It fails with error if either the supplied weights have different dimensions than the existing layer weights or if the passed in weights matrix is nil or if the layer is an INPUT layer: INPUT layer has no weights matrix.

func (*Layer) Weights

func (l *Layer) Weights() *mat64.Dense

Weights returns layer's eights matrix

type LayerKind

type LayerKind uint

LayerKind defines type of neural network layer There are three kinds available: INPUT, HIDDEN and OUTPUT

const (
	// INPUT is input network layer
	INPUT LayerKind = iota + 1
	// HIDDEN is hidden network layer
	HIDDEN
	// OUTPUT is output network layer
	OUTPUT
)

func (LayerKind) String

func (l LayerKind) String() string

String implements Stringer interface for nice LayerKind printing

type LogLikelihood

type LogLikelihood struct{}

LogLikelihood implements Cost interface

func (LogLikelihood) CostFunc

func (c LogLikelihood) CostFunc(inMx, outMx, labelsMx mat64.Matrix) float64

CostFunc implements log-likelihood cost function. C = -sum(sum(out_k.*log(out)))

func (LogLikelihood) Delta

func (c LogLikelihood) Delta(outMx, expMx mat64.Matrix) mat64.Matrix

Delta calculates the error of the last layer and returns it D = (out_k - out)

type Network

type Network struct {
	// contains filtered or unexported fields
}

Network represents Neural Network

func NewNetwork

func NewNetwork(c *config.NetConfig) (*Network, error)

NewNetwork creates new Neural Network based on the passed in configuration parameters. It fails with error if either the requested network type is not supported or if any of the neural network layers failed to be created.

func (*Network) AddLayer

func (n *Network) AddLayer(layer *Layer) error

AddLayer adds a neural layer to neural network or fails with error AddLayer places restrictions on adding new layers to the network: 1. INPUT layer - there can only be one INPUT layer 2. HIDDEN layer - new HIDDEN layer is appened after the last HIDDEN layer 3. OUTPUT layer - there can only be one OUTPUT layer AddLayer fails with error if either 1. or 3. are not satisfied

func (*Network) BackProp

func (n *Network) BackProp(inMx, errMx mat64.Matrix, fromLayer int) error

BackProp performs back propagation of neural network. It traverses neural network recursively from layer specified via parameter and calculates error deltas for each network layer. It fails with error if either the supplied input and delta matrices are nil or if the specified from boundary goes beyond the first network layer that can have output errors calculated

func (*Network) Classify

func (n *Network) Classify(inMx mat64.Matrix) (mat64.Matrix, error)

Classify classifies the provided data vector to a particular label class. It returns a matrix that contains probabilities of the input belonging to a particular class It returns error if the network forward propagation fails at any point during classification.

func (*Network) ForwardProp

func (n *Network) ForwardProp(inMx mat64.Matrix, toLayer int) (mat64.Matrix, error)

ForwardProp performs forward propagation for a given input up to a specified network layer. It recursively activates all layers in the network and returns the output in a matrix It fails with error if requested end layer index is beyond all available layers or if the supplied input data is nil.

func (Network) ID

func (n Network) ID() string

ID returns neural network id

func (Network) Kind

func (n Network) Kind() NetworkKind

Kind returns kind of neural network

func (Network) Layers

func (n Network) Layers() []*Layer

Layers returns network layers in slice sorted from INPUT to OUTPUT layer

func (*Network) Train

func (n *Network) Train(c *config.TrainConfig, inMx *mat64.Dense, labelsVec *mat64.Vector) error

Train trains feedforward neural network per configuration passed in as parameter. It returns error if either the training configuration is invalid ot the training fails.

func (*Network) Validate

func (n *Network) Validate(valInMx *mat64.Dense, valOut *mat64.Vector) (float64, error)

Validate runs forward propagation on the validation data set through neural network. It returns the percentage of successful classifications or error.

type NetworkKind

type NetworkKind uint

NetworkKind defines a type of neural network

const (
	// FEEDFWD is a feed forward Neural Network
	FEEDFWD NetworkKind = iota + 1
)

func (NetworkKind) String

func (n NetworkKind) String() string

String implements Stringer interface for pretty printing