neural

Documentation

Overview

Package neural contains Neural Network functions.

Index

• type ActivationFunction
• type LayerFunc
• type MultiLayerNet
  • func NewMultiLayerNet(layers []int) *MultiLayerNet
  • func (m *MultiLayerNet) Fit(X base.FixedDataGrid)
  • func (m *MultiLayerNet) Predict(X base.FixedDataGrid) base.FixedDataGrid
  • func (m *MultiLayerNet) String() string
• type Network
  • func NewNetwork(size int, input int, f NeuralFunction) *Network
  • func (n *Network) Activate(with *mat64.Dense, maxIterations int)
  • func (n *Network) Error(outArg, errArg *mat64.Dense, maxIterations int) *mat64.Dense
  • func (n *Network) GetBias(node int) float64
  • func (n *Network) GetWeight(src, target int) float64
  • func (n *Network) SetBias(node int, v float64)
  • func (n *Network) SetWeight(src, target int, v float64)
  • func (n *Network) String() string
  • func (n *Network) UpdateBias(err *mat64.Dense, learnRate float64)
  • func (n *Network) UpdateWeights(out, err *mat64.Dense, learnRate float64)
• type NeuralFunction

Types

type ActivationFunction

type ActivationFunction func(float64) float64

type LayerFunc

type LayerFunc func(*mat64.Dense)

LayerFuncs are vectorised layer value transformation functions (e.g. sigmoid). They must operate in-place.

type MultiLayerNet

type MultiLayerNet struct {
	Convergence   float64
	MaxIterations int
	LearningRate  float64
	// contains filtered or unexported fields
}

MultiLayerNet creates a new Network which is conceptually organised into layers, zero or more of which are hidden.

Within each layer, no neurons are connected.

No neurons in a given layer are connected with any neurons in a previous layer.

Neurons can only be connected to neurons in the layer above.

func NewMultiLayerNet

func NewMultiLayerNet(layers []int) *MultiLayerNet

NewMultiLayerNet returns an underlying Network conceptuallyorganised into layers

Layers variable = slice of integers representing node count at each layer.

func (*MultiLayerNet) Fit

func (m *MultiLayerNet) Fit(X base.FixedDataGrid)

Fit trains the neural network on the given fixed datagrid.

Training stops when the mean-squared error acheived is less than the Convergence value, or when back-propagation has occured more times than the value set by MaxIterations.

func (*MultiLayerNet) Predict

func (m *MultiLayerNet) Predict(X base.FixedDataGrid) base.FixedDataGrid

Predict uses the underlying network to produce predictions for the class variables of X.

Can only predict one CategoricalAttribute at a time, or up to n FloatAttributes. Set or unset ClassAttributes to work around this limitation.

func (*MultiLayerNet) String

func (m *MultiLayerNet) String() string

String returns a human-readable summary of this network.

type Network

type Network struct {
	// contains filtered or unexported fields
}

Network represents the most general neural network possible Weights are stored in a dense matrix, each can have its own NeuralFunction.

func NewNetwork

func NewNetwork(size int, input int, f NeuralFunction) *Network

NewNetwork creates a new Network containing size neurons, with a certain number dedicated to input, and a pre-defined neural function applied to the rest.

Input nodes are set to have a Linear NeuralFunction and are connected to themselves for propagation.

func (*Network) Activate

func (n *Network) Activate(with *mat64.Dense, maxIterations int)

Activate propagates the given input matrix (with) across the network a certain number of times (up to maxIterations).

The with matrix should be size * size elements, with only the values of input neurons set (everything else should be zero).

If the network is conceptually organised into layers, maxIterations should be set to the number of layers.

This function overwrites whatever's stored in its first argument.

func (*Network) Error

func (n *Network) Error(outArg, errArg *mat64.Dense, maxIterations int) *mat64.Dense

Error computes the back-propagation error from a given size * 1 output vector and a size * 1 error vector for a given number of iterations.

outArg should be the response from Activate.

errArg should be the difference between the output neuron's output and that expected, and should be zero everywhere else.

If the network is conceptually organised into n layers, maxIterations should be set to n.

func (*Network) GetBias

func (n *Network) GetBias(node int) float64

GetBias returns the bias at a given neuron (counted from 1).

func (*Network) GetWeight

func (n *Network) GetWeight(src, target int) float64

GetWeight returns the weight between a given source and target neuron (counted from 1).

func (*Network) SetBias

func (n *Network) SetBias(node int, v float64)

SetBias sets the bias at a given neuron (counted from 1).

func (*Network) SetWeight

func (n *Network) SetWeight(src, target int, v float64)

SetWeight sets the weight between a given source and target neuron (counted from 1).

func (*Network) String

func (n *Network) String() string

String gets a human-readable representation of this network.

func (*Network) UpdateBias

func (n *Network) UpdateBias(err *mat64.Dense, learnRate float64)

UpdateBias computes B = B + l.E and updates the bias weights from a size * 1 back-propagated error vector.

func (*Network) UpdateWeights

func (n *Network) UpdateWeights(out, err *mat64.Dense, learnRate float64)

UpdateWeights takes an output size * 1 output vector and a size * 1 back-propagated error vector, as well as a learnRate and updates the internal weights matrix.

type NeuralFunction

type NeuralFunction struct {
	Forward  ActivationFunction
	Backward ActivationFunction
}

First function is always the forward activation function Second function is always the backward activation function

var Linear NeuralFunction = NeuralFunction{
	func(v float64) float64 { return v },
	func(v float64) float64 { return 1.0 },
}

LinearFunction doesn't modify the value

var Sigmoid NeuralFunction = NeuralFunction{
	func(v float64) float64 { return 1.0 / (1.0 + math.Exp(-v)) },
	func(v float64) float64 { return v * (1 - v) },
}

SigmoidForward function does S(t) = \frac{1}{1 + e^{-t}}.

See http://en.wikipedia.org/wiki/Sigmoid_function

var SoftplusRectifier NeuralFunction = NeuralFunction{
	func(v float64) float64 { return math.Log(1 + math.Exp(v)) },
	func(v float64) float64 { return v * (1 - v) },
}

Rectified Linear function https://www.wikiwand.com/en/Rectifier_(neural_networks)