network

package
v0.0.0-...-8022e88 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 16, 2023 License: GPL-3.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Connection

type Connection struct {
	// To is the neuron that this Connection's owning neuron is considered to be
	// connected "to". This neuron should be in the layer previous to the
	// Connection's owning neuron.
	To *Neuron
	// contains filtered or unexported fields
}

Connection represents a line of communication between the output of one Neuron to the input of another. Every Neuron has a set of Connections to every Neuron in the previous Layer. Connections are one way structures. They only record the connected relationship of the owning Neuron "to" a Neuron in the previous Layer, but not the other way around.

func NewConnection

func NewConnection(pn *Neuron) *Connection

NewConnection creates a new connection assigning To to pn, which should be a Neuron from the previous Layer of the Network relative to the Layer the owning Neuron is in. The weight of this Connection is randomized.

func (*Connection) Equals

func (c *Connection) Equals(c2 *Connection) error

func (*Connection) MarshalJSON

func (c *Connection) MarshalJSON() ([]byte, error)

func (*Connection) SetWeight

func (c *Connection) SetWeight(weight float64)

func (*Connection) UnmarshalJSON

func (c *Connection) UnmarshalJSON(j []byte) error

type Deserializer

type Deserializer interface {
	Deserialize([]byte) (Network, error)
	MustDeserialize([]byte) Network
}

type Layer

type Layer []*Neuron

Layer is an isolated set of neurons in the network that constitute a single layer. A layer can be input, output, or hidden, a distinction which is only recognized by its position in the Network.

func MustNewLayer

func MustNewLayer(qn int, pl Layer, activationFunctionName activationfunction.Name) Layer

MustNewLayer calls NewLayer but panics if an error is encountered.

func NewLayer

func NewLayer(qn int, pl Layer, activationFunctionName activationfunction.Name) (Layer, error)

NewLayer will construct a new Layer with qn neurons each of which with the activation function corresponding to activationFunctionName. The new layer will have its new neurons connected to all neurons in pl, which should be the previous layer in the network.

If the network to be created is the input layer and does not have a previous layer, nil is an acceptable value to provide for pl.

func (Layer) ConnectNeurons

func (l Layer) ConnectNeurons(pl Layer) error

ConnectNeurons connects all neurons in l to all neurons in pl using the existing connections. It only updates what each neurons Connection.To points to. All other values are preserved.

func (Layer) ConnectTo

func (l Layer) ConnectTo(pl Layer)

ConnectTo connects all the neurons in l to all the neurons in pl using brand new connections.

func (Layer) ConnectWith

func (l Layer) ConnectWith(pl Layer, cs [][]*Connection) error

ConnectWith connects all the neurons in l to all the neurons in pl using the provided connections. cs is an MxN slice of connections where M is the number of neurons in l, and n is the number of neurons in pl. If this is not honored, an error will be returned.

func (Layer) Equals

func (l Layer) Equals(l2 Layer) error

func (Layer) FirstNeuron

func (l Layer) FirstNeuron() *Neuron

func (Layer) GetNeuron

func (l Layer) GetNeuron(i int) (*Neuron, error)

func (Layer) GetNeurons

func (l Layer) GetNeurons(i, j int) ([]*Neuron, error)

func (Layer) LastNeuron

func (l Layer) LastNeuron() *Neuron

func (Layer) MustGetNeuron

func (l Layer) MustGetNeuron(i int) *Neuron

MustGetNeuron calls GetNeuron but panics if an error is encountered.

func (Layer) MustGetNeurons

func (l Layer) MustGetNeurons(i, j int) []*Neuron

MustGetNeurons calls GetNeurons but panics if an error is encountered.

func (Layer) MustSetActivationFunctions

func (l Layer) MustSetActivationFunctions(activationFunctionNames []activationfunction.Name)

MustSetActivationFunctions calls SetActivationFunctions but panics if an error is encountered.

func (Layer) MustSetConnectionWeights

func (l Layer) MustSetConnectionWeights(weights [][]float64)

MustSetConnectionWeights calls SetConnectionWeights but panics if an error is encountered.

func (Layer) MustSetNeuron

func (l Layer) MustSetNeuron(i int, n *Neuron, pl Layer)

MustSetNeuron calls SetNeuron but panics if an error is encountered.

func (Layer) MustSetNeuronActivationFunctionsTo

func (l Layer) MustSetNeuronActivationFunctionsTo(activationFunctionName activationfunction.Name)

MustSetNeuronActivationFunctionsTo calls SetNeuronActivationFunctionsTo but panics if an error is encountered.

func (Layer) MustSetNeuronBiases

func (l Layer) MustSetNeuronBiases(biases []float64)

MustSetNeuronBiases calls SetNeuronBiases but panics if an error is encountered.

func (Layer) MustSetNeuronLabels

func (l Layer) MustSetNeuronLabels(labels []string)

MustSetNeuronLabels calls SetNeuronLabels but panics if an error is encountered.

func (Layer) MustSetNeuronValues

func (l Layer) MustSetNeuronValues(values []float64)

MustSetNeuronValues calls SetNeuronValues but panics if an error is encountered.

func (Layer) MustSetNeurons

func (l Layer) MustSetNeurons(i, j int, ns []*Neuron, pl Layer)

MustSetNeurons calls SetNeurons but panics if an error is encountered.

func (Layer) MustSwapOutNeuron

func (l Layer) MustSwapOutNeuron(i int, n *Neuron, pl Layer) *Neuron

MustSwapOutNeuron calls SwapOutNeuron but panics if an error is encountered.

func (Layer) MustSwapOutNeurons

func (l Layer) MustSwapOutNeurons(i, j int, ns []*Neuron, pl Layer) []*Neuron

MustSwapOutNeurons calls SwapOutNeurons but panics if an error is encountered.

func (Layer) NumNeurons

func (l Layer) NumNeurons() int

func (Layer) SetActivationFunctions

func (l Layer) SetActivationFunctions(activationFunctionNames []activationfunction.Name) error

func (Layer) SetConnectionWeights

func (l Layer) SetConnectionWeights(weights [][]float64) error

func (Layer) SetConnectionWeightsTo

func (l Layer) SetConnectionWeightsTo(weight float64)

func (Layer) SetNeuron

func (l Layer) SetNeuron(i int, n *Neuron, pl Layer) error

func (Layer) SetNeuronActivationFunctionsTo

func (l Layer) SetNeuronActivationFunctionsTo(activationFunctionName activationfunction.Name) error

func (Layer) SetNeuronBiases

func (l Layer) SetNeuronBiases(biases []float64) error

func (Layer) SetNeuronBiasesTo

func (l Layer) SetNeuronBiasesTo(value float64)

func (Layer) SetNeuronLabels

func (l Layer) SetNeuronLabels(labels []string) error

func (Layer) SetNeuronLabelsTo

func (l Layer) SetNeuronLabelsTo(label string)

func (Layer) SetNeuronValues

func (l Layer) SetNeuronValues(values []float64) error

func (Layer) SetNeuronValuesTo

func (l Layer) SetNeuronValuesTo(value float64)

func (Layer) SetNeurons

func (l Layer) SetNeurons(i, j int, ns []*Neuron, pl Layer) error

func (Layer) SwapOutNeuron

func (l Layer) SwapOutNeuron(i int, n *Neuron, pl Layer) (*Neuron, error)

func (Layer) SwapOutNeurons

func (l Layer) SwapOutNeurons(i, j int, ns []*Neuron, pl Layer) ([]*Neuron, error)

type Network

type Network []Layer

Network is the top level type for interacting with the neural networks this package provides.

func From

func From(spec Spec) (Network, error)

From creates a new fully-connected Network from the construction details in spec and returns an error if spec is invalid.

func MustFrom

func MustFrom(spec Spec) Network

MustFrom calls From but panics if an error is encountered.

func (Network) AdjustWeights

func (nw Network) AdjustWeights(learningRate float64)

AdjustWeights will nudge all the weights and biases across the entire network in the direction of progress towards minimizing the loss function by taking the average value across that neuron or connections nudge-values and pushing the weight in that direction scaled against learningRate.

This should be called after executing a forward and backward pass for an entire mini batch.

func (Network) BackwardPass

func (nw Network) BackwardPass(truth []float64) error

BackwardPass executes a backward pass on nw with truth compared to nw's output layer index-wise in order to calculate back propagation of the loss value across nw's constituent parts.

nw is mutated during this process to track the weighted sum of all inputs as its fed through the network as well as the calculus required to do back propagation and adjust the weights accordingly.

if len(truth) != len(nw.LastLayer()) then an error will be returned. TODO(justin): Break up

func (Network) CalculateLoss

func (nw Network) CalculateLoss(truth []float64) (float64, error)

CalculateLoss returns the loss value of the current state of nw's output layer as compared with the values in truth. This should be used after running ForwardPass to see the true value of loss for a given input.

If len(truth) != len(nw.LastLayer()) then an error will be returned.

func (Network) Equals

func (nw Network) Equals(nw2 Network) error

func (Network) FirstLayer

func (nw Network) FirstLayer() Layer

func (Network) ForwardPass

func (nw Network) ForwardPass(input []float64) error

ForwardPass executes a forward pass on nw with input fed into nw's input layer index-wise.

nw is mutated during this process to track the weighted sum of all inputs as its fed through the network as well as the calculus required to do back propagation and adjust the weights accordingly.

if len(input) != len(nw.FirstLayer()) then an error will be returned.

func (Network) GetLayer

func (nw Network) GetLayer(i int) (Layer, error)

func (Network) GetLayers

func (nw Network) GetLayers(i, j int) ([]Layer, error)

GetLayers returns the subset of layers from i to j, i inclusive and j exclusive.

func (Network) HighestConfidenceNeuron

func (nw Network) HighestConfidenceNeuron() *Neuron

HighestConfidenceNeuron iterates over every neuron in nw's last layer and returns the one with the highest value (confidence).

func (Network) IsFullyConnected

func (nw Network) IsFullyConnected() bool

IsFullyConnected reports whether all neurons in the network are connected to another neuron by checking that the neuron has the same number of connections as there are neurons in the previous layer and that each of those connections has its To field populated with a non-nil neuron.

func (Network) LastLayer

func (nw Network) LastLayer() Layer

func (Network) MustBackwardPass

func (nw Network) MustBackwardPass(truth []float64)

MustBackwardPass calls BackwardPass but panics if an error is encountered.

func (Network) MustCalculateLoss

func (nw Network) MustCalculateLoss(truth []float64) float64

MustCalculateLoss calls CalculateLoss but panics if an error is encountered.

func (Network) MustForwardPass

func (nw Network) MustForwardPass(input []float64)

MustForwardPass calls ForwardPass but panics if an error is encountered.

func (Network) MustGetLayer

func (nw Network) MustGetLayer(i int) Layer

MustGetLayer calls GetLayer but panics if an error is encountered.

func (Network) MustGetLayers

func (nw Network) MustGetLayers(i, j int) []Layer

MustGetLayers calls GetLayers but panics if an error is encountered.

func (Network) MustPredict

func (nw Network) MustPredict(input []float64) (string, float64)

MustPredict calls Predict but panics if an error is encountered.

func (Network) MustSetConnectionWeights

func (nw Network) MustSetConnectionWeights(weights [][][]float64)

MustSetConnectionWeights calls SetConnectionWeights but panics if an error is encountered.

func (Network) MustSetInputNeuronLabels

func (nw Network) MustSetInputNeuronLabels(labels []string)

MustSetInputNeuronLabels calls SetInputNeuronLabels but panics if an error is encountered.

func (Network) MustSetLayer

func (nw Network) MustSetLayer(i int, l Layer)

MustSetLayer calls SetLayer but panics if an error is encountered.

func (Network) MustSetLayers

func (nw Network) MustSetLayers(i, j int, ls []Layer)

MustSetLayers calls SetLayers but panics if an error is encountered.

func (Network) MustSetNeuronActivationFunctions

func (nw Network) MustSetNeuronActivationFunctions(activationFunctionNames [][]activationfunction.Name)

MustSetNeuronActivationFunctions calls SetNeuronActivationFunctions but panics if an error is encountered.

func (Network) MustSetNeuronActivationFunctionsTo

func (nw Network) MustSetNeuronActivationFunctionsTo(activationFunctionName activationfunction.Name)

MustSetNeuronActivationFunctionsTo calls SetNeuronActivationFunctionsTo but panics if an error is encountered.

func (Network) MustSetNeuronBiases

func (nw Network) MustSetNeuronBiases(biases [][]float64)

MustSetNeuronBiases calls SetNeuronBiases but panics if an error is encountered.

func (Network) MustSetNeuronLabels

func (nw Network) MustSetNeuronLabels(labels [][]string)

MustSetNeuronLabels calls SetNeuronLabels but panics if an error is encountered.

func (Network) MustSetNeuronValues

func (nw Network) MustSetNeuronValues(values [][]float64)

MustSetNeuronValues calls SetNeuronValues but panics if an error is encountered.

func (Network) MustSetOutputNeuronLabels

func (nw Network) MustSetOutputNeuronLabels(labels []string)

MustSetOutputNeuronLabels calls SetOutputNeuronLabels but panics if an error is encountered.

func (Network) MustSwapOutLayer

func (nw Network) MustSwapOutLayer(i int, l Layer) Layer

MustSwapOutLayer calls SwapOutLayer but panics if an error is encountered.

func (Network) MustSwapOutLayers

func (nw Network) MustSwapOutLayers(i, j int, ls []Layer) []Layer

MustSwapOutLayers calls SwapOutLayers but panics if an error is encountered.

func (Network) NumLayers

func (nw Network) NumLayers() int

func (Network) Predict

func (nw Network) Predict(input []float64) (string, float64, error)

Predict will execute a ForwardPass on nw using input as its input, then returns the label and value of the neuron with the highest confidence. If you wish to see all output neurons instead of just the neuron with highest confidence then use LastLayer to inspect them all.

If len(input) != len(nw.FirstLayer()) then an error will be returned.

func (Network) Reconnect

func (nw Network) Reconnect()

Reconnect connects all the neurons in nw to all the neurons in their previous layers using brand new connections. This function will scan over the entire network and recreate all connections between contiguous layers.

func (Network) ReconnectNeurons

func (nw Network) ReconnectNeurons() error

ReconnectNeurons connects all neurons in nw to all neurons in their previous layers using the existing connections. It only updates what each neurons Connection.To points to, all other values are preserved.

func (Network) ReconnectWith

func (nw Network) ReconnectWith(cs [][][]*Connection) error

ReconnectWith connects all the neurons in nw to all neurons in their previous layers using the provided connections. cs is an LxMxN slice of connections where L is the number of layers in nw, M is the number of neurons in l, and n is the number of neurons in pl. If this is not honored, an error will be returned.

func (Network) RecordNudges

func (nw Network) RecordNudges()

RecordNudges will record the values calculated for gradient descent in nw's relevant places so that after a Forward and Backward pass the direction of progress towards minimizing the loss function can be recorded.

This should be called for each time a forward and backward pass is executed in a mini batch.

TODO(justin): I feel like RecordNudges should be part of or called in BackwardPass since its only ever used directly after calling that and wouldn't have much value to an end user.

func (Network) ResetFromBatch

func (nw Network) ResetFromBatch()

ResetFromBatch will reset the entire network from the perspective of having just ran an entire mini batch. This differs from ResetFromPass in that it will reset all the nudges recorded on the bias and weights of the network, not just the calculus used for back propagation.

Call this function after executing a forward and backward pass for an entire mini batch.

func (Network) ResetFromPass

func (nw Network) ResetFromPass()

ResetFromPass will reset the entire network from the perspective of having just ran a forward and backward pass. This differs from ResetFromBatch in that it will only reset the calculus used for back propagation, not the nudges recorded on the bias and weights of each neuron.

Call this function after executing a forward and backward pass on the network once.

func (Network) SetConnectionWeights

func (nw Network) SetConnectionWeights(weights [][][]float64) error

func (Network) SetConnectionWeightsTo

func (nw Network) SetConnectionWeightsTo(weight float64)

func (Network) SetInputNeuronLabels

func (nw Network) SetInputNeuronLabels(labels []string) error

func (Network) SetInputNeuronLabelsTo

func (nw Network) SetInputNeuronLabelsTo(label string)

func (Network) SetLayer

func (nw Network) SetLayer(i int, l Layer) error

func (Network) SetLayers

func (nw Network) SetLayers(i, j int, ls []Layer) error

SetLayers sets the subset of layers from i to j, i inclusive and j exclusive, to ls.

func (Network) SetNeuronActivationFunctions

func (nw Network) SetNeuronActivationFunctions(activationFunctionNames [][]activationfunction.Name) error

func (Network) SetNeuronActivationFunctionsTo

func (nw Network) SetNeuronActivationFunctionsTo(activationFunctionName activationfunction.Name) error

func (Network) SetNeuronBiases

func (nw Network) SetNeuronBiases(biases [][]float64) error

func (Network) SetNeuronBiasesTo

func (nw Network) SetNeuronBiasesTo(val float64)

func (Network) SetNeuronLabels

func (nw Network) SetNeuronLabels(labels [][]string) error

func (Network) SetNeuronLabelsTo

func (nw Network) SetNeuronLabelsTo(label string)

func (Network) SetNeuronValues

func (nw Network) SetNeuronValues(values [][]float64) error

func (Network) SetNeuronValuesTo

func (nw Network) SetNeuronValuesTo(value float64)

func (Network) SetOutputNeuronLabels

func (nw Network) SetOutputNeuronLabels(labels []string) error

func (Network) SetOutputNeuronLabelsTo

func (nw Network) SetOutputNeuronLabelsTo(label string)

func (Network) SwapOutLayer

func (nw Network) SwapOutLayer(i int, l Layer) (Layer, error)

func (Network) SwapOutLayers

func (nw Network) SwapOutLayers(i, j int, ls []Layer) ([]Layer, error)

func (*Network) UnmarshalJSON

func (nw *Network) UnmarshalJSON(data []byte) error

type Neuron

type Neuron struct {
	// Connections is the set of weighted connections from this Neuron to the
	// previous layer's neurons. Connections refer to the "owning neuron", and
	// that refers to this neuron, not the neuron this connection leads to.
	Connections []*Connection

	// ActivationFunctionName is the activationfunction.Name that corresponds to
	// the activation function this Neuron should use.
	ActivationFunctionName activationfunction.Name
	// contains filtered or unexported fields
}

Neuron represents the base unit of a neural network. Each neuron should have a set of connections to every neuron in the previous layer via its Connections field. Additionally every neuron has an activation function associated with it for non-linearization of the weighted inputs from the previous layer.

func MustNewNeuron

func MustNewNeuron(pl Layer, activationFunctionName activationfunction.Name) *Neuron

MustNewNeuron calls NewNeuron but panics if an error is encountered.

func NewNeuron

func NewNeuron(pl Layer, activationFunctionName activationfunction.Name) (*Neuron, error)

NewNeuron creates a new Neuron and connects it to pl with randomized weights and a randomized bias value. activationFunctionName is used to assign an activation function to this Neuron. If an activation function can't be found matching the provided activationfunction.Name, an error is returned.

func (*Neuron) ConnectNeurons

func (n *Neuron) ConnectNeurons(pl Layer) error

ConnectNeurons connects n to all neurons in pl using the existing connections. It only updates what n.Connection.To points to. All other values are preserved.

func (*Neuron) ConnectTo

func (n *Neuron) ConnectTo(pl Layer)

ConnectTo connects n to all the neurons in pl using brand new connections.

func (*Neuron) ConnectWith

func (n *Neuron) ConnectWith(pl Layer, pcs []*Connection) error

ConnectWith connects n to all neurons in pl using the provided connections. You must provide the same number of connections are there are neurons in the previous layer.

func (*Neuron) Equals

func (n *Neuron) Equals(n2 *Neuron) error

func (*Neuron) FirstConnection

func (n *Neuron) FirstConnection() *Connection

func (*Neuron) GetConnection

func (n *Neuron) GetConnection(i int) (*Connection, error)

func (*Neuron) GetConnections

func (n *Neuron) GetConnections(i, j int) ([]*Connection, error)

func (*Neuron) LastConnection

func (n *Neuron) LastConnection() *Connection

func (*Neuron) MarshalJSON

func (n *Neuron) MarshalJSON() ([]byte, error)

func (*Neuron) MustGetConnection

func (n *Neuron) MustGetConnection(i int) *Connection

MustGetConnection calls GetConnection but panics if an error is encountered.

func (*Neuron) MustGetConnections

func (n *Neuron) MustGetConnections(i, j int) []*Connection

MustGetConnections calls GetConnections but panics if an error is encountered.

func (*Neuron) MustSetActivationFunction

func (n *Neuron) MustSetActivationFunction(activationFunctionName activationfunction.Name)

MustSetActivationFunction calls SetActivationFunction but panics if an error is encountered.

func (*Neuron) MustSetConnection

func (n *Neuron) MustSetConnection(i int, c *Connection)

MustSetConnection calls SetConnection but panics if an error is encountered.

func (*Neuron) MustSetConnectionWeights

func (n *Neuron) MustSetConnectionWeights(weights []float64)

MustSetConnectionWeights calls SetConnectionWeights but panics if an error is encountered.

func (*Neuron) MustSetConnections

func (n *Neuron) MustSetConnections(i, j int, cs []*Connection)

MustSetConnections calls SetConnections but panics if an error is encountered.

func (*Neuron) MustSwapOutConnection

func (n *Neuron) MustSwapOutConnection(i int, c *Connection) *Connection

MustSwapOutConnection calls SwapOutConnection but panics if an error is encountered.

func (*Neuron) MustSwapOutConnections

func (n *Neuron) MustSwapOutConnections(i, j int, cs []*Connection) []*Connection

MustSwapOutConnections calls SwapOutConnections but panics if an error is encountered.

func (*Neuron) NumConnections

func (n *Neuron) NumConnections() int

func (*Neuron) SetActivationFunction

func (n *Neuron) SetActivationFunction(activationFunctionName activationfunction.Name) error

func (*Neuron) SetBias

func (n *Neuron) SetBias(bias float64)

func (*Neuron) SetConnection

func (n *Neuron) SetConnection(i int, c *Connection) error

func (*Neuron) SetConnectionWeights

func (n *Neuron) SetConnectionWeights(weights []float64) error

func (*Neuron) SetConnectionWeightsTo

func (n *Neuron) SetConnectionWeightsTo(weight float64)

func (*Neuron) SetConnections

func (n *Neuron) SetConnections(i, j int, cs []*Connection) error

func (*Neuron) SetLabel

func (n *Neuron) SetLabel(label string)

func (*Neuron) SetValue

func (n *Neuron) SetValue(value float64)

func (*Neuron) SwapOutConnection

func (n *Neuron) SwapOutConnection(i int, c *Connection) (*Connection, error)

func (*Neuron) SwapOutConnections

func (n *Neuron) SwapOutConnections(i, j int, cs []*Connection) ([]*Connection, error)

func (*Neuron) UnmarshalJSON

func (n *Neuron) UnmarshalJSON(j []byte) error

type Serializer

type Serializer interface {
	Serialize(Network) ([]byte, error)
	MustSerialize(Network) []byte
}

type Spec

type Spec struct {
	// NeuronMap is intended to detail the number of layers in this network as
	// well as the number of neurons in each Layer. For example if index 3
	// contains the value 5, that would mean that the third Layer of the network
	// should contain 5 neurons.
	NeuronMap []int
	// InputLabels defines the labels for each of the input neurons.
	// len(InputLabels) must equal NeuronMap[0].
	InputLabels []string
	// OutputLabels defines the labels for each of the output neurons.
	// len(OutputLabels) must equal NeuronMap[len(NeuronMap)-1].
	OutputLabels []string
	// ActivationFunctionName is a name corresponding to an ActivationFunction
	// found in the activationfunction package. All neurons created for this
	// network will use this activation function.
	ActivationFunctionName activationfunction.Name
}

Spec defines the details for the construction of a Network.

type Translator

type Translator interface {
	Serializer
	Deserializer
}

func NewGobTranslator

func NewGobTranslator(opts ...TranslatorOption) Translator

func NewJsonTranslator

func NewJsonTranslator(opts ...TranslatorOption) Translator

func NewProtoTranslator

func NewProtoTranslator(opts ...TranslatorOption) Translator

type TranslatorOption

type TranslatorOption struct {
	Serialize   func(bs []byte) ([]byte, error)
	Deserialize func(bs []byte) ([]byte, error)
}

func WithBase64

func WithBase64() TranslatorOption

func WithCompression

func WithCompression() TranslatorOption

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL