bls

Documentation

Overview

Package bls provides an implementation of the Broad Learning System (BLS) described in "Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture" by C. L. Philip Chen and Zhulin Liu, 2017. (https://ieeexplore.ieee.org/document/7987745)

The "Model" contains only the inference part of the Broad Learning System. The ridge regression approximation training is performed by the "BroadLearningAlgorithm".

Since the forward pass is built using the computational graph ("ag" a.k.a. auto-grad package), you can train the BLS through the gradient-descent learning method, like other neural models, updating all the parameters and not just the output weights as in the original implementation. Cool, isn't it? ;)

Index

• type BroadLearningAlgorithm
  • func (l *BroadLearningAlgorithm) Do()
• type Config
• type Model
  • func New(c Config) *Model
  • func (m *Model) Forward(xs ...ag.Node) []ag.Node

Types

type BroadLearningAlgorithm

type BroadLearningAlgorithm struct {
	Model                  *Model
	Input                  []mat.Matrix
	DesiredOutput          []mat.Matrix
	Penalty                mat.Float
	OptimizeFeaturesWeight bool // skip optimization if you don't want to
	Verbose                bool
}

BroadLearningAlgorithm performs the ridge regression approximation to optimize the output params (Wo). The parameters for feature mapping (Wz) can also be optimized through the alternating direction method of multipliers (ADMM) method (Goldstein et al. 2014). The parameters of the enhanced nodes remain the initial ones and are not optimized.

func (*BroadLearningAlgorithm) Do

func (l *BroadLearningAlgorithm) Do()

Do runs the board learning algorithm.

type Config

type Config struct {
	InputSize                    int
	FeaturesSize                 int
	NumOfFeatures                int
	EnhancedNodesSize            int
	OutputSize                   int
	FeaturesActivation           ag.OpName
	EnhancedNodesActivation      ag.OpName
	OutputActivation             ag.OpName
	KeepFeaturesParamsFixed      bool
	KeepEnhancedNodesParamsFixed bool
	FeaturesDropout              mat.Float
	EnhancedNodesDropout         mat.Float
}

Config provides configuration settings for a BLS Model.

type Model

type Model struct {
	nn.BaseModel
	Config
	Wz []nn.Param `spago:"type:weights"`
	Bz []nn.Param `spago:"type:biases"`
	Wh nn.Param   `spago:"type:weights"`
	Bh nn.Param   `spago:"type:biases"`
	W  nn.Param   `spago:"type:weights"`
	B  nn.Param   `spago:"type:biases"`
}

Model contains the serializable parameters.

func New

func New(c Config) *Model

New returns a new model with parameters initialized to zeros.

func (*Model) Forward

func (m *Model) Forward(xs ...ag.Node) []ag.Node

Forward performs the forward step for each input node and returns the result.