Documentation
¶
Overview ¶
Package layers provides neural network layer implementations for the Zerfoo ML framework. It is organized into sub-packages by functional category, with a central registry that wires every layer into the model builder.
Sub-packages by category ¶
Activations:
- github.com/zerfoo/zerfoo/layers/activations — Activation functions (GELU, FastGELU, Sigmoid, Softmax, Tanh, Erf).
Attention:
- github.com/zerfoo/zerfoo/layers/attention — Attention mechanisms (Grouped Query Attention, Global Attention, Multi-Head Latent Attention).
Core math and tensor operations:
- github.com/zerfoo/zerfoo/layers/core — Arithmetic (Add, Sub, Mul, Div, Pow, Neg, Sqrt, Mod), comparison (Equal, Greater, LessOrEqual, Or, Where), shape manipulation (Reshape, Unsqueeze, Squeeze, Expand, Concat, Slice, Pad, Tile, Shape, Cast), linear algebra (MatMul, Gemm, Conv2d, GlobalAveragePool), rotary embeddings, FFN, Mixture of Experts, and more.
- github.com/zerfoo/zerfoo/layers/gather — Gather operation for index-based element selection.
- github.com/zerfoo/zerfoo/layers/reducesum — ReduceSum along specified axes.
- github.com/zerfoo/zerfoo/layers/transpose — Transpose for axis permutation.
Embeddings:
- github.com/zerfoo/zerfoo/layers/embeddings — Token and positional embedding layers.
Normalization:
- github.com/zerfoo/zerfoo/layers/normalization — RMSNorm, LayerNormalization, SimplifiedLayerNormalization, SkipSimplifiedLayerNormalization, and BatchNormalization.
Regularization:
- github.com/zerfoo/zerfoo/layers/regularization — Dropout and FeatureDropout.
Transformer:
- github.com/zerfoo/zerfoo/layers/transformer — Transformer building blocks (encoder/decoder Block).
Sequence modeling:
- github.com/zerfoo/zerfoo/layers/sequence — Sequence layers such as State Space Models.
Recurrent:
- github.com/zerfoo/zerfoo/layers/recurrent — Recurrent neural network layers.
Higher-order / composite:
- github.com/zerfoo/zerfoo/layers/components — Reusable composite components built from lower-level layers.
- github.com/zerfoo/zerfoo/layers/features — Feature extraction layers.
- github.com/zerfoo/zerfoo/layers/hrm — Hierarchical Reasoning Model layers.
- github.com/zerfoo/zerfoo/layers/tokenizers — Tokenizer integration layers.
Registry:
- github.com/zerfoo/zerfoo/layers/registry — Central registration point that maps layer names to builder functions via [registry.RegisterAll].
Layer registry ¶
The registry sub-package calls [model.RegisterLayer] for every built-in layer, associating a string name (e.g. "MatMul", "Softmax") with a generic builder function. Call [registry.RegisterAll] once at startup to make all layers available to the model builder.
Adding a new layer ¶
To add a new layer:
- Create the layer in the appropriate sub-package (or a new sub-package if no existing category fits). Implement the builder function with the signature expected by [model.RegisterLayer].
- Register the layer in [registry.RegisterAll] by adding a [model.RegisterLayer] call with a unique name and the builder function.
- Write tests in the same sub-package.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package activations provides activation function layers.
|
Package activations provides activation function layers. |
|
Package attention provides attention mechanisms for neural networks.
|
Package attention provides attention mechanisms for neural networks. |
|
Package audio provides audio-related neural network layers.
|
Package audio provides audio-related neural network layers. |
|
Package components provides reusable components for neural network layers.
|
Package components provides reusable components for neural network layers. |
|
Package core provides core neural network layer implementations.
|
Package core provides core neural network layer implementations. |
|
Package embeddings provides neural network embedding layers for the Zerfoo ML framework.
|
Package embeddings provides neural network embedding layers for the Zerfoo ML framework. |
|
Package gather provides the Gather layer for the Zerfoo ML framework.
|
Package gather provides the Gather layer for the Zerfoo ML framework. |
|
Package hrm implements the Hierarchical Reasoning Model.
|
Package hrm implements the Hierarchical Reasoning Model. |
|
Package normalization provides various normalization layers for neural networks.
|
Package normalization provides various normalization layers for neural networks. |
|
Package reducesum provides the ReduceSum layer for the Zerfoo ML framework.
|
Package reducesum provides the ReduceSum layer for the Zerfoo ML framework. |
|
Package registry provides a central registration point for all layer builders.
|
Package registry provides a central registration point for all layer builders. |
|
Package regularization provides regularization layers for neural networks.
|
Package regularization provides regularization layers for neural networks. |
|
Package sequence provides sequence modeling layers such as State Space Models.
|
Package sequence provides sequence modeling layers such as State Space Models. |
|
Package ssm implements state space model layers.
|
Package ssm implements state space model layers. |
|
Package transformer provides transformer building blocks such as the Transformer `Block` used in encoder/decoder stacks.
|
Package transformer provides transformer building blocks such as the Transformer `Block` used in encoder/decoder stacks. |
|
Package transpose provides the Transpose layer for the Zerfoo ML framework.
|
Package transpose provides the Transpose layer for the Zerfoo ML framework. |
|
Package vision provides vision-related neural network layers.
|
Package vision provides vision-related neural network layers. |
Click to show internal directories.
Click to hide internal directories.