Documentation
¶
Overview ¶
Package layers provides neural network layer implementations for the Zerfoo ML framework. It is organized into sub-packages by functional category, with a central registry that wires every layer into the model builder. (Stability: stable)
Sub-packages by category ¶
Activations:
- github.com/zerfoo/zerfoo/layers/activations — Activation functions (GELU, FastGELU, Sigmoid, Softmax, Tanh, Erf).
Attention:
- github.com/zerfoo/zerfoo/layers/attention — Attention mechanisms (Grouped Query Attention, Global Attention, Multi-Head Latent Attention).
Core math and tensor operations:
- github.com/zerfoo/zerfoo/layers/core — Arithmetic (Add, Sub, Mul, Div, Pow, Neg, Sqrt, Mod), comparison (Equal, Greater, LessOrEqual, Or, Where), shape manipulation (Reshape, Unsqueeze, Squeeze, Expand, Concat, Slice, Pad, Tile, Shape, Cast), linear algebra (MatMul, Gemm, Conv2d, GlobalAveragePool), rotary embeddings, FFN, Mixture of Experts, and more.
- github.com/zerfoo/zerfoo/layers/gather — Gather operation for index-based element selection.
- github.com/zerfoo/zerfoo/layers/reducesum — ReduceSum along specified axes.
- github.com/zerfoo/zerfoo/layers/transpose — Transpose for axis permutation.
Embeddings:
- github.com/zerfoo/zerfoo/layers/embeddings — Token and positional embedding layers.
Normalization:
- github.com/zerfoo/zerfoo/layers/normalization — RMSNorm, LayerNormalization, SimplifiedLayerNormalization, SkipSimplifiedLayerNormalization, and BatchNormalization.
Regularization:
- github.com/zerfoo/zerfoo/layers/regularization — Dropout and FeatureDropout.
Transformer:
- github.com/zerfoo/zerfoo/layers/transformer — Transformer building blocks (encoder/decoder Block).
State space models:
- github.com/zerfoo/zerfoo/layers/ssm — State space model layers (Mamba, RWKV, S4, MIMO SSM, complex state, B/C normalization).
Recurrent:
- github.com/zerfoo/zerfoo/layers/recurrent — Recurrent neural network layers.
Higher-order / composite:
- github.com/zerfoo/zerfoo/layers/components — Reusable composite components built from lower-level layers.
- github.com/zerfoo/zerfoo/layers/hrm — Hierarchical Reasoning Model layers.
Registry:
- github.com/zerfoo/zerfoo/layers/registry — Central registration point that maps layer names to builder functions via [registry.RegisterAll].
Layer registry ¶
The registry sub-package calls [model.RegisterLayer] for every built-in layer, associating a string name (e.g. "MatMul", "Softmax") with a generic builder function. Call [registry.RegisterAll] once at startup to make all layers available to the model builder.
Adding a new layer ¶
To add a new layer:
- Create the layer in the appropriate sub-package (or a new sub-package if no existing category fits). Implement the builder function with the signature expected by [model.RegisterLayer].
- Register the layer in [registry.RegisterAll] by adding a [model.RegisterLayer] call with a unique name and the builder function.
- Write tests in the same sub-package.
Stability: stable
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package activations provides activation function layers.
|
Package activations provides activation function layers. |
|
Package attention provides attention mechanisms for neural networks.
|
Package attention provides attention mechanisms for neural networks. |
|
Package audio provides audio-related neural network layers.
|
Package audio provides audio-related neural network layers. |
|
Package components provides reusable components for neural network layers.
|
Package components provides reusable components for neural network layers. |
|
Package core provides core neural network layer implementations.
|
Package core provides core neural network layer implementations. |
|
Package embeddings provides neural network embedding layers.
|
Package embeddings provides neural network embedding layers. |
|
Package gather provides the Gather layer for embedding-table lookup.
|
Package gather provides the Gather layer for embedding-table lookup. |
|
Package hrm implements the Hierarchical Reasoning Model.
|
Package hrm implements the Hierarchical Reasoning Model. |
|
Package normalization provides normalization layers for neural networks.
|
Package normalization provides normalization layers for neural networks. |
|
Package recurrent provides recurrent neural network layers.
|
Package recurrent provides recurrent neural network layers. |
|
Package reducesum provides the ReduceSum layer for axis-wise reduction.
|
Package reducesum provides the ReduceSum layer for axis-wise reduction. |
|
Package registry provides a central registration point for all layer builders.
|
Package registry provides a central registration point for all layer builders. |
|
Package regularization provides regularization layers for neural networks.
|
Package regularization provides regularization layers for neural networks. |
|
Package residual provides residual connection layers for neural networks.
|
Package residual provides residual connection layers for neural networks. |
|
Package ssm implements state space model layers.
|
Package ssm implements state space model layers. |
|
Package timeseries provides time-series specific neural network layers.
|
Package timeseries provides time-series specific neural network layers. |
|
Package transformer provides transformer building blocks such as the Transformer `Block` used in encoder/decoder stacks.
|
Package transformer provides transformer building blocks such as the Transformer `Block` used in encoder/decoder stacks. |
|
Package transpose provides the Transpose layer for axis permutation.
|
Package transpose provides the Transpose layer for axis permutation. |
|
Package vision provides vision-related neural network layers.
|
Package vision provides vision-related neural network layers. |
Click to show internal directories.
Click to hide internal directories.