regularization

Documentation

Overview

Package regularization provides regularization layers for neural networks.

Stability: stable

Package regularization provides regularization layers for neural networks.

Index

• func BuildDropout[T tensor.Float](engine compute.Engine[T], ops numeric.Arithmetic[T], _ string, ...) (graph.Node[T], error)
• func BuildFeatureDropout[T tensor.Float](engine compute.Engine[T], ops numeric.Arithmetic[T], _ string, ...) (graph.Node[T], error)
• type Dropout
  • func NewDropout[T tensor.Float](engine compute.Engine[T], ops numeric.Arithmetic[T], rate T, ...) *Dropout[T]
  • func (d *Dropout[T]) Attributes() map[string]any
  • func (d *Dropout[T]) Backward(ctx context.Context, _ types.BackwardMode, dOut *tensor.TensorNumeric[T], ...) ([]*tensor.TensorNumeric[T], error)
  • func (d *Dropout[T]) Forward(ctx context.Context, inputs ...*tensor.TensorNumeric[T]) (*tensor.TensorNumeric[T], error)
  • func (d *Dropout[T]) IsTraining() bool
  • func (d *Dropout[T]) OpType() string
  • func (d *Dropout[T]) OutputShape() []int
  • func (d *Dropout[T]) SetSaver(sv graph.Saver[T])
  • func (d *Dropout[T]) SetTraining(training bool)
• type DropoutOption
  • func WithDropoutSeed[T tensor.Float](seed uint64) DropoutOption[T]
  • func WithDropoutSource[T tensor.Float](src rand.Source) DropoutOption[T]
  • func WithEngineDropout[T tensor.Float]() DropoutOption[T]
• type FeatureDropout
  • func NewFeatureDropout[T tensor.Float](engine compute.Engine[T], ops numeric.Arithmetic[T], rate T, ...) *FeatureDropout[T]
  • func (d *FeatureDropout[T]) Attributes() map[string]any
  • func (d *FeatureDropout[T]) Backward(ctx context.Context, _ types.BackwardMode, dOut *tensor.TensorNumeric[T], ...) ([]*tensor.TensorNumeric[T], error)
  • func (d *FeatureDropout[T]) Forward(ctx context.Context, inputs ...*tensor.TensorNumeric[T]) (*tensor.TensorNumeric[T], error)
  • func (d *FeatureDropout[T]) IsTraining() bool
  • func (d *FeatureDropout[T]) OpType() string
  • func (d *FeatureDropout[T]) OutputShape() []int
  • func (d *FeatureDropout[T]) SetSaver(sv graph.Saver[T])
  • func (d *FeatureDropout[T]) SetTraining(training bool)
• type FeatureDropoutOption
  • func WithFeatureDropoutSeed[T tensor.Float](seed uint64) FeatureDropoutOption[T]
  • func WithFeatureDropoutSource[T tensor.Float](src rand.Source) FeatureDropoutOption[T]

Functions

func BuildDropout

func BuildDropout[T tensor.Float](
	engine compute.Engine[T],
	ops numeric.Arithmetic[T],
	_ string,
	_ map[string]*graph.Parameter[T],
	attributes map[string]any,
) (graph.Node[T], error)

BuildDropout constructs a Dropout node from the provided attributes.

func BuildFeatureDropout

func BuildFeatureDropout[T tensor.Float](
	engine compute.Engine[T],
	ops numeric.Arithmetic[T],
	_ string,
	_ map[string]*graph.Parameter[T],
	attributes map[string]any,
) (graph.Node[T], error)

BuildFeatureDropout constructs a FeatureDropout node from the provided attributes.

Types

type Dropout

type Dropout[T tensor.Float] struct {
	graph.NoParameters[T]
	// contains filtered or unexported fields
}

Dropout implements inverted dropout regularization. During training, each element is zeroed with probability `rate` and the surviving elements are scaled by 1/(1-rate) so that expected values are preserved. During evaluation (the default mode) the input is returned unchanged.

func NewDropout

func NewDropout[T tensor.Float](engine compute.Engine[T], ops numeric.Arithmetic[T], rate T, opts ...DropoutOption[T]) *Dropout[T]

NewDropout creates a new Dropout layer with the given drop rate. The rate must be in [0, 1). A rate of 0 disables dropout entirely.

By default the dropout mask is drawn from the unseeded package-global math/rand/v2 source, so masks are not reproducible across runs. Pass WithDropoutSeed or WithDropoutSource to make the mask deterministic.

func (*Dropout[T]) Attributes

func (d *Dropout[T]) Attributes() map[string]any

Attributes returns the non-tensor attributes of the layer.

func (*Dropout[T]) Backward

func (d *Dropout[T]) Backward(ctx context.Context, _ types.BackwardMode, dOut *tensor.TensorNumeric[T], inputs ...*tensor.TensorNumeric[T]) ([]*tensor.TensorNumeric[T], error)

Backward computes the backward pass. In evaluation mode the upstream gradient is returned unchanged. In training mode the upstream gradient is multiplied by the cached mask from the most recent Forward call.

func (*Dropout[T]) Forward

func (d *Dropout[T]) Forward(ctx context.Context, inputs ...*tensor.TensorNumeric[T]) (*tensor.TensorNumeric[T], error)

Forward computes the forward pass. In evaluation mode the input is returned unchanged. In training mode each element is independently zeroed with probability rate, and surviving elements are scaled by 1/(1-rate) (inverted dropout).

func (*Dropout[T]) IsTraining

func (d *Dropout[T]) IsTraining() bool

IsTraining returns whether the layer is in training mode.

func (*Dropout[T]) OpType

func (d *Dropout[T]) OpType() string

OpType returns the operation type.

func (*Dropout[T]) OutputShape

func (d *Dropout[T]) OutputShape() []int

OutputShape returns the output shape from the most recent Forward call.

func (*Dropout[T]) SetSaver

func (d *Dropout[T]) SetSaver(sv graph.Saver[T])

SetSaver implements graph.SaverAware.

func (*Dropout[T]) SetTraining

func (d *Dropout[T]) SetTraining(training bool)

SetTraining enables or disables training mode.

type DropoutOption

type DropoutOption[T tensor.Float] func(*Dropout[T])

DropoutOption configures optional behavior of a Dropout layer.

func WithDropoutSeed

func WithDropoutSeed[T tensor.Float](seed uint64) DropoutOption[T]

WithDropoutSeed returns a DropoutOption that makes the dropout mask reproducible by drawing it from a deterministic source seeded with the given value. Two layers constructed with the same seed produce identical masks for identical inputs and call sequences. Without this option (the default) masks are drawn from the unseeded package-global source and are not reproducible.

func WithDropoutSource

func WithDropoutSource[T tensor.Float](src rand.Source) DropoutOption[T]

WithDropoutSource returns a DropoutOption that draws the dropout mask from the provided source, allowing callers to supply their own seeded or shared rand.Source for reproducible training. A nil source is ignored, leaving the default unseeded behavior in place.

func WithEngineDropout

func WithEngineDropout[T tensor.Float]() DropoutOption[T]

WithEngineDropout returns a DropoutOption that routes the dropout mask through the engine's capture-safe Dropout op (compute.Dropouter) instead of generating the mask on the host. The op derives the mask deterministically on-device from a per-step counter-based seed (Philox) and regenerates it in Backward, so no mask tensor is pinned for the backward pass — this is what makes dropout eligible for CUDA-graph capture on the GPU, where a host-generated random mask is capture-ineligible.

If the engine does not implement compute.Dropouter, the layer transparently falls back to the host mask path. Combine with WithDropoutSeed to make the engine-op masks reproducible across runs; without it a distinct base seed is drawn once at construction so multiple layers decorrelate.

type FeatureDropout

type FeatureDropout[T tensor.Float] struct {
	graph.NoParameters[T]
	// contains filtered or unexported fields
}

FeatureDropout implements feature-level (column-wise) inverted dropout. During training, entire feature columns are zeroed with probability rate, and surviving columns are scaled by 1/(1-rate). During evaluation the input is returned unchanged.

func NewFeatureDropout

func NewFeatureDropout[T tensor.Float](engine compute.Engine[T], ops numeric.Arithmetic[T], rate T, opts ...FeatureDropoutOption[T]) *FeatureDropout[T]

NewFeatureDropout creates a new FeatureDropout layer with the given drop rate. The rate must be in [0, 1). A rate of 0 disables dropout entirely.

By default the per-feature mask is drawn from the unseeded package-global math/rand/v2 source, so masks are not reproducible across runs. Pass WithFeatureDropoutSeed or WithFeatureDropoutSource to make the mask deterministic.

func (*FeatureDropout[T]) Attributes

func (d *FeatureDropout[T]) Attributes() map[string]any

Attributes returns the non-tensor attributes of the layer.

func (*FeatureDropout[T]) Backward

func (d *FeatureDropout[T]) Backward(ctx context.Context, _ types.BackwardMode, dOut *tensor.TensorNumeric[T], inputs ...*tensor.TensorNumeric[T]) ([]*tensor.TensorNumeric[T], error)

Backward computes the backward pass. In evaluation mode the upstream gradient is returned unchanged. In training mode the upstream gradient is multiplied by the cached mask from the most recent Forward call.

func (*FeatureDropout[T]) Forward

func (d *FeatureDropout[T]) Forward(ctx context.Context, inputs ...*tensor.TensorNumeric[T]) (*tensor.TensorNumeric[T], error)

Forward computes the forward pass. In evaluation mode the input is returned unchanged. In training mode entire feature columns (axis=1) are independently zeroed with probability rate, and surviving columns are scaled by 1/(1-rate).

func (*FeatureDropout[T]) IsTraining

func (d *FeatureDropout[T]) IsTraining() bool

IsTraining returns whether the layer is in training mode.

func (*FeatureDropout[T]) OpType

func (d *FeatureDropout[T]) OpType() string

OpType returns the operation type.

func (*FeatureDropout[T]) OutputShape

func (d *FeatureDropout[T]) OutputShape() []int

OutputShape returns the output shape from the most recent Forward call.

func (*FeatureDropout[T]) SetSaver

func (d *FeatureDropout[T]) SetSaver(sv graph.Saver[T])

SetSaver implements graph.SaverAware.

func (*FeatureDropout[T]) SetTraining

func (d *FeatureDropout[T]) SetTraining(training bool)

SetTraining enables or disables training mode.

type FeatureDropoutOption

type FeatureDropoutOption[T tensor.Float] func(*FeatureDropout[T])

FeatureDropoutOption configures optional behavior of a FeatureDropout layer.

func WithFeatureDropoutSeed

func WithFeatureDropoutSeed[T tensor.Float](seed uint64) FeatureDropoutOption[T]

WithFeatureDropoutSeed returns a FeatureDropoutOption that makes the per-feature dropout mask reproducible by drawing it from a deterministic source seeded with the given value. Without this option (the default) masks are drawn from the unseeded package-global source and are not reproducible.

func WithFeatureDropoutSource

func WithFeatureDropoutSource[T tensor.Float](src rand.Source) FeatureDropoutOption[T]

WithFeatureDropoutSource returns a FeatureDropoutOption that draws the mask from the provided source, allowing callers to supply their own seeded or shared rand.Source for reproducible training. A nil source is ignored, leaving the default unseeded behavior in place.