scheduler

Documentation

Overview

Package scheduler provides learning rate scheduling strategies for optimizers.

Index

• type CosineAnnealing
  • func NewCosineAnnealing[T tensor.Numeric](cfg CosineAnnealingConfig[T]) *CosineAnnealing[T]
  • func (c *CosineAnnealing[T]) GetLR() T
  • func (c *CosineAnnealing[T]) Step(epoch int, _ float64)
• type CosineAnnealingConfig
• type ReduceOnPlateau
  • func NewReduceOnPlateau[T tensor.Numeric](cfg ReduceOnPlateauConfig[T]) *ReduceOnPlateau[T]
  • func (r *ReduceOnPlateau[T]) GetLR() T
  • func (r *ReduceOnPlateau[T]) Step(_ int, metric float64)
• type ReduceOnPlateauConfig
• type Scheduler

Types

type CosineAnnealing

type CosineAnnealing[T tensor.Numeric] struct {
	// contains filtered or unexported fields
}

CosineAnnealing implements cosine annealing learning rate scheduling.

func NewCosineAnnealing

func NewCosineAnnealing[T tensor.Numeric](cfg CosineAnnealingConfig[T]) *CosineAnnealing[T]

NewCosineAnnealing creates a new CosineAnnealing scheduler.

func (*CosineAnnealing[T]) GetLR

func (c *CosineAnnealing[T]) GetLR() T

GetLR returns the current learning rate.

func (*CosineAnnealing[T]) Step

func (c *CosineAnnealing[T]) Step(epoch int, _ float64)

Step computes the learning rate for the given epoch.

type CosineAnnealingConfig

type CosineAnnealingConfig[T tensor.Numeric] struct {
	// EtaMax is the initial (maximum) learning rate.
	EtaMax T

	// EtaMin is the minimum learning rate.
	EtaMin float64

	// TMax is the number of epochs for a full cosine cycle.
	TMax int

	// WarmRestarts enables cosine annealing with warm restarts. When true,
	// the epoch counter resets every TMax epochs.
	WarmRestarts bool
}

CosineAnnealingConfig holds configuration for the CosineAnnealing scheduler.

type ReduceOnPlateau

type ReduceOnPlateau[T tensor.Numeric] struct {
	// contains filtered or unexported fields
}

ReduceOnPlateau reduces the learning rate when a metric has stopped improving.

func NewReduceOnPlateau

func NewReduceOnPlateau[T tensor.Numeric](cfg ReduceOnPlateauConfig[T]) *ReduceOnPlateau[T]

NewReduceOnPlateau creates a new ReduceOnPlateau scheduler.

func (*ReduceOnPlateau[T]) GetLR

func (r *ReduceOnPlateau[T]) GetLR() T

GetLR returns the current learning rate.

func (*ReduceOnPlateau[T]) Step

func (r *ReduceOnPlateau[T]) Step(_ int, metric float64)

Step checks the metric and reduces LR if no improvement for patience epochs.

type ReduceOnPlateauConfig

type ReduceOnPlateauConfig[T tensor.Numeric] struct {
	// InitialLR is the starting learning rate.
	InitialLR T

	// Factor is the multiplier applied to the LR when reducing (e.g. 0.1).
	Factor float64

	// Patience is the number of epochs with no improvement before reducing.
	Patience int

	// MinLR is the minimum learning rate. LR will not be reduced below this value.
	MinLR float64

	// Mode is "min" (lower metric is better) or "max" (higher metric is better).
	Mode string
}

ReduceOnPlateauConfig holds configuration for the ReduceOnPlateau scheduler.

type Scheduler

type Scheduler[T tensor.Numeric] interface {
	// Step advances the scheduler by one epoch. The metric value is used
	// by metric-aware schedulers such as ReduceOnPlateau; schedulers that
	// do not use a metric may ignore it.
	Step(epoch int, metric float64)

	// GetLR returns the current learning rate.
	GetLR() T
}

Scheduler adjusts the learning rate over the course of training.