adaptivethrottler

Documentation

Overview

Package adaptivethrottler provides AdaptiveThrottler and PriorityThrottler policies.

Index

• Variables
• func NewPrioritizer() priority.Prioritizer
• func NewPrioritizerBuilder() priority.PrioritizerBuilder
• type AdaptiveThrottler
  • func NewWithDefaults[R any]() AdaptiveThrottler[R]
• type Builder
  • func NewBuilder[R any]() Builder[R]
• type Metrics
• type PriorityThrottler

Variables

var ErrExceeded = errors.New("failure rate exceeded")

ErrExceeded is returned when an execution exceeds the current failure rate.

Functions

func NewPrioritizer

func NewPrioritizer() priority.Prioritizer

NewPrioritizer returns a new Prioritizer.

func NewPrioritizerBuilder

func NewPrioritizerBuilder() priority.PrioritizerBuilder

NewPrioritizerBuilder returns a new PrioritizerBuilder.

Types

type AdaptiveThrottler

type AdaptiveThrottler[R any] interface {
	failsafe.ResultAgnosticPolicy[R]
	Metrics

	// AcquirePermit attempts to acquire a permit to perform an execution via the throttler, returning ErrExceeded if one
	// could not be acquired.
	AcquirePermit() error

	// TryAcquirePermit attempts to acquire a permit to perform an execution via the throttler, returning whether one could be acquired.
	TryAcquirePermit() bool

	// RecordResult records an execution result as a success or failure based on the failure handling configuration.
	RecordResult(result R)

	// RecordError records an error as a success or failure based on the failure handling configuration.
	RecordError(err error)

	// RecordSuccess records an execution success.
	RecordSuccess()

	// RecordFailure records an execution failure.
	RecordFailure()
}

AdaptiveThrottler throttles load probabalistically based on recent failures. This approach is described in the Google SRE book: https://sre.google/sre-book/handling-overload/#client-side-throttling-a7sYUg

func NewWithDefaults

func NewWithDefaults[R any]() AdaptiveThrottler[R]

NewWithDefaults returns a new AdaptiveThrottler with a failureRateThreshold of .1, a thresholdingPeriod of 1 minute, and a maxRejectionRate of .9. To configure additional options on am AdaptiveThrottler, use NewBuilder() instead.

type Builder

type Builder[R any] interface {
	failsafe.FailurePolicyBuilder[Builder[R], R]

	// WithFailureRateThreshold configures the failure rate threshold and thresholding period for the throttler. The
	// throttler will increase rejection probability when the failure rate exceeds this threshold over the specified time
	// period. The number of executions must also exceed the executionThreshold within the thresholdingPeriod
	// before any executions will be rejected.
	// Panics if failureRateThreshold < 0 or > 1.
	WithFailureRateThreshold(failureRateThreshold float64, executionThreshold uint, thresholdingPeriod time.Duration) Builder[R]

	// WithMaxRejectionRate configures the max allowed rejection rate, which defaults to .9.
	// Panics if maxRejectionRate < 0 or > 1.
	WithMaxRejectionRate(maxRejectionRate float64) Builder[R]

	// Build returns a new AdaptiveThrottler using the builder's configuration.
	Build() AdaptiveThrottler[R]

	// BuildPrioritized returns a new PrioritizedThrottler using the builder's configuration. This prioritizes rejections of
	// executions when throttling occurs. Rejections are performed using the Prioritizer, which sets a rejection threshold
	// based on all the throttlers being used by the Prioritizer. The Prioritizer can and should be shared across all
	// throttler instances that need to coordinate prioritization.
	//
	// Prioritized rejection is disabled by default, which means no executions will block when the throttler is full.
	BuildPrioritized(prioritizer priority.Prioritizer) PriorityThrottler[R]
}

Builder builds AdaptiveThrottler instances.

This type is not concurrency safe.

func NewBuilder

func NewBuilder[R any]() Builder[R]

NewBuilder returns an AdaptiveThrottler builder that defaults to a failureRateThreshold of .1, a thresholdingPeriod of 1 minute, and a maxRejectionRate of .9.

type Metrics

type Metrics interface {
	// RejectionRate returns the current rate, from 0 to 1, at which executions will be rejected, based on recent failures.
	RejectionRate() float64
}

type PriorityThrottler

type PriorityThrottler[R any] interface {
	failsafe.Policy[R]
	Metrics

	// AcquirePermit attempts to acquire a permit for an execution at the priority or level contained in the context,
	// returning ErrExceeded if one could not be acquired. A priority must be stored in the context using the PriorityKey,
	// or a level must be stored in the context using the LevelKey. The priority or level must be greater than the current
	// rejection threshold for admission. Levels must be between 0 and 499.
	//
	// Example usage:
	//   ctx := priority.ContextWithPriority(context.Background(), priority.High)
	//   permit, err := throttler.AcquirePermit(ctx)
	AcquirePermit(ctx context.Context) error

	// AcquirePermitWithPriority attempts to acquire a permit for an execution at the given priority, returning ErrExceeded
	// if one could not be acquired. The priority must be greater than the current rejection threshold for
	// admission.
	AcquirePermitWithPriority(priority priority.Priority) error

	// AcquirePermitWithLevel attempts to acquire a permit for an execution at the given level, returning ErrExceeded if one
	// could not be acquired. The level must be greater than the current rejection threshold for admission.
	AcquirePermitWithLevel(level int) error

	// TryAcquirePermit attempts to acquire a permit for an execution at the priority or level contained in the context,
	// returning one could be acquired. A priority must be stored in the context using the PriorityKey, or a level must be
	// stored in the context using the LevelKey. The priority or level must be greater than the current rejection threshold
	// for admission. Levels must be between 0 and 499.
	//
	// Example usage:
	//   ctx := priority.ContextWithPriority(context.Background(), priority.High)
	//   permit, err := throttler.AcquirePermit(ctx)
	TryAcquirePermit(ctx context.Context) bool

	// TryAcquirePermitWithPriority attempts to acquire a permit for an execution at the given priority, returning whether
	// one could be acquired. The priority must be greater than the current rejection threshold for admission.
	TryAcquirePermitWithPriority(priority priority.Priority) bool

	// TryAcquirePermitWithLevel attempts to acquire a permit for an execution at the given level, returning whether one
	// could be acquired. The level must be greater than the current rejection threshold for admission.
	TryAcquirePermitWithLevel(level int) bool

	// RecordResult records an execution result as a success or failure based on the failure handling configuration.
	RecordResult(result R)

	// RecordError records an error as a success or failure based on the failure handling configuration.
	RecordError(err error)

	// RecordSuccess records an execution success.
	RecordSuccess()

	// RecordFailure records an execution failure.
	RecordFailure()
}

PriorityThrottler is an adaptive throttler that throttles load probabalistically based on recent failures. When throttling is needed, it uses a Prioritizer to determine which priority levels should be rejected, allowing higher-priority executions to proceed while shedding lower-priority load.

R is the execution result type. This type is concurrency safe.