stormbreak

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 2, 2026 License: MIT Imports: 10 Imported by: 0

README

STORMBREAK

Shared retry budgets for resilient Go applications.

CI CodeQL Go Reference

Stormbreak prevents failing services from being overwhelmed by the retries intended to save them.

Project status: release candidate for v0.1.0. The API is intentionally small but remains pre-1.0; review the changelog when upgrading minor versions.

Ordinary retry loops make decisions in isolation. Under load, hundreds of HTTP requests, workers, or database operations can all retry together, amplifying the failure. Stormbreak keeps retry policy local to an operation while sharing a concurrency-safe token budget across every caller that depends on the same resource.

Installation

Stormbreak requires Go 1.23 or newer.

go get github.com/magnexis/stormbreak

It has no third-party runtime dependencies.

Basic usage

budget, err := stormbreak.NewBudget(stormbreak.Config{
    Capacity:       100,
    RefillRate:     10,
    RefillInterval: time.Second,
})
if err != nil {
    return err
}

result, err := stormbreak.Do(
    ctx,
    budget,
    stormbreak.Policy{
        MaxAttempts: 5,
        BaseDelay:   200 * time.Millisecond,
        MaxDelay:    5 * time.Second,
        Multiplier:  2,
        Jitter:      true,
    },
    func(ctx context.Context) (string, error) {
        return fetchRemoteResource(ctx)
    },
)

The first attempt is always free. Each retry consumes one token. Tokens return lazily at the configured rate, without a refill goroutine.

Share budgets by dependency

One budget should represent one constrained dependency or failure domain, not one call site. A registry is useful when an application has several of them:

registry := stormbreak.NewRegistry()

githubBudget, err := registry.Create("github-api", stormbreak.Config{
    Capacity: 30, RefillRate: 3, RefillInterval: time.Second,
})
if err != nil {
    return err
}

databaseBudget, err := registry.Create("database", stormbreak.Config{
    Capacity: 10, RefillRate: 1, RefillInterval: time.Second,
})

Pass githubBudget to every GitHub caller and databaseBudget to every database caller. Registry.Snapshot and TokenBudget.Snapshot expose immutable state for monitoring.

HTTP client

The optional httpretry package retries temporary network failures and status codes 408, 425, 429, 500, 502, 503, and 504 by default.

client := &http.Client{
    Timeout: 10 * time.Second,
    Transport: &httpretry.Transport{
        Budget:        budget,
        Policy:        stormbreak.DefaultPolicy(),
        MaxRetryAfter: 30 * time.Second, // zero defaults to one minute
    },
}

The transport honors Retry-After, closes intermediate response bodies, and replays bodies only through Request.GetBody. Server-requested delays are bounded by MaxRetryAfter so an untrusted header cannot create an effectively unbounded timer; the request context remains the ultimate deadline. It does not retry authentication or general validation failures by default.

HTTP retries do not make writes safe. The default transport retries idempotent methods. POST and PATCH additionally require an Idempotency-Key header and a replayable body, but the upstream server must actually enforce that key. A custom Transport.Classifier can alter response/error classification; request safety checks still apply.

The same hooks used by Do can observe the HTTP transport:

transport := &httpretry.Transport{
    Budget: budget,
    Policy: stormbreak.DefaultPolicy(),
    Hooks: stormbreak.Hooks{
        OnFailure: func(event stormbreak.FailureEvent) {
            var statusErr *httpretry.StatusError
            if errors.As(event.Error, &statusErr) {
                metrics.RecordRetryableStatus(statusErr.StatusCode)
            }
        },
    },
}

Retryable HTTP responses appear in hooks as *httpretry.StatusError. A final retryable response is still returned normally after policy attempts are used, following net/http conventions. If the shared budget stops the request first, the status error is retained as RetryError.LastError.

Error handling

When policy attempts or shared capacity run out, Stormbreak returns a *RetryError. Both its cause and final operation error participate in errors.Is and errors.As.

result, err := stormbreak.Do(ctx, budget, policy, operation)
if errors.Is(err, stormbreak.ErrBudgetExhausted) {
    // Fail fast, shed work, or surface dependency pressure.
}
if errors.Is(err, stormbreak.ErrAttemptsExhausted) {
    // This operation used every permitted attempt.
}

var retryErr *stormbreak.RetryError
if errors.As(err, &retryErr) {
    log.Printf("attempts=%d last_error=%v", retryErr.Attempts, retryErr.LastError)
}
_ = result

Invalid budgets and policies wrap ErrInvalidBudget and ErrInvalidPolicy with descriptive context. A policy multiplier of zero uses the safe default of 2, which keeps concise policy literals useful; any explicit multiplier must be at least 1.

Retry classification

Ordinary errors are retryable by default unless the context is canceled. Mark known permanent failures at their source:

return stormbreak.Permanent(errors.New("invalid credentials"))

Or install a domain classifier:

err := stormbreak.DoVoid(ctx, budget, policy, operation,
    stormbreak.WithClassifier(func(err error) bool {
        return errors.Is(err, ErrTemporarilyUnavailable)
    }),
)

AlwaysRetry and NeverRetry are provided for explicit policies. Permanent errors stop immediately, before any retry token is consumed.

Database writes

Retry only errors the driver or database identifies as transient. A retried write may need a transaction restart, uniqueness constraint, or application idempotency key.

err := stormbreak.DoVoid(
    ctx,
    databaseBudget,
    stormbreak.Policy{
        MaxAttempts: 4,
        BaseDelay:   100 * time.Millisecond,
        MaxDelay:    time.Second,
        Multiplier:  2,
        Jitter:      true,
    },
    func(ctx context.Context) error {
        return repository.Save(ctx, record)
    },
    stormbreak.WithClassifier(isTemporaryDatabaseError),
)

Worker pools

Give every worker that reaches the same dependency the same budget:

err := stormbreak.DoVoid(ctx, queueBudget, workerPolicy,
    func(ctx context.Context) error {
        return processor.Handle(ctx, message)
    },
)

If that dependency fails, only the shared token capacity can become retries—not the number of workers multiplied by each worker's maximum attempts. Message acknowledgement, deduplication, and dead-letter behavior remain queue concerns.

Hooks and observability

hooks := stormbreak.Hooks{
    OnRetry: func(event stormbreak.RetryEvent) {
        retries.Add(ctx, 1)
        logger.Printf("attempt=%d delay=%s remaining=%d err=%v",
            event.Attempt, event.Delay, event.BudgetRemaining, event.Error)
    },
}

result, err := stormbreak.Do(ctx, budget, policy, operation,
    stormbreak.WithHooks(hooks),
)

Hooks are optional, nil-safe, synchronous, and dependency-free. They should return quickly and must not block critical application paths. If exporting telemetry can block, enqueue it in a bounded application-owned channel.

Concurrency guarantees

  • TokenBudget methods are safe for concurrent use.
  • Refills are calculated lazily while holding the budget lock; no background goroutine or ticker is created.
  • Registry operations and snapshots are safe during concurrent creation, lookup, and deletion.
  • Do keeps all execution state local to the call.
  • Hooks and operations are invoked synchronously; their own shared state remains the application's responsibility.
  • httpretry.Transport is safe for concurrent requests when its fields are not mutated after first use and its Base, budget, classifier, and hooks satisfy their own concurrency requirements. Hooks may run concurrently across calls.

Why not a basic loop?

Capability Basic retry loop Stormbreak
Exponential backoff Manual Yes
Context cancellation Often missing Yes
Shared retry limits No Yes
Retry classification Manual Yes
Jitter Manual Yes
Observability hooks Manual Yes
HTTP integration Manual Yes
Concurrent safety Uncertain Yes
Dependency-level budgets No Yes

Stormbreak is intentionally compatible with other resilience techniques. Its distinct role is coordinating retry pressure across callers.

Design philosophy

  • The first attempt is not a retry and never consumes budget.
  • Capacity is local and explicit; there is no global registry or mutable policy.
  • Full jitter spreads retry timing between zero and the exponential delay cap.
  • Context cancellation wins before execution and during backoff.
  • Standard-library primitives keep the dependency and allocation footprint small.
  • Invalid configuration fails at construction or execution instead of silently changing behavior.

The implementation invariants and HTTP request lifecycle are described in ARCHITECTURE.md.

Examples

Public API examples also run as Go example tests and appear in generated package documentation.

Limitations

The initial release coordinates retries only within one process. It is not a circuit breaker, rate limiter, scheduler, queue, or distributed budget. It cannot guarantee idempotency, prevent a remote operation from completing after a network timeout, or coordinate independent application replicas. Refill uses the process clock; Reset intentionally discards the current refill window.

Benchmarks

Benchmarks cover serial and parallel budget access, first-attempt success, one retry, and registry lookup. Results depend heavily on CPU, Go version, and race instrumentation; run them on the target environment instead of treating checked figures as a guarantee:

go test -run '^$' -bench . -benchmem ./...

No throughput or allocation target is part of the public compatibility promise.

Reference run on 2026-08-02 using Go 1.26.3 on Windows/amd64 and an AMD Ryzen 9 9950X (32 logical CPUs):

Benchmark ns/op B/op allocs/op
BenchmarkBudgetAllow-32 11.26 0 0
BenchmarkBudgetAllowParallel-32 39.18 0 0
BenchmarkDoSuccess-32 25.03 64 1
BenchmarkDoSingleRetry-32 123.4 96 4
BenchmarkRegistryGet-32 10.26 0 0

These are one local reference run, not a performance guarantee. Compiler, hardware, contention, hooks, classifiers, and operation behavior can materially change results.

Roadmap

Possible future additions include circuit-breaker interoperability, OpenTelemetry helpers, Prometheus adapters, Redis-backed distributed budgets, adaptive refill rates, server-provided retry budgets, gRPC interceptors, SQL and queue adapters, per-error retry costs, and dynamic policy updates. Distributed coordination will remain optional and outside the core package.

Contributing and release

See CONTRIBUTING.md, SUPPORT.md, SECURITY.md, and RELEASING.md. The normal release checks are:

go test -count=1 -shuffle=on ./...
go test -race -count=1 ./...
go vet ./...

After updating the changelog, a maintainer can create the first release with:

git tag v0.1.0
git push origin v0.1.0

License

Stormbreak is available under the MIT License.

Documentation

Overview

Package stormbreak coordinates retry pressure through shared, concurrency-safe token budgets. Operations run once without cost; every subsequent attempt must consume shared capacity before applying context-aware exponential backoff.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// ErrBudgetExhausted indicates that shared retry capacity was unavailable.
	ErrBudgetExhausted = errors.New("stormbreak: retry budget exhausted")
	// ErrAttemptsExhausted indicates that every policy attempt was used.
	ErrAttemptsExhausted = errors.New("stormbreak: retry attempts exhausted")
	// ErrInvalidPolicy indicates invalid retry policy configuration.
	ErrInvalidPolicy = errors.New("stormbreak: invalid retry policy")
	// ErrInvalidBudget indicates invalid retry budget configuration.
	ErrInvalidBudget = errors.New("stormbreak: invalid retry budget configuration")
)

Functions

func AlwaysRetry 

func AlwaysRetry(err error) bool

AlwaysRetry accepts every non-nil error.

func Backoff 

func Backoff(policy Policy, attempt int) time.Duration

Backoff computes the delay before retry number attempt. Attempt one returns BaseDelay.

func Do 

func Do[T any](ctx context.Context, budget Budget, policy Policy, operation func(context.Context) (T, error), opts ...Option) (T, error)

Do executes operation immediately and gates every subsequent attempt through budget.

Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"github.com/magnexis/stormbreak"
)

func main() {
	budget, _ := stormbreak.NewBudget(stormbreak.Config{Capacity: 1})
	policy := stormbreak.Policy{MaxAttempts: 2, Multiplier: 1}
	attempts := 0

	value, err := stormbreak.Do(context.Background(), budget, policy, func(context.Context) (int, error) {
		attempts++
		if attempts == 1 {
			return 0, errors.New("temporary failure")
		}
		return 42, nil
	})

	fmt.Println(value, err, attempts, budget.Remaining())
}

Output:
42 <nil> 2 0

func DoVoid 

func DoVoid(ctx context.Context, budget Budget, policy Policy, operation func(context.Context) error, options ...Option) error

DoVoid is the error-only form of Do.

func IsPermanent 

func IsPermanent(err error) bool

IsPermanent reports whether err or one of its wrapped errors was marked permanent.

func NeverRetry 

func NeverRetry(error) bool

NeverRetry rejects every error.

func Permanent 

func Permanent(err error) error

Permanent marks err as non-retryable. A nil error remains nil.

Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"github.com/magnexis/stormbreak"
)

func main() {
	budget, _ := stormbreak.NewBudget(stormbreak.Config{Capacity: 5})
	attempts := 0
	err := stormbreak.DoVoid(context.Background(), budget, stormbreak.DefaultPolicy(), func(context.Context) error {
		attempts++
		return stormbreak.Permanent(errors.New("invalid credentials"))
	})

	fmt.Println(stormbreak.IsPermanent(err), attempts, budget.Remaining())
}

Output:
true 1 5

Types

type AttemptEvent 

type AttemptEvent struct{ Attempt int }

AttemptEvent is emitted immediately before an operation attempt.

type Budget 

type Budget interface {
	// Allow consumes one retry token when capacity is available.
	Allow() bool
	// Remaining returns the currently available retry tokens.
	Remaining() int64
	// Capacity returns the maximum retry tokens.
	Capacity() int64
	// Reset restores the budget to full capacity.
	Reset()
}

Budget controls whether a retry may proceed.

type BudgetEvent 

type BudgetEvent struct {
	Attempts  int
	LastError error
	Remaining int64
}

BudgetEvent is emitted when a retry cannot proceed because the budget is empty.

type BudgetSnapshot 

type BudgetSnapshot struct {
	Capacity       int64
	Remaining      int64
	RefillRate     int64
	RefillInterval time.Duration
	Exhausted      bool
}

BudgetSnapshot is an immutable view of a TokenBudget.

type Classifier 

type Classifier func(error) bool

Classifier reports whether an operation error is retryable.

type Config 

type Config struct {
	// Capacity is the maximum and initial token count.
	Capacity int64
	// RefillRate is the number of tokens restored per refill interval.
	RefillRate int64
	// RefillInterval controls how frequently elapsed time restores tokens.
	RefillInterval time.Duration
}

Config configures a token-based retry budget.

type FailureEvent 

type FailureEvent struct {
	Attempt int
	Error   error
}

FailureEvent is emitted when an operation attempt fails.

type Hooks 

type Hooks struct {
	OnAttempt         func(AttemptEvent)
	OnRetry           func(RetryEvent)
	OnSuccess         func(SuccessEvent)
	OnFailure         func(FailureEvent)
	OnBudgetExhausted func(BudgetEvent)
}

Hooks contains optional synchronous observability callbacks. Hook functions should return quickly and must not block critical paths.

type Option 

type Option func(*options)

Option customizes a retry execution.

func WithClassifier 

func WithClassifier(classifier Classifier) Option

WithClassifier replaces the default retry classifier.

func WithHooks 

func WithHooks(hooks Hooks) Option

WithHooks installs synchronous, nil-safe observability hooks.

Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"github.com/magnexis/stormbreak"
)

func main() {
	budget, _ := stormbreak.NewBudget(stormbreak.Config{Capacity: 1})
	policy := stormbreak.Policy{MaxAttempts: 2, Multiplier: 1}
	attempts := 0
	hooks := stormbreak.Hooks{
		OnRetry: func(event stormbreak.RetryEvent) {
			fmt.Printf("retry attempt=%d remaining=%d\n", event.Attempt, event.BudgetRemaining)
		},
	}

	_ = stormbreak.DoVoid(context.Background(), budget, policy, func(context.Context) error {
		attempts++
		if attempts == 1 {
			return errors.New("temporary")
		}
		return nil
	}, stormbreak.WithHooks(hooks))
}

Output:
retry attempt=2 remaining=0

func WithRandomSource 

func WithRandomSource(source func() float64) Option

WithRandomSource supplies values in [0, 1) for deterministic jitter. It is primarily useful in tests; the function must be safe for its caller's use.

type Policy 

type Policy struct {
	// MaxAttempts includes the initial attempt.
	MaxAttempts int
	// BaseDelay is the delay cap before the first retry.
	BaseDelay time.Duration
	// MaxDelay caps exponential backoff.
	MaxDelay time.Duration
	// Multiplier controls exponential growth. Zero selects the default of two.
	Multiplier float64
	// Jitter randomizes each delay uniformly between zero and its calculated cap.
	Jitter bool
}

Policy configures retry attempts and exponential backoff.

func DefaultPolicy 

func DefaultPolicy() Policy

DefaultPolicy returns conservative defaults suitable for network operations.

func (Policy) Validate 

func (p Policy) Validate() error

Validate checks whether a policy can be executed safely.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry stores independently named retry budgets. It has no global instance.

Example 
package main

import (
	"fmt"

	"github.com/magnexis/stormbreak"
)

func main() {
	registry := stormbreak.NewRegistry()
	_, _ = registry.Create("database", stormbreak.Config{Capacity: 10})
	_, _ = registry.Create("github-api", stormbreak.Config{Capacity: 20})

	fmt.Println(registry.Names())
}

Output:
[database github-api]

func NewRegistry 

func NewRegistry() *Registry

NewRegistry creates an empty registry.

func (*Registry) Create 

func (r *Registry) Create(name string, config Config) (*TokenBudget, error)

Create validates and registers a new uniquely named budget.

func (*Registry) Delete 

func (r *Registry) Delete(name string) bool

Delete removes a named budget.

func (*Registry) Get 

func (r *Registry) Get(name string) (*TokenBudget, bool)

Get retrieves a budget by its exact name.

func (*Registry) Names 

func (r *Registry) Names() []string

Names returns sorted registered names.

func (*Registry) Snapshot 

func (r *Registry) Snapshot() map[string]BudgetSnapshot

Snapshot returns independent snapshots of all registered budgets.

type RetryError 

type RetryError struct {
	Attempts  int
	LastError error
	Cause     error
}

RetryError describes why a retry operation stopped.

func (*RetryError) Error 

func (e *RetryError) Error() string

Error returns a summary containing the attempt count, terminal cause, and last error.

func (*RetryError) Unwrap 

func (e *RetryError) Unwrap() error

Unwrap exposes both the terminal cause and last operation error to errors.Is/As.

type RetryEvent 

type RetryEvent struct {
	Attempt         int
	Delay           time.Duration
	Error           error
	BudgetRemaining int64
}

RetryEvent is emitted after a retry token is consumed and before backoff.

type SuccessEvent 

type SuccessEvent struct {
	Attempts int
	Elapsed  time.Duration
}

SuccessEvent is emitted after an operation succeeds.

type TokenBudget 

type TokenBudget struct {
	// contains filtered or unexported fields
}

TokenBudget is a concurrency-safe, lazily refilled retry budget.

func NewBudget 

func NewBudget(config Config) (*TokenBudget, error)

NewBudget validates config and creates a full retry budget.

func (*TokenBudget) Allow 

func (b *TokenBudget) Allow() bool

Allow consumes one retry token when capacity is available.

func (*TokenBudget) Capacity 

func (b *TokenBudget) Capacity() int64

Capacity returns the maximum token count.

func (*TokenBudget) Remaining 

func (b *TokenBudget) Remaining() int64

Remaining returns the current token count after applying lazy refill.

func (*TokenBudget) Reset 

func (b *TokenBudget) Reset()

Reset restores full capacity and restarts the refill window.

func (*TokenBudget) Snapshot 

func (b *TokenBudget) Snapshot() BudgetSnapshot

Snapshot returns an immutable view after applying lazy refill.

Source Files

View all Source files

Directories

Path Synopsis
examples
basic command
database-retry command
http-client command
shared-budget command
worker-pool command
Package httpretry provides an HTTP transport protected by a shared stormbreak budget.
 Package httpretry provides an HTTP transport protected by a shared stormbreak budget.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.