backoff

package module
v0.0.0-...-f54ae69 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 27, 2026 License: MIT Imports: 6 Imported by: 0

README

Exponential Backoff GoDoc

This is a Go port of the exponential backoff algorithm from Google's HTTP Client Library for Java.

Exponential backoff is an algorithm that uses feedback to multiplicatively decrease the rate of some process, in order to gradually find an acceptable rate. The retries exponentially increase and stop increasing when a certain threshold is met.

Install

go get github.com/andig/backoff

Usage

For most cases, wrap the operation you want to retry in Retry:

result, err := backoff.Retry(func() (string, error) {
	resp, err := http.Get("https://www.example.com")
	if err != nil {
		return "", err // transient: Retry will try again
	}
	defer resp.Body.Close()

	switch {
	case resp.StatusCode >= 500:
		return "", fmt.Errorf("server error: %s", resp.Status) // retried
	case resp.StatusCode >= 400:
		// client errors won't fix themselves, so stop retrying.
		return "", backoff.Permanent(fmt.Errorf("client error: %s", resp.Status))
	}
	return "ok", nil
}, backoff.WithMaxTries(5))

Retry runs the operation at least once and keeps retrying with exponential backoff until it succeeds, returns a Permanent error, or a limit is reached. See example_test.go for a fuller example, and the package docs for the available options (WithBackOff, WithMaxTries, WithMaxElapsedTime, WithNotify).

When the operation has no result to return, use RetryError, which takes a func() error and returns only an error:

err := backoff.RetryError(func() error {
	return db.Ping()
}, backoff.WithMaxTries(5))

To bound retrying with a context.Context, use the Ctx variants — RetryCtx and RetryErrorCtx. Cancelling the context stops further attempts and interrupts the wait between them:

result, err := backoff.RetryCtx(ctx, operation)

Retry and RetryError are those functions with context.Background().

If Retry does not fit your needs, copy it from retry.go and adapt it.

Handling errors

On failure, Retry always returns an *Error. It carries the last operation error (LastErr) and the reason retrying stopped (Cause). Inspect it with errors.Is, or reach the struct with errors.As:

result, err := backoff.RetryCtx(ctx, operation)
switch {
case errors.Is(err, backoff.ErrPermanent):
	// the operation returned a Permanent error
case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
	// the caller's context was cancelled or its deadline expired
case errors.Is(err, backoff.ErrMaxElapsedTime):
	// the WithMaxElapsedTime budget was exhausted
case errors.Is(err, backoff.ErrExhausted):
	// WithMaxTries was reached or the backoff policy returned Stop
}

// The last operation error is always available, whatever the cause:
var re *backoff.Error
if errors.As(err, &re) {
	log.Printf("gave up after last error: %v", re.LastErr)
}

Printing the error prints your operation's own message, undecorated: the cause is not prefixed onto it. Match the cause with errors.Is or read the Cause field instead.

Mark an error non-retriable with backoff.Permanent(err); Retry stops immediately and returns an *Error whose Cause is ErrPermanent and whose LastErr is err.

Bounding total time

Two independent limits cap how long Retry runs, and they behave differently:

  • A context deadline (context.WithTimeout) is reactive: it interrupts the wait between attempts and — if your operation observes the context — can abort an in-flight attempt. Retry reports it as context.DeadlineExceeded.
  • WithMaxElapsedTime bounds only retry scheduling: it is checked between attempts, never interrupts a running operation, and is reported as ErrMaxElapsedTime.

WithMaxElapsedTime defaults to 15 minutes, so both limits are active unless you override it — pass backoff.WithMaxElapsedTime(0) to rely solely on the context.

Contributing

  • I would like to keep this library as small as possible.
  • Please don't send a PR without opening an issue and discussing it first.
  • If proposed change is not a common use case, I will probably not accept it.

Documentation

Overview

Package backoff implements backoff algorithms for retrying operations.

Use Retry function for retrying operations that may fail. Use RetryError for operations that return no result, and the RetryCtx and RetryErrorCtx variants to bound retrying with a context. If Retry does not meet your needs, copy/paste the function into your project and modify as you wish.

On failure Retry returns an *Error reporting the last operation error and why it stopped; see Error and the ErrPermanent, ErrExhausted, and ErrMaxElapsedTime causes.

There is also Ticker type similar to time.Ticker. You can use it if you need to work with channels.

See Examples section below for usage examples.

Index

Examples

Constants

View Source 
const (
	DefaultInitialInterval     = 500 * time.Millisecond
	DefaultRandomizationFactor = 0.5
	DefaultMultiplier          = 1.5
	DefaultMaxInterval         = 60 * time.Second
)

Default values for ExponentialBackOff.

View Source 
const DefaultMaxElapsedTime = 15 * time.Minute

DefaultMaxElapsedTime sets a default limit for the total retry duration.

View Source 
const Stop time.Duration = -1

Stop indicates that no more retries should be made for use in NextBackOff().

Variables

View Source 
var (
	// ErrPermanent is the cause when the operation returned a Permanent error.
	ErrPermanent = errors.New("backoff: permanent error")

	// ErrExhausted is the cause when retrying stops because WithMaxTries was
	// reached or the backoff policy returned Stop.
	ErrExhausted = errors.New("backoff: retries exhausted")

	// ErrMaxElapsedTime is the cause when retrying stops because
	// WithMaxElapsedTime was reached.
	ErrMaxElapsedTime = errors.New("backoff: maximum elapsed time exceeded")
)

Cause values reported by Error.Cause. Match them with errors.Is.

Functions

func Permanent 

func Permanent(err error) error

Permanent wraps err to signal that Retry should stop immediately instead of retrying. Retry then returns an *Error with Cause ErrPermanent and LastErr set to err. Permanent(nil) returns nil.

func Retry 

func Retry[T any](operation Operation[T], opts ...RetryOption) (T, error)

Retry is RetryCtx with context.Background(): retrying is bounded only by WithMaxElapsedTime (15 minutes by default), WithMaxTries, and the backoff policy. Use RetryCtx to bound it with a context as well.

Example 
package main

import (
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"
	"strconv"
	"time"

	"github.com/andig/backoff"
)

func main() {
	// A stand-in for the remote service. In real code this is the endpoint you
	// are calling; here it is an in-process test server so the example is
	// self-contained and does not depend on the network.
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
	}))
	defer server.Close()

	// Define an operation function that returns a value and an error.
	// The value can be any type.
	// We'll pass this operation to Retry function.
	operation := func() (string, error) {
		// An example request that may fail.
		resp, err := http.Get(server.URL)
		if err != nil {
			return "", err
		}
		defer resp.Body.Close()

		// If we are being rate limited, return a RetryAfter to specify how long to wait.
		// This will also reset the backoff policy.
		if resp.StatusCode == http.StatusTooManyRequests {
			seconds, err := strconv.ParseInt(resp.Header.Get("Retry-After"), 10, 64)
			if err == nil {
				return "", backoff.RetryAfter(time.Duration(seconds)*time.Second, fmt.Errorf("rate limited: %s", resp.Status))
			}
		}

		// In case of non-retriable error, return Permanent error to stop retrying.
		// For this HTTP example, client errors are non-retriable.
		if resp.StatusCode >= 400 && resp.StatusCode < 500 {
			return "", backoff.Permanent(errors.New("bad request"))
		}

		// Return successful response.
		return "hello", nil
	}

	result, err := backoff.Retry(operation, backoff.WithBackOff(backoff.NewExponentialBackOff()))
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	// Operation is successful after retries.

	fmt.Println(result)
}

Output:
hello
Example (Outcomes) 
package main

import (
	"context"
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"
	"time"

	"github.com/andig/backoff"
)

func main() {
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
	}))
	defer server.Close()

	operation := func() (string, error) {
		resp, err := http.Get(server.URL)
		if err != nil {
			return "", err
		}
		defer resp.Body.Close()
		return "ok", nil
	}

	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	result, err := backoff.RetryCtx(ctx, operation,
		backoff.WithMaxElapsedTime(10*time.Second),
		backoff.WithMaxTries(5),
	)

	switch {
	case err == nil:
		// Operation succeeded.
		fmt.Println(result)

	case errors.Is(err, backoff.ErrPermanent):
		// The operation returned a Permanent (non-retriable) error.
		fmt.Println("permanent:", err)

	case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
		// The caller's context was cancelled or its deadline expired.
		fmt.Println("context done:", err)

	case errors.Is(err, backoff.ErrMaxElapsedTime):
		// The WithMaxElapsedTime budget was exhausted.
		fmt.Println("timed out:", err)

	case errors.Is(err, backoff.ErrExhausted):
		// WithMaxTries was reached or the backoff policy stopped.
		fmt.Println("retries exhausted:", err)
	}

	// The last operation error is always available, whatever the cause:
	var re *backoff.Error
	if errors.As(err, &re) {
		fmt.Println("last error:", re.LastErr)
	}
}

Output:
ok

func RetryAfter 

func RetryAfter(d time.Duration, cause error) error

RetryAfter returns a RetryAfterError that tells Retry to wait the given duration before the next attempt. cause is the error that triggered the wait; it is preserved as Error.LastErr if retrying stops. Pass a non-nil cause so the failure reason is not lost; nil is allowed but discouraged.

func RetryCtx 

func RetryCtx[T any](ctx context.Context, operation Operation[T], opts ...RetryOption) (T, error)

RetryCtx attempts the operation until it succeeds, returns a Permanent error, or backoff completes. It ensures the operation is executed at least once.

On success it returns the operation result and a nil error. On any failure it returns the last result and an *Error whose Cause reports why it stopped — ErrPermanent, ErrExhausted, ErrMaxElapsedTime, or the context cancellation cause — and whose LastErr holds the last operation error. See Error.

ctx bounds the retry loop: its cancellation or deadline stops further attempts and interrupts the wait between them. The operation receives no context, so capture ctx inside the operation if you want cancellation to abort an in-flight attempt. To bound only how long backoff keeps retrying, without affecting in-flight attempts, use WithMaxElapsedTime instead.

func RetryError 

func RetryError(operation func() error, opts ...RetryOption) error

RetryError is RetryErrorCtx with context.Background(), as Retry is to RetryCtx.

func RetryErrorCtx 

func RetryErrorCtx(ctx context.Context, operation func() error, opts ...RetryOption) error

RetryErrorCtx is RetryCtx for an operation that returns no result, only an error. It behaves identically to RetryCtx in every other respect, including the options it accepts and the *Error it returns on failure.

Types

type BackOff 

type BackOff interface {
	// NextBackOff returns the duration to wait before retrying the operation,
	// backoff.Stop to indicate that no more retries should be made.
	//
	// Example usage:
	//
	//     duration := backoff.NextBackOff()
	//     if duration == backoff.Stop {
	//         // Do not retry operation.
	//     } else {
	//         // Sleep for duration and retry operation.
	//     }
	//
	NextBackOff() time.Duration

	// Reset to initial state.
	Reset()
}

BackOff is a backoff policy for retrying an operation.

type ConstantBackOff 

type ConstantBackOff struct {
	Interval time.Duration
}

ConstantBackOff is a backoff policy that always returns the same backoff delay. This is in contrast to an exponential backoff policy, which returns a delay that grows longer as you call NextBackOff() over and over again.

func NewConstantBackOff 

func NewConstantBackOff(d time.Duration) *ConstantBackOff

func (*ConstantBackOff) NextBackOff 

func (b *ConstantBackOff) NextBackOff() time.Duration

func (*ConstantBackOff) Reset 

func (b *ConstantBackOff) Reset()

type Error 

type Error struct {
	// LastErr is the error returned by the final operation attempt. For a
	// permanent failure it is the error passed to Permanent.
	LastErr error
	// Cause reports why retrying stopped: ErrPermanent, ErrExhausted,
	// ErrMaxElapsedTime, or a context cancellation cause (see context.Cause).
	Cause error
}

Error is the error returned by Retry for every failure. It records the last error returned by the operation (LastErr) together with the reason retrying stopped (Cause), so callers never lose either piece of information.

Inspect it with errors.Is and errors.As: 

result, err := backoff.RetryCtx(ctx, op)
switch {
case errors.Is(err, backoff.ErrPermanent):
	// operation returned a Permanent error
case errors.Is(err, context.Canceled):
	// caller cancelled ctx
case errors.Is(err, backoff.ErrMaxElapsedTime):
	// ran out of the WithMaxElapsedTime budget
case errors.Is(err, backoff.ErrExhausted):
	// hit WithMaxTries or the backoff policy stopped
}

var re *backoff.Error
if errors.As(err, &re) {
	log.Printf("gave up after last error: %v", re.LastErr)
}

Because Error implements Unwrap() []error, errors.Unwrap (the single error form) returns nil for it; use errors.Is or errors.As.

func (*Error) Error 

func (e *Error) Error() string

Error returns the last operation error's message, undecorated by the cause, falling back to the cause when there is no operation error. Read Cause, or match it with errors.Is, to learn why retrying stopped.

func (*Error) Unwrap 

func (e *Error) Unwrap() []error

Unwrap returns the cause and the last operation error so both can be matched with errors.Is and errors.As.

type ExponentialBackOff 

type ExponentialBackOff struct {
	InitialInterval     time.Duration
	RandomizationFactor float64
	Multiplier          float64
	MaxInterval         time.Duration
	// contains filtered or unexported fields
}

ExponentialBackOff is a backoff implementation that increases the backoff period for each retry attempt using a randomization function that grows exponentially.

NextBackOff() is calculated using the following formula: 

randomized interval =
    RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])

In other words NextBackOff() will range between the randomization factor percentage below and above the retry interval.

For example, given the following parameters: 

RetryInterval = 2
RandomizationFactor = 0.5
Multiplier = 2

the actual backoff period used in the next retry attempt will range between 1 and 3 seconds, multiplied by the exponential, that is, between 2 and 6 seconds.

Note: MaxInterval caps the RetryInterval and not the randomized interval.

Example: Given the following default arguments, for 9 tries the sequence will be: 

Request #  RetryInterval (seconds)  Randomized Interval (seconds)

 1          0.5                     [0.25,   0.75]
 2          0.75                    [0.375,  1.125]
 3          1.125                   [0.562,  1.687]
 4          1.687                   [0.8435, 2.53]
 5          2.53                    [1.265,  3.795]
 6          3.795                   [1.897,  5.692]
 7          5.692                   [2.846,  8.538]
 8          8.538                   [4.269, 12.807]
 9         12.807                   [6.403, 19.210]

Note: Implementation is not thread-safe.

func NewExponentialBackOff 

func NewExponentialBackOff(opts ...ExponentialBackOffOption) *ExponentialBackOff

NewExponentialBackOff creates an instance of ExponentialBackOff using default values, changed by opts.

func (*ExponentialBackOff) NextBackOff 

func (b *ExponentialBackOff) NextBackOff() time.Duration

NextBackOff calculates the next backoff interval using the formula: 

Randomized interval = RetryInterval * (1 ± RandomizationFactor)

func (*ExponentialBackOff) Reset 

func (b *ExponentialBackOff) Reset()

Reset the interval back to the initial retry interval and restarts the timer. Reset must be called before using b.

type ExponentialBackOffOption 

type ExponentialBackOffOption func(*ExponentialBackOff)

ExponentialBackOffOption changes a setting of an ExponentialBackOff created by NewExponentialBackOff. Settings that are not given keep their default.

func WithInitialInterval 

func WithInitialInterval(d time.Duration) ExponentialBackOffOption

WithInitialInterval sets the interval of the first retry.

func WithMaxInterval 

func WithMaxInterval(d time.Duration) ExponentialBackOffOption

WithMaxInterval caps the interval between retries.

func WithMultiplier 

func WithMultiplier(f float64) ExponentialBackOffOption

WithMultiplier sets the factor the interval grows by on every retry.

func WithRandomizationFactor 

func WithRandomizationFactor(f float64) ExponentialBackOffOption

WithRandomizationFactor sets how far an interval is randomized around its nominal value, e.g. 0.5 for ±50%.

type Notify 

type Notify func(error, time.Duration)

Notify is called after a failed attempt that will be retried, with the operation error and the backoff duration before the next attempt. It is called once per retry, not for the final error that stops Retry (a permanent error, an exhausted limit, or a cancelled context).

type Operation 

type Operation[T any] func() (T, error)

Operation is the function Retry calls. It is invoked at least once and may be retried on error. Return a Permanent error to stop retrying immediately, or a RetryAfterError to control the delay before the next attempt.

type RetryAfterError 

type RetryAfterError struct {
	Duration time.Duration
	// contains filtered or unexported fields
}

RetryAfterError signals that the operation should be retried after the given duration. When an operation returns one (directly or wrapped), Retry waits that duration before the next attempt and resets the backoff policy, so the backoff sequence restarts afterward.

The error that triggered the wait (passed to RetryAfter) is available via Unwrap, so errors.Is and errors.As see through the RetryAfterError. If retrying later stops because a limit is reached or the context ends, Retry reports that error as Error.LastErr instead of the RetryAfterError itself, so the underlying cause is not lost.

func (*RetryAfterError) Error 

func (e *RetryAfterError) Error() string

Error returns a string representation of the RetryAfter error.

func (*RetryAfterError) Unwrap 

func (e *RetryAfterError) Unwrap() error

Unwrap returns the error that triggered the retry, if one was provided.

type RetryOption 

type RetryOption func(*retryOptions)

RetryOption configures the behavior of Retry.

func WithBackOff 

func WithBackOff(b BackOff) RetryOption

WithBackOff configures the backoff policy used between attempts. The default is NewExponentialBackOff.

Retry calls Reset on the policy before the first attempt, so a previously used policy may be passed. A BackOff is stateful and not safe for concurrent use: give each concurrent Retry call its own BackOff rather than sharing one.

func WithMaxElapsedTime 

func WithMaxElapsedTime(d time.Duration) RetryOption

WithMaxElapsedTime limits the total wall-clock time spent retrying, measured from when Retry is called. When the limit is reached, Retry returns a *Error with Cause ErrMaxElapsedTime.

The limit is checked only between attempts: it gates whether another attempt is scheduled. It does not interrupt an operation that is already running, nor a backoff wait already in progress, and Retry stops early rather than starting a backoff that would overrun the limit.

This differs from bounding Retry with a context deadline (e.g. context.WithTimeout): a context deadline is reactive — it interrupts the backoff wait and, if the operation observes the context, can abort an in-flight attempt — and Retry reports it with Cause context.DeadlineExceeded.

The default is DefaultMaxElapsedTime (15 minutes), so both limits are active at once unless overridden. Pass 0 to disable the elapsed-time limit and rely on the context (or WithMaxTries) instead.

func WithMaxTries 

func WithMaxTries(n uint) RetryOption

WithMaxTries limits the total number of attempts, not retries: WithMaxTries(1) runs the operation once and does not retry. When the limit is reached, Retry returns an *Error with Cause ErrExhausted. The default, 0, means no limit.

func WithNotify 

func WithNotify(n Notify) RetryOption

WithNotify sets a function called after each failed attempt that will be retried. See Notify for exactly when it fires.

type StopBackOff 

type StopBackOff struct{}

StopBackOff is a fixed backoff policy that always returns backoff.Stop for NextBackOff(), meaning that the operation should never be retried.

func (*StopBackOff) NextBackOff 

func (b *StopBackOff) NextBackOff() time.Duration

func (*StopBackOff) Reset 

func (b *StopBackOff) Reset()

type Ticker 

type Ticker struct {
	C <-chan time.Time
	// contains filtered or unexported fields
}

Ticker holds a channel that delivers `ticks' of a clock at times reported by a BackOff.

Ticks will continue to arrive when the previous operation is still running, so operations that take a while to fail could run in quick succession.

Example 
package main

import (
	"fmt"
	"log"

	"github.com/andig/backoff"
)

func main() {
	// An operation that may fail.
	operation := func() (string, error) {
		return "hello", nil
	}

	ticker := backoff.NewTicker(backoff.NewExponentialBackOff())
	defer ticker.Stop()

	var result string
	var err error

	// Ticks will continue to arrive when the previous operation is still running,
	// so operations that take a while to fail could run in quick succession.
	for range ticker.C {
		if result, err = operation(); err != nil {
			log.Println(err, "will retry...")
			continue
		}

		break
	}

	if err != nil {
		// Operation has failed.
		fmt.Println("Error:", err)
		return
	}

	// Operation is successful after retries.

	fmt.Println(result)
}

Output:
hello

func NewTicker 

func NewTicker(b BackOff) *Ticker

NewTicker returns a new Ticker containing a channel that will send the time at times specified by the BackOff argument. Ticker is guaranteed to tick at least once. The channel is closed when Stop method is called or BackOff stops. It is not safe to manipulate the provided backoff policy (notably calling NextBackOff or Reset) while the ticker is running.

func (*Ticker) Stop 

func (t *Ticker) Stop()

Stop turns off a ticker. After Stop, no more ticks will be sent.

type ZeroBackOff 

type ZeroBackOff struct{}

ZeroBackOff is a fixed backoff policy whose backoff time is always zero, meaning that the operation is retried immediately without waiting, indefinitely.

func (*ZeroBackOff) NextBackOff 

func (b *ZeroBackOff) NextBackOff() time.Duration

func (*ZeroBackOff) Reset 

func (b *ZeroBackOff) Reset()

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.