backoff

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 8, 2026 License: MIT Imports: 6 Imported by: 0

README

Exponential Backoff GoDoc

This is a Go port of the exponential backoff algorithm from Google's HTTP Client Library for Java.

Exponential backoff is an algorithm that uses feedback to multiplicatively decrease the rate of some process, in order to gradually find an acceptable rate. The retries exponentially increase and stop increasing when a certain threshold is met.

Install

go get github.com/aliykh/backoff

Usage

For most cases, wrap the operation you want to retry in Retry:

result, err := backoff.Retry(ctx, func() (string, error) {
	resp, err := http.Get("https://www.example.com")
	if err != nil {
		return "", err // transient: Retry will try again
	}
	defer resp.Body.Close()

	switch {
	case resp.StatusCode >= 500:
		return "", fmt.Errorf("server error: %s", resp.Status) // retried
	case resp.StatusCode >= 400:
		// client errors won't fix themselves, so stop retrying.
		return "", backoff.Permanent(fmt.Errorf("client error: %s", resp.Status))
	}
	return "ok", nil
}, backoff.WithMaxTries(5))

Retry runs the operation at least once and keeps retrying with exponential backoff until it succeeds, returns a Permanent error, or a limit is reached. See example_test.go for a fuller example, and the package docs for the available options (WithBackOff, WithMaxTries, WithMaxElapsedTime, WithNotify).

If Retry does not fit your needs, copy it from retry.go and adapt it.

Handling errors

On failure, Retry always returns a *RetryError. It carries the last operation error (LastErr) and the reason retrying stopped (Cause). Inspect it with errors.Is, or reach the struct with AsRetryError:

result, err := backoff.Retry(ctx, operation)
switch {
case errors.Is(err, backoff.ErrPermanent):
	// the operation returned a Permanent error
case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
	// the caller's context was cancelled or its deadline expired
case errors.Is(err, backoff.ErrMaxElapsedTime):
	// the WithMaxElapsedTime budget was exhausted
case errors.Is(err, backoff.ErrExhausted):
	// WithMaxTries was reached or the backoff policy returned Stop
}

// The last operation error is always available, whatever the cause:
if re := backoff.AsRetryError(err); re != nil {
	log.Printf("gave up after last error: %v", re.LastErr)
}

Mark an error non-retriable with backoff.Permanent(err); Retry stops immediately and returns a *RetryError whose Cause is ErrPermanent and whose LastErr is err.

Bounding total time

Two independent limits cap how long Retry runs, and they behave differently:

  • A context deadline (context.WithTimeout) is reactive: it interrupts the wait between attempts and — if your operation observes the context — can abort an in-flight attempt. Retry reports it as context.DeadlineExceeded.
  • WithMaxElapsedTime bounds only retry scheduling: it is checked between attempts, never interrupts a running operation, and is reported as ErrMaxElapsedTime.

WithMaxElapsedTime defaults to 15 minutes, so both limits are active unless you override it — pass backoff.WithMaxElapsedTime(0) to rely solely on the context.

Contributing

  • I would like to keep this library as small as possible.
  • Please don't send a PR without opening an issue and discussing it first.
  • If proposed change is not a common use case, I will probably not accept it.

Documentation

Overview

Package backoff implements backoff algorithms for retrying operations.

Use Retry function for retrying operations that may fail. If Retry does not meet your needs, copy/paste the function into your project and modify as you wish.

On failure Retry returns a *RetryError reporting the last operation error and why it stopped; see RetryError, AsRetryError, and the ErrPermanent, ErrExhausted, and ErrMaxElapsedTime causes.

There is also Ticker type similar to time.Ticker. You can use it if you need to work with channels.

See Examples section below for usage examples.

Index

Examples

Constants

View Source 
const (
	DefaultInitialInterval     = 500 * time.Millisecond
	DefaultRandomizationFactor = 0.5
	DefaultMultiplier          = 1.5
	DefaultMaxInterval         = 60 * time.Second
)

Default values for ExponentialBackOff.

View Source 
const DefaultMaxElapsedTime = 15 * time.Minute

DefaultMaxElapsedTime sets a default limit for the total retry duration.

View Source 
const Stop time.Duration = -1

Stop indicates that no more retries should be made for use in NextBackOff().

Variables

View Source 
var (
	// ErrPermanent is the cause when the operation returned a Permanent error.
	ErrPermanent = errors.New("backoff: permanent error")

	// ErrExhausted is the cause when retrying stops because WithMaxTries was
	// reached or the backoff policy returned Stop.
	ErrExhausted = errors.New("backoff: retries exhausted")

	// ErrMaxElapsedTime is the cause when retrying stops because
	// WithMaxElapsedTime was reached.
	ErrMaxElapsedTime = errors.New("backoff: maximum elapsed time exceeded")
)

Cause values reported by RetryError.Cause. Match them with errors.Is.

Functions

func Permanent 

func Permanent(err error) error

Permanent wraps err to signal that Retry should stop immediately instead of retrying. Retry then returns a *RetryError with Cause ErrPermanent and LastErr set to err. Permanent(nil) returns nil.

func Retry 

func Retry[T any](ctx context.Context, operation Operation[T], opts ...RetryOption) (T, error)

Retry attempts the operation until it succeeds, returns a Permanent error, or backoff completes. It ensures the operation is executed at least once.

On success it returns the operation result and a nil error. On any failure it returns the last result and a *RetryError whose Cause reports why it stopped — ErrPermanent, ErrExhausted, ErrMaxElapsedTime, or the context cancellation cause — and whose LastErr holds the last operation error. See RetryError and AsRetryError.

ctx bounds the retry loop: its cancellation or deadline stops further attempts and interrupts the wait between them. The operation receives no context, so capture ctx inside the operation if you want cancellation to abort an in-flight attempt. To bound only how long backoff keeps retrying, without affecting in-flight attempts, use WithMaxElapsedTime instead.

Example 
package main

import (
	"context"
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"
	"strconv"
	"time"

	"github.com/aliykh/backoff"
)

func main() {
	// A stand-in for the remote service. In real code this is the endpoint you
	// are calling; here it is an in-process test server so the example is
	// self-contained and does not depend on the network.
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
	}))
	defer server.Close()

	// Define an operation function that returns a value and an error.
	// The value can be any type.
	// We'll pass this operation to Retry function.
	operation := func() (string, error) {
		// An example request that may fail.
		resp, err := http.Get(server.URL)
		if err != nil {
			return "", err
		}
		defer resp.Body.Close()

		// If we are being rate limited, return a RetryAfter to specify how long to wait.
		// This will also reset the backoff policy.
		if resp.StatusCode == http.StatusTooManyRequests {
			seconds, err := strconv.ParseInt(resp.Header.Get("Retry-After"), 10, 64)
			if err == nil {
				return "", backoff.RetryAfter(time.Duration(seconds)*time.Second, fmt.Errorf("rate limited: %s", resp.Status))
			}
		}

		// In case of non-retriable error, return Permanent error to stop retrying.
		// For this HTTP example, client errors are non-retriable.
		if resp.StatusCode >= 400 && resp.StatusCode < 500 {
			return "", backoff.Permanent(errors.New("bad request"))
		}

		// Return successful response.
		return "hello", nil
	}

	result, err := backoff.Retry(context.TODO(), operation, backoff.WithBackOff(backoff.NewExponentialBackOff()))
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	// Operation is successful after retries.

	fmt.Println(result)
}

Output:
hello
Example (Outcomes) 
package main

import (
	"context"
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"
	"time"

	"github.com/aliykh/backoff"
)

func main() {
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
	}))
	defer server.Close()

	operation := func() (string, error) {
		resp, err := http.Get(server.URL)
		if err != nil {
			return "", err
		}
		defer resp.Body.Close()
		return "ok", nil
	}

	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	result, err := backoff.Retry(ctx, operation,
		backoff.WithMaxElapsedTime(10*time.Second),
		backoff.WithMaxTries(5),
	)

	switch {
	case err == nil:
		// Operation succeeded.
		fmt.Println(result)

	case errors.Is(err, backoff.ErrPermanent):
		// The operation returned a Permanent (non-retriable) error.
		fmt.Println("permanent:", err)

	case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
		// The caller's context was cancelled or its deadline expired.
		fmt.Println("context done:", err)

	case errors.Is(err, backoff.ErrMaxElapsedTime):
		// The WithMaxElapsedTime budget was exhausted.
		fmt.Println("timed out:", err)

	case errors.Is(err, backoff.ErrExhausted):
		// WithMaxTries was reached or the backoff policy stopped.
		fmt.Println("retries exhausted:", err)
	}

	// The last operation error is always available, whatever the cause:
	if re := backoff.AsRetryError(err); re != nil {
		fmt.Println("last error:", re.LastErr)
	}
}

Output:
ok

func RetryAfter 

func RetryAfter(d time.Duration, cause error) error

RetryAfter returns a RetryAfterError that tells Retry to wait the given duration before the next attempt. cause is the error that triggered the wait; it is preserved as RetryError.LastErr if retrying stops. Pass a non-nil cause so the failure reason is not lost; nil is allowed but discouraged.

Types

type BackOff 

type BackOff interface {
	// NextBackOff returns the duration to wait before retrying the operation,
	// backoff.Stop to indicate that no more retries should be made.
	//
	// Example usage:
	//
	//     duration := backoff.NextBackOff()
	//     if duration == backoff.Stop {
	//         // Do not retry operation.
	//     } else {
	//         // Sleep for duration and retry operation.
	//     }
	//
	NextBackOff() time.Duration

	// Reset to initial state.
	Reset()
}

BackOff is a backoff policy for retrying an operation.

type ConstantBackOff 

type ConstantBackOff struct {
	Interval time.Duration
}

ConstantBackOff is a backoff policy that always returns the same backoff delay. This is in contrast to an exponential backoff policy, which returns a delay that grows longer as you call NextBackOff() over and over again.

func NewConstantBackOff 

func NewConstantBackOff(d time.Duration) *ConstantBackOff

func (*ConstantBackOff) NextBackOff 

func (b *ConstantBackOff) NextBackOff() time.Duration

func (*ConstantBackOff) Reset 

func (b *ConstantBackOff) Reset()

type ExponentialBackOff 

type ExponentialBackOff struct {
	InitialInterval     time.Duration
	RandomizationFactor float64
	Multiplier          float64
	MaxInterval         time.Duration
	// contains filtered or unexported fields
}

ExponentialBackOff is a backoff implementation that increases the backoff period for each retry attempt using a randomization function that grows exponentially.

NextBackOff() is calculated using the following formula: 

randomized interval =
    RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])

In other words NextBackOff() will range between the randomization factor percentage below and above the retry interval.

For example, given the following parameters: 

RetryInterval = 2
RandomizationFactor = 0.5
Multiplier = 2

the actual backoff period used in the next retry attempt will range between 1 and 3 seconds, multiplied by the exponential, that is, between 2 and 6 seconds.

Note: MaxInterval caps the RetryInterval and not the randomized interval.

Example: Given the following default arguments, for 9 tries the sequence will be: 

Request #  RetryInterval (seconds)  Randomized Interval (seconds)

 1          0.5                     [0.25,   0.75]
 2          0.75                    [0.375,  1.125]
 3          1.125                   [0.562,  1.687]
 4          1.687                   [0.8435, 2.53]
 5          2.53                    [1.265,  3.795]
 6          3.795                   [1.897,  5.692]
 7          5.692                   [2.846,  8.538]
 8          8.538                   [4.269, 12.807]
 9         12.807                   [6.403, 19.210]

Note: Implementation is not thread-safe.

func NewExponentialBackOff 

func NewExponentialBackOff() *ExponentialBackOff

NewExponentialBackOff creates an instance of ExponentialBackOff using default values.

func (*ExponentialBackOff) NextBackOff 

func (b *ExponentialBackOff) NextBackOff() time.Duration

NextBackOff calculates the next backoff interval using the formula: 

Randomized interval = RetryInterval * (1 ± RandomizationFactor)

func (*ExponentialBackOff) Reset 

func (b *ExponentialBackOff) Reset()

Reset the interval back to the initial retry interval and restarts the timer. Reset must be called before using b.

type Notify 

type Notify func(error, time.Duration)

Notify is called after a failed attempt that will be retried, with the operation error and the backoff duration before the next attempt. It is called once per retry, not for the final error that stops Retry (a permanent error, an exhausted limit, or a cancelled context).

type Operation 

type Operation[T any] func() (T, error)

Operation is the function Retry calls. It is invoked at least once and may be retried on error. Return a Permanent error to stop retrying immediately, or a RetryAfterError to control the delay before the next attempt.

type RetryAfterError 

type RetryAfterError struct {
	Duration time.Duration
	// contains filtered or unexported fields
}

RetryAfterError signals that the operation should be retried after the given duration. When an operation returns one (directly or wrapped), Retry waits that duration before the next attempt and resets the backoff policy, so the backoff sequence restarts afterward.

The error that triggered the wait (passed to RetryAfter) is available via Unwrap, so errors.Is and errors.As see through the RetryAfterError. If retrying later stops because a limit is reached or the context ends, Retry reports that error as RetryError.LastErr instead of the RetryAfterError itself, so the underlying cause is not lost.

func (*RetryAfterError) Error 

func (e *RetryAfterError) Error() string

Error returns a string representation of the RetryAfter error.

func (*RetryAfterError) Unwrap 

func (e *RetryAfterError) Unwrap() error

Unwrap returns the error that triggered the retry, if one was provided.

type RetryError 

type RetryError struct {
	// LastErr is the error returned by the final operation attempt. For a
	// permanent failure it is the error passed to Permanent.
	LastErr error
	// Cause reports why retrying stopped: ErrPermanent, ErrExhausted,
	// ErrMaxElapsedTime, or a context cancellation cause (see context.Cause).
	Cause error
}

RetryError is the error returned by Retry for every failure. It records the last error returned by the operation (LastErr) together with the reason retrying stopped (Cause), so callers never lose either piece of information.

Inspect it with errors.Is, errors.As, or AsRetryError: 

result, err := backoff.Retry(ctx, op)
switch {
case errors.Is(err, backoff.ErrPermanent):
	// operation returned a Permanent error
case errors.Is(err, context.Canceled):
	// caller cancelled ctx
case errors.Is(err, backoff.ErrMaxElapsedTime):
	// ran out of the WithMaxElapsedTime budget
case errors.Is(err, backoff.ErrExhausted):
	// hit WithMaxTries or the backoff policy stopped
}

if re := backoff.AsRetryError(err); re != nil {
	log.Printf("gave up after last error: %v", re.LastErr)
}

Because RetryError implements Unwrap() []error, errors.Unwrap (the single error form) returns nil for it; use errors.Is, errors.As, or AsRetryError.

func AsRetryError 

func AsRetryError(err error) *RetryError

AsRetryError returns the *RetryError in err's chain, or nil if there is none (including when err is nil). It is a convenience wrapper around errors.As.

func (*RetryError) Error 

func (e *RetryError) Error() string

Error returns a single-line representation of the cause and last error.

func (*RetryError) Unwrap 

func (e *RetryError) Unwrap() []error

Unwrap returns the cause and the last operation error so both can be matched with errors.Is and errors.As.

type RetryOption 

type RetryOption func(*retryOptions)

RetryOption configures the behavior of Retry.

func WithBackOff 

func WithBackOff(b BackOff) RetryOption

WithBackOff configures the backoff policy used between attempts. The default is NewExponentialBackOff.

Retry calls Reset on the policy before the first attempt, so a previously used policy may be passed. A BackOff is stateful and not safe for concurrent use: give each concurrent Retry call its own BackOff rather than sharing one.

func WithMaxElapsedTime 

func WithMaxElapsedTime(d time.Duration) RetryOption

WithMaxElapsedTime limits the total wall-clock time spent retrying, measured from when Retry is called. When the limit is reached, Retry returns a *RetryError with Cause ErrMaxElapsedTime.

The limit is checked only between attempts: it gates whether another attempt is scheduled. It does not interrupt an operation that is already running, nor a backoff wait already in progress, and Retry stops early rather than starting a backoff that would overrun the limit.

This differs from bounding Retry with a context deadline (e.g. context.WithTimeout): a context deadline is reactive — it interrupts the backoff wait and, if the operation observes the context, can abort an in-flight attempt — and Retry reports it with Cause context.DeadlineExceeded.

The default is DefaultMaxElapsedTime (15 minutes), so both limits are active at once unless overridden. Pass 0 to disable the elapsed-time limit and rely on the context (or WithMaxTries) instead.

func WithMaxTries 

func WithMaxTries(n uint) RetryOption

WithMaxTries limits the total number of attempts, not retries: WithMaxTries(1) runs the operation once and does not retry. When the limit is reached, Retry returns a *RetryError with Cause ErrExhausted. The default, 0, means no limit.

func WithNotify 

func WithNotify(n Notify) RetryOption

WithNotify sets a function called after each failed attempt that will be retried. See Notify for exactly when it fires.

type StopBackOff 

type StopBackOff struct{}

StopBackOff is a fixed backoff policy that always returns backoff.Stop for NextBackOff(), meaning that the operation should never be retried.

func (*StopBackOff) NextBackOff 

func (b *StopBackOff) NextBackOff() time.Duration

func (*StopBackOff) Reset 

func (b *StopBackOff) Reset()

type Ticker 

type Ticker struct {
	C <-chan time.Time
	// contains filtered or unexported fields
}

Ticker holds a channel that delivers `ticks' of a clock at times reported by a BackOff.

Ticks will continue to arrive when the previous operation is still running, so operations that take a while to fail could run in quick succession.

Example 
package main

import (
	"fmt"
	"log"

	"github.com/aliykh/backoff"
)

func main() {
	// An operation that may fail.
	operation := func() (string, error) {
		return "hello", nil
	}

	ticker := backoff.NewTicker(backoff.NewExponentialBackOff())
	defer ticker.Stop()

	var result string
	var err error

	// Ticks will continue to arrive when the previous operation is still running,
	// so operations that take a while to fail could run in quick succession.
	for range ticker.C {
		if result, err = operation(); err != nil {
			log.Println(err, "will retry...")
			continue
		}

		break
	}

	if err != nil {
		// Operation has failed.
		fmt.Println("Error:", err)
		return
	}

	// Operation is successful after retries.

	fmt.Println(result)
}

Output:
hello

func NewTicker 

func NewTicker(b BackOff) *Ticker

NewTicker returns a new Ticker containing a channel that will send the time at times specified by the BackOff argument. Ticker is guaranteed to tick at least once. The channel is closed when Stop method is called or BackOff stops. It is not safe to manipulate the provided backoff policy (notably calling NextBackOff or Reset) while the ticker is running.

func (*Ticker) Stop 

func (t *Ticker) Stop()

Stop turns off a ticker. After Stop, no more ticks will be sent.

type ZeroBackOff 

type ZeroBackOff struct{}

ZeroBackOff is a fixed backoff policy whose backoff time is always zero, meaning that the operation is retried immediately without waiting, indefinitely.

func (*ZeroBackOff) NextBackOff 

func (b *ZeroBackOff) NextBackOff() time.Duration

func (*ZeroBackOff) Reset 

func (b *ZeroBackOff) Reset()

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.