rehttp

package module
Version: v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 19, 2021 License: BSD-3-Clause Imports: 9 Imported by: 36

README

rehttp build status Go Reference

Package rehttp implements an HTTP Transport (an http.RoundTripper) that handles retries. See the godoc for details.

Please note that rehttp requires Go1.6+, because it uses the http.Request.Cancel field to check for cancelled requests. It should work on Go1.5, but only if there is no timeout set on the *http.Client. Go's stdlib will return an error on the first request if that's the case, because it requires a RoundTripper that implements the (now deprecated in Go1.6) CancelRequest method.

On Go1.7+, it uses the context returned by http.Request.Context to check for cancelled requests.

Installation

$ go get github.com/PuerkitoBio/rehttp

License

The BSD 3-Clause license.

Documentation

Overview

Package rehttp implements an HTTP transport that handles retries. An HTTP client can be created with a *rehttp.Transport as RoundTripper and it will apply the retry strategy to its requests.

The retry strategy is provided by the Transport, which determines whether or not the request should be retried, and if so, what delay to apply before retrying, based on the RetryFn and DelayFn functions passed to NewTransport.

The package offers common delay strategies as ready-made functions that return a DelayFn:

  • ConstDelay(delay time.Duration) DelayFn
  • ExpJitterDelay(base, max time.Duration) DelayFn

It also provides common retry helpers that return a RetryFn:

  • RetryIsErr(func(error) bool) RetryFn
  • RetryHTTPMethods(methods ...string) RetryFn
  • RetryMaxRetries(max int) RetryFn
  • RetryStatuses(statuses ...int) RetryFn
  • RetryStatusInterval(fromStatus, toStatus int) RetryFn
  • RetryTemporaryErr() RetryFn

Those can be combined with RetryAny or RetryAll as needed. RetryAny enables retries if any of the RetryFn return true, while RetryAll enables retries if all RetryFn return true. Typically, the RetryFn of the Transport should use at least RetryMaxRetries and some other retry condition(s), combined using RetryAll.

By default, the Transport will buffer the request's body in order to be able to retry the request, as a request attempt will consume and close the existing body. Sometimes this is not desirable, so it can be prevented by setting PreventRetryWithBody to true on the Transport. Doing so will disable retries when a request has a non-nil body.

This package requires Go version 1.6+, since it uses the new http.Request.Cancel field in order to cancel requests. It doesn't implement the deprecated http.Transport.CancelRequest method (https://golang.org/pkg/net/http/#Transport.CancelRequest).

On Go1.7+, it uses the context returned by http.Request.Context to check for cancelled requests.

It should work on Go1.5, but only if there is no timeout set on the *http.Client. Go's stdlib will return an error on the first request if that's the case, because it requires a RoundTripper that implements the CancelRequest method.

Example
package main

import (
	"net/http"
	"time"

	"github.com/PuerkitoBio/rehttp"
)

func main() {
	tr := rehttp.NewTransport(
		nil, // will use http.DefaultTransport
		rehttp.RetryAll(rehttp.RetryMaxRetries(3), rehttp.RetryTemporaryErr()), // max 3 retries for Temporary errors
		rehttp.ConstDelay(time.Second),                                         // wait 1s between retries
	)
	client := &http.Client{
		Transport: tr,
		Timeout:   10 * time.Second, // Client timeout applies to all retries as a whole
	}
	res, err := client.Get("http://example.com")
	if err != nil {
		// handle err
	}
	// handle response
	res.Body.Close()
}
Output:

Index

Examples

Constants

This section is empty.

Variables

View Source
var PRNG = rand.New(rand.NewSource(time.Now().UnixNano()))

PRNG is the *math.Rand value to use to add jitter to the backoff algorithm used in ExpJitterDelay. By default it uses a *rand.Rand initialized with a source based on the current time in nanoseconds.

Functions

This section is empty.

Types

type Attempt

type Attempt struct {
	// Index is the attempt index starting at 0.
	Index int

	// Request is the request for this attempt. If a non-nil Response
	// is present, this is the same as Response.Request, but since a
	// Response may not be available, it is guaranteed to be set on this
	// field.
	Request *http.Request

	// Response is the response for this attempt. It may be nil if an
	// error occurred an no response was received.
	Response *http.Response

	// Error is the error returned by the attempt, if any.
	Error error
}

Attempt holds the data describing a RoundTrip attempt.

type DelayFn

type DelayFn func(attempt Attempt) time.Duration

DelayFn is the signature for functions that return the delay to apply before the next retry.

func ConstDelay

func ConstDelay(delay time.Duration) DelayFn

ConstDelay returns a DelayFn that always returns the same delay.

func ExpJitterDelay

func ExpJitterDelay(base, max time.Duration) DelayFn

ExpJitterDelay returns a DelayFn that returns a delay between 0 and base * 2^attempt capped at max (an exponential backoff delay with jitter).

See the full jitter algorithm in: http://www.awsarchitectureblog.com/2015/03/backoff.html

type RetryFn

type RetryFn func(attempt Attempt) bool

RetryFn is the signature for functions that return whether a retry should be done for the request.

func RetryAll

func RetryAll(retryFns ...RetryFn) RetryFn

RetryAll returns a RetryFn that allows a retry if all retryFns return true. If retryFns is empty, it always returns true.

func RetryAny

func RetryAny(retryFns ...RetryFn) RetryFn

RetryAny returns a RetryFn that allows a retry as long as one of the retryFns returns true. If retryFns is empty, it always returns false.

func RetryHTTPMethods

func RetryHTTPMethods(methods ...string) RetryFn

RetryHTTPMethods returns a RetryFn that retries if the request's HTTP method is one of the provided methods. It is meant to be used in conjunction with another RetryFn such as RetryTemporaryErr combined using RetryAll, otherwise this function will retry any successful request made with one of the provided methods.

func RetryIsErr

func RetryIsErr(fn func(error) bool) RetryFn

RetryIsErr returns a RetryFn that retries if the provided error predicate - a function that receives an error and returns a boolean - returns true for the error associated with the Attempt. Note that fn may be called with a nil error.

func RetryMaxRetries

func RetryMaxRetries(max int) RetryFn

RetryMaxRetries returns a RetryFn that retries if the number of retries is less than or equal to max.

func RetryStatusInterval

func RetryStatusInterval(fromStatus, toStatus int) RetryFn

RetryStatusInterval returns a RetryFn that retries if the response's status code is in the provided half-closed interval [fromStatus, toStatus) (that is, it retries if fromStatus <= Response.StatusCode < toStatus, so RetryStatusInterval(400, 500) would retry for any 4xx code, but not for 500).

func RetryStatuses

func RetryStatuses(statuses ...int) RetryFn

RetryStatuses returns a RetryFn that retries if the response's status code is one of the provided statuses.

func RetryTemporaryErr

func RetryTemporaryErr() RetryFn

RetryTemporaryErr returns a RetryFn that retries if the Attempt's error is a temporary error. A temporary error is one that implements the Temporary() bool method. Most errors from the net package implement this.

type Transport

type Transport struct {
	http.RoundTripper

	// PreventRetryWithBody prevents retrying if the request has a body. Since
	// the body is consumed on a request attempt, in order to retry a request
	// with a body, the body has to be buffered in memory. Setting this
	// to true avoids this buffering: the retry logic is bypassed if the body
	// is non-nil.
	PreventRetryWithBody bool
	// contains filtered or unexported fields
}

Transport wraps a RoundTripper such as *http.Transport and adds retry logic.

func NewTransport

func NewTransport(rt http.RoundTripper, retry RetryFn, delay DelayFn) *Transport

NewTransport creates a Transport with a retry strategy based on retry and delay to control the retry logic. It uses the provided RoundTripper to execute the requests. If rt is nil, http.DefaultTransport is used.

func (*Transport) RoundTrip

func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper for the Transport type. It calls its underlying http.RoundTripper to execute the request, and adds retry logic as per its configuration.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL