httpx

package module
v3.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 19, 2026 License: GPL-2.0, GPL-3.0 Imports: 17 Imported by: 0

README

httpx

Go Reference Go version Test coverage Mutation OpenSSF Best Practices OpenSSF Scorecard

Resilient outbound-HTTP toolkit for Go: retry, backoff, transient-error classification, and more.

A resilient outbound-HTTP toolkit for Go providing jittered exponential backoff, transient-error classification, Retry-After parsing, HTTP status mapping, secret redaction, body draining, a transparent retrying http.RoundTripper with body replay, and a configurable redirect allowlist. Zero dependencies beyond the Go standard library and pgregory.net/rapid (test only).

v3 presents the toolkit as three retry doors sharing one option vocabulary:

  • Do[T] retries a typed operation you own (any closure returning (T, error)).
  • GetBytes retries an HTTP GET and returns bounded, redaction-safe body bytes.
  • NewRetryRoundTripper retries transparently inside a stdlib http.RoundTripper.

NewRetryClient assembles the retrying client (transport + an explicit, required redirect policy) in one call.

Install

go get github.com/cplieger/httpx/v3@latest

Usage

// Bounded-bytes GET with retry
body, err := httpx.GetBytes(ctx, client, url,
    httpx.WithMaxAttempts(3),
    httpx.WithBaseDelay(time.Second),
)

// Generic typed retry
result, err := httpx.Do(ctx, func(ctx context.Context) (T, error) {
    return doWork(ctx)
}, httpx.WithMaxAttempts(3), httpx.WithLabel("fetch"))

// Retry rate limits too (opt-in): a *RateLimitError is retried after
// min(its Retry-After hint, maxWait)
result, err := httpx.Do(ctx, fn,
    httpx.WithRateLimitRetry(30*time.Second),
)

// Retry ONLY rate limits (transients fail fast; e.g. when transient retry is
// handled by an outer layer)
_, err := httpx.Do(ctx, func(ctx context.Context) (struct{}, error) {
    return struct{}{}, download(ctx)
}, httpx.WithRateLimitOnly(5*time.Minute), httpx.WithMaxAttempts(3))

// Retrying client: transport retry + an explicit redirect policy in one call.
// The policy parameter is required (there is no safe universal default; nil
// panics). No Client.Timeout is set — bound totals with a context deadline or
// TransportConfig.MaxElapsedTime, and single attempts on the base transport.
client := httpx.NewRetryClient(nil, httpx.DefaultRedirectPolicy, httpx.TransportConfig{
    MaxAttempts: 4,
    BaseDelay:   time.Second,
})

// Transparent retrying RoundTripper (inspired by hashicorp/go-retryablehttp)
rt := httpx.NewRetryRoundTripper(http.DefaultTransport, httpx.TransportConfig{
    MaxAttempts: 4,
    OnRetry: func(attempt int, req *http.Request, resp *http.Response, err error) {
        log.Printf("retry #%d for %s", attempt, req.URL)
    },
    PrepareRetry: func(req *http.Request) error {
        req.Header.Set("Authorization", "Bearer "+freshToken())
        return nil
    },
})
client := &http.Client{Transport: rt, CheckRedirect: httpx.DefaultRedirectPolicy}

// Retry POST/PUT with body replay (opt-in, requires GetBody)
client := httpx.NewRetryClient(nil, httpx.DefaultRedirectPolicy, httpx.TransportConfig{
    MaxAttempts:        4,
    RetryNonIdempotent: true,
})
payload := []byte(`{"key":"value"}`)
req, _ := http.NewRequest("POST", url, bytes.NewReader(payload))
req.GetBody = func() (io.ReadCloser, error) {
    return io.NopCloser(bytes.NewReader(payload)), nil
}
resp, err := client.Do(req)

// PermanentError — signal "do not retry" (mirrors cenkalti/backoff)
if configErr != nil {
    return httpx.Permanent(configErr) // will not be retried
}

// Custom redirect policy
policy := httpx.RedirectPolicyFunc(
    httpx.WithAllowedHosts("api.example.com"),
    httpx.WithAllowedSuffixes(".cdn.example.com"),
    httpx.WithMaxHops(3),
)

// Pin a private / self-signed CA as the SOLE trust anchor (verification stays
// ON, TLS 1.2 minimum). The caller reads the PEM bytes (file, secret, env),
// keeping the helper I/O-free.
tr, err := httpx.CATransport(pemBytes)
client := &http.Client{Transport: tr, CheckRedirect: httpx.DefaultRedirectPolicy}
// ...or compose the pinned transport with retry:
client = httpx.NewRetryClient(tr, httpx.DefaultRedirectPolicy, httpx.TransportConfig{MaxAttempts: 3})

// Transient error classification
if httpx.IsTransient(err) { /* safe to retry */ }

// Limit response body size
rc := httpx.LimitedBody(resp, 1<<20) // 1 MB cap
defer rc.Close()

Migrating from v2

One option vocabulary replaces v2's three config dialects; two loop doors replace three retry functions; the retrying client gains a required redirect policy. Mechanical mapping:

v2 v3
Retry(ctx, client, url, opts...) GetBytes(ctx, client, url, opts...) — identical contract and option names
RetryWithBackoff[T](ctx, n, d, label, fn) Do[T](ctx, fn, WithMaxAttempts(n), WithBaseDelay(d), WithLabel(label))
RetryOnRateLimit(ctx, n, maxWait, fn) Do[struct{}](ctx, wrap(fn), WithRateLimitOnly(maxWait), WithMaxAttempts(n)) — wrap adapts func(ctx) error to (struct{}, error)
NewRetryRoundTripper(base, WithRTMaxAttempts(4), ...) NewRetryRoundTripper(base, TransportConfig{MaxAttempts: 4, ...})
WithRTMaxAttempts(0) (try once) TransportConfig{MaxAttempts: -1} — zero now means unset (default 3); NEGATIVE means exactly one attempt
rt.StandardClient() + manual CheckRedirect NewRetryClient(base, policy, cfg) — policy is a required argument
WithBackoffFunc / Backoff / BackoffStop / NewExponentialBackoff removed — the equal-jitter progression configured by BaseDelay is the strategy; MaxElapsedTime is the hard ceiling

Do keeps v2's generic-loop semantics exactly: total attempts (minimum 1, WithMaxAttempts(0) still means one attempt), transient-only default classification, RetryAfterHint honored, context checked after each failure.

API

Retry doors
  • Do[T] — generic retry with jittered exponential backoff; when a transient error implements RetryAfterHint, its pre-capped duration replaces the backoff for the next wait (the exponential base keeps advancing). Rate-limit handling is opt-in per call: WithRateLimitRetry(maxWait) adds *RateLimitError to the retryable set; WithRateLimitOnly(maxWait) retries nothing else. Counts total attempts (a non-positive count clamps to 1).
  • GetBytes — HTTP GET with exponential backoff on 429/5xx and transient transport errors (timeouts, connection resets, DNS failures — see IsTransient); 4xx (non-429) and non-transient transport errors return immediately. Honors Retry-After (capped at RetryAfterCap). Counts total attempts.
  • NewRetryRoundTripper(base, TransportConfig{...}) — create a retrying http.RoundTripper. TransportConfig{} is ready to use (3 attempts, 1s base delay, default policy); MaxAttempts: -1 means exactly one attempt.
Loop options

Shared by both loop doors (Option): WithMaxAttempts, WithBaseDelay, WithLogger. Do-only (DoOption): WithLabel, WithRateLimitRetry, WithRateLimitOnly. GetBytes-only (GetOption): WithHeaders, WithMaxBodyBytes. Passing an option to the wrong door is a compile error. A non-positive rate-limit maxWait falls back to RetryAfterCap (60s), so the inter-attempt wait is always positive (never a hot spin); supplying both rate-limit modes is a configuration error.

Clients
  • NewClient(timeout) — simple client with the same-host DefaultRedirectPolicy preinstalled.
  • NewRetryClient(base, policy, cfg) — retrying client; policy is required (nil panics — a nil CheckRedirect would silently mean net/http's follow-anywhere default). Sets no Client.Timeout (it would cap the whole retry sequence); bound totals with a context deadline or TransportConfig.MaxElapsedTime, and single attempts on the base transport (e.g. ResponseHeaderTimeout on a CloneDefaultTransport()).
TLS transports
  • CATransport(pem) — build an *http.Transport (cloned from http.DefaultTransport, so pooling/timeouts/proxy are preserved) that pins the CA certificate(s) in pem as the sole trust anchors. Verification stays on (InsecureSkipVerify is never set) with a TLS 1.2 minimum. Returns the concrete, mutable transport so it composes with NewRetryRoundTripper.
  • ErrNoCertsInPEM — returned by CATransport when pem yields no certificates (a loud error instead of a silently-empty pool). The caller reads the PEM bytes, keeping the helper I/O-free.
  • CloneDefaultTransport() — a private clone of http.DefaultTransport (pooling/timeouts/HTTP2/proxy preserved) that is yours to mutate — set a per-attempt ResponseHeaderTimeout, tune MaxIdleConnsPerHost, or use it as the base of NewRetryRoundTripper — without reconfiguring every other client in the process. Errors when http.DefaultTransport has been replaced by a non-*http.Transport (nothing concrete to clone). The building block CATransport is assembled on.
Test helpers (certtest subpackage)

The github.com/cplieger/httpx/v3/certtest subpackage supplies throwaway self-signed CA material for tests — the companion to CATransport. It lives in a separate package so the certificate-generation code is never linked into a production binary (only the _test.go files that import it pull it in, exactly as the standard library ships net/http/httptest alongside net/http).

  • certtest.SelfSignedCA(tb) — a fresh, throwaway self-signed CA certificate, PEM-encoded ([]byte); feed it to CATransport or an x509.CertPool. A new key each call, so two certs are mutually untrusted (handy for asserting a pin is enforced).
  • certtest.WriteSelfSignedCA(tb) — the same certificate written to a ca.pem file (mode 0o600) under tb.TempDir(), returning the path — for code under test that reads its CA from a file path.
Transport hooks & policies (TransportConfig fields)
  • CheckRetry — pluggable retry policy: func(ctx, resp, err) (bool, error). The default retries transient transport errors and 429/502/503/504 (deliberately narrower than GetBytes, which retries every 5xx).
  • OnRetry — per-attempt callback for observability/metrics (the transport's only seam; it logs nothing itself)
  • PrepareRetry — mutate request before retry (e.g., re-sign tokens)
  • MaxElapsedTime — hard total-time ceiling across retries, including honored Retry-After (checked between attempts)
  • RetryNonIdempotent — opt-in POST/PUT/PATCH/DELETE retry with GetBody replay
Classification & Parsing
  • IsTransient — classify errors as transient (retryable); respects PermanentError
  • RetryAfterHint is an interface (RetryAfterHint() time.Duration) an error implements to supply the next retry wait; Do honors it when the error is transient and the duration is positive (the implementer must cap the value, since httpx applies no ceiling of its own here)
  • CheckHTTPStatus — map HTTP status to typed errors
  • ParseRetryAfter / ParseRetryAfterResponse — parse Retry-After header (capped at RetryAfterCap / raw)
Error Control
  • Permanent(err) — wrap error to signal "do not retry" (mirrors cenkalti/backoff)
  • IsPermanent(err) — check if error is wrapped as permanent
  • PermanentError — the wrapper type (supports errors.Is/errors.As/Unwrap)
Backoff Primitives
  • JitteredBackoff — equal jitter [backoff/2, backoff]
  • SafeDouble / SleepCtx — overflow-safe doubling, context-aware sleep
Body Helpers
  • Drain / DrainClose — body drain for connection reuse (64 KB limit)
  • LimitedBody — wrap response body with a size cap
  • ReadLimitedBody — read a body to a cap (closing it) with overflow detection, returning *ResponseTooLargeError instead of a silently truncated body
Conditional GET
  • Validators{ETag, LastModified} — the cache validators captured from a previous 200, replayed on the next request
  • ConditionalResult{Validators, Body, NotModified} — one conditional-request outcome
  • DoConditional(client, req, v, maxBodyBytes) — a single conditional attempt: sets If-None-Match / If-Modified-Since from v (empty fields unsent) and classifies the response — 304 (NotModified, body drained, zero Validators: keep the ones you sent), 200 (bounded body + the response's fresh validators), anything else an error (the CheckHTTPStatus mapping for >= 400, a plain non-transient error for a status that is neither content nor a revalidation). Deliberately single-shot so the caller owns retry and cache policy: wrap it in Do (transient classification composes through the returned errors), rebuild the request per attempt, persist body and validators together, and send the zero Validators when the cached body is unusable so an empty cache can never be "revalidated" into a 304 with nothing to reuse.
Redirect Policies
  • DefaultRedirectPolicy — same-host-only redirect policy (used by NewClient). It also refuses a same-host https->http scheme downgrade and allows an http->https upgrade; it delegates to RedirectPolicyFunc(WithSameHost()), so the two cannot drift.
  • RefuseAllRedirects — follows no redirect: returns http.ErrUseLastResponse, so the client surfaces the 3xx response itself (nil error) instead of following. The policy for a token-bearing client of an API that issues no redirects — Go forwards custom headers (X-Plex-Token, X-Api-Key) across redirects, so a hostile 302 would exfiltrate the credential.
  • DockerGitHubRedirectPolicy — optional example policy for docker.com/github.com; like the other policies it refuses an https->http downgrade, even to an allowlisted host
  • RedirectPolicyFunc — build a custom redirect allowlist (functional options: WithAllowedHosts, WithAllowedSuffixes, WithSameHost, WithAllowSchemeDowngrade, WithMaxHops)
    • WithSameHost additionally allows a redirect whose target host equals the original request's host (layered on any allowlisted hosts or suffixes); it is the building block for a same-origin policy.
    • WithAllowSchemeDowngrade(bool) permits an https->http downgrade redirect. The default false refuses it, so a custom auth header is never forwarded onto a cleartext hop; an http->https upgrade is always allowed. The downgrade is judged against the original request's scheme, and the guard applies to allowlisted and same-host targets alike.
  • CheckRedirect — the http.Client.CheckRedirect function shape as a type alias; every shipped policy is one.
Secret Redaction
  • RedactTransportError / RedactSecret / RedactSecretString — secret redaction (error- and string-level)
  • LogSafeError — reduce a URL-embedding transport *url.Error to its underlying cause (everything else passes through, errors.Is/As preserved). The same reduction httpx applies to every transport error it logs; equivalent to RedactTransportError(err, "", "").
Error Types
  • AuthError / RateLimitError / HTTPStatusError / StatusError
  • ResponseTooLargeError — returned by GetBytes when the response exceeds WithMaxBodyBytes (carries Limit; no body is returned)
  • ErrRateLimited / ErrServerError — sentinel errors
  • PermanentError — do-not-retry sentinel wrapper

Logging

Do and GetBytes log via log/slog and accept WithLogger to override the default logger per call. Per-attempt "retrying" lines are logged at Debug — a retry that recovers is normal operation, not a degraded state. Only the terminal "retries exhausted" / "rate limit retries exhausted" lines are at Warn. GetBytes also emits a Warn "slow upstream response" when a single attempt's response takes longer than 10s (timed per attempt, so backoff sleeps are not counted as upstream latency). The RetryRoundTripper logs nothing itself — observe its retries through the OnRetry hook.

URL redaction in logs and errors

To avoid leaking credentials into logs (CWE-532, the class of go-retryablehttp CVE-2024-6104), GetBytes never logs or returns a raw request URL:

  • Every logged url attribute is redacted — the userinfo password is masked (like url.URL.Redacted) and query values are replaced with REDACTED (query values commonly carry api keys, tokens, and signatures — the same default .NET 9's IHttpClientFactory adopted). Query keys, scheme, host, and path are kept for debugging.
  • StatusError.Error() renders that same redacted URL, so the secret stays out of returned errors too; the raw StatusError.URL field remains available for programmatic use.
  • Transport errors (*url.Error, which embed the full URL) are reduced to their underlying cause before logging — the reduction is exported as LogSafeError so callers wrapping transport errors into their own messages can apply the same one.

The RoundTripper performs no URL logging of its own — wire any logging through its OnRetry hook, where redaction is the caller's responsibility.

Retry exhaustion

GetBytes and the RetryRoundTripper report exhaustion differently — match your error handling to the one you use:

  • GetBytes returns nil body and a wrapped error: retries exhausted after <elapsed>: <lastErr> (unwrap with errors.Is/errors.As). A response that overflows WithMaxBodyBytes returns *ResponseTooLargeError (no body).
  • RetryRoundTripper returns the last response with a nil error, even when that response is a retryable 5xx (e.g. a 503) — mirroring how a non-retried request behaves. A caller that checks only err != nil will treat an exhausted 503 as success, so inspect resp.StatusCode and close the body. (A budget abort via MaxElapsedTime does return an error.)

Timeouts and deadlines

httpx retries transient failures, not budget expiry, and that distinction drives how you should bound a retried call. IsTransient classifies context.DeadlineExceeded and context.Canceled as non-transient (checked first), while a transport-level net.Error timeout, a connection reset, a DNS error, and a 429/5xx are transient. So a context deadline means "the budget is exhausted, stop", and a transport timeout means "this attempt failed, try again".

  • Total budget: a context deadline. Pass a context.WithTimeout (or a caller-supplied deadline) as the single authoritative bound over the whole operation. GetBytes / Do stop the moment ctx is done, and SleepCtx caps the backoff by it, so the deadline spans every attempt and every backoff sleep. On expiry the call ends; it is terminal, not retried.
  • Per-attempt bound. Where a per-attempt cap lives depends on the retry entry point, because that is what decides whether http.Client.Timeout is per-attempt or total:
    • With the one-shot GetBytes / Do (the retry loop runs outside client.Do), an http.Client.Timeout bounds each attempt and fires as a net.Error timeout, so it is retried; a context.WithTimeout wrapped inside the retry fn is instead a context deadline and is terminal. Choose the stall behavior you want.
    • With NewRetryRoundTripper / NewRetryClient (the retry loop runs inside client.Do), http.Client.Timeout is NOT per-attempt: it caps the whole retry sequence, and because it is a context deadline a slow attempt that trips it aborts the remaining retries. This is why NewRetryClient sets no Client.Timeout. For a per-attempt bound here, set a transport timeout such as ResponseHeaderTimeout on the base transport (it fires as a retryable net.Error); bound the total with the caller's context or TransportConfig.MaxElapsedTime. Note that neither the between-attempt MaxElapsedTime check nor an expired context can interrupt an attempt already stalled inside the base transport — only a transport-level timeout can.

Recommended: give the operation a context deadline as its total budget (honored end-to-end, and it keeps a slow attempt from running unbounded), and add a per-attempt bound only when a single try needs its own cap. Through the one-shot GetBytes helper a bare http.Client.Timeout with no context deadline is fine for a simple call to a trusted or local endpoint (there it is per-attempt, so a retried call can run up to maxAttempts times its value); under NewRetryClient reach for ResponseHeaderTimeout plus a total from the context or MaxElapsedTime instead.

A per-attempt timeout that is itself retried (a stalled attempt abandoned and a fresh one tried within the remaining total budget, the gRPC per-try-timeout model) is not built into the retry primitives today. Approximate it by pairing a per-attempt bound (a context.WithTimeout inside the one-shot fn, or ResponseHeaderTimeout under the RoundTripper) with an outer context deadline or MaxElapsedTime for the total.

Unsupported by Design (SKIP List)

The following features are intentionally not provided:

Feature Rationale
Circuit breaker Orthogonal pattern excluded by all comparables. Compose externally with sony/gobreaker.
Retry budget / token bucket None of the comparables implement it. Disproportionate complexity (~150 LOC + shared mutable state) for a focused library.
Multiple jitter strategies (full, decorrelated) Equal jitter is the recommended default per AWS Builders' Library. Full jitter risks near-zero delays.
ErrorHandler for exhaustion Current fmt.Errorf("retries exhausted: %w", lastErr) is sufficient. Callers unwrap.
Response body on error Adds API complexity (ownership of body close). Use Do[T] with custom logic.
Idempotency key injection Application-level concern, not a retry library's responsibility.
Configurable Retry-After cap A raisable cap would regress the fixed-60s DoS ceiling (ParseRetryAfter); rate-limit waits are capped by the caller-owned maxWait arguments.

Disclaimer

This project is built with care and follows security best practices, but it is intended for personal / self-hosted use. No guarantees of fitness for production environments. Use at your own risk.

This project was built with AI-assisted tooling using Claude Opus and Kiro. The human maintainer defines architecture, supervises implementation, and makes all final decisions.

License

GPL-3.0 — see LICENSE.

Documentation

Overview

Package httpx provides a resilient outbound-HTTP toolkit: transient-error classification, generic typed retry (Do) and a bounded-bytes GET (GetBytes) over one jittered-exponential-backoff loop, a transparent retrying RoundTripper, Retry-After parsing, HTTP status mapping, secret redaction, body draining, custom-CA TLS transports, and a configurable redirect allowlist.

The package deliberately keeps these concerns together: they compose into a single net/http.Client, whose configuration surface (Transport, CheckRedirect, per-request contexts) spans exactly this set. This overview maps the surface; the README carries usage examples, the v2 migration table, and the timeout model.

Retry doors

Three entry points with different ownership contracts (who builds the request, who owns the body, who sees the response). They share one option vocabulary and the same equal-jitter backoff progression; passing an option to the wrong door does not compile.

Shared loop options (Option): WithMaxAttempts, WithBaseDelay, WithLogger.

Clients

NewClient (timeout + same-host redirect policy preinstalled), NewRetryClient (retry transport + REQUIRED explicit redirect policy), ContextWithDefaultTimeout (request-deadline helper).

Classification and error control

IsTransient decides retryability; extend it for your own error types via the Transient and RetryAfterHint interfaces. Permanent (and PermanentError, IsPermanent) marks an error non-retryable. CheckHTTPStatus maps response codes to typed errors: AuthError, RateLimitError, StatusError, HTTPStatusError, ResponseTooLargeError, and the ErrRateLimited and ErrServerError sentinels.

Retry-After

ParseRetryAfter and ParseRetryAfterResponse parse the header with a 60-second cap (RetryAfterCap). RateLimitError.RetryAfter carries the RAW uncapped hint; callers sleeping on it must bound it (the rate-limit retry modes do).

Redirect policies

A CheckRedirect policy is required knowledge the caller supplies: DefaultRedirectPolicy (same host), RefuseAllRedirects, DockerGitHubRedirectPolicy, or a custom allowlist built with RedirectPolicyFunc and RedirectOption (WithAllowedHosts, WithAllowedSuffixes, WithSameHost, WithMaxHops, WithAllowSchemeDowngrade).

TLS transports

CATransport pins PEM CA certificate(s) as the sole trust anchors on a cloned default transport (ErrNoCertsInPEM on empty input); CloneDefaultTransport yields a private mutable transport clone. The certtest subpackage generates throwaway CA material for tests.

Secret redaction

RedactSecret, RedactSecretString, RedactTransportError, and LogSafeError keep credentials out of logs and returned errors. GetBytes never logs or returns a raw URL.

Body helpers

Drain, DrainClose, LimitedBody, ReadLimitedBody.

Conditional GET

DoConditional with Validators and ConditionalResult implements ETag/Last-Modified revalidation over one bounded read.

Backoff primitives

JitteredBackoff, SafeDouble, and SleepCtx are the exported building blocks the doors are made of, for callers composing their own loops.

Index

Examples

Constants

View Source 
const (
	// DefaultBaseDelay is the production base for exponential-backoff retry.
	DefaultBaseDelay = time.Second
	// DefaultMaxAttempts caps the retry doors at three total attempts.
	DefaultMaxAttempts = 3
	// DefaultMaxBodyBytes caps response bodies at 10 MB.
	DefaultMaxBodyBytes int64 = 10 << 20
	// RetryAfterCap is the maximum Retry-After honor duration.
	RetryAfterCap = 60 * time.Second
)

Variables

View Source 
var ErrNoCertsInPEM = errors.New("httpx: no PEM-encoded certificates found")

ErrNoCertsInPEM is returned by CATransport when the supplied PEM data contains no parseable certificates. A misconfigured or empty CA file therefore fails loudly rather than silently producing a transport that trusts nothing (which would reject every connection with an opaque error).

View Source 
var ErrRateLimited = errors.New("rate limited")

ErrRateLimited is a sentinel callers use with errors.Is to detect 429 responses.

View Source 
var ErrServerError = errors.New("server error")

ErrServerError is a sentinel for upstream 5xx responses.

Functions

func CATransport 

func CATransport(pem []byte) (*http.Transport, error)

CATransport builds an *http.Transport that trusts ONLY the CA certificate(s) in pem for TLS verification (pinning). It is cloned from http.DefaultTransport (via CloneDefaultTransport), so it keeps the standard connection pooling, dial/keepalive timeouts, HTTP/2 negotiation, and proxy support (ProxyFromEnvironment). A FRESH TLS config is then installed — RootCAs from pem, a TLS 1.2 minimum, and verification always ENABLED (InsecureSkipVerify is not set). Any TLS settings already on http.DefaultTransport are intentionally NOT carried over, so the returned transport's trust posture cannot be weakened by a program that globally mutated the default transport's *tls.Config (e.g. set InsecureSkipVerify or an accept-all VerifyPeerCertificate hook).

The supplied CA(s) are the SOLE trust anchors: the transport rejects any host not chaining to them, including public-CA hosts. This is the right setup for a known self-hosted endpoint presenting a private or self-signed certificate (a Plex server, an internal API).

The returned *http.Transport is concrete and mutable, so callers may tune fields such as MaxIdleConnsPerHost or pass it as the base RoundTripper of NewRetryRoundTripper: 

tr, err := httpx.CATransport(pem)
client := httpx.NewRetryClient(tr, httpx.DefaultRedirectPolicy, httpx.TransportConfig{MaxAttempts: 3})

It returns ErrNoCertsInPEM when pem yields no certificates. The caller owns reading the PEM bytes (from a file, a secret, an env var), which keeps this function I/O-free and lets the caller bound the read as it sees fit.

CATransport requires http.DefaultTransport to be the standard library's *http.Transport (the default). If your program has replaced it with a wrapping RoundTripper (for example request instrumentation), CATransport returns an error.

func CheckHTTPStatus 

func CheckHTTPStatus(resp *http.Response) error

CheckHTTPStatus maps HTTP error status codes to typed errors. Returns nil for 2xx/3xx. 401/403 → *AuthError, 429 → *RateLimitError, others ≥400 → *HTTPStatusError.

func CloneDefaultTransport 

func CloneDefaultTransport() (*http.Transport, error)

CloneDefaultTransport returns a private clone of http.DefaultTransport, keeping the standard connection pooling, dial/keepalive timeouts, HTTP/2 negotiation, and proxy support (ProxyFromEnvironment). The clone is the caller's to mutate — set a per-attempt ResponseHeaderTimeout, tune MaxIdleConnsPerHost, or pass it as the base RoundTripper of NewRetryRoundTripper — without the global footgun of mutating http.DefaultTransport itself, which would silently reconfigure every other client in the process.

It returns an error when http.DefaultTransport has been replaced by a non-*http.Transport RoundTripper (request instrumentation, a test stub): a wrapping RoundTripper offers no concrete transport to clone, and failing loudly beats silently dropping the wrapper's behavior. It is the building block CATransport is assembled on.

func ContextWithDefaultTimeout added in v3.1.0 

func ContextWithDefaultTimeout(ctx context.Context, def time.Duration) (context.Context, context.CancelFunc)

ContextWithDefaultTimeout bounds ctx by def ONLY when ctx carries no deadline; a deadline the caller already set is the authoritative budget and is never undercut (or extended). A def <= 0 means "no default" and leaves ctx unbounded. The returned cancel is always non-nil and must be called (a no-op on the passthrough paths), matching context.WithTimeout's contract.

It is the per-request timeout rule for an API client: per-attempt bounds live on the transport (ResponseHeaderTimeout), the total budget is the caller's ctx, and this default applies only when the caller brought no budget of its own. Extracted from the identical requestContext helpers in the plexapi and arrapi client libraries.

func DefaultRedirectPolicy 

func DefaultRedirectPolicy(req *http.Request, via []*http.Request) error

DefaultRedirectPolicy is the default redirect policy: it allows a redirect only to the same host as the original request, and refuses a same-host https->http scheme downgrade (which would forward a custom auth header onto a cleartext hop). A cross-host redirect is refused (Go forwards a custom header across a redirect, so it would leak) and an http->https upgrade is allowed. It delegates to RedirectPolicyFunc(WithSameHost()), with one addition: a call with an empty via chain (which net/http never produces — via always carries at least the original request) is allowed rather than refused. Use RedirectPolicyFunc for a custom allowlist, a higher hop cap (WithMaxHops), or to permit downgrades (WithAllowSchemeDowngrade).

func Do 

func Do[T any](ctx context.Context, fn func(ctx context.Context) (T, error), opts ...DoOption) (T, error)

Do calls fn up to WithMaxAttempts times (total, including the first call) with jittered exponential backoff, returning the first success. Non-retryable errors are returned immediately. By default the retryable set is IsTransient (a *RateLimitError is deliberately NOT transient; a generic operation must not blindly re-fire a rate-limited call) and a transient error carrying a positive RetryAfterHint waits that hint instead of the backoff, the exponential base still advancing. WithRateLimitRetry and WithRateLimitOnly opt into rate-limit retry per their docs.

Under the default and WithRateLimitRetry modes a context canceled after a failed attempt returns ctx.Err(); under WithRateLimitOnly the final attempt's error wins (the v2 RetryOnRateLimit contract). Logging goes to WithLogger (default slog.Default()): per-attempt lines at Debug, the terminal exhaustion at Warn.

Example 
package main

import (
	"context"
	"fmt"
	"time"

	"github.com/cplieger/httpx/v3"
)

func main() {
	attempts := 0
	result, err := httpx.Do(context.Background(), func(_ context.Context) (string, error) {
		attempts++
		if attempts < 2 {
			return "", &httpx.HTTPStatusError{Code: 503}
		}
		return "done", nil
	}, httpx.WithMaxAttempts(3), httpx.WithBaseDelay(time.Millisecond), httpx.WithLabel("example"))
	if err != nil {
		fmt.Println("error:", err)
		return
	}
	fmt.Println(result)
}

Output:
done

func DockerGitHubRedirectPolicy 

func DockerGitHubRedirectPolicy(req *http.Request, via []*http.Request) error

DockerGitHubRedirectPolicy is an OPTIONAL example redirect policy allowing docker.com and github.com hosts. Like every shipped policy it refuses an https->http scheme downgrade (judged against the original request's scheme, see WithAllowSchemeDowngrade), so a custom auth header never rides a cleartext hop even to an allowlisted host. Use it by assigning to Client.CheckRedirect or pass RedirectOption values to RedirectPolicyFunc for other allowlists.

func Drain 

func Drain(body io.ReadCloser)

Drain reads and discards up to 64 KB of a response body to enable HTTP connection reuse.

func DrainClose 

func DrainClose(rc io.ReadCloser)

DrainClose reads remaining bytes (up to drainLimit) from rc before closing it.

func GetBytes 

func GetBytes(ctx context.Context, client *http.Client, reqURL string, opts ...GetOption) ([]byte, error)

GetBytes performs an HTTP GET with bounded exponential-backoff retry on 429 and 5xx responses and on transient transport errors (timeouts, connection resets, DNS failures - see IsTransient). 4xx (non-429) and non-transient transport errors are returned immediately. Honors Retry-After (capped at RetryAfterCap). The response body is read to WithMaxBodyBytes and returned; an over-limit body fails loud with *ResponseTooLargeError (no body). Every logged url attribute and every returned error is redacted (see the package's URL redaction docs).

GetBytes deliberately keeps its own retry loop rather than delegating to RetryRoundTripper.RoundTrip. It is a decorator over the same shared primitives (resolveWait, JitteredBackoff, SafeDouble, SleepCtx, ParseRetryAfter, IsTransient, Drain), not a thin wrapper over the RoundTripper cycle, because GetBytes carries behavior the transparent RoundTripper has no equivalent for and which must stay byte-for-byte stable for existing consumers:

  • []byte return with the body capped at WithMaxBodyBytes (the RoundTripper hands back an *http.Response and never reads the body);
  • URL/secret redaction on every log "url" attr (redactURL) and every returned/wrapped error (LogSafeError, StatusError.Error()), the CWE-532 hardening the RoundTripper path does not perform;
  • rich per-attempt slog logging plus the "retries exhausted after %s: %w" wrapper, which the RoundTripper exposes only as an OnRetry hook;
  • classification of every 5xx (not just 502/503/504) as retryable and of any non-2xx (3xx included) as a permanent *StatusError. A 2xx response returns the body; GetBytes cannot surface a redirect, so 3xx is an error.

Routing GetBytes through RoundTrip would silently change one or more of these, so the loop is intentionally not merged.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/httptest"
	"time"

	"github.com/cplieger/httpx/v3"
)

func main() {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		fmt.Fprint(w, "hello")
	}))
	defer srv.Close()

	body, err := httpx.GetBytes(context.Background(), srv.Client(), srv.URL,
		httpx.WithMaxAttempts(3),
		httpx.WithBaseDelay(100*time.Millisecond),
	)
	if err != nil {
		fmt.Println("error:", err)
		return
	}
	fmt.Println(string(body))
}

Output:
hello

func IsPermanent 

func IsPermanent(err error) bool

IsPermanent reports whether err (or any wrapped error) is a *PermanentError.

func IsTransient 

func IsTransient(err error) bool

IsTransient returns true for errors likely caused by temporary server or network issues worth retrying. Auth, rate-limit, permanent, and context errors are never transient.

func JitteredBackoff 

func JitteredBackoff(backoff time.Duration) time.Duration

JitteredBackoff returns a duration in [backoff/2, backoff] using the "equal jitter" strategy (per AWS Builders' Library). Full jitter and decorrelated jitter are intentionally not provided — equal jitter is the recommended default for HTTP retry as it avoids thundering herd while maintaining a minimum backoff floor.

func LimitedBody 

func LimitedBody(resp *http.Response, limit int64) io.ReadCloser

LimitedBody wraps resp.Body with an io.LimitReader capped at limit bytes, preserving the original Close method.

func LogSafeError 

func LogSafeError(err error) error

LogSafeError returns an error whose message is safe to log. A transport *url.Error embeds the full request URL (with any userinfo/query secrets), so it is reduced to its underlying cause. Nil returns nil; *StatusError already renders a redacted URL via Error(), so it (and everything else) passes through unchanged — preserving errors.Is/As chains for callers.

httpx applies this reduction to every transport error it logs or wraps; it is exported so a caller wrapping transport errors into its own messages can apply the same one (equivalent to RedactTransportError(err, "", "") — reach for that variant when a known secret must also be scrubbed from the text).

func NewClient 

func NewClient(timeout time.Duration) *http.Client

NewClient returns an *http.Client with the given timeout and the DefaultRedirectPolicy (same-host only). For custom redirect allowlists, configure CheckRedirect with RedirectPolicyFunc or assign DockerGitHubRedirectPolicy.

func NewRetryClient 

func NewRetryClient(base http.RoundTripper, policy CheckRedirect, cfg TransportConfig) *http.Client

NewRetryClient returns an *http.Client whose Transport is a RetryRoundTripper over base (nil base means http.DefaultTransport) and whose CheckRedirect is policy. It is the one-call form of the composition plexapi-style consumers assemble by hand: retry transport plus an explicit redirect policy.

policy is REQUIRED and must be non-nil: NewRetryClient panics on a nil policy, because a nil CheckRedirect silently means net/http's default follow-anywhere behavior (up to 10 hops to any host, custom auth headers forwarded) — exactly the unsafe omission this constructor exists to prevent. Pass DefaultRedirectPolicy (same-host), RefuseAllRedirects, or a RedirectPolicyFunc allowlist.

The returned client sets no Client.Timeout: a Client.Timeout above a retrying transport caps the WHOLE retry sequence and defeats the retries beneath it. Note that neither MaxElapsedTime nor the request context can interrupt a stalled in-flight attempt from between attempts: bound single attempts on the base transport (e.g. ResponseHeaderTimeout on a CloneDefaultTransport()) and bound the total with a context deadline (http.NewRequestWithContext) or TransportConfig.MaxElapsedTime.

func ParseRetryAfter 

func ParseRetryAfter(h string) time.Duration

ParseRetryAfter parses a Retry-After header value (delta-seconds or HTTP-date). Returns zero for missing/malformed values. Caps at RetryAfterCap for safety (prevents unbounded waits in retry loops). For raw uncapped values, use ParseRetryAfterResponse.

func ParseRetryAfterResponse 

func ParseRetryAfterResponse(resp *http.Response) time.Duration

ParseRetryAfterResponse parses the Retry-After header from an *http.Response. Returns zero if absent or unparseable. Does NOT cap — preserves the raw duration so callers (e.g., CheckHTTPStatus) can make their own decisions. For capped values suitable for retry loops, use ParseRetryAfter.

func Permanent 

func Permanent(err error) error

Permanent wraps err to indicate it should never be retried. Mirrors cenkalti/backoff.Permanent().

func ReadLimitedBody 

func ReadLimitedBody(body io.ReadCloser, limit int64) ([]byte, error)

ReadLimitedBody reads body up to limit bytes, always closes body, and returns the bytes read. It reads one byte past limit to detect an over-limit body and returns *ResponseTooLargeError (with nil bytes) rather than a silently truncated payload — a truncated body indistinguishable from a complete one is a corruption hazard. A limit of math.MaxInt64 means "effectively unlimited" and is guarded against probe-size overflow.

It is the read-all-with-overflow-detection companion to LimitedBody (which only caps the stream and leaves reading and overflow handling to the caller), and is the same cap+1 read GetBytes applies internally — exposed for callers that issue their own request and decode outside GetBytes but still want the fail-loud size bound. On any error the body is already closed.

func RedactSecret 

func RedactSecret(err error, secret string) error

RedactSecret replaces occurrences of secret in err's message with "REDACTED".

func RedactSecretString 

func RedactSecretString(s, secret string) string

RedactSecretString replaces every occurrence of secret in s with "REDACTED" and returns the result. It is the string-level building block behind RedactSecret and RedactTransportError, exposed for callers that must redact a secret from a plain string — a captured HTTP response body destined for an error field or a log line — rather than from an error value. An empty secret is a no-op (s is returned unchanged), matching the error-shaped variants.

func RedactTransportError 

func RedactTransportError(err error, prefix, secret string) error

RedactTransportError unwraps *url.Error and redacts the secret from the error message. Returns nil for nil input.

func RedirectPolicyFunc 

func RedirectPolicyFunc(opts ...RedirectOption) func(*http.Request, []*http.Request) error

RedirectPolicyFunc returns a CheckRedirect function configured with the given options. A redirect is followed only when its target host is allowed — an exact WithAllowedHosts entry, a WithAllowedSuffixes match, or (with WithSameHost) the original request's own host — and, unless WithAllowSchemeDowngrade is set, the redirect does not downgrade https->http. With no allowlist and no WithSameHost, all redirects are refused. The hop cap is WithMaxHops (default 5).

func RefuseAllRedirects 

func RefuseAllRedirects(*http.Request, []*http.Request) error

RefuseAllRedirects is a CheckRedirect policy that follows NO redirect: it returns http.ErrUseLastResponse, so the client hands the caller the redirect response itself (status 3xx, body open, nil error) instead of the followed hop. It is the policy for a token-bearing client of an API that issues no redirects: Go's client forwards custom request headers (an X-Plex-Token, an X-Api-Key) across redirects — only Authorization, Cookie, and WWW-Authenticate are stripped, and only on a cross-domain hop — so a hostile 302 (MITM, DNS poisoning) would exfiltrate the credential to an attacker-chosen origin. With the hop refused, the credential never leaves the configured host and the unexpected 3xx surfaces to the caller's own status handling.

func SafeDouble 

func SafeDouble(d time.Duration) time.Duration

SafeDouble doubles a duration, guarding against int64 overflow.

func SleepCtx 

func SleepCtx(ctx context.Context, d time.Duration) error

SleepCtx sleeps for d or returns early on context cancellation.

Types

type AuthError 

type AuthError struct{ Msg string }

AuthError indicates invalid or expired credentials.

func (*AuthError) Error 

func (e *AuthError) Error() string

type CheckRedirect 

type CheckRedirect = func(req *http.Request, via []*http.Request) error

CheckRedirect is the http.Client.CheckRedirect function shape. It is a type alias, so values are assignable in both directions; every shipped policy (DefaultRedirectPolicy, RefuseAllRedirects, DockerGitHubRedirectPolicy, and anything built with RedirectPolicyFunc) is a CheckRedirect.

type CheckRetry 

type CheckRetry func(ctx context.Context, resp *http.Response, err error) (bool, error)

CheckRetry is the signature for a retry policy function. It receives the context, the response (may be nil on transport error), and the error (nil on successful response). It returns whether to retry and an optional error to short-circuit with. Mirrors hashicorp/go-retryablehttp CheckRetry.

type ConditionalResult 

type ConditionalResult struct {
	// Validators are the fresh validators captured from a 200 response's ETag /
	// Last-Modified headers (either may be empty when the server sent none).
	// Zero on a 304: the caller keeps the validators it already holds.
	Validators Validators
	// Body is the full response body of a 200, bounded by the maxBodyBytes
	// given to DoConditional. Nil on a 304.
	Body []byte
	// NotModified reports a 304: the cached representation is still current.
	NotModified bool
}

ConditionalResult is one conditional-request outcome. Fields are ordered largest-alignment-first for govet fieldalignment.

func DoConditional 

func DoConditional(client *http.Client, req *http.Request, v Validators, maxBodyBytes int64) (ConditionalResult, error)

DoConditional executes req as a single conditional request: it sets If-None-Match / If-Modified-Since from v (an empty field is not sent), performs the request on client, and classifies the response.

  • 304 -> NotModified=true (body drained and closed; Validators zero — keep the ones you sent).
  • 200 -> the bounded body plus the response's fresh validators. A body over maxBodyBytes fails loud with *ResponseTooLargeError rather than being silently truncated; maxBodyBytes <= 0 means DefaultMaxBodyBytes.
  • Anything else -> an error: the CheckHTTPStatus mapping for >= 400 (*AuthError, *RateLimitError — non-transient; *HTTPStatusError — transient for 502/503/504), or a plain non-transient error for a status that is neither usable content nor a revalidation (a 204, a 3xx from a redirect-refusing client). The body is always closed.

It is deliberately a SINGLE attempt so the caller owns the retry and cache policy: wrap it in Do (transient classification composes through the returned errors), rebuild req per attempt, and decide app-side when a cached copy may be reused on failure (stale-on-error) and whether validators may be sent at all (send the zero Validators when the cached body is unusable, so an empty cache can never be "revalidated" into a 304 with nothing to reuse). Intended for GET (or HEAD, where Body stays empty).

type DoOption 

type DoOption interface {
	// contains filtered or unexported methods
}

DoOption configures Do. Options constructed as Option values apply to both loop doors; Do-only options (WithLabel, WithRateLimitRetry, WithRateLimitOnly) implement only this interface, so passing one to GetBytes is a compile error.

func WithLabel 

func WithLabel(s string) DoOption

WithLabel sets the operation label used in Do's log lines ("<label> failed, retrying", "<label> retries exhausted"). Default: "operation".

func WithRateLimitOnly 

func WithRateLimitOnly(maxWait time.Duration) DoOption

WithRateLimitOnly makes Do retry ONLY *RateLimitError (matched through wrapped errors); every other error, including transient transport errors, is returned immediately. It absorbs v2's RetryOnRateLimit: the wait semantics match WithRateLimitRetry, the terminal Warn is "rate limit retries exhausted", and, matching the v2 contract, the FINAL attempt's error wins even under an already-canceled context (cancellation is observed in the always-positive inter-attempt sleep instead). Mutually exclusive with WithRateLimitRetry.

func WithRateLimitRetry 

func WithRateLimitRetry(maxWait time.Duration) DoOption

WithRateLimitRetry makes Do additionally treat *RateLimitError as retryable, alongside the transient set. The wait before a rate-limit retry is min(err.RetryAfter, maxWait) when the error carries a positive hint, else maxWait; a non-positive maxWait falls back to RetryAfterCap, so the wait is always positive and a canceled context is observed before every retry. Transient errors keep their jittered-backoff waits (with the RetryAfterHint override), and the exponential base advances on every retry. Mutually exclusive with WithRateLimitOnly (Do returns a configuration error when both are supplied).

type GetOption 

type GetOption interface {
	// contains filtered or unexported methods
}

GetOption configures GetBytes. GetBytes-only options (WithHeaders, WithMaxBodyBytes) implement only this interface, so passing one to Do is a compile error.

func WithHeaders 

func WithHeaders(fn func(*http.Request)) GetOption

WithHeaders sets a function that is called to set headers on each request.

func WithMaxBodyBytes 

func WithMaxBodyBytes(n int64) GetOption

WithMaxBodyBytes sets the maximum response body size to read. Default: DefaultMaxBodyBytes (10 MB). A non-positive value falls back to the default.

type HTTPStatusError 

type HTTPStatusError struct {
	Code int
}

HTTPStatusError represents a non-2xx HTTP response not covered by AuthError or RateLimitError. Implements the Transient interface for 502/503/504.

func (*HTTPStatusError) Error 

func (e *HTTPStatusError) Error() string

func (*HTTPStatusError) IsClientError 

func (e *HTTPStatusError) IsClientError() bool

IsClientError reports whether the status code is 4xx.

func (*HTTPStatusError) IsServerError 

func (e *HTTPStatusError) IsServerError() bool

IsServerError reports whether the status code is 5xx.

func (*HTTPStatusError) IsTransient 

func (e *HTTPStatusError) IsTransient() bool

IsTransient reports whether the status code is a retryable server failure (502/503/504).

type OnRetry 

type OnRetry func(attempt int, req *http.Request, resp *http.Response, err error)

OnRetry is called before each retry attempt (not the initial request). attempt is 1-indexed (first retry = 1). resp may be nil on transport errors.

type Option 

type Option interface {
	DoOption
	GetOption
}

Option is a retry option accepted by BOTH loop doors, Do and GetBytes: WithMaxAttempts, WithBaseDelay, and WithLogger construct Option values.

func WithBaseDelay 

func WithBaseDelay(d time.Duration) Option

WithBaseDelay sets the initial backoff delay. Default: DefaultBaseDelay (1s). A non-positive value falls back to the default.

func WithLogger 

func WithLogger(l *slog.Logger) Option

WithLogger sets the logger for retry diagnostics. Default: slog.Default(). A nil logger falls back to the default.

func WithMaxAttempts 

func WithMaxAttempts(n int) Option

WithMaxAttempts sets the maximum number of attempts (TOTAL, including the first call). Default: DefaultMaxAttempts (3). A value below 1 is treated as 1, so the operation always runs at least once (never a silent no-op).

type PermanentError 

type PermanentError struct {
	Err error
}

PermanentError wraps an error to signal that it should NOT be retried, regardless of other retry policies. Mirrors cenkalti/backoff.PermanentError. Use Permanent(err) to wrap.

func (*PermanentError) Error 

func (e *PermanentError) Error() string

func (*PermanentError) Is 

func (e *PermanentError) Is(target error) bool

Is allows errors.Is matching against other PermanentErrors.

func (*PermanentError) Unwrap 

func (e *PermanentError) Unwrap() error

type PrepareRetry 

type PrepareRetry func(req *http.Request) error

PrepareRetry is called before each retry to allow request mutation (e.g., re-signing with a fresh token). Mirrors go-retryablehttp PrepareRetry.

type RateLimitError 

type RateLimitError struct {
	Msg        string
	RetryAfter time.Duration
}

RateLimitError indicates a rate limit was exceeded. RetryAfter, when non-zero, is the RAW, UNCAPPED hint from the upstream's Retry-After header (populated via ParseRetryAfterResponse). The upstream controls this value; a hostile or misconfigured server can supply a very large duration (CWE-400 uncontrolled resource consumption). Callers that sleep on it directly MUST bound it first, e.g. min(err.RetryAfter, cap). Do's rate-limit modes (WithRateLimitRetry, WithRateLimitOnly) already do this (they cap at their maxWait argument). For a pre-capped value use ParseRetryAfter (bounded at RetryAfterCap = 60s).

func (*RateLimitError) Error 

func (e *RateLimitError) Error() string

type RedirectOption 

type RedirectOption func(*redirectCfg)

RedirectOption configures a redirect policy created by RedirectPolicyFunc.

func WithAllowSchemeDowngrade 

func WithAllowSchemeDowngrade(allow bool) RedirectOption

WithAllowSchemeDowngrade permits a redirect that downgrades the scheme (https on the original request -> http on the target). The default (false) refuses such a downgrade so a credential carried in a custom request header (which Go forwards across a redirect, stripping only Authorization/Cookie) is never sent over a cleartext hop. A scheme upgrade (http->https) is always allowed regardless of this setting. The downgrade is judged against the ORIGINAL request's scheme (via[0]).

func WithAllowedHosts 

func WithAllowedHosts(hosts ...string) RedirectOption

WithAllowedHosts sets the exact hostnames allowed as redirect targets.

func WithAllowedSuffixes 

func WithAllowedSuffixes(suffixes ...string) RedirectOption

WithAllowedSuffixes sets the domain suffixes allowed (e.g. ".docker.com").

func WithMaxHops 

func WithMaxHops(n int) RedirectOption

WithMaxHops sets the maximum number of redirect hops. Default: 5.

func WithSameHost 

func WithSameHost() RedirectOption

WithSameHost additionally allows a redirect whose target host equals the original request's host (ASCII case-insensitive, RFC 3986 §6.2.2.1), in addition to any WithAllowedHosts / WithAllowedSuffixes entries. It is the building block for a same-origin policy: combined with the default scheme-downgrade refusal (see WithAllowSchemeDowngrade), it follows a service's own same-host redirects (including an http->https upgrade) while refusing a cross-host hop that would forward a custom auth header to another origin. A policy built with only WithSameHost (no allowlisted hosts) permits exactly the same-host set.

type ResponseTooLargeError 

type ResponseTooLargeError struct {
	Limit int64
}

ResponseTooLargeError is returned by GetBytes when the response body exceeds the configured maximum (WithMaxBodyBytes, default DefaultMaxBodyBytes). The body is not returned: a truncated payload indistinguishable from a complete one is a silent-corruption hazard, so GetBytes fails loud instead. Limit is the cap that was exceeded, mirroring the stdlib *http.MaxBytesError shape.

func (*ResponseTooLargeError) Error 

func (e *ResponseTooLargeError) Error() string

type RetryAfterHint 

type RetryAfterHint interface {
	RetryAfterHint() time.Duration
}

RetryAfterHint is implemented by errors that carry an explicit wait duration for the next retry, typically a parsed and capped Retry-After. When fn's returned error is transient AND implements this interface with a positive duration, Do waits that duration before the next attempt instead of its jittered exponential backoff. The exponential base still advances, so a later transient error without a hint resumes the normal progression. The hint MUST already be capped by the implementer (e.g. via ParseRetryAfter); Do sleeps on it verbatim and applies no ceiling of its own, so an uncapped value is an unbounded-wait hazard.

type RetryRoundTripper 

type RetryRoundTripper struct {
	// contains filtered or unexported fields
}

RetryRoundTripper implements http.RoundTripper with automatic retry.

By default, only idempotent methods (GET, HEAD, OPTIONS, TRACE) are retried. Use TransportConfig.RetryNonIdempotent to also retry POST/PUT/PATCH/DELETE when the request has a GetBody function for body replay.

Inspired by hashicorp/go-retryablehttp, but operates directly on stdlib *http.Request without a custom request type, and counts TOTAL attempts (TransportConfig.MaxAttempts) rather than go-retryablehttp's retries-beyond-first.

func NewRetryRoundTripper 

func NewRetryRoundTripper(next http.RoundTripper, cfg TransportConfig) *RetryRoundTripper

NewRetryRoundTripper creates a RetryRoundTripper wrapping next with the given configuration. If next is nil, http.DefaultTransport is used. TransportConfig{} gives the defaults (see its field docs).

func (*RetryRoundTripper) RoundTrip 

func (rt *RetryRoundTripper) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper. It retries eligible requests on transient failures with jittered exponential backoff, honoring Retry-After. Per the http.RoundTripper contract, the caller's request is never mutated.

When retries are exhausted the final response is returned, not an error: if the last attempt produced a retryable response (e.g. a 503), RoundTrip returns that *http.Response with a nil error, exactly as a non-retried request would. The caller owns the returned body (must Close it) and MUST inspect resp.StatusCode - a nil error does not imply a 2xx response.

type StatusError 

type StatusError struct {
	URL  string
	Code int
}

StatusError represents a non-2xx response with URL context. Used by GetBytes. Supports errors.Is matching against ErrRateLimited and ErrServerError.

func (*StatusError) Error 

func (e *StatusError) Error() string

func (*StatusError) Is 

func (e *StatusError) Is(target error) bool

Is reports whether this StatusError matches ErrRateLimited or ErrServerError.

type Transient 

type Transient interface {
	IsTransient() bool
}

Transient is the interface for errors that can report whether they represent a transient (retryable) failure.

type TransportConfig 

type TransportConfig struct {
	// CheckRetry overrides the retry policy. nil means the default policy:
	// transient transport errors plus 429/502/503/504 responses (deliberately
	// narrower than GetBytes, which retries every 5xx; supply a custom policy
	// to broaden the set when an upstream returns transient 500s).
	CheckRetry CheckRetry
	// OnRetry is a hook called before each retry attempt, the transport's only
	// observability seam (it logs nothing itself).
	OnRetry OnRetry
	// PrepareRetry is called before each retry to mutate the cloned request
	// (e.g. refresh an auth token).
	PrepareRetry PrepareRetry
	// MaxAttempts is the TOTAL attempt count including the initial request.
	// Zero means unset and takes DefaultMaxAttempts (3); a NEGATIVE value
	// means exactly one attempt (the "try once" configuration — v2 expressed
	// it as WithRTMaxAttempts(0), but a zero struct field cannot distinguish
	// absent from zero, so v3 moves try-once to negatives).
	MaxAttempts int
	// BaseDelay is the initial backoff delay; non-positive means
	// DefaultBaseDelay (1s). Waits are equal-jitter with overflow-safe
	// doubling, and a retryable response's Retry-After header (capped at
	// RetryAfterCap) overrides the computed wait.
	BaseDelay time.Duration
	// MaxElapsedTime is a hard ceiling on total time across retries,
	// including any honored Retry-After: when now+wait would meet or pass it,
	// the round-tripper aborts with an error instead of sleeping. It is
	// checked BETWEEN attempts and cannot interrupt a stalled in-flight
	// attempt; bound single attempts on the base transport (e.g.
	// ResponseHeaderTimeout) and the whole call with the request context.
	// Zero means no ceiling.
	MaxElapsedTime time.Duration
	// RetryNonIdempotent enables retry of non-idempotent methods (POST, PUT,
	// PATCH, DELETE) when the request has a GetBody function for body replay.
	RetryNonIdempotent bool
}

TransportConfig configures a RetryRoundTripper (and, through NewRetryClient, the retrying client). The zero value is ready to use and behaves exactly like an unconfigured v2 round-tripper: three total attempts, one-second base delay, the default retry policy, no hooks, no elapsed ceiling, no non-idempotent replay.

type Validators 

type Validators struct {
	// ETag is replayed as If-None-Match.
	ETag string
	// LastModified is replayed as If-Modified-Since.
	LastModified string
}

Validators carries the cache validators captured from a previous 200 response, replayed on the next conditional request so an unchanged resource is a cheap 304 instead of a re-download. Persist them alongside the cached body; the zero value sends no conditional headers (forcing a full 200).

Source Files

View all Source files

Directories

Path Synopsis
Package certtest provides throwaway self-signed certificate material for tests.
 Package certtest provides throwaway self-signed certificate material for tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.