gttp

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 1, 2026 License: MIT Imports: 19 Imported by: 0

README

gttp

ci Go Reference

A robust HTTP client for Go with good defaults and tunable behavior. Called gttp because it's a "go http" client library.

gttp.New() returns a standard *http.Client with sensible timeouts, connection pooling, retry logic, and safety guards built in.

Features

  • Retry with exponential backoff + full jitter, honoring Retry-After / RateLimit-Reset
  • HTTP/2 health-check pings, TLS 1.2+ enforced, session cache
  • Request/response body idle-timeout (stops slow-loris)
  • Minimum transfer rate watchdog (opt-in; curl --speed-limit equivalent)
  • Decompression-bomb guard (1000:1 ratio default)
  • Response-body size cap (opt-in)
  • Redirect loop detection, scheme-downgrade refusal
  • SSRF filter on redirects (loopback / private / CGNAT / NAT64 / link-local / IMDS)
  • Sensitive-header scrubbing on cross-origin redirects
  • Typed error sentinels; every failure mode has an errors.Is target

Usage

// Use the defaults:
client := gttp.New()
resp, err := client.Get("https://example.com")

// Tune for your environment:
client := gttp.New(
    gttp.WithTimeout(10 * time.Second),
    gttp.WithRetries(5),
    gttp.WithIdleTimeout(10 * time.Second),
    gttp.WithMaxResponseBodyBytes(100 << 20),
    gttp.WithStrictSSRFProtection(),
)

See godoc for the full option list.

Documentation

Overview

Package gttp provides a robust HTTP client with reasonable defaults and tunable behavior.

The returned *http.Client is fully standard — callers use client.Do, client.Get, etc. Built-in protections:

  • Retry with exponential backoff + jitter, Retry-After honoring
  • HTTP/2 health-check pings (detects black-holed connections)
  • TLS 1.2+ minimum, session cache by default
  • Idle timeout on request-body writes and response-body reads (30s)
  • Decompression-bomb guard (1000:1 ratio)
  • Redirect loop detection, scheme-downgrade refusal, SSRF filter on private / loopback / link-local / CGNAT / NAT64 / IMDS addresses

All defaults can be overridden via Option values. See the With* options below.

Basic usage: 

client := gttp.New() // be sure to reuse this single object across multiple requests!
resp, err := client.Get("https://example.com")

With options: 

client := gttp.New(
    gttp.WithTimeout(10 * time.Second),
    gttp.WithRetries(5),
    gttp.WithAdditionalRetryableStatusCodes(500),
)

Index

Constants

View Source 
const (
	DefaultTimeout               = 30 * time.Second
	DefaultMaxRedirects          = 5
	DefaultMaxIdleConns          = 20
	DefaultMaxIdleConnsPerHost   = 20
	DefaultMaxConnsPerHost       = 100
	DefaultIdleConnTimeout       = 90 * time.Second
	DefaultTLSHandshakeTimeout   = 5 * time.Second
	DefaultResponseHeaderTimeout = 10 * time.Second
	DefaultDialTimeout           = 5 * time.Second
	DefaultDialKeepAlive         = 30 * time.Second
	DefaultMaxRetries            = 3
	DefaultRetryWaitMin          = 250 * time.Millisecond
	DefaultRetryWaitMax          = 2 * time.Second
	DefaultExpectContinueTimeout = 2 * time.Second
	DefaultMaxRetryBodyBytes     = 4 << 20 // 4 MiB
	DefaultMaxRetryAfter         = 1 * time.Minute
	DefaultIdleTimeout           = 30 * time.Second
	DefaultMaxCompressionRatio   = 1000.0
)

Default configuration values. All can be overridden via Option values.

View Source 
const (
	DefaultHTTP2ReadIdleTimeout = 30 * time.Second
	DefaultHTTP2PingTimeout     = 15 * time.Second
)

Default HTTP/2 health-check values. ReadIdleTimeout triggers a PING when no frame has been received on the connection for that long; PingTimeout governs how long we wait for the PING response before tearing the connection down. Without these, a dead half-open HTTP/2 connection (e.g. killed silently by a load balancer) sits idle in the pool and returns errors only on the next use — the classic "black-hole" failure mode.

Variables

View Source 
var (
	// ErrBodyTooLarge is returned when a request body exceeds the retry
	// buffer limit (see WithMaxRetryBodyBytes) and the body does not
	// already provide a GetBody function for rewinding.
	ErrBodyTooLarge = errors.New("gttp: request body exceeds retry buffer limit")

	// ErrBodyRead is returned when reading the request body into the retry
	// buffer fails.
	ErrBodyRead = errors.New("gttp: reading request body for retry")

	// ErrBodyClose is returned when closing the original request body (after
	// buffering it for retry) fails.
	ErrBodyClose = errors.New("gttp: closing request body")

	// ErrBodyRewind is returned when rewinding the request body between
	// retry attempts fails (req.GetBody returned an error).
	ErrBodyRewind = errors.New("gttp: rewinding request body")

	// ErrTooManyRedirects is returned when the redirect chain exceeds the
	// configured maximum (see WithRedirectPolicy).
	ErrTooManyRedirects = errors.New("gttp: too many redirects")

	// ErrBodyIdleTimeout is returned when a response body read or a request
	// body write stalls for longer than the configured idle timeout
	// (see WithIdleTimeout).
	ErrBodyIdleTimeout = errors.New("gttp: body idle timeout")

	// ErrBodyTransferTooSlow is returned when the rolling average transfer
	// rate of the response body falls below the configured floor
	// (see WithMinTransferRate).
	ErrBodyTransferTooSlow = errors.New("gttp: body transfer rate below minimum")

	// ErrResponseTooLarge is returned when the decompressed response body
	// exceeds the configured maximum size (see WithMaxResponseBodyBytes).
	ErrResponseTooLarge = errors.New("gttp: response body exceeds max size")

	// ErrDecompressionBomb is returned when the ratio of decompressed to
	// compressed bytes exceeds the configured maximum
	// (see WithMaxCompressionRatio).
	ErrDecompressionBomb = errors.New("gttp: decompression ratio exceeded")

	// ErrRedirectLoop is returned when a redirect would revisit a URL
	// already seen in the current chain.
	ErrRedirectLoop = errors.New("gttp: redirect loop detected")

	// ErrSchemeDowngrade is returned when a redirect would move from https
	// to http without WithAllowSchemeDowngrade.
	ErrSchemeDowngrade = errors.New("gttp: redirect downgrades scheme https to http")

	// ErrBlockedByIPPolicy is returned when a redirect target's resolved IP
	// falls within one of the default-blocked ranges (private, loopback,
	// link-local, multicast, unique-local v6, CGNAT, NAT64, or IMDS addresses).
	ErrBlockedByIPPolicy = errors.New("gttp: target resolves to blocked IP range")
)

Sentinel errors returned by gttp. Callers can use errors.Is to distinguish them. All are wrapped with %w when returned, so the underlying cause (if any) remains reachable via errors.Unwrap / errors.As / errors.AsType.

Functions

func New 

func New(opts ...Option) *http.Client

New creates a new *http.Client with good defaults. All defaults can be overridden via Option values. As with all http.Clients, be sure to use the returned client across the lifetime of multiple requests.

Types

type BodyObservation 

type BodyObservation struct {
	URL               string
	StatusCode        int
	CompressedBytes   int64
	UncompressedBytes int64
	Duration          time.Duration
}

BodyObservation reports response-body byte counts when a guarded response body is closed. It is intentionally generic: callers that need request-specific labels can derive them from URL.

type Option 

type Option func(*config)

Option configures the HTTP client.

func WithAdditionalRetryableMethods 

func WithAdditionalRetryableMethods(methods ...string) Option

WithAdditionalRetryableMethods adds HTTP methods to the default retryable set without replacing it. For example, to also retry POST and PUT: 

gttp.New(gttp.WithAdditionalRetryableMethods("POST", "PUT"))

func WithAdditionalRetryableStatusCodes 

func WithAdditionalRetryableStatusCodes(codes ...int) Option

WithAdditionalRetryableStatusCodes adds status codes to the default retryable set without replacing it. For example, to also retry on 500: 

gttp.New(gttp.WithAdditionalRetryableStatusCodes(500))

func WithAllowPrivateRedirects 

func WithAllowPrivateRedirects() Option

WithAllowPrivateRedirects opts out of the SSRF redirect guard. When set, redirects to loopback / private / link-local / CGNAT / NAT64 / IMDS addresses are allowed.

func WithAllowSchemeDowngrade 

func WithAllowSchemeDowngrade() Option

WithAllowSchemeDowngrade opts out of refusing https -> http redirects.

func WithBodyObserver 

func WithBodyObserver(fn func(BodyObservation)) Option

WithBodyObserver registers a callback invoked when a guarded response body is closed. This is intended for temporary diagnostics where callers need precise response byte counts without changing higher-level APIs.

func WithCheckRetry 

func WithCheckRetry(fn func(req *http.Request, resp *http.Response, err error) bool) Option

WithCheckRetry provides a custom function to determine if a request should be retried. When set, this overrides the default status-code and error classification logic, but the method check still applies first — only methods in the retryable set are candidates for retry. Return true to retry, false to stop.

func WithDialContext 

func WithDialContext(fn func(ctx context.Context, network, address string) (net.Conn, error)) Option

WithDialContext provides a custom function for establishing TCP connections. When set, WithDialTimeout and WithDialKeepAlive are ignored since they configure the default dialer that this replaces.

WithResolver is likewise ignored in the general case, but not in strict direct mode: when WithStrictSSRFProtection and WithNoProxy are set without WithAllowPrivateRedirects, gttp resolves the request hostname itself with the configured resolver, validates the full answer set, and calls this function only with an already-validated literal IP address. In that mode WithResolver controls the validation lookup, and a custom dialer that performs its own name resolution never receives the original hostname. This is required to close the DNS-rebinding window between validation and dial.

This option is ignored when WithTransport is used.

func WithDialKeepAlive 

func WithDialKeepAlive(d time.Duration) Option

WithDialKeepAlive sets the TCP keep-alive interval for connections. Default: 30s.

func WithDialTimeout 

func WithDialTimeout(d time.Duration) Option

WithDialTimeout sets the maximum time to establish a TCP connection. Default: 5s (stdlib default is 30s).

func WithDisableCompression 

func WithDisableCompression() Option

WithDisableCompression disables transparent gzip decompression. The client will not add Accept-Encoding: gzip and will not decompress responses automatically. This can be useful when Content-Length must match the actual body size.

func WithDisableKeepAlives 

func WithDisableKeepAlives() Option

WithDisableKeepAlives disables HTTP keep-alives, making each request use a new connection. Useful for short-lived CLI tools.

func WithExpectContinueTimeout 

func WithExpectContinueTimeout(d time.Duration) Option

WithExpectContinueTimeout sets the maximum time to wait for a server's first response headers after fully writing the request headers if the request has an "Expect: 100-continue" header. Default: 2s.

func WithForceHTTP2 

func WithForceHTTP2(force bool) Option

WithForceHTTP2 controls whether HTTP/2 is attempted when a custom TLS config is set. Default: true.

func WithHTTP2PingTimeout 

func WithHTTP2PingTimeout(d time.Duration) Option

WithHTTP2PingTimeout sets how long to wait for a response to an HTTP/2 health-check PING before tearing the connection down. Default: 15s. Has no effect when WithForceHTTP2(false) or WithTransport is used.

func WithHTTP2ReadIdleTimeout 

func WithHTTP2ReadIdleTimeout(d time.Duration) Option

WithHTTP2ReadIdleTimeout sets the duration after which an HTTP/2 health-check PING is sent when no frame has been received on a connection. This detects silently-dropped connections (e.g., by a load balancer) that would otherwise sit in the idle pool forever. Set to 0 to disable health checks. Default: 30s. Has no effect when WithForceHTTP2(false) or WithTransport is used.

func WithIdleConnTimeout 

func WithIdleConnTimeout(d time.Duration) Option

WithIdleConnTimeout sets how long idle connections remain in the pool. Default: 90s.

func WithIdleTimeout 

func WithIdleTimeout(d time.Duration) Option

WithIdleTimeout sets the idle timeout applied to both response-body reads and request-body writes. If no bytes flow in either direction for this long, the request is cancelled with ErrBodyIdleTimeout. 0 disables. Default: 30s.

func WithMaxCompressionRatio 

func WithMaxCompressionRatio(r float64) Option

WithMaxCompressionRatio sets the maximum allowed decompressed:compressed ratio when gzip decoding is in effect. The guard activates only once at least 64 KiB of compressed bytes have been read, to avoid false positives on small responses. 0 disables. Default: 1000.

func WithMaxConnsPerHost 

func WithMaxConnsPerHost(n int) Option

WithMaxConnsPerHost sets the maximum total connections per host. 0 means unlimited. Default: 100.

func WithMaxIdleConns 

func WithMaxIdleConns(n int) Option

WithMaxIdleConns sets the maximum number of idle connections across all hosts. Default: 20.

func WithMaxIdleConnsPerHost 

func WithMaxIdleConnsPerHost(n int) Option

WithMaxIdleConnsPerHost sets the maximum number of idle connections per host. Default: 20 (stdlib default is 2).

func WithMaxResponseBodyBytes 

func WithMaxResponseBodyBytes(n int64) Option

WithMaxResponseBodyBytes sets a hard cap on the decompressed response body size. Reads past this limit fail with ErrResponseTooLarge. 0 disables (unlimited). Default: 0.

func WithMaxResponseHeaderBytes 

func WithMaxResponseHeaderBytes(n int64) Option

WithMaxResponseHeaderBytes sets the maximum number of response bytes that the transport will read looking for the header. 0 means no limit. This option is ignored when WithTransport is used.

func WithMaxRetryAfter 

func WithMaxRetryAfter(d time.Duration) Option

WithMaxRetryAfter sets the maximum duration that a server-directed wait hint will be respected. If the server requests a longer delay, it will be capped at this value. Applies to Retry-After, RateLimit-Reset (RFC 9745 draft), and X-RateLimit-Reset (vendor-specific). Values are also floored at the minimum retry wait time (see WithRetryWait). Default: 1 minute.

func WithMaxRetryBodyBytes 

func WithMaxRetryBodyBytes(n int64) Option

WithMaxRetryBodyBytes sets the maximum request body size (in bytes) that will be buffered into memory for retry support. Bodies larger than this limit cause an error when retries are enabled and the body is not already seekable. Set to 0 for no limit. Default: 4 MiB.

func WithMinTransferRate 

func WithMinTransferRate(bps int64, window time.Duration) Option

WithMinTransferRate sets a minimum average transfer rate (bytes per second) for the response body, measured over the given rolling window. If the observed rate stays below bps for a full window, the read fails with ErrBodyTransferTooSlow. Default: disabled. Matches curl --speed-limit / --speed-time.

func WithNoProxy 

func WithNoProxy() Option

WithNoProxy disables proxy support, making all connections direct. This option is ignored when WithTransport is used.

func WithNoRedirects 

func WithNoRedirects() Option

WithNoRedirects disables following redirects.

func WithNoRetries 

func WithNoRetries() Option

WithNoRetries disables retry logic entirely.

func WithProxy 

func WithProxy(fn func(*http.Request) (*url.URL, error)) Option

WithProxy sets a custom proxy function for the transport. The default is http.ProxyFromEnvironment. Use WithNoProxy to disable proxy support entirely. This option is ignored when WithTransport is used.

func WithRedirectPolicy 

func WithRedirectPolicy(n int) Option

WithRedirectPolicy sets the maximum number of redirects to follow. Set to 0 to disable redirects. Default: 5.

func WithResolver 

func WithResolver(r *net.Resolver) Option

WithResolver sets a custom DNS resolver on the default dialer. This is useful for directing DNS queries to a specific server (e.g., 1.1.1.1) without replacing the entire dial function. Example: 

gttp.New(gttp.WithResolver(&net.Resolver{
    PreferGo: true,
    Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
        return (&net.Dialer{}).DialContext(ctx, "udp", "1.1.1.1:53")
    },
}))

This option is ignored when WithTransport is used, and when WithDialContext is used outside strict direct mode. In strict direct mode (WithStrictSSRFProtection and WithNoProxy without WithAllowPrivateRedirects) it drives the validation lookup even alongside WithDialContext; see WithDialContext for details.

func WithResponseHeaderTimeout 

func WithResponseHeaderTimeout(d time.Duration) Option

WithResponseHeaderTimeout sets the maximum time to wait for response headers after the request is fully written. 0 means no limit. Default: 10s.

func WithRetries 

func WithRetries(n int) Option

WithRetries sets the maximum number of retries. 0 disables retries. Default: 3.

func WithRetryObserver 

func WithRetryObserver(fn func(attempt int, req *http.Request, resp *http.Response, err error)) Option

WithRetryObserver registers a callback that is invoked before each retry attempt. The attempt number is 0-indexed (0 = first failed attempt that will be retried). This is not called on the final exhausted attempt — only when a retry will actually follow. This is useful for logging or metrics.

func WithRetryWait 

func WithRetryWait(minWait, maxWait time.Duration) Option

WithRetryWait sets the minimum and maximum wait times between retries. Backoff is exponential with full jitter within these bounds. Default: 250ms min, 2s max.

func WithRetryableMethods 

func WithRetryableMethods(methods ...string) Option

WithRetryableMethods replaces the default retryable HTTP methods. Default: GET, HEAD, OPTIONS.

func WithRetryableStatusCodes 

func WithRetryableStatusCodes(codes ...int) Option

WithRetryableStatusCodes replaces the default retryable status codes. Default: 408, 425, 429, 502, 503, 504.

func WithSensitiveHeaders 

func WithSensitiveHeaders(names ...string) Option

WithSensitiveHeaders marks additional header names to strip when a redirect crosses origins. The stdlib already strips Authorization and cookies; this extends the set for bearer tokens, API keys, and so on.

func WithStrictSSRFProtection 

func WithStrictSSRFProtection() Option

WithStrictSSRFProtection also applies the IP policy to the initial request URL (not just redirects). Useful for services accepting attacker-controlled URLs.

Combine this with WithNoProxy to bind validation to the actual network connection: gttp resolves each hostname once per new connection, validates every returned address, and dials an approved literal address. With a proxy or custom transport, gttp cannot control the target dial and therefore retains request-time DNS preflight checks instead. The preflight path is also retained with WithAllowPrivateRedirects, whose redirect-specific exception cannot be represented safely by a connection-wide dial policy.

func WithTLSConfig 

func WithTLSConfig(cfg *tls.Config) Option

WithTLSConfig sets a custom TLS configuration on the default transport. A minimum TLS version of 1.2 is enforced regardless of the provided config. This option is ignored when WithTransport is used.

func WithTLSHandshakeTimeout 

func WithTLSHandshakeTimeout(d time.Duration) Option

WithTLSHandshakeTimeout sets the maximum time for TLS handshakes. Default: 5s.

func WithTimeout 

func WithTimeout(d time.Duration) Option

WithTimeout sets the overall client timeout (dial + TLS + headers + body). Default: 30s.

func WithTransport 

func WithTransport(rt http.RoundTripper) Option

WithTransport provides a custom base RoundTripper, bypassing the default transport construction. Retry logic and response-body guards (idle timeout, size cap, min-rate) are still applied on top, but note:

The decompression-bomb guard (WithMaxCompressionRatio) is effectively disabled when a custom transport is supplied, because gttp can no longer control the base transport's DisableCompression setting. The caller's transport is presumed to handle Accept-Encoding / gzip decoding itself, and once stdlib's default transport auto-decodes, the response arrives without a Content-Encoding header for gttp to act on. If you need the bomb guard, use the default transport.

func WithUserAgent 

func WithUserAgent(ua string) Option

WithUserAgent sets the User-Agent header on requests that don't already have one. By default, no User-Agent override is applied (the stdlib default is used).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.