httpkit

README

go-httpkit

---

HTTP client plumbing: capped requests, typed status and rate-limit errors, Retry-After, redirect policies, and resumable verified file downloads. Package httpkit, module github.com/Rethunk-Tech/go-httpkit.

A program that talks to several sites and downloads large files needs the same few things every time, and each is easy to get subtly wrong: a body read with no size bound, a Retry-After that overflows, an API key that follows a redirect to another host, a download that restarts from zero after a dropped connection. This module is the one correct copy of each.

Quick start

go get github.com/Rethunk-Tech/go-httpkit

A worked example and the contracts: HUMANS.md.

Features

• DoCapped reads an answer at most N bytes and turns every other status into a *StatusError that errors.Is matches against ErrNotFound and ErrRateLimited.
• RetryAfter reads both header forms and refuses values that would overflow a Duration.
• NoDowngrade, SameOrigin and NoFollow keep credentials where they belong; WithRedirect adds one to a client without losing the client's own.
• Transport sets a default User-Agent and reports every round trip.
• download.File resumes, verifies and size-limits a download, fails a stalled body with ErrIdle, and shares one bandwidth Limiter across downloads.
• download.RangeReader reads a remote file's parts through ranged GETs.

Documentation

Doc	Audience
HUMANS.md	Running and using the package
AGENTS.md	Internals, invariants, and why
CONTRIBUTING.md	Process — commits, tests, docs
SECURITY.md	Vulnerability disclosure
CHANGELOG.md	Release notes

License

MIT — see LICENSE. © 2026 Rethunk.Tech, LLC.

Documentation

Overview

Package httpkit is the HTTP client plumbing a program needs once it talks to more than one site: a size-capped request, typed status and rate-limit errors, one Retry-After reader, redirect policies that keep credentials with their origin, and a Transport that sets a default User-Agent and reports every round trip.

Resumable file downloads live in the download subpackage.

Index

• Variables
• func DoCapped(hc *http.Client, req *http.Request, limit int64, ...) ([]byte, error)
• func NoDowngrade(req *http.Request, via []*http.Request) error
• func NoFollow(*http.Request, []*http.Request) error
• func RetryAfter(v string) (at time.Time, ok bool)
• func SameOrigin(req *http.Request, via []*http.Request) error
• func WithRedirect(hc *http.Client, policy func(*http.Request, []*http.Request) error) *http.Client
• type RateLimitError
  • func (e *RateLimitError) Error() string
  • func (e *RateLimitError) Is(target error) bool
• type StatusError
  • func (e *StatusError) Error() string
  • func (e *StatusError) Is(target error) bool
• type Transport
  • func (t *Transport) CloseIdleConnections()
  • func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error)

Variables

var (
	// ErrNotFound is matched by a 404 *StatusError.
	ErrNotFound = errors.New("not found")
	// ErrRateLimited is matched by a 429 *StatusError and by every *RateLimitError.
	ErrRateLimited = errors.New("rate limited")
)

Functions

func DoCapped

func DoCapped(hc *http.Client, req *http.Request, limit int64, check func(*http.Response) error) ([]byte, error)

DoCapped sends req with hc (nil means http.DefaultClient) and returns the body of a 200 answer, at most limit bytes. check (nil means none) runs on every response before the status test; a non-nil error from it is returned after the body is closed. Any other status is a *StatusError; a body over limit is safefs.ErrTooLarge. The body is always closed.

func NoDowngrade

func NoDowngrade(req *http.Request, via []*http.Request) error

NoDowngrade refuses a redirect from https to any other scheme: net/http keeps the Authorization header across a same-host redirect, scheme change included. A first hop that is not https may redirect freely.

func NoFollow

func NoFollow(*http.Request, []*http.Request) error

NoFollow returns the redirect answer itself instead of following it.

func RetryAfter

func RetryAfter(v string) (at time.Time, ok bool)

RetryAfter reads a Retry-After value (RFC 9110 section 10.2.3): delta-seconds from now, or an HTTP-date. ok is false when v is empty, unparseable, "0" or negative, or more than 365 days of seconds (so the Duration cannot overflow). A past HTTP-date is returned as is; the caller decides what a reset not after now means.

func SameOrigin

func SameOrigin(req *http.Request, via []*http.Request) error

SameOrigin refuses a redirect to another scheme or host than the first request's, so a header such as an API key stays with its origin.

func WithRedirect

func WithRedirect(hc *http.Client, policy func(*http.Request, []*http.Request) error) *http.Client

WithRedirect returns a shallow copy of hc (nil means http.DefaultClient) whose CheckRedirect runs hc's own policy first, when it has one, and then policy. hc is never modified.

Types

type RateLimitError

type RateLimitError struct{ Reset time.Time }

RateLimitError is a server refusing calls until Reset (zero when it did not say).

func (*RateLimitError) Error

func (e *RateLimitError) Error() string

func (*RateLimitError) Is

func (e *RateLimitError) Is(target error) bool

Is matches ErrRateLimited.

type StatusError

type StatusError struct {
	Code   int
	Status string // resp.Status, e.g. "502 Bad Gateway".
}

StatusError is an answer whose status the caller did not accept.

func (*StatusError) Error

func (e *StatusError) Error() string

func (*StatusError) Is

func (e *StatusError) Is(target error) bool

Is matches ErrNotFound for a 404 and ErrRateLimited for a 429.

type Transport

type Transport struct {
	Base      http.RoundTripper // Nil means http.DefaultTransport.
	UserAgent string            // Set only when the request has none.
	// OnResult sees every round trip's response or transport error; nil means none. It must not read resp.Body.
	OnResult func(req *http.Request, resp *http.Response, err error)
}

Transport sets a default User-Agent and reports every round trip.

func (*Transport) CloseIdleConnections

func (t *Transport) CloseIdleConnections()

CloseIdleConnections forwards to Base when Base has the method, so http.Client.CloseIdleConnections still works.

func (*Transport) RoundTrip

func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip never modifies req: when it sets the User-Agent it does so on req.Clone(req.Context()).