httpx

Documentation

Overview

Package httpx provides a shared, security-hardened HTTP client and helpers for use by secret verifiers.

It is intentionally placed under internal/verifier/internal so that it can only be imported by packages within internal/verifier.

Security rationale:

• Verifiers send provider credentials in custom headers (for example x-api-key, PRIVATE-TOKEN, DD-API-KEY) or embedded in the request URL (for example Telegram and Infura). On a cross-domain 3xx redirect, the Go standard library strips the Authorization header but NOT custom headers, and it re-sends the full URL — which would leak the credential to an attacker-controlled redirect target. To prevent this, the shared client does NOT follow redirects: it returns the 3xx response so the verifier can decide how to map it (see IsRedirect).

• Response bodies are read through a bounded reader (LimitReader) so a malicious or misbehaving endpoint cannot exhaust memory.

• The client does NOT set an http.Client.Timeout wall-clock ceiling. Request duration is governed solely by the per-request context deadline the verification engine applies (derived from the operator-configured verification.timeout). A client-level Timeout would silently cap that configured value and cannot be reconciled with it here, so it is omitted; callers MUST pass a context with a deadline.

• The client asserts an explicit TLS 1.2 minimum version rather than relying on the crypto/tls default, as defense-in-depth and self-documentation.

The shared token helper performs at most one Retry-After-aware HTTP 429 retry for safe GET/HEAD probes. The wait is strictly bounded, context-aware, and passes through an engine-owned admission gate immediately before every actual send when verification runs through the engine. Format-only, empty-input, and missing-context paths consume no limiter capacity. Unsafe methods and missing/invalid Retry-After responses are never retried.

Index

• Constants
• func BaseURL(override, fallback string) string
• func Client() *http.Client
• func IsRedirect(statusCode int) bool
• func LimitReader(r io.Reader) io.Reader
• func RateLimited(ctx context.Context, name, retryAfter string) finding.VerificationResult
• func RedactError(err error, secret string) string
• func UnexpectedStatus(ctx context.Context, name string, code int) finding.VerificationResult
• func VerifyToken(ctx context.Context, client *http.Client, token string, spec TokenSpec) finding.VerificationResult
• func WithRequestGate(ctx context.Context, gate RequestGate) context.Context
• func WithRetryGate(ctx context.Context, gate RetryGate) context.Context
• type DecodeFunc
• type InactiveDecodeFunc
• type Request
• type RequestGate
• type ResponseDecodeFunc
• type RetryGate
• type TokenSpec

Constants

const MaxBodyBytes int64 = 1 << 20

MaxBodyBytes is the maximum number of response-body bytes a verifier reads. It caps memory usage when decoding provider responses. 1 MiB is far larger than any legitimate verification response.

Functions

func BaseURL

func BaseURL(override, fallback string) string

BaseURL returns override when it is non-empty, otherwise fallback. Verifiers use it to honor a test-injected API base URL while defaulting to the real one.

func Client

func Client() *http.Client

Client returns the shared, security-hardened HTTP client.

The returned client is safe for concurrent use and is shared across all verifiers. Callers MUST NOT mutate it. Tests that need to point a verifier at a stub server should inject their own *http.Client through the verifier's test seam instead of mutating this client.

func IsRedirect

func IsRedirect(statusCode int) bool

IsRedirect reports whether the given HTTP status code is a 3xx redirect.

Because Client does not follow redirects, verifiers observe 3xx responses directly. A redirect from an API endpoint generally means the credential context is wrong (for example a wrong host or a login redirect), so it should NOT be treated as a successful verification.

func LimitReader

func LimitReader(r io.Reader) io.Reader

LimitReader wraps r so that at most MaxBodyBytes are read from it. Verifiers should decode response bodies through this reader (for example json.NewDecoder(httpx.LimitReader(resp.Body))) to bound memory usage.

func RateLimited

func RateLimited(ctx context.Context, name, retryAfter string) finding.VerificationResult

RateLimited returns a distinguished StatusVerifyError result for an HTTP 429 (Too Many Requests) response, so a provider-side rate limit is never conflated with a genuine verification bug or an inactive secret. Only a syntactically valid delta-seconds or HTTP-date Retry-After value is emitted; arbitrary provider-controlled header text is never copied into logs or results.

func RedactError

func RedactError(err error, secret string) string

RedactError returns err.Error() with every occurrence of secret replaced by "[REDACTED]".

Transport errors from net/http wrap a *url.Error whose message embeds the full request URL. When a verifier places a credential in the request URL (for example Telegram and Infura embed the token in the path, and Teams uses the webhook URL itself), a DNS, TLS, or proxy failure would otherwise echo that credential into logs and the returned VerificationResult.Message. Callers MUST route such error text through this helper before logging or returning it.

If secret is empty the original message is returned unchanged, since an empty match would otherwise corrupt the string. The returned text is safe to log.

func UnexpectedStatus

func UnexpectedStatus(ctx context.Context, name string, code int) finding.VerificationResult

UnexpectedStatus returns the canonical StatusVerifyError result for a response status code that a verifier does not recognize.

func VerifyToken

func VerifyToken(ctx context.Context, client *http.Client, token string, spec TokenSpec) finding.VerificationResult

VerifyToken performs the verification described by spec and maps the response to a VerificationResult. The token is checked for emptiness first (an empty credential is StatusUnverified, never an HTTP call). client may be nil, in which case the shared hardened Client is used.

func WithRequestGate

func WithRequestGate(ctx context.Context, gate RequestGate) context.Context

WithRequestGate returns a child context carrying the engine-owned admission gate used immediately before every HTTP attempt, including a bounded retry.

func WithRetryGate

func WithRetryGate(ctx context.Context, gate RetryGate) context.Context

WithRetryGate returns a child context carrying an engine-owned admission gate used only for an HTTP 429 retry. It supports manually gated multi-request verifiers whose initial/fallback sends call their verifier.RequestGate directly; standard httpx verifiers use WithRequestGate instead.

Types

type DecodeFunc

type DecodeFunc func(body io.Reader) (extra map[string]string, downgradeMessage string, err error)

DecodeFunc inspects an active-status (typically 200) response body. It returns the ExtraData to attach to a verified-active result. When it returns a non-empty downgradeMessage, VerifyToken instead reports verified-inactive with that message — used by APIs that return 200 with an "ok":false / "valid":false body. A non-nil error yields StatusVerifyError.

The reader passed to a DecodeFunc is already bounded by LimitReader.

type InactiveDecodeFunc

type InactiveDecodeFunc func(body io.Reader) error

InactiveDecodeFunc validates that an inactive-status response is definitive for the provider. A nil error permits verified-inactive; an error keeps the outcome fail-conservative as StatusVerifyError. It is intended for providers whose 401 class also includes challenges such as DPoP that do not prove a credential is invalid.

type Request

type Request struct {
	// Method defaults to GET when empty.
	Method string
	// URL is the fully-formed request URL.
	URL string
	// Body, when non-nil, is sent as the request body.
	Body []byte
	// Header holds additional request headers (for example the provider auth
	// header). User-Agent is always set automatically.
	Header map[string]string
	// BasicAuthUser and BasicAuthPass, when either is non-empty, set HTTP Basic
	// auth on the request (req.SetBasicAuth).
	BasicAuthUser string
	BasicAuthPass string
}

Request describes the logical HTTP probe a verifier sends to a provider. VerifyToken builds and performs it through the shared, security-hardened client; a safe GET/HEAD probe may be replayed once after a bounded HTTP 429.

type RequestGate

type RequestGate func() *finding.VerificationResult

RequestGate is installed by the verification engine for standard httpx verifiers. It admits each actual send through both provider and global rate limiters. Returning a non-nil result rejects the request before client.Do.

type ResponseDecodeFunc

type ResponseDecodeFunc func(header http.Header, body io.Reader) (extra map[string]string, downgradeMessage string, err error)

ResponseDecodeFunc is the header-aware counterpart to DecodeFunc. It is for provider metadata that is available only in response headers (for example GitHub OAuth scopes and token expiry). Header values are untrusted provider input and must never be copied wholesale into ExtraData.

type RetryGate

type RetryGate func() *finding.VerificationResult

RetryGate is installed by the verification engine and admits one retry at its actual send point through both the provider and global rate limiters. Returning a non-nil result rejects the retry without sending it.

type TokenSpec

type TokenSpec struct {
	// Name identifies the verifier in structured logs, for example "openai".
	Name string

	// Request is the provider request to send.
	Request Request

	// Redact is an optional additional sensitive value stripped from error text.
	// VerifyToken always redacts its token argument; use this field only when a
	// request contains another credential-bearing representation.
	Redact string

	// ActiveStatuses are the HTTP status codes mapped to verified-active.
	// Defaults to {200} when nil.
	ActiveStatuses []int

	// InactiveStatuses are the HTTP status codes mapped to verified-inactive.
	// Defaults to {401} when nil. Pass a non-nil empty slice ([]int{}) for
	// verifiers that decide inactive solely from the response body, so that no
	// status code maps to inactive (any unexpected code is a verify error).
	InactiveStatuses []int

	// ActiveMessage and InactiveMessage are the result messages for an
	// active / inactive outcome.
	ActiveMessage   string
	InactiveMessage string

	// ActiveExtra is attached to an active result when Decode is nil. Use it for
	// verifiers that report static ExtraData (for example a key type) without
	// reading the response body. Ignored when Decode is set.
	ActiveExtra map[string]string

	// Decode, when non-nil, is invoked on an active-status response body to
	// extract ExtraData (and optionally downgrade the result). When nil, an
	// active-status response yields a bare active result without reading the body.
	Decode DecodeFunc

	// DecodeResponse is mutually exclusive with Decode and receives both the
	// active response headers and bounded body. Use it only for explicitly
	// validated, non-secret provider metadata.
	DecodeResponse ResponseDecodeFunc

	// DecodeInactive, when non-nil, must positively validate an inactive-status
	// response body before the credential is classified as inactive. The body is
	// read completely through the same strict size bound used for active bodies.
	DecodeInactive InactiveDecodeFunc

	// RequireCompleteBody reads the full active response through a strict
	// MaxBodyBytes+1 bound before Decode runs. Responses over the bound are
	// rejected instead of letting a truncated prefix appear valid.
	RequireCompleteBody bool

	// RequireJSONContentType rejects an active response unless its media type is
	// application/json or an application/*+json subtype. The raw header is never
	// reflected into logs or result messages.
	RequireJSONContentType bool
}

TokenSpec describes a standard token verification probe: the request to send and how each response status maps to a VerificationResult.

The shared flow — User-Agent, no-redirect handling, bounded body, error redaction, and the canonical "unexpected status code" / "failed to decode" results — lives in VerifyToken, so each verifier declares only what is provider-specific. This keeps the ~50 verifier packages free of the near-identical request/response boilerplate they previously duplicated.