handler

Documentation

Index

• Constants
• Variables
• func DefaultProxyTransport() *http.Transport
• func NewAnthropicProxyHandler(upstream *url.URL, transport http.RoundTripper) http.Handler
• func NewAuthSwapTransport(next http.RoundTripper, token string) http.RoundTripper
• func NewDisableTraceHandler() http.Handler
• func NewEnableTraceHandler() http.Handler
• func NewHealthzHandler() http.Handler
• func NewLoggingRoundTripper(inner http.RoundTripper, bodySampler liblog.Sampler, ...) http.RoundTripper
• func NewModelRouter(routes []ModelRoute, defaultProviderName string, defaultHandler http.Handler, ...) http.Handler
• func NewNotFoundHandler() http.Handler
• func NewRootLivenessHandler() http.Handler
• func NewSetLoglevelHandler() http.Handler
• func NewSetLoglevelHandlerWithRevert(autoRevert time.Duration) http.Handler
• func NewTraceMiddleware(next http.Handler, traceDir string, traceState *TraceState, ...) http.Handler
• func RedactBearerTokensInBody(b []byte) []byte
• func RedactHeadersForLog(h http.Header) map[string]string
• type Metrics
  • func NewMetrics(aliases map[string]string) *Metrics
  • func (m *Metrics) ObserveAliasResolution(alias, resolved string)
  • func (m *Metrics) ObserveRequest(provider, model string, status int, latencySeconds float64, isRouterError bool)
  • func (m *Metrics) ObserveTokens(provider, model, direction string, count int)
  • func (m *Metrics) Register(reg prometheus.Registerer) error
• type ModelRoute
• type TokenUsage
  • func ExtractUsage(tail []byte, contentType, contentEncoding string) (usage TokenUsage)
• type TraceState
  • func DefaultTraceState() *TraceState
  • func NewTraceState() *TraceState
  • func NewTraceStateWithTTL(ttl time.Duration) *TraceState
  • func (ts *TraceState) Disable()
  • func (ts *TraceState) Enable()
  • func (ts *TraceState) IsEnabled() bool

Constants

const (
	SetLoglevelDefault    = 1
	SetLoglevelAutoRevert = 5 * time.Minute
)

SetLoglevelDefault is the verbosity we revert to after autoRevertAfter. V(1) keeps the structured per-request line; the operator bumps to 2/3 for alias/route detail or to higher tiers for deep debug.

const BodySampleMaxBytes = 4 << 10 // 4 KB

BodySampleMaxBytes is the maximum number of bytes captured from a request or response body for V(4) body-sample logging. 4 KB is large enough to see the full JSON envelope of any ordinary Anthropic /messages request while staying small enough to avoid log-line bloat for large streaming bodies.

const MaxRequestBodyBytes = 32 << 20 // 32 MB

MaxRequestBodyBytes caps inbound /v1/* request bodies at 32 MB to match the Anthropic API ceiling. Long Claude Code sessions (full conversation history + tool definitions + sub-agent results) routinely exceed 1 MB, so a tighter cap surfaces as confusing 413s that read as upstream errors. 32 MB still bounds memory exhaustion from an accidental multi-GB upload via io.ReadAll. On overflow, the wrapped body returns *http.MaxBytesError; the router responds with HTTP 413 Request Entity Too Large + a generic body (no internal state leaked).

const TailBufferBytes = 2 << 20 // 2 MiB

TailBufferBytes caps the number of response-body bytes retained for post-request usage extraction. The bound must hold the FULL response body when the upstream sends `Content-Encoding: gzip` because gzip is not self-synchronizing — decompression requires the start-of-stream bytes, so a mid-stream tail is unrecoverable. Real Anthropic 200 responses have been observed at ~500 KB gzipped through Cloudflare (spec 006 Reproduction). 2 MiB is a frozen constant chosen to cover the observed p99 with headroom; it is NOT a YAML field (spec 006 Non-goals). A full 5 MB streaming response therefore occupies at most TailBufferBytes of additional memory.

const TraceTTLDefault = 5 * time.Minute

TraceTTLDefault is the production TTL window for trace enable. The TTL timer turns tracing off automatically after this duration. The 5-minute constant is a frozen invariant; TRACE_TTL is a test-only override and is never read by production code.

const UnknownModelLabel = "_unknown_"

UnknownModelLabel is the sentinel used in place of an empty model string when the request body has no `model` field (probe traffic, misshapen bodies) or the router-side early-return paths reject before body parse. The leading + trailing underscore avoids collision with real Anthropic + OpenRouter model names, which never start with `_`. This constant is a MODEL-label sentinel; the same string is also used as the provider label value from the three router-side early-return paths in NewModelRouter (see spec 007 Desired Behavior 6) — that reuse is deliberate and scoped, NOT a general fallback for other labels.

Variables

var LatencyBucketsSeconds = []float64{0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60}

LatencyBucketsSeconds covers expected LLM end-to-end latency distribution: sub-second for cached / token-counter calls, multi- second for streaming completions, up to ~60s for long contexts. Bucket boundaries match what a Grafana p95/p99 panel on `histogram_quantile(0.95, ...ccrouter_request_duration_seconds...)` can meaningfully resolve.

Functions

func DefaultProxyTransport

func DefaultProxyTransport() *http.Transport

DefaultProxyTransport returns an http.Transport with explicit timeouts suitable for upstream LLM API calls. Generous ResponseHeaderTimeout because long-generation requests (`/compact` on a large session, big code-gen prompts) can delay 60-300s before Anthropic sends the first byte of headers; short Dial because connections are quick HTTPS to api.anthropic.com.

Observed 2026-06-28: previous 60s ResponseHeaderTimeout produced `net/http: timeout awaiting response headers` 502s on `/compact` that took several minutes total. 300s (5 min) is generous enough for the worst observed case while still bounding a genuinely-wedged connection.

func NewAnthropicProxyHandler

func NewAnthropicProxyHandler(upstream *url.URL, transport http.RoundTripper) http.Handler

NewAnthropicProxyHandler returns an HTTP handler that reverse-proxies every incoming request to upstream (typically https://api.anthropic.com).

The Authorization header passes through unchanged — this is what lets Claude Code's subscription OAuth bearer travel through the router to Anthropic without the router ever holding it. No body parsing, no model-based routing: that's task 3. v1 of task 2 = single upstream, verbatim forward.

Upstream errors (connection refused, 5xx before body, etc.) are logged server-side with the full error string for debugging, but the client sees only a generic "502 Bad Gateway / upstream unavailable" — the internal error details (IPs, TLS handshake failures, connection strings) are not leaked.

If transport is nil, DefaultProxyTransport is used.

func NewAuthSwapTransport

func NewAuthSwapTransport(next http.RoundTripper, token string) http.RoundTripper

NewAuthSwapTransport wraps next so each outbound request has its Authorization header replaced with `Bearer <token>`. Used by the model router to swap the client's subscription OAuth bearer for a per-provider API token (MiniMax, Ollama, vLLM) before forwarding.

If token is empty, the wrapper is a no-op and returns next.

func NewDisableTraceHandler

func NewDisableTraceHandler() http.Handler

NewDisableTraceHandler returns a handler that turns per-request trace logging off immediately and cancels any in-flight TTL timer so no late reset can flip tracing back on. Mirrors the /setloglevel pattern.

func NewEnableTraceHandler

func NewEnableTraceHandler() http.Handler

NewEnableTraceHandler returns a handler that turns per-request trace logging on for a bounded 5-minute window (TraceTTLDefault). The window auto-disables on expiry; repeated calls reset the window. Mirrors the /setloglevel pattern: operator-local, no auth, short plaintext body.

func NewHealthzHandler

func NewHealthzHandler() http.Handler

NewHealthzHandler returns a handler that responds 200 with body "OK".

func NewLoggingRoundTripper

func NewLoggingRoundTripper(
	inner http.RoundTripper,
	bodySampler liblog.Sampler,
	currentDateTime libtime.CurrentDateTimeGetter,
) http.RoundTripper

NewLoggingRoundTripper wraps inner with upstream-call logging at three verbosity tiers:

• V(3) [upstream.headers]: emitted before the inner RoundTrip call; dumps the outbound request headers (after the auth-swap transport has applied its Authorization rewrite) as a JSON object with credential-shaped values redacted via RedactHeadersForLog. Useful for confirming exactly what token / headers reached the provider. Enable via `curl http://127.0.0.1:8788/setloglevel/3`.

• V(4) [upstream.start] / [upstream.end]: method+path on start; on end, adds TTFB (time-to-first-byte from when inner.RoundTrip was invoked until it returned with response headers) + status code (or error). Useful for debugging slow upstream behavior — distinguishes "Anthropic took 90s to send first byte" (high TTFB) from "body streaming was slow" (low TTFB, high total latency in the surrounding [req] line). Enable via `curl http://127.0.0.1:8788/setloglevel/4`.

• V(4) [upstream.req.body] / [upstream.resp.body]: body-sample lines gated by bodySampler (typically SamplerList{NewSampleTime(1s), NewSamplerGlogLevel(5)}). Captures up to BodySampleMaxBytes (4 KB) of the request body before forwarding it, and wraps the response body so the same size prefix is logged on Close. Bearer tokens are redacted via RedactBearerTokensInBody before the line is emitted.

If inner is nil, http.DefaultTransport is used (matches the nil-default pattern in NewAnthropicProxyHandler).

Silent at default V(1)-V(2).

func NewModelRouter

func NewModelRouter(
	routes []ModelRoute,
	defaultProviderName string,
	defaultHandler http.Handler,
	aliases map[string]string,
	sampler liblog.Sampler,
	metrics *Metrics,
	currentDateTime libtime.CurrentDateTimeGetter,
) http.Handler

NewModelRouter returns an HTTP handler that body-parses each request's JSON `model` field, resolves it through the aliases map (single-hop, case-sensitive exact match), then dispatches to the first matching ModelRoute. Unmatched models (and non-JSON / no-model requests) fall through to defaultHandler (logged as provider=defaultProviderName). The body is fully read and replayed for the downstream handler — fine for /v1/messages JSON payloads (typically <100 KB); not suitable for unbounded upload bodies.

aliases may be nil or empty — both mean "no alias rewriting". On a hit, the body's top-level .model field is re-marshaled to the resolved value before route dispatch, so the upstream sees the full model name.

One structured `[req]` log line per request at V(1):

[req] POST /v1/messages model=m3 alias=MiniMax-M3-highspeed provider=minimax status=200 latency=842ms

Non-200 responses are ALWAYS logged; 200 responses are gated by the sampler. `log.DefaultSamplerFactory` gives the canonical OR-combo: at most once per 10s, OR unconditionally when glog `-v` ≥ 4. This keeps the steady-state log readable while preserving every error event and giving full visibility once the operator bumps verbosity via `/setloglevel/4`.

At V(2), alias resolution and route match get their own `[alias]` / `[route]` detail lines (independent of the sampler — V(2) detail is already operator-opt-in, additional gating buys nothing).

match route → dispatch → observe metrics → emit logs. Each step's branching is local and reads sequentially; extracting any of it into a helper buys nothing (the prior `logReq` extraction was a naive line-count fix per architecture audit 2026-06-28 — inlined back). If a second log-event shape ever needs the same data, introduce a `requestLogger` struct holding `sampler` + `metrics` then.

func NewNotFoundHandler

func NewNotFoundHandler() http.Handler

NewNotFoundHandler returns a 404 handler that logs the unknown path before responding. Registered at `/` in the factory's mux so it catches everything not matched by a more specific route (`/v1/`, `/healthz`, `/readiness`, `/metrics`, `/setloglevel/`, `/gc`).

Logged at glog V(1) — same level as `[req]` — so unknown-path probes surface in the operator's default log alongside real traffic. Useful for catching misconfigured clients (wrong base URL, typo in `/v1/messages`) and any probing of the listener.

func NewRootLivenessHandler

func NewRootLivenessHandler() http.Handler

NewRootLivenessHandler returns a 200 OK handler for `HEAD /`.

Claude Code's HTTP client probes the base URL with `HEAD /` before dispatching its first /v1/messages request on a fresh connection. Without this handler the probe falls through to NewNotFoundHandler and logs a `[404] HEAD /` line ahead of every real request, which is noisy and obscures the actual `[req]` traffic.

func NewSetLoglevelHandler

func NewSetLoglevelHandler() http.Handler

NewSetLoglevelHandler returns a handler that flips glog's -v verbosity at runtime. Convenience wrapper for the production default (SetLoglevelAutoRevert = 5 min); tests use NewSetLoglevelHandlerWithRevert with a short window.

func NewSetLoglevelHandlerWithRevert

func NewSetLoglevelHandlerWithRevert(autoRevert time.Duration) http.Handler

NewSetLoglevelHandlerWithRevert returns a handler that flips glog's -v verbosity at runtime. URL shape is `/setloglevel/<level>`; the integer suffix is parsed and passed to `log.LogLevelSetter.Set`, which auto- reverts to SetLoglevelDefault after autoRevert so a forgotten bump can't leave the router in verbose mode indefinitely.

The LogLevelSetter is created once at handler construction (single instance, shared across requests); `Set()` itself spawns the auto- revert goroutine per call — that's upstream bborbe/log behavior, and `resetLogLevel` is idempotent so overlapping timers are harmless.

Level validation: negative values are rejected with 400; glog itself accepts any int32, but a negative verbosity is meaningless and almost always a typo (`-1` instead of `1`).

Example:

$ curl http://127.0.0.1:8788/setloglevel/3
set loglevel to 3 completed

Stdlib-mux compatible: parses the level from URL.Path directly rather than relying on gorilla/mux path vars.

func NewTraceMiddleware

func NewTraceMiddleware(
	next http.Handler,
	traceDir string,
	traceState *TraceState,
	configAlwaysOn bool,
) http.Handler

NewTraceMiddleware wraps next in a handler that, for each request, captures the full request (method, path, headers, body) and full response (status, headers, body) and writes one JSON file to traceDir — BUT only when trace is enabled. Trace is enabled when either the injected traceState flag is true (toggled via /enabletrace, bounded by a 5-min TTL) OR configAlwaysOn is true (the legacy trace: config flag, deprecated). Authorization and x-api-key request headers are redacted to "***" (case-insensitive); all other headers and bodies are logged verbatim. Trace file writes are best-effort: a write failure logs glog.Warningf and the request still succeeds. The trace directory is created on demand (MkdirAll, 0o700).

func RedactBearerTokensInBody

func RedactBearerTokensInBody(b []byte) []byte

RedactBearerTokensInBody returns a copy of b with every "Bearer <token>" substring replaced by "Bearer <redacted>". The replacement is case-insensitive on the "Bearer" keyword. Returns b unchanged (same slice, no allocation) when no Bearer token is found.

func RedactHeadersForLog

func RedactHeadersForLog(h http.Header) map[string]string

RedactHeadersForLog returns a flat map of header name → value suitable for JSON marshaling. Multi-value headers are joined with ", ". Credential-shaped headers (see isCredentialHeader) have their joined value replaced with "<redacted len=N>" where N is the byte-length of the original joined string.

This helper is designed for V(3) upstream-header logging so operators can see exactly what went on the wire without leaking tokens into log files.

Types

type Metrics

type Metrics struct {
	RequestsTotal    *prometheus.CounterVec
	RequestDuration  *prometheus.HistogramVec
	AliasResolutions *prometheus.CounterVec
	TokensTotal      *prometheus.CounterVec
}

Metrics groups the Prometheus collectors emitted from the model router: one CounterVec for request totals labeled by provider + model + status_class (`2xx`/`3xx`/`4xx_auth`/`4xx_rate_limited`/ `4xx_bad_request`/`5xx_upstream`/`5xx_router`), one HistogramVec for request latency labeled by provider + model with LLM-shaped buckets, one CounterVec for alias resolutions (operator-side observability for `/model qwen`-style short names), and one CounterVec for token counts labeled by provider + model + direction (`input`/`output`).

Cardinality budget: 5 providers × ~15 active models × 7 status_classes = 525 series ceiling for the requests counter (~450 in practice because some tuples never fire); 5 × 15 × 2 = 150 series for the tokens counter; histogram adds 5×15×len(buckets) = 750 series; aliases counter bounded by the YAML config (≤10). Total ~1.5k series — fine for a local Prometheus scrape.

func NewMetrics

func NewMetrics(aliases map[string]string) *Metrics

NewMetrics constructs the four collectors but does NOT register them. Call Register on a *prometheus.Registry to expose them; that split lets tests verify behavior against a fresh registry per spec without colliding on the global default registry.

aliases is used to pre-initialize the alias_resolutions counter so that `rate(...) > X` alerts evaluate to 0 (not no-data) for aliases that haven't been hit yet. A nil aliases map is safe — ranging a nil map yields zero iterations. TokensTotal is NOT pre-initialized: there is no closed set of (provider, model, direction) tuples known at boot; cardinality is bounded by real traffic.

func (*Metrics) ObserveAliasResolution

func (m *Metrics) ObserveAliasResolution(alias, resolved string)

ObserveAliasResolution increments the alias counter on each hit; labels are bounded by the YAML config's `aliases:` map size.

func (*Metrics) ObserveRequest

func (m *Metrics) ObserveRequest(
	provider, model string,
	status int,
	latencySeconds float64,
	isRouterError bool,
)

ObserveRequest is the call-site shorthand used by NewModelRouter after each /v1/* dispatch: increments the request counter with the 7-value status_class enum (see statusClass) and observes the latency on the histogram. isRouterError distinguishes 5xx router-side rejections (body-too-large, alias-rewrite-fail) from 5xx upstream faults — see spec 007 Desired Behavior 2 and 6. Callers on the happy path pass isRouterError=false.

func (*Metrics) ObserveTokens

func (m *Metrics) ObserveTokens(provider, model, direction string, count int)

ObserveTokens increments the ccrouter_tokens_total counter by count for the given (provider, model, direction) tuple. Drop rules — the call is a no-op (no series created) when:

• count <= 0 (zero-drop rule; zero is not a data point, negative is a schema violation)
• direction is not "input" or "output" (bounded-enum rule; keeps cardinality contract)

These drops are silent: token counting is best-effort observability, never on the request-serving critical path. See spec 007 Failure Modes for the sentinel/negative/zero rows this method absorbs.

func (*Metrics) Register

func (m *Metrics) Register(reg prometheus.Registerer) error

Register registers all collectors against reg. Pass a fresh registry in tests; pass the registry the /metrics endpoint scrapes in production. Returns the first registration error (if any) so caller can decide whether to abort startup.

type ModelRoute

type ModelRoute struct {
	Pattern      string
	ProviderName string
	Handler      http.Handler
}

ModelRoute pairs a glob pattern (filepath.Match syntax) with the provider name + handler to invoke when an incoming request's `model` field matches. ProviderName is what appears in the structured log (`provider=minimax`) and is the same key as in the YAML config's `providers:` map.

type TokenUsage

type TokenUsage struct {
	Input  string
	Output string
}

TokenUsage holds the input/output token counts extracted from an upstream response body. When extraction fails or no usage is present, Input and Output are the empty string and the caller logs the sentinel "-" for each (see logLineValue below). The empty string is the "no data" signal — a real zero-token count from the upstream is reported as "0" (the extractor reports what it parsed).

func ExtractUsage

func ExtractUsage(tail []byte, contentType, contentEncoding string) (usage TokenUsage)

extractUsage pulls input/output token counts out of a response-body tail buffer. SSE responses (Content-Type: text/event-stream OR bytes containing "event: message_") are scanned for the LAST `event: message_start` event (which carries input_tokens) and the LAST `event: message_delta` event (which carries output_tokens) and the two are combined into a single TokenUsage. Anthropic splits the two counts across these two events; the terminal message_delta is always the last chunk of a stream, so it lives in the tail. JSON responses are parsed for a top-level `usage` object.

SSE detection is by Content-Type OR content scan: the primary signal is strings.Contains(contentType, "text/event-stream"), but reverse-proxied Anthropic responses have been observed with an empty or wrong Content-Type on the sniffed rec.Header() (see spec 005). The fallback bytes.Contains(tail, "event: message_") is cheap (single scan over ≤ 64 KB) and specific enough to Anthropic's protocol to keep the false-positive risk low. JSON parsing is the fallback for any other content type.

Extraction is best-effort: truncated tails, malformed JSON/SSE, missing usage, or zero-token usage all yield the noUsage sentinel ("-" / "-") and never an error — the caller's [req] log line must never be aborted by a parse failure.

Presence detection: the extractor first unmarshals the `usage` JSON object into json.RawMessage; if that RawMessage is nil or "null", the field is treated as absent and noUsage is returned. Otherwise the inner input_tokens/output_tokens integers are parsed. This means a present `{"input_tokens":0,"output_tokens":0}` is correctly reported as "0"/"0", and a missing usage block returns the sentinel.

Partial-data policy:

• Both message_start and message_delta parseable -> TokenUsage{Input:"<N>", Output:"<M>"}
• Only message_start parseable -> TokenUsage{Input:"<N>", Output:""} (logs "in=<N> out=-")
• Only message_delta parseable -> TokenUsage{Input:"", Output:"<M>"} (logs "in=- out=<M>")
• Neither parseable -> noUsage (logs "in=- out=-")

type TraceState

type TraceState struct {
	// contains filtered or unexported fields
}

TraceState owns the process-global trace-enabled flag and its TTL timer. It guarantees exactly one live timer at any time: Enable() cancels the prior timer before starting a fresh window, and Disable() cancels the in-flight timer so no concurrent expiry can re-enable tracing. The flag is atomic for lock-free IsEnabled() reads on the request hot path.

func DefaultTraceState

func DefaultTraceState() *TraceState

DefaultTraceState returns the process-global trace-state instance.

func NewTraceState

func NewTraceState() *TraceState

NewTraceState returns a TraceState whose TTL window is the production constant TraceTTLDefault (5 minutes). Tests use NewTraceStateWithTTL with a shorter window.

func NewTraceStateWithTTL

func NewTraceStateWithTTL(ttl time.Duration) *TraceState

NewTraceStateWithTTL returns a TraceState whose TTL window is ttl. Tests use a short ttl for fast expiry assertions; production uses NewTraceState (TraceTTLDefault = 5 minutes).

func (*TraceState) Disable

func (ts *TraceState) Disable()

Disable sets the trace-enabled flag to false and cancels any in-flight TTL timer so no later expiry can flip tracing back on.

func (*TraceState) Enable

func (ts *TraceState) Enable()

Enable sets the trace-enabled flag to true, cancels any in-flight TTL timer, and starts a fresh TTL window. Repeated calls are idempotent on the flag but always reset the window to a fresh full ttl.

func (*TraceState) IsEnabled

func (ts *TraceState) IsEnabled() bool

IsEnabled returns the current trace-enabled flag value. This is the hot path consulted per request by the trace middleware; it performs an atomic load without locking.