phoebe

module

v0.0.0-...-9924381 Latest Latest Go to latest Published: Jul 6, 2026 License: MIT

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/saturncloud/phoebe

Links

Open Source Insights

README ¶

Phoebe

Phoebe is the token-metering interceptor for Saturn Cloud's token factory. It is a thin, tenant-aware reverse proxy that sits behind Traefik and in front of the (optional) inference router / vLLM engine:

Traefik → atlas-auth (ForwardAuth) → Phoebe → [vLLM prod router | llm-d] → vLLM / SGLang / TensorRT-LLM

Phoebe is the always-present capture point for token metering and billing. It does not authenticate or authorize — that happens at the edge — it trusts the identity headers atlas-auth injects.

What it does

Identity (read, don't re-auth). Reads the trusted X-Saturn-User-Id, X-Saturn-Group-Id, X-Saturn-Resource-Id, X-Saturn-Resource-Type headers. See internal/identity.
Per-model dispatch. Resolves the target model from X-Saturn-Resource-Id to an upstream URL, with no redeploy needed for new models and a clean 404/410 for torn-down ones. See internal/registry.
Token metering capture. Captures token counts from the engine's own usage block (the authority — never re-tokenizes), forces stream_options.include_usage=true, and emits one immutable, idempotent metering event per request keyed by request_id. See internal/metering.
Streaming correctness. Forward-then-inspect SSE: streams each chunk to the client immediately, captures the trailing usage chunk, handles client aborts. See internal/proxy.

It is topology-independent: it behaves identically whether the upstream is an engine directly (Shape A) or a router (Shape B), and acts as the stable contract that makes everything below it swappable.

What it deliberately does NOT do

Authentication / authorization (edge, via atlas-auth).
Cache-aware routing across replicas (the inference router's job).
Format translation (the engines already serve OpenAI shapes).

Layout

cmd/interceptor/    main entrypoint
internal/config/    YAML settings (defaults → unmarshal → parse)
internal/identity/  trusted X-Saturn-* header extraction
internal/logging/   leveled logger
internal/metering/  billing event schema + Emitter contract
internal/proxy/     the reverse-proxy core (streaming tee, dispatch)
internal/registry/  model → upstream resolution
config/             example settings

Local dev

make test    # go test ./...
make vet     # go vet ./...
make run     # build + run against config/settings.example.yaml
make build   # build to bin/phoebe

Status

Walking skeleton: identity → registry → upstream dispatch and the server lifecycle are wired and tested. The SSE forward-then-inspect tee, the include_usage forcing, client-abort handling, and the durable metering emitter (Kafka / Redis Streams) are stubbed with TODO(milestone-...) markers in internal/proxy and internal/metering.

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
cmd
drainer command Command drainer is the consumer side of phoebe's metering durability ladder.	Command drainer is the consumer side of phoebe's metering durability ladder.
engine-conformance command Command engine-conformance verifies that Phoebe's usage-block parsing (internal/metering.Usage) matches what a REAL OpenAI-compatible engine (vLLM / SGLang / TensorRT-LLM) actually emits — especially the cached-token field, which internal/metering flags as needing live verification.	Command engine-conformance verifies that Phoebe's usage-block parsing (internal/metering.Usage) matches what a REAL OpenAI-compatible engine (vLLM / SGLang / TensorRT-LLM) actually emits — especially the cached-token field, which internal/metering flags as needing live verification.
interceptor command
price-fetch command Command price-fetch is phoebe's price-book SYNC step: it pulls the customer's effective token prices from the central pricing service (saturn-aws-manager's GET /customer/token-prices) and installs them as the local YAML price file the rater (cmd/rater) loads.	Command price-fetch is phoebe's price-book SYNC step: it pulls the customer's effective token prices from the central pricing service (saturn-aws-manager's GET /customer/token-prices) and installs them as the local YAML price file the rater (cmd/rater) loads.
rater command Command rater is phoebe's REVENUE path batch job.	Command rater is phoebe's REVENUE path batch job.
token-push command Command token-push is phoebe's install-side hourly billing PUSH job.	Command token-push is phoebe's install-side hourly billing PUSH job.
internal
capture Package capture defines what the streaming tee extracts from an upstream response — the frozen handoff between the proxy's forward-then-inspect tee (M1) and the metering emitter (M2).	Package capture defines what the streaming tee extracts from an upstream response — the frozen handoff between the proxy's forward-then-inspect tee (M1) and the metering emitter (M2).
config Package config loads interceptor settings from a YAML file, applying defaults then parsing/validating.	Package config loads interceptor settings from a YAML file, applying defaults then parsing/validating.
drain Package drain is the CONSUMER side of the metering durability ladder.	Package drain is the CONSUMER side of the metering durability ladder.
emit Package emit provides a durable metering.Emitter that ships events via a three-level durability ladder:	Package emit provides a durable metering.Emitter that ships events via a three-level durability ladder:
identity Package identity reads the trusted identity headers that atlas-auth (Traefik ForwardAuth) injects upstream.	Package identity reads the trusted identity headers that atlas-auth (Traefik ForwardAuth) injects upstream.
iolog Package iolog is M5: per-tenant, opt-in, sampled, short-retention capture of request and response BODIES, forked from the same upstream bytes the metering tee already sees but written to a SEPARATE store with its own retention and access controls.	Package iolog is M5: per-tenant, opt-in, sampled, short-retention capture of request and response BODIES, forked from the same upstream bytes the metering tee already sees but written to a SEPARATE store with its own retention and access controls.
logging Package logging provides a simple leveled logger, mirroring the convention used across Saturn's Go services (see auth-server's util.Logger).	Package logging provides a simple leveled logger, mirroring the convention used across Saturn's Go services (see auth-server's util.Logger).
metering Package metering defines the immutable billing event and the Emitter contract.	Package metering defines the immutable billing event and the Emitter contract.
proxy Package proxy is the core of the interceptor: a thin, tenant-aware reverse proxy that sits behind Traefik and in front of the inference router/engine.	Package proxy is the core of the interceptor: a thin, tenant-aware reverse proxy that sits behind Traefik and in front of the inference router/engine.
rating Package rating is phoebe's REVENUE path.	Package rating is phoebe's REVENUE path.