gateway

Documentation

Overview

Package gateway is the embeddable serving core: the OpenAI-compat /v1 surface (chat completions, the models control plane, advisor and web tools) over the shared provider adapters, with a handful of small, generic extension points.

The doctrine this package encodes: the core carries the PRODUCT — serving, translation, the wire's typed error vocabulary — and nothing that exists only because someone operates a paid service around it. Anything service-shaped (who may call, what they may spend, where usage reports go, where per-user provider keys live) enters through the interfaces below. The in-tree implementations are the self-host ones: a local shared token, allow-everything authorization, usage as stdout JSON lines, provider keys from the environment. A hosted operator imports this package and supplies its own implementations; the composition is a Go dependency, never a fork.

The extension API is v0 and may change between minor releases.

Index

• Variables
• func IdentityOf(r *http.Request) (org, user string, keyedVendors []string)
• type Access
  • func (a Access) Keyed(vendor string) bool
• type AllowAll
  • func (AllowAll) Access(*Identity) Access
  • func (AllowAll) Authorize(*http.Request, *Identity) error
  • func (AllowAll) GateVendor(context.Context, *Identity, string) error
• type Authenticator
• type Authorizer
• type Extensions
  • func Resolve(opts ...Option) Extensions
• type Identity
• type KeySource
• type Option
  • func WithAuthenticator(a Authenticator) Option
  • func WithAuthorizer(a Authorizer) Option
  • func WithKeySource(k KeySource) Option
  • func WithRoutes(rs ...Route) Option
  • func WithUsageSink(s UsageSink) Option
• type Route
• type StatusError
  • func (e *StatusError) Error() string
• type UsageEvent
• type UsageSink

Variables

var ErrKeyInvalid = errors.New("key recently rejected by the vendor")

ErrKeyInvalid is returned by KeySource.Key when the key was recently proven bad (a vendor auth rejection) — the serving path fast-fails with the same message instead of re-burning a vendor call.

Functions

func IdentityOf

func IdentityOf(r *http.Request) (org, user string, keyedVendors []string)

IdentityOf returns the principal + user + keyed vendors stamped on the request by the auth middleware (empty strings before auth).

Types

type Access

type Access struct {
	// KeyedVendors are vendors this identity holds serving keys for via the
	// KeySource ("anthropic", "openai", …). Drives the per-model byok flags
	// on /v1/models and the keyed-first serving preference.
	KeyedVendors []string
	// LimitToKeyed means serving should require keyed vendors (the wire's
	// credits_exhausted fact). Always false in the default composition.
	LimitToKeyed bool
}

Access is the authorization/routing state the serving mechanism and the /v1/models control plane consume — separate from identity, produced by the Authorizer. The core maps it onto the wire's field names (credits_exhausted, per-model byok flags) as pure protocol translation; what makes the values true is the composition's business.

func (Access) Keyed

func (a Access) Keyed(vendor string) bool

Keyed reports whether vendor is in KeyedVendors.

type AllowAll

type AllowAll struct{}

AllowAll is the self-host Authorizer: everything an authenticated identity asks for is permitted, and no routing limits apply.

func (AllowAll) Access

func (AllowAll) Access(*Identity) Access

func (AllowAll) Authorize

func (AllowAll) Authorize(*http.Request, *Identity) error

func (AllowAll) GateVendor

func (AllowAll) GateVendor(context.Context, *Identity, string) error

type Authenticator

type Authenticator interface {
	Authenticate(r *http.Request) (*Identity, error)
}

Authenticator turns a request into an identity. A nil *Identity or a non-nil error refuses the request (401). Implementations own their own caching.

type Authorizer

type Authorizer interface {
	// Authorize is the door: called once per request after authentication.
	Authorize(r *http.Request, id *Identity) error
	// GateVendor is called on the serving path before a vendor serves a
	// non-keyed call.
	GateVendor(ctx context.Context, id *Identity, vendor string) error
	// Access reports the identity's routing/capability state.
	Access(id *Identity) Access
}

Authorizer approves work for an authenticated identity. Errors returned from Authorize/GateVendor surface as the wire's typed responses (the error VOCABULARY — 402 insufficient_credits and friends — is protocol and lives in the core; the policy deciding to raise them lives in the implementation).

type Extensions

type Extensions struct {
	Authenticator Authenticator
	Authorizer    Authorizer
	UsageSink     UsageSink
	KeySource     KeySource
	Routes        []Route
}

Extensions is the resolved option set (exported for the internal server's consumption; construct via Options).

func Resolve

func Resolve(opts ...Option) Extensions

Resolve applies opts over the self-host defaults.

type Identity

type Identity struct {
	// ID is the stable principal id (an org, a deploy key, or "selfhost").
	ID string
	// User sub-scopes ID when the principal distinguishes users ("" = none).
	User string
}

Identity is IDENTITY ONLY — who a request runs as. No capability, billing, or policy state rides on it (see Access).

type KeySource

type KeySource interface {
	Key(ctx context.Context, org, user, vendor string) (key, version string, err error)
	MarkInvalid(org, user, vendor string)
}

KeySource supplies per-identity provider keys — the mechanism behind the wire's byok lanes. Nil means the environment keys are the only keys.

type Option

type Option func(*Extensions)

Option configures New.

func WithAuthenticator

func WithAuthenticator(a Authenticator) Option

func WithAuthorizer

func WithAuthorizer(a Authorizer) Option

func WithKeySource

func WithKeySource(k KeySource) Option

func WithRoutes

func WithRoutes(rs ...Route) Option

func WithUsageSink

func WithUsageSink(s UsageSink) Option

type Route

type Route struct {
	Pattern string // e.g. "POST /v1/byok"
	Handler http.Handler
	Authed  bool
}

Route mounts an operator-specific endpoint on the server (composed via gateway/serve.New). Authed routes run behind the Authenticator/Authorizer like every core route.

type StatusError

type StatusError struct {
	Status int
	Code   string
	Msg    string
}

StatusError lets an Authorizer map a refusal onto a specific wire status + code (e.g. a hosted composition's 402 insufficient_credits). A plain error from Authorize is a 403.

func (*StatusError) Error

func (e *StatusError) Error() string

type UsageEvent

type UsageEvent struct {
	RequestID      string
	Purpose        string
	RequestedModel string
	ServedModel    string
	Backend        string
	InputTokens    int
	OutputTokens   int
	CacheRead      int
	CacheWrite     int
	SearchCount    int
	LatencyMS      int64
	// ViaKeySource marks a call served on a key the KeySource supplied for
	// this identity (rather than the gateway's own environment keys).
	ViaKeySource bool
	KeyVendor    string
}

UsageEvent is the OPERATIONALLY GENERIC record of one served call: counts, models, latency. It deliberately carries no pricing or commercial accounting — a composition's sink derives that on its own side.

type UsageSink

type UsageSink interface {
	Record(id *Identity, u UsageEvent)
}

UsageSink receives each served call's usage, after the response (asynchronous, best-effort). The core always also emits its stdout JSON usage line; a sink is for operators who need the event elsewhere.