modelproxy

Documentation

Overview

Package modelproxy is the host-side model egress proxy: it listens on a unix socket bound into the sandbox and forwards to the model API with a destination allowlist. It is the single outbound path — the sandbox has network=none.

The proxy is also the sole authenticator: the sandbox holds no model credentials, so the proxy strips any inbound auth header and injects the host-held credential for the upstream. The key lives only on the host and never enters the sandbox image or its environment.

Production hardening (see hardening.go) layers per-session/token rate caps, request/response audit records, and response secret redaction on top of the allowlist — all opt-in.

Index

• type AuditRecord
• type AuditSink
• type CredentialSource
• type Credentials
• type GcloudTokenSource
  • func (g *GcloudTokenSource) Token() (string, error)
• type Injector
  • func AnthropicInjector(apiKey, version string) Injector
  • func AzureKeyInjector(apiKey string) Injector
  • func AzureTokenInjector(ts TokenSource) Injector
  • func BedrockInjector(cs CredentialSource) Injector
  • func GeminiInjector(apiKey string) Injector
  • func GroqInjector(apiKey string) Injector
  • func LocalInjector(host, apiKey string) Injector
  • func MultiInjector(injectors ...Injector) Injector
  • func OllamaInjector(host, apiKey string) Injector
  • func OpenAIInjector(apiKey string) Injector
  • func OpenRouterInjector(apiKey string) Injector
  • func VertexInjector(ts TokenSource) Injector
• type Option
  • func WithAudit(sink AuditSink) Option
  • func WithIdentity(keyFn func(*http.Request) string) Option
  • func WithInjector(f Injector) Option
  • func WithInsecureUpstreams(hosts ...string) Option
  • func WithRateCap(rps float64, burst int) Option
  • func WithRedactedSecrets(secrets ...string) Option
  • func WithTransport(rt http.RoundTripper) Option
  • func WithUpstreamGateway(proxyURL string, insecureTLS bool) Option
• type Proxy
  • func New(allowedHosts []string, opts ...Option) *Proxy
  • func (p *Proxy) Handler() http.Handler
  • func (p *Proxy) Serve(ctx context.Context, socketPath string) error
• type StaticCredentials
  • func (s StaticCredentials) Credentials() (Credentials, error)
• type StaticTokenSource
  • func (s StaticTokenSource) Token() (string, error)
• type TokenSource

Types

type AuditRecord

type AuditRecord struct {
	Time          time.Time     `json:"time"`
	Session       string        `json:"session,omitempty"`
	Method        string        `json:"method"`
	Host          string        `json:"host"`
	Path          string        `json:"path"`
	Status        int           `json:"status"`
	Allowed       bool          `json:"allowed"`
	RateLimited   bool          `json:"rateLimited,omitempty"`
	RequestBytes  int64         `json:"requestBytes"`
	ResponseBytes int64         `json:"responseBytes"`
	Duration      time.Duration `json:"durationNanos"`
}

AuditRecord is one forwarded (or rejected) model-egress request. It carries no payload bytes — only metadata — so audit logs never become a secret sink.

type AuditSink

type AuditSink func(AuditRecord)

AuditSink receives one record per request. Implementations must not block the request path for long (write to a buffered logger/file).

type CredentialSource

type CredentialSource interface {
	Credentials() (Credentials, error)
}

CredentialSource yields the current AWS credential for Bedrock signing. Implementations own any refresh/caching (temporary credentials rotate); the injector calls it once per forwarded request. The credential never leaves the host.

type Credentials

type Credentials struct {
	AccessKeyID     string
	SecretAccessKey string
	SessionToken    string
}

Credentials is a host-held AWS credential used to SigV4-sign Bedrock requests. SessionToken is set only for temporary credentials (STS / IAM role); it becomes the X-Amz-Security-Token header. The credential lives only on the host — it is used to sign the outbound request and never enters the sandbox.

type GcloudTokenSource

type GcloudTokenSource struct {
	// TTL bounds how long a fetched token is reused before a refresh. gcloud tokens
	// last ~1h; a conservative default leaves headroom. Zero uses gcloudDefaultTTL.
	TTL time.Duration
	// contains filtered or unexported fields
}

GcloudTokenSource is a TokenSource that obtains a Vertex AI access token from the host's Application Default Credentials by execing `gcloud auth print-access-token` and caching it until shortly before it would expire. It adds no Go dependency and covers the standard local ADC paths (gcloud user login, a service-account key via GOOGLE_APPLICATION_CREDENTIALS, or the GCE metadata server). gcloud runs only on the host, never in the sandbox.

func (*GcloudTokenSource) Token

func (g *GcloudTokenSource) Token() (string, error)

Token returns a cached gcloud access token, refreshing it when the cache is empty or older than TTL. Concurrent callers serialize on the mutex; a refresh failure surfaces to the caller (VertexInjector treats it as "leave unauthenticated").

type Injector

type Injector func(upstreamHost string, req *http.Request)

Injector authenticates an outbound request to upstreamHost by setting the host-held credential headers on req. It runs after any sandbox-supplied auth has been stripped, so the proxy is the sole authenticator.

func AnthropicInjector

func AnthropicInjector(apiKey, version string) Injector

AnthropicInjector returns an Injector that authenticates requests to the Anthropic Messages API with a host-held API key and API version. The key lives only on the host (e.g. from ANTHROPIC_API_KEY) and never enters the sandbox.

func AzureKeyInjector

func AzureKeyInjector(apiKey string) Injector

AzureKeyInjector returns an Injector that authenticates requests to Azure OpenAI ({resource}.openai.azure.com) with a host-held API key via the `api-key` header (e.g. from AZURE_OPENAI_API_KEY). The key lives only on the host and never enters the sandbox. It self-guards on the Azure host suffix so it no-ops for any other provider — safe to compose through MultiInjector. An empty key is a no-op.

func AzureTokenInjector

func AzureTokenInjector(ts TokenSource) Injector

AzureTokenInjector returns an Injector that authenticates requests to Azure OpenAI with a host-held Microsoft Entra ID (Azure AD) OAuth2 bearer token obtained from ts, for deployments configured for Entra auth instead of a static key. Like VertexInjector the token is short-lived and refreshed host-side by ts; the sandbox never holds it. A token-source error leaves the request unauthenticated (the upstream rejects with 401) rather than failing closed inside the proxy; the error is never logged here to avoid leaking token material. It self-guards on the Azure host suffix so it no-ops for any other provider — safe to compose through MultiInjector.

func BedrockInjector

func BedrockInjector(cs CredentialSource) Injector

BedrockInjector returns an Injector that authenticates requests to the AWS Bedrock Runtime (bedrock-runtime.{region}.amazonaws.com) with AWS Signature Version 4, using a host-held credential from cs. The region is parsed from the upstream host so a single injector serves whatever region is allowlisted. Unlike the static-header providers, SigV4 signs the request body and selected headers, so the injector reads and re-buffers the body and stamps X-Amz-Date, X-Amz-Content-Sha256, the optional X-Amz-Security-Token, and Authorization.

It self-guards on the Bedrock host so it no-ops for any other provider — safe to compose through MultiInjector. It also strips any sandbox-supplied X-Amz-* headers before signing so the sandbox cannot inject headers into the signed set. A credential-source error or missing key leaves the request unsigned (the upstream rejects with 403) rather than failing closed inside the proxy; no credential material is ever logged.

func GeminiInjector

func GeminiInjector(apiKey string) Injector

GeminiInjector returns an Injector that authenticates requests to the Google Generative Language API (Google AI Studio / Gemini) with a host-held API key via the x-goog-api-key header (e.g. from GOOGLE_API_KEY or GEMINI_API_KEY). Like the others the key lives only on the host and never enters the sandbox. It self-guards on the upstream host so it no-ops for any other provider — safe to compose through MultiInjector.

func GroqInjector

func GroqInjector(apiKey string) Injector

GroqInjector returns an Injector that authenticates requests to the Groq API — an OpenAI-compatible inference provider — with a host-held API key via the Bearer scheme (e.g. from GROQ_API_KEY). It self-guards on the upstream host so it no-ops for any other provider — safe to compose through MultiInjector.

func LocalInjector

func LocalInjector(host, apiKey string) Injector

LocalInjector returns an Injector that authenticates requests to a local OpenAI-compatible model server (Ollama, LM Studio, vLLM, llama.cpp) with an OPTIONAL host-held API key via the Bearer scheme. Most local servers need no credential at all — pass an empty apiKey and this injector is a no-op (the proxy forwards with no Authorization header). A non-empty key is for the rare local server configured to require one (e.g. a vLLM deployment). It self-guards on the exact upstream host so it never leaks the key to any other provider. host is the allowlist entry (host or host:port); the key, when set, lives only on the host and never enters the sandbox.

func MultiInjector

func MultiInjector(injectors ...Injector) Injector

MultiInjector composes several provider injectors into one. Each injector self-guards on the upstream host, so for any given request exactly the matching provider's credential is stamped and the rest no-op. This is how the proxy authenticates a multi-provider allowlist (per-agent-group provider selection) with a single Injector. nil injectors are skipped.

func OllamaInjector

func OllamaInjector(host, apiKey string) Injector

OllamaInjector returns an Injector for the Ollama provider. Ollama needs NO credential — the whole point of the ollama backend is the zero-key local path — so with an empty apiKey this is a no-op and the proxy forwards requests to Ollama with no Authorization header. A non-empty key covers the rare case of an Ollama reached through an authenticating reverse proxy (e.g. a shared/remote Ollama behind a gateway that requires a Bearer token); the key then lives only on the host and never enters the sandbox.

Ollama is OpenAI-wire-compatible and authenticates (when it does at all) via the same Bearer scheme, so this delegates to LocalInjector, which self-guards on the exact upstream host and no-ops for every other provider — safe to compose through MultiInjector. It is a named alias so the ollama provider's auth path is explicit and greppable rather than piggybacking on "local".

func OpenAIInjector

func OpenAIInjector(apiKey string) Injector

OpenAIInjector returns an Injector that authenticates requests to the OpenAI API with a host-held API key via the Bearer scheme. Like AnthropicInjector the key lives only on the host (e.g. from OPENAI_API_KEY) and never enters the sandbox. It self-guards on the upstream host so it no-ops for any other provider — safe to compose through MultiInjector.

func OpenRouterInjector

func OpenRouterInjector(apiKey string) Injector

OpenRouterInjector returns an Injector that authenticates requests to the OpenRouter API — an OpenAI-compatible multi-model gateway — with a host-held API key via the Bearer scheme (e.g. from OPENROUTER_API_KEY). It self-guards on the upstream host so it no-ops for any other provider.

func VertexInjector

func VertexInjector(ts TokenSource) Injector

VertexInjector returns an Injector that authenticates requests to Google Cloud Vertex AI ({location}-aiplatform.googleapis.com) with a host-held OAuth2 bearer token obtained from ts. Unlike the static-API-key providers, the credential is a short-lived bearer that ts refreshes host-side; the sandbox never holds it. The injector self-guards on the Vertex host suffix so it no-ops for any other provider — safe to compose through MultiInjector. A token-source error leaves the request unauthenticated (the upstream rejects with 401) rather than failing closed inside the proxy; the error string is never logged here to avoid leaking token material.

type Option

type Option func(*Proxy)

Option configures a Proxy at construction.

func WithAudit

func WithAudit(sink AuditSink) Option

WithAudit installs an audit sink invoked once per request (allowed, denied, or rate-limited).

func WithIdentity

func WithIdentity(keyFn func(*http.Request) string) Option

WithIdentity sets how a request maps to a rate-limit / audit key (e.g. a session header). Default keys everything to "" (a single per-proxy bucket).

func WithInjector

func WithInjector(f Injector) Option

WithInjector sets the credential injector (the host-side authenticator).

func WithInsecureUpstreams

func WithInsecureUpstreams(hosts ...string) Option

WithInsecureUpstreams marks allowlisted hosts that the proxy reaches over plain HTTP instead of HTTPS, preserving any explicit port. This is for a local, loopback OpenAI-compatible model server (Ollama at localhost:11434, LM Studio, vLLM, llama.cpp) that serves the API without TLS — the "100% local, zero cloud credential" path. Hosts must also be on the allowlist (pass them to New). Never use this for a real remote upstream: plain HTTP would expose the request.

func WithRateCap

func WithRateCap(rps float64, burst int) Option

WithRateCap caps model egress at rps requests/second with a burst of burst, keyed by session/token identity (see WithIdentity; default is one bucket for the whole proxy, which equals a per-session cap since a proxy serves one sandbox). rps <= 0 disables it.

func WithRedactedSecrets

func WithRedactedSecrets(secrets ...string) Option

WithRedactedSecrets registers exact secret strings (e.g. the host API key) to scrub from forwarded response bodies and headers, so an upstream that echoes a credential cannot leak it into the sandbox. Empty strings are ignored.

func WithTransport

func WithTransport(rt http.RoundTripper) Option

WithTransport overrides the upstream RoundTripper (used in tests).

func WithUpstreamGateway

func WithUpstreamGateway(proxyURL string, insecureTLS bool) Option

WithUpstreamGateway routes every forwarded upstream request through an HTTP CONNECT proxy — an operator-vetted credential gateway (e.g. OneCLI) — instead of dialing the upstream directly. The gateway injects the real provider credential, so the sandbox AND this control-plane stay credential-free for that provider (set no Injector for those hosts). proxyURL may embed Basic credentials in its userinfo (e.g. http://x:<agent-token>@127.0.0.1:10255); net/http sends them as the Proxy-Authorization header on CONNECT. When insecureTLS is set, upstream TLS verification is skipped — required when the gateway terminates TLS (MITM) with its own CA. Intended for a loopback gateway only.

type Proxy

type Proxy struct {
	// contains filtered or unexported fields
}

Proxy forwards sandbox-originated requests to allowlisted upstream model hosts over a unix-domain socket. Any request to a host not on the allowlist is rejected with 403.

func New

func New(allowedHosts []string, opts ...Option) *Proxy

New constructs a Proxy whose allowlist is the given set of hostnames (host or host:port). Comparison is case-insensitive on the host portion.

func (*Proxy) Handler

func (p *Proxy) Handler() http.Handler

Handler returns the http.Handler that enforces the allowlist and reverse-proxies allowed requests to their upstream over HTTPS. It is exported so it can be mounted in tests without a real socket.

func (*Proxy) Serve

func (p *Proxy) Serve(ctx context.Context, socketPath string) error

Serve listens on socketPath (a unix-domain socket bound into the sandbox) and serves the allowlist-enforcing handler until ctx is cancelled. The socket file is removed on start (stale cleanup) and on stop.

type StaticCredentials

type StaticCredentials struct {
	AccessKeyID     string
	SecretAccessKey string
	SessionToken    string
}

StaticCredentials is a CredentialSource that always returns the same credential. Use it when the operator supplies AWS keys via the environment (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / optional AWS_SESSION_TOKEN). For temporary credentials the operator refreshes them out of band (e.g. a sidecar that re-execs the control-plane, or a rotating secret mount).

func (StaticCredentials) Credentials

func (s StaticCredentials) Credentials() (Credentials, error)

Credentials returns the fixed credential.

type StaticTokenSource

type StaticTokenSource string

StaticTokenSource is a TokenSource that always returns the same token. Use it when the operator supplies a Vertex access token via the environment (GOOGLE_VERTEX_ACCESS_TOKEN) and refreshes it out of band (e.g. a sidecar running `gcloud auth print-access-token` on a timer). The token expires on Google's schedule (~1h); for unattended auto-refresh prefer GcloudTokenSource.

func (StaticTokenSource) Token

func (s StaticTokenSource) Token() (string, error)

Token returns the fixed token.

type TokenSource

type TokenSource interface {
	Token() (string, error)
}

TokenSource yields the current OAuth2 bearer token for Vertex AI. Implementations are responsible for refresh and caching; Token is called once per forwarded request, so a static source returns a fixed value and an auto-refreshing source returns a cached, still-valid token. The token lives only on the host — it is injected into the upstream request and never enters the sandbox.