proxy

package
v0.1.117 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 21, 2026 License: MIT Imports: 49 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	AccountImportEnabled  = "enabled"
	AccountImportDisabled = "disabled"
)

Account import states reported by /_subrouter/health.

View Source 
const AzureCodexDefaultDeploymentKey = "*"

AzureCodexDefaultDeploymentKey maps every unlisted model onto one deployment.

Variables

This section is empty.

Functions

func NewBedrockQuotaBumper added in v0.1.30 

func NewBedrockQuotaBumper(cfg aws.Config, logger *slog.Logger) *bedrockQuotaBumper

NewBedrockQuotaBumper builds a quota bumper that reacts to Bedrock throttling.

func NewOutboundTransport added in v0.1.14 

func NewOutboundTransport() *http.Transport

Types

type AccountRef added in v0.1.6 

type AccountRef struct {
	// contains filtered or unexported fields
}

func NewAccountRef added in v0.1.6 

func NewAccountRef(store accounts.CodexStore, initial []accounts.Account, client *http.Client) *AccountRef

func OpenAccountRef added in v0.1.52 

func OpenAccountRef(store accounts.CodexStore, claudeStore agentclaude.Store, client *http.Client) (*AccountRef, error)

OpenAccountRef loads accounts and their disk generation under the same cross-process transaction lock. Production callers use this constructor so worker startup can never make a stale snapshot look current.

func OpenAccountRefContext added in v0.1.52 

func OpenAccountRefContext(ctx context.Context, store accounts.CodexStore, claudeStore agentclaude.Store, client *http.Client) (*AccountRef, error)

OpenAccountRefContext loads one account snapshot while honoring cancellation if another process is currently committing an import transaction.

func (*AccountRef) All added in v0.1.6 

func (r *AccountRef) All() []accounts.Account

func (*AccountRef) FetchUsageWindowsCached added in v0.1.14 

func (r *AccountRef) FetchUsageWindowsCached(ctx context.Context, client *http.Client, account accounts.Account) ([]accounts.UsageWindow, bool, error)

FetchUsageWindowsCached is the single path for reading an account's usage windows. Every consumer (scheduler scoring, the usage-status sweep, auto-switch) used to fetch live, and with many pooled accounts the combined rate tripped the upstream usage endpoints' per-IP limits, which then cascaded: failed fetches zeroed scores and made healthy accounts look exhausted. Fresh-within-TTL returns the cache; a transient fetch failure falls back to the last-known-good windows instead of erroring.

The returned bool reports whether the windows are FRESH (a recent successful fetch) versus a STALE last-known-good fallback. Callers must not treat stale windows as a confident exhaustion signal: stale cooked data was overwriting healthy accounts' scores and routing traffic to dead accounts.

func (*AccountRef) Generation added in v0.1.52 

func (r *AccountRef) Generation() uint64

func (*AccountRef) InvalidateUsageStatusCache added in v0.1.14 

func (r *AccountRef) InvalidateUsageStatusCache()

func (*AccountRef) Refresh added in v0.1.6 

func (r *AccountRef) Refresh(ctx context.Context, account accounts.Account) (accounts.Account, error)

func (*AccountRef) Reload added in v0.1.9 

func (r *AccountRef) Reload() ([]accounts.Account, error)

func (*AccountRef) ReloadSnapshot added in v0.1.52 

func (r *AccountRef) ReloadSnapshot() ([]accounts.Account, uint64, error)

func (*AccountRef) ResolvedAccount added in v0.1.21 

func (r *AccountRef) ResolvedAccount(ctx context.Context, email string) (accounts.Account, bool, error)

ResolvedAccount returns a refreshed, token-bearing OAuth account for the given email, refreshing the stored token if it has expired. The bool is false (with no error) when no stored OAuth account matches email. It is the single path the rate-limit-reset handler uses to get a live credential for a server-owned account without going through the usage-status sweep.

func (*AccountRef) Snapshot added in v0.1.52 

func (r *AccountRef) Snapshot() ([]accounts.Account, uint64)

func (*AccountRef) Statuses added in v0.1.7 

func (r *AccountRef) Statuses(ctx context.Context, forceRefresh bool) []AccountStatus

func (*AccountRef) UsageStatuses added in v0.1.14 

func (r *AccountRef) UsageStatuses(ctx context.Context) []AccountUsageStatus

UsageStatuses serves a cached sweep when one is fresh, and backfills accounts whose live usage fetch transiently failed (the upstream usage endpoints rate-limit bursts) with their last-known-good windows, so brief 429s do not blank a healthy account's quota display.

type AccountStatus added in v0.1.7 

type AccountStatus struct {
	ID          string            `json:"id"`
	Provider    accounts.Provider `json:"provider"`
	AuthMode    accounts.AuthMode `json:"auth_mode"`
	Email       string            `json:"email,omitempty"`
	Source      string            `json:"source"`
	AuthChecked bool              `json:"auth_checked"`
	AuthValid   bool              `json:"auth_valid"`
	Refreshed   bool              `json:"refreshed,omitempty"`
	Error       string            `json:"error,omitempty"`
}

type AccountUsageStatus added in v0.1.14 

type AccountUsageStatus struct {
	AccountStatus
	Active             bool                             `json:"active,omitempty"`
	PlanType           string                           `json:"plan_type,omitempty"`
	Windows            []accounts.UsageWindow           `json:"windows,omitempty"`
	Credits            *accounts.CreditsInfo            `json:"credits,omitempty"`
	ComplimentaryReset *accounts.ComplimentaryResetInfo `json:"complimentary_reset,omitempty"`
	UsageFresh         bool                             `json:"-"`
}

type ActiveSessions added in v0.1.14 

type ActiveSessions struct {
	// contains filtered or unexported fields
}

func NewActiveSessions added in v0.1.14 

func NewActiveSessions() *ActiveSessions

func (*ActiveSessions) Active added in v0.1.14 

func (a *ActiveSessions) Active(agentType, sessionID string) bool

func (*ActiveSessions) Begin added in v0.1.14 

func (a *ActiveSessions) Begin(agentType, sessionID string) func()

type AzureCodexConfig added in v0.1.82 

type AzureCodexConfig struct {
	Endpoints []AzureCodexEndpoint
	// Models limits which requested models the fallback serves. Empty serves
	// every model. An entry matches the model exactly (case-insensitive), or
	// as a prefix when it ends with "*" ("gpt-5.6*"). The gate exists because
	// the fallback is metered and Azure trails the ChatGPT catalog: without
	// it, a default deployment quietly answers a request for one model with a
	// different one.
	Models []string
	// Transport is the outbound RoundTripper. Nil uses the server transport.
	Transport http.RoundTripper
	// PinStorePath persists which sessions are pinned to Azure. Without it the
	// pins live only in memory, so a restart sends a thread that has been
	// answered by Azure back to OpenAI carrying reasoning OpenAI cannot read.
	PinStorePath string
	// CostLogPath is the JSONL file each served request is priced into. Empty
	// disables cost accounting. Azure is metered, unlike the subscription pool,
	// so a fallback that silently spends money needs the same per-request
	// record the Bedrock gateway keeps.
	CostLogPath string
}

AzureCodexConfig is the Codex fallback route. It serves a Codex Responses request from Azure OpenAI after the subscription pool has failed, and pins the session to Azure afterwards so the follow-up turns keep hitting the same prompt cache instead of flapping between two providers.

type AzureCodexEndpoint added in v0.1.82 

type AzureCodexEndpoint struct {
	// Name labels the endpoint in logs (usually the region or resource name).
	Name string
	// BaseURL is the Azure OpenAI v1 base, e.g.
	// https://my-resource.openai.azure.com/openai/v1.
	BaseURL *url.URL
	// APIKey authenticates the call. Sent as both api-key and Authorization
	// bearer because the v1 surface accepts either.
	APIKey string
	// Deployments maps a requested model to the Azure deployment name. A model
	// with no entry uses its own name, which is how Foundry deployments are
	// named by default.
	Deployments map[string]string
}

AzureCodexEndpoint is one Azure OpenAI resource that can serve Codex Responses traffic. BaseURL is the v1 surface documented for Codex CLI (https://<resource>.openai.azure.com/openai/v1), which takes the same request body as api.openai.com with model set to the deployment name.

type BedrockConfig added in v0.1.30 

type BedrockConfig struct {
	// Regions is the ordered set of Bedrock runtime regions to try. List only
	// regions where the target inference profile has model access and TPM quota;
	// Bedrock 4xx model-access failures are terminal and are not retried.
	Regions     []string
	Credentials aws.CredentialsProvider
	Sources     []BedrockCredentialSource
	// GatewayToken, when non-empty, must be presented by clients via the
	// Authorization: Bearer header (Claude Code's ANTHROPIC_AUTH_TOKEN). Empty
	// means the endpoint relies on network-level trust like the rest of the proxy.
	GatewayToken string
	Transport    http.RoundTripper
	// CostLogPath is the JSONL file where per-request token usage and estimated
	// cost are appended. Empty disables cost tracking.
	CostLogPath string
	// Bumper, when set, requests a Service Quotas increase when Bedrock throttles
	// (HTTP 429), deduped per quota with a cooldown.
	Bumper *bedrockQuotaBumper
	// contains filtered or unexported fields
}

BedrockConfig configures the Bedrock signing gateway. When enabled, requests to /bedrock/* are re-signed with the team's AWS credentials (SigV4) and forwarded to bedrock-runtime, so clients (e.g. Claude Code in Bedrock gateway mode with CLAUDE_CODE_SKIP_BEDROCK_AUTH=1) never need AWS credentials.

type BedrockCredentialSource added in v0.1.30 

type BedrockCredentialSource struct {
	Name        string
	Credentials aws.CredentialsProvider
	Bumper      *bedrockQuotaBumper
}

type CredentialBroker added in v0.1.50 

type CredentialBroker interface {
	Lease(context.Context, broker.LeaseRequest) (broker.Lease, error)
	Report(context.Context, string, broker.LeaseReport) error
}

type Lifecycle added in v0.1.14 

type Lifecycle struct {
	// contains filtered or unexported fields
}

func NewLifecycle added in v0.1.14 

func NewLifecycle() *Lifecycle

func (*Lifecycle) ActiveProxyRequests added in v0.1.14 

func (l *Lifecycle) ActiveProxyRequests() int64

func (*Lifecycle) BeginProxyRequest added in v0.1.14 

func (l *Lifecycle) BeginProxyRequest() func()

func (*Lifecycle) Drain added in v0.1.14 

func (l *Lifecycle) Drain()

func (*Lifecycle) Draining added in v0.1.14 

func (l *Lifecycle) Draining() bool

func (*Lifecycle) Status added in v0.1.14 

func (l *Lifecycle) Status() map[string]any

type MultiTenant added in v0.1.50 

type MultiTenant struct {
	// Base is the template every tenant Server is copied from: upstreams,
	// transport, logger, request limits, usage-score TTL, and the shared
	// Lifecycle. Per-tenant state (accounts, sessions, scheduler, transcripts,
	// caches) is replaced per tenant.
	Base     Server
	Registry *tenant.Registry
	// TranscriptDir, when set, scopes each tenant's transcripts under
	// <TranscriptDir>/tenants/<id>.
	TranscriptDir string
	// Enabled is retained for --multi-tenant CLI compatibility. Tenant-shaped
	// credentials now always fail closed when they do not resolve.
	Enabled bool
	// StackVerifier enables normal-user tenant exchange at
	// /_subrouter/auth/stack. StackTenantKeySecret deterministically derives
	// the tenant path key after the verifier binds the request to a Stack team.
	StackVerifier interface {
		Verify(context.Context, string) (stackauth.Claims, error)
	}
	StackTeams interface {
		ListTeams(context.Context, string) ([]stackauth.Team, error)
	}
	StackProjectID         string
	StackTenantKeySecret   []byte
	StackTenantDeleteToken []byte
	StackLegacyKeyCutoff   time.Time
	PublicURL              string
	Now                    func() time.Time
	// contains filtered or unexported fields
}

MultiTenant routes requests carrying a tenant key to a per-tenant Server whose account store, scheduler, sticky sessions, and transcripts live under <state-dir>/tenants/<id>/. A key arrives either as a /t/<key>/... URL path prefix (agent CLIs can only override base URLs) or as an Authorization Bearer / x-api-key value (Claude Code sends ANTHROPIC_AUTH_TOKEN there). Requests without a tenant key fall through to the legacy single-tenant handler unchanged.

func (*MultiTenant) Handler added in v0.1.50 

func (m *MultiTenant) Handler(fallback http.Handler) http.Handler

Handler wraps the legacy single-tenant handler with tenant routing and the admin tenant CRUD endpoints.

type RateLimitResetResult added in v0.1.21 

type RateLimitResetResult struct {
	Email         string                         `json:"email"`
	Eligible      bool                           `json:"eligible"`
	Reset         bool                           `json:"reset"`
	DryRun        bool                           `json:"dry_run,omitempty"`
	Credit        *accounts.RateLimitResetCredit `json:"credit,omitempty"`
	WindowsBefore []accounts.UsageWindow         `json:"windows_before,omitempty"`
	WindowsAfter  []accounts.UsageWindow         `json:"windows_after,omitempty"`
	Error         string                         `json:"error,omitempty"`
}

RateLimitResetResult is the per-account outcome of a rate-limit reset request, whether the account was a single target or part of an --all sweep.

type ResetCreditsAccount added in v0.1.37 

type ResetCreditsAccount struct {
	Email   string                          `json:"email"`
	Count   int                             `json:"count"`
	Credits []accounts.RateLimitResetCredit `json:"credits,omitempty"`
	Error   string                          `json:"error,omitempty"`
}

ResetCreditsAccount is one account's redeemable rate-limit reset credits, including each credit's expiry so callers can see when they lapse.

type Server 

type Server struct {
	Upstream       *url.URL
	CodexUpstream  *url.URL
	APIUpstream    *url.URL
	ClaudeUpstream *url.URL
	KimiUpstream   *url.URL
	ZAIUpstream    *url.URL
	Accounts       []accounts.Account
	AccountRef     *AccountRef
	Sessions       *session.Store
	Scheduler      selectacct.Scheduler
	SchedulerRef   *selectacct.SchedulerRef
	UsageScoreTTL  time.Duration
	ScoreAccounts  func(context.Context, []accounts.Account) ([]selectacct.Score, int)
	// RefreshAccountFn, when set, replaces the default OAuth refresh path. Test
	// seam for simulating dead/expired refresh tokens; nil in production.
	RefreshAccountFn func(context.Context, accounts.Account) (accounts.Account, error)
	// CredentialBroker selects a team account and returns an access-only,
	// short-lived lease. When configured, local refresh-token stores and the
	// local scheduler are bypassed entirely.
	CredentialBroker    CredentialBroker
	Transport           http.RoundTripper
	Logger              *slog.Logger
	ActiveSessions      *ActiveSessions
	RequireSessionLease bool
	// ForwardSessionHeaders preserves the selected session identity across an
	// explicitly configured Subrouter-to-Subrouter delegation hop.
	ForwardSessionHeaders bool

	// StreamDrops counts dropped response streams by which side ended them,
	// so the expected client-hangup case is countable without a log line each.
	StreamDrops *StreamDropStats
	Lifecycle   *Lifecycle
	AdminToken  string
	// AccountImportToken authorizes only the protected account-import endpoint.
	// It is intentionally distinct from AdminToken, which can read operational
	// state and transcripts.
	AccountImportToken string
	// TailnetAuth authenticates callers with the tailnet itself instead of a
	// token, for self-hosted servers whose port is already restricted to a
	// tailnet by ACL. Identity comes from this machine's tailscaled, so it is
	// an assertion about a WireGuard-authenticated peer rather than a claim
	// carried in the request. Nil disables the mode, which is the default and
	// the only supported configuration for shared cloud deployments.
	TailnetAuth TailnetAuthorizer
	// LocalProxyToken protects provider proxy routes in cloud mode. Health and
	// readiness stay unauthenticated so supervisors can probe the daemon.
	LocalProxyToken string
	MaxBodyBytes    int64
	Transcripts     *transcript.Recorder
	// CacheFlight collapses identical concurrent requests to read-heavy
	// polling endpoints into one upstream fetch. Nothing is stored between
	// requests; see request_coalesce.go for why there is no response cache.
	CacheFlight *singleFlight
	// Bedrock, when set, enables the /bedrock/* SigV4 signing gateway.
	Bedrock *BedrockConfig
	// ClaudeFableAPIKey, when set, serves Claude Fable requests via this Anthropic
	// API key (x-api-key) instead of the subscription pool or Bedrock. It applies
	// ONLY to Fable; Opus/Sonnet/etc. continue to use the OAuth pool and never
	// touch this key.
	ClaudeFableAPIKey string

	// ClaudeFableCacheTTLUpgradeOff disables the Bedrock-path rewrite of bare
	// ephemeral cache_control blocks to the 1-hour TTL (see
	// upgradeEphemeralCacheTTL). Default off = upgrade enabled.
	ClaudeFableCacheTTLUpgradeOff bool
	// AzureCodex, when set, serves Codex Responses requests from Azure OpenAI
	// after the subscription pool has spent its retry budget and still failed,
	// and pins the session to Azure afterwards so later turns keep hitting the
	// same prompt cache. It never preempts the pool.
	AzureCodex *AzureCodexConfig

	// FableBedrockPrimary, when true, routes Claude Fable requests to AWS Bedrock
	// FIRST, before the subscription pool, instead of using Bedrock only as a
	// fallback. It only takes effect when the Bedrock gateway is configured; a
	// non-2xx Bedrock response (or an unreachable Bedrock) falls through to the
	// normal pool path, which keeps its own Bedrock/API-key fallback. Applies
	// ONLY to Fable; other Claude models are unaffected.
	FableBedrockPrimary bool
	// contains filtered or unexported fields
}

func (Server) AccountImportState added in v0.1.64 

func (s Server) AccountImportState() string

AccountImportState reports whether this server can accept `sr add` uploads. Account import denies every request when no credential is configured, and nothing else on the server reflects that, so a host whose binary updated past the credential requirement while its configuration stayed behind looks healthy right up until someone tries to add an account.

func (Server) AuthMode added in v0.1.67 

func (s Server) AuthMode() string

AuthMode describes how this server authenticates non-loopback callers, for health output and startup logging.

func (Server) Handler 

func (s Server) Handler() http.Handler

type StreamDropSnapshot added in v0.1.38 

type StreamDropSnapshot struct {
	Client    uint64 `json:"client"`
	Proxy     uint64 `json:"proxy"`
	Upstream  uint64 `json:"upstream"`
	Unknown   uint64 `json:"unknown"`
	Total     uint64 `json:"total"`
	Since     string `json:"since,omitempty"`
	LastProxy string `json:"last_proxy_drop,omitempty"`
}

StreamDropSnapshot is a point-in-time read of the counters.

type StreamDropStats added in v0.1.38 

type StreamDropStats struct {
	// contains filtered or unexported fields
}

StreamDropStats counts response-stream terminations by which side ended them.

Storage is the constraint this type exists to solve. A dropped stream happens roughly 1000 times a day at current volume, and ~94% of those are the client hanging up and immediately retrying, which is expected behavior rather than a defect. Writing an ERROR line for each one buys nothing and is a large share of an unrotated log. Counting them costs a few atomic adds and stays flat forever, while the rare canceled_by=proxy case still gets a full line each.

func (*StreamDropStats) Observe added in v0.1.38 

func (s *StreamDropStats) Observe(canceledBy string, now time.Time)

Observe records one dropped stream. Safe on a nil receiver so the counters stay optional for callers that do not wire them up.

func (*StreamDropStats) Snapshot added in v0.1.38 

func (s *StreamDropStats) Snapshot() StreamDropSnapshot

Snapshot reads the counters without resetting them.

type TailnetAuthorizer added in v0.1.67 

type TailnetAuthorizer interface {
	Authorize(ctx context.Context, remoteAddr string) (tailnet.Identity, bool)
}

TailnetAuthorizer resolves the tailnet principal behind a request. It is an interface so the proxy never depends on how identity is obtained, and so tests can assert the authorization rule without a running tailscaled.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.