provider

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 10, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package provider defines the model boundaries the gateway talks to, plus the model-selection doctrine (ResolveModel, ResolveAlias, EffectiveModel). The wire types live in the shared protocol package (common); the vendor clients (anthropic, fireworks, openai, gemini, grok) and the hybrid router live here alongside the routing rules.

Index

Constants

View Source 
const (
	// EnvProvider selects the backend: "" | "anthropic" (default) | "openai" |
	// "gemini" | "grok" | "fireworks" (cheap lane only) | "hybrid" (OpenAI
	// strong tier + the cheap lane — the target architecture; needs both
	// backends' env). Hybrid routes per-turn and can switch the strong tier
	// via Intent.Vendor.
	EnvProvider = "MEMCODE_PROVIDER"
	// EnvFireworksURL is the cheap lane's OpenAI-compatible /v1 root, e.g.
	// https://api.fireworks.ai/inference/v1. Required for fireworks/hybrid.
	EnvFireworksURL = "MEMCODE_FIREWORKS_URL"
	// EnvFireworksKey is the Fireworks API key. Required for fireworks/hybrid.
	EnvFireworksKey = "MEMCODE_FIREWORKS_KEY"
	// EnvFireworksModel is the served model id backing requests that arrive
	// without one; Hybrid retargets per resolved role.
	EnvFireworksModel = "MEMCODE_FIREWORKS_MODEL"
)

Backend-selection environment. memcode has ONE cheap lane — Fireworks, a hosted OpenAI-compatible token API — plus the frontier vendor APIs (OpenAI, Anthropic, Gemini, Grok). Hybrid routes between them per turn (the prod architecture); the pure single-vendor modes run the WHOLE session on one backend, chosen by env. (The self-hosted vLLM/RunPod era ended at the 2026-06-12 Fireworks cutover; there is no self-hosted backend.)

View Source 
const (
	EnvOpenAIKey = openai.EnvOpenAIKey
	EnvGrokKey   = openai.EnvGrokKey
	EnvAPIKey    = anthropic.EnvAnthropicKey
	EnvGeminiKey = gemini.EnvGeminiKey
	EnvGCPSAKey  = gemini.EnvGCPSAKey
)

Key env vars ride with the shared adapters (one definition).

Variables

View Source 
var ErrKeyInvalid = gateway.ErrKeyInvalid

ErrKeyInvalid re-exports the shared sentinel: a KeySource.Key rejection meaning the key was recently proven bad (fast-fail).

Functions

func APIKeyFromEnv 

func APIKeyFromEnv() string

APIKeyFromEnv returns the Anthropic API key from the environment.

Keys are read from the environment ONLY — never from .memcode/config.json or the state database. A project-root .env (gitignored) may be loaded into the environment by the CLI before this is read; OS keychain / credential-helper support can be added later.

func ByokVendors 

func ByokVendors() []string

ByokVendors enumerates every vendor a user could bring their own key for — derived from the models.json catalog (via owningVendor), never hardcoded, so a new catalog vendor shows up in /apikeys and the www API Keys page without code changes. Fireworks (the cheap lane) is always included. Order is stable (catalog order, fireworks last unless the catalog surfaced it already).

func ConfiguredVendors 

func ConfiguredVendors() []string

ConfiguredVendors reports which strong-tier vendors the gateway has keys for (and can therefore serve), reported on /v1/models. The FIRST entry is the deployment's default vendor — the CLI's selection policy uses it as the unkeyed-session tier preference.

func EffectiveModel 

func EffectiveModel(s string) string

EffectiveModel resolves a configured tier value (alias or model id) to the wire model for the ACTIVE backend. On Anthropic (the default) it is ResolveAlias; on fireworks every tier collapses to the single served model. Pricing follows the returned id, so the ledger stays honest whichever backend is live.

func HasGeminiCreds 

func HasGeminiCreds() bool

HasGeminiCreds reports whether real Gemini credentials are configured (a valid SA key or a Developer API key) — used by ConfiguredVendors so the /model selector never offers a vendor the gateway can't actually serve.

func IsContextOverflow 

func IsContextOverflow(err error) bool

IsContextOverflow reports whether err is (or wraps) a context-window overflow from any backend — the shared overflow error, or a lane 4xx flagged Overflow. The server maps it to 413 context_overflow.

func LookupServable 

func LookupServable(label string) (string, bool)

LookupServable is the compat endpoint's STRICT model gate: it resolves a client-facing catalog label ("sonnet", "glm-5p2", "gpt-oss-120b") to its raw model id — ok only for known CHAT labels (Window > 0; embedding/image rows don't serve chat) whose owning backend has credentials in this deployment. Anything else — "auto", vendor names, typos, retired labels — must 400 (unknown_model): the gateway serves exactly what the agent asked for, and a typo must never silently reroute a session.

func ResolveAlias 

func ResolveAlias(s string) string

ResolveAlias maps the short aliases (opus|sonnet|haiku) to model ids. Any other value is treated as a literal model id and returned unchanged.

func SanitizeModelID 

func SanitizeModelID(id string) string

SanitizeModelID returns the client-facing short name for a model id: its declared catalog label (models.json) if the model is known — e.g. "claude-haiku-4-5-20251001" → "haiku", "accounts/fireworks/models/glm-5p1" → "glm-5p1" — else its last path segment as a defensive fallback. Either way the inference vendor's path never reaches the client. The gateway keeps the RAW id internally (cost, metering, catalog lookup); this is only for the wire.

func SanitizeResponse 

func SanitizeResponse(resp *wire.Response)

SanitizeResponse strips provider identity from a response just before it is sent to the client: model ids lose their vendor paths, and the cheap lane's internal backend tag ("fireworks" — honest inside the gateway) becomes the vendor-neutral wire value "cheap". Call it AFTER metering (which needs the raw ids/backends) and BEFORE writing.

func SetDefaultVendor 

func SetDefaultVendor(v string)

SetDefaultVendor pins the deployment default. Called by NewFromEnv per backend.

func SetModels 

func SetModels(planner, reviewer, standard string, classify ...string)

SetModels pins the configured role ids for the /v1/models role report. Called by NewFromEnv, so the role story is single-sourced here. classify is variadic for back-compat: a single-model backend (anthropic/fireworks) passes only the first three and the classifier collapses onto the standard lane.

Types

type Anthropic 

type Anthropic = anthropic.Anthropic

The Anthropic Messages adapter — shared implementation.

func NewAnthropic 

func NewAnthropic(apiKey string) *Anthropic

NewAnthropic returns the shared Messages adapter on the given key.

type ByokError 

type ByokError struct {
	Vendor string
	Auth   bool  // vendor rejected the key (401/403-shaped) — mark invalid
	Err    error // underlying cause; not always user-safe, message below is
	// contains filtered or unexported fields
}

ByokError is a turn-fatal BYOK failure. The server layer maps it to a distinct non-retryable HTTP error, and flips the key's metadata status when Auth is true.

func AsByokError 

func AsByokError(err error) *ByokError

AsByokError unwraps a turn error to its BYOK failure, if that's what it is.

func (*ByokError) Error 

func (e *ByokError) Error() string

func (*ByokError) Unwrap 

func (e *ByokError) Unwrap() error

type ByokKeys 

type ByokKeys interface {
	Key(ctx context.Context, org, user, vendor string) (key, version string, err error)
	MarkInvalid(org, user, vendor string)
}

ByokKeys is the key-resolution seam the Hybrid router consults, implemented by the composition's KeySource (a hosted vault + TTL caches). Key returns the live key and a stable version tag (cache key material); ErrKeyInvalid means a recent call proved the key bad — fail fast with the same message instead of re-burning a vendor call every turn.

type CapabilityError 

type CapabilityError struct {
	Capability string // "vision" | "document"
	Model      string // client-facing label
}

CapabilityError is the typed refusal for a turn the requested model cannot take: an image on a no-vision model, a document on a model without native PDF input. The CLI pre-checks these from the same catalog before sending — this is the enforcement backstop (and the honest answer for third-party clients). Mapped to HTTP 400 with code "model_capability"; never absorbed.

func AsCapabilityError 

func AsCapabilityError(err error) *CapabilityError

AsCapabilityError unwraps a turn error to its capability refusal, if that's what it is.

func (*CapabilityError) Error 

func (e *CapabilityError) Error() string

type ContextOverflowError 

type ContextOverflowError = provcore.ContextOverflowError

ContextOverflowError is the shared overflow signal (provcore) — aliased so errors.As matches across the gateway and the shared adapters identically.

type CreditsExhaustedError 

type CreditsExhaustedError struct{ Vendor string }

CreditsExhaustedError is a turn-fatal refusal: serving is limited to keyed the request was admitted only because the user has BYOK keys — but THIS lane would serve on memcode's keys. The server maps it to a 402 with the insufficient_credits code. No vendor call happened.

func AsCreditsExhausted 

func AsCreditsExhausted(err error) *CreditsExhaustedError

AsCreditsExhausted unwraps a turn error to its credits refusal, if that's what it is.

func (*CreditsExhaustedError) Error 

func (e *CreditsExhaustedError) Error() string

type Fireworks 

type Fireworks struct {
	// contains filtered or unexported fields
}

Fireworks wraps the shared engine with the lane's default model.

func NewFireworks 

func NewFireworks(baseURL, apiKey, model string) *Fireworks

NewFireworks returns the cheap-lane client: the shared compat engine against the Fireworks OpenAI-compatible API, lane mode + salvage on.

func (*Fireworks) Complete 

func (f *Fireworks) Complete(ctx context.Context, r wire.Request) (wire.Response, error)

Complete serves one lane call on the shared engine. Backend is stamped "fireworks" — honest internal naming for metering (the engine's generic "endpoint" tag is for backends it can't identify).

func (*Fireworks) Model 

func (f *Fireworks) Model() string

Model returns the lane's default served model id.

func (*Fireworks) Stream 

func (f *Fireworks) Stream(ctx context.Context, r wire.Request, h wire.StreamHandler) (wire.Response, error)

Stream serves one lane call, forwarding deltas (the router buffers cheap serves today; the capability exists for parity).

type Gemini 

type Gemini = gemini.Gemini

The Gemini adapter — shared implementation, both backends.

func NewGemini 

func NewGemini(apiKey string) *Gemini

NewGemini returns the shared Gemini adapter on a Developer-API key.

func NewGeminiVertex 

func NewGeminiVertex(serviceAccountJSON []byte, project, location string) *Gemini

NewGeminiVertex returns the shared Gemini adapter on Vertex AI credentials (the service-account JSON is RESOLVED here, gateway-side — the adapter just takes bytes).

type Grok 

type Grok = openai.Grok

Grok — the same Responses dialect pointed at api.x.ai.

func NewGrok 

func NewGrok(apiKey string) *Grok

NewGrok returns the shared Grok variant on the given key.

type Hybrid 

type Hybrid struct {
	// contains filtered or unexported fields
}

Hybrid is the gateway's backend: the strong-vendor providers (OpenAI / Anthropic / Gemini / Grok, whichever keys are configured) + the cheap lane (an OpenAI-compatible token API — Fireworks). Since the all-policy-client-side migration it is a pure SERVING map: the request names a concrete model (the compat gate resolved the catalog label to a raw id), route() hands it to the provider that owns it, and any mismatch — a vision turn on a text model, a document on a lane without PDF input, a prompt past the model's window — returns a TYPED error instead of being absorbed onto a different model. Model choice, escalation, and recovery are the CLI's (cli/internal/llm); the gateway serves exactly what was asked or declines with a machine-readable reason. BYOK key injection is serving, not routing: the same model, the user's key.

func NewHybrid 

func NewHybrid(strong StrongTiers, cheapURL, cheapKey, cheapModel string) *Hybrid

NewHybrid builds the serving map. strong is the vendor → StrongTier map (at least the default vendor should be present). cheapURL/cheapKey describe the OpenAI-compatible endpoint (Fireworks); cheapModel is the defensive default when a request arrives modelless (internal callers only — the compat gate always stamps one).

func (*Hybrid) Complete 

func (h *Hybrid) Complete(ctx context.Context, req wire.Request) (wire.Response, error)

func (*Hybrid) SetByok 

func (h *Hybrid) SetByok(k ByokKeys)

SetByok wires the key-resolution seam into the router. Nil (the default) disables BYOK entirely — the zero-BYOK invariant's hard floor.

func (*Hybrid) Stream 

func (h *Hybrid) Stream(ctx context.Context, req wire.Request, sh wire.StreamHandler) (wire.Response, error)

func (*Hybrid) WebFetch 

func (h *Hybrid) WebFetch(ctx context.Context, url string) (string, wire.Response, error)

func (*Hybrid) WebSearch 

func (h *Hybrid) WebSearch(ctx context.Context, query string) (string, wire.Response, error)

WebSearch / WebFetch delegate to the default strong provider (the side-channel has no per-turn model context).

type LaneRequestError 

type LaneRequestError = compat.LaneRequestError

LaneRequestError is the shared lane error contract (compatwire) — aliased so errors.As matches identically across the gateway and the shared engine.

type ModelProvider 

type ModelProvider interface {
	Complete(ctx context.Context, r wire.Request) (wire.Response, error)
}

ModelProvider performs reasoning/generation calls (Claude in v1).

func NewFromEnv 

func NewFromEnv() (ModelProvider, error)

NewFromEnv constructs the active ModelProvider from the environment — the ONE place backend selection and its credential story live. Call it at the cmd boundary after LoadDotEnv; the error text tells the user exactly what to set.

type ModelSpec 

type ModelSpec struct {
	ID        string
	Label     string // client-facing short name (e.g. "haiku", "glm-5p1") — the only id the CLI sees
	Name      string // friendly display name ("Sonnet 5") — the /model picker's name column
	Desc      string // one-line picker description ("1M context · Efficient for routine tasks")
	Window    int
	Vision    bool
	PDF       bool // accepts PDFs natively on the LLM call; without it a document turn absorbs
	Reasoning bool
	Pinnable  bool   // offered in the /model picker; Intent.Pin only honors these
	Group     string // picker display family ("OpenAI", "Claude", "Kimi", …)
}

ModelSpec is a model's static capabilities — the gateway's view of one shared- catalog entry (see catalog.CatalogModel; same fields, provider-local name kept so call sites read naturally).

type OpenAI 

type OpenAI = openai.OpenAI

The OpenAI Responses adapter — shared implementation, gateway construction.

func NewOpenAI 

func NewOpenAI(apiKey string) *OpenAI

NewOpenAI returns the shared Responses adapter on the given key.

type RoleModel 

type RoleModel struct {
	Role   string `json:"role"`             // planner | reviewer | standard | classify
	ID     string `json:"id"`               // sanitized label — the raw provider id never leaves the server
	Label  string `json:"label"`            // short human name (catalog label)
	Window int    `json:"window,omitempty"` // context window (tokens), from models.json
	Vision bool   `json:"vision,omitempty"` // reads images natively
}

RoleModel is one configured role for the /v1/models surface: which model plays which job, with the catalog facts the client can show.

func ConfiguredModels 

func ConfiguredModels() []RoleModel

ConfiguredModels reports the model this deployment plays for each role — the CLI's semantic ladder maps role lanes onto these labels. Falls back to the OpenAI tier labels when a role is unset (single-vendor modes).

type ServableModel 

type ServableModel struct {
	Label     string
	Name      string
	Desc      string
	Group     string // display family — presentation only
	Vendor    string // authoritative serving vendor — the selection identity
	Window    int
	Vision    bool
	PDF       bool
	Reasoning bool
	Pinnable  bool // picker fact; serving accepts every listed label
	Byok      bool // the requesting user brought their own key for this model's vendor
}

ServableModel is one /v1/models entry: a model this deployment can actually serve, with the control-plane facts the CLI's selection policy reads.

func ServableModelsFor 

func ServableModelsFor(who identity.Info) []ServableModel

ServableModelsFor reports every model /v1/models lists for one user: all chat-capable catalog rows (Window > 0) whose serving backend has credentials, in models.json order, Byok stamped from the user's presence gate. This is the hosted routing control plane — the dataset CLI-side selection runs on.

type Streamer 

type Streamer interface {
	Stream(ctx context.Context, r wire.Request, h wire.StreamHandler) (wire.Response, error)
}

Streamer is an optional capability: a provider that can stream a completion, emitting text/usage as they arrive while still returning the fully assembled Response. Callers type-assert for it and fall back to Complete otherwise.

(Capability interfaces live at their consuming boundary — here, the gateway — not in the shared protocol package; the wire types they reference are common's.)

type StrongProvider 

type StrongProvider interface {
	ModelProvider
	Streamer
	WebSearcher
	WebFetcher
	Model() string
}

StrongProvider is the capability surface a strong-tier backend must satisfy: a ModelProvider that can also stream, answer web searches, fetch URLs, and report its default model id. The Hybrid router holds one of these (swappable per-turn via Intent.Vendor) instead of a concrete *OpenAI — so Anthropic, Gemini, and Grok can each serve as the strong tier behind the same router. Model() returns the vendor's default (balanced-tier) model id, used for display and fallback.

type StrongTier 

type StrongTier struct {
	Vendor   string
	Provider StrongProvider
}

StrongTier bundles a strong provider with its vendor label.

type StrongTiers 

type StrongTiers map[string]StrongTier

StrongTiers is the vendor → StrongTier map the Hybrid router consults. A vendor is present ONLY when its API key was set at NewFromEnv time — so /v1/models never lists a label the gateway can't actually serve.

func (StrongTiers) StrongTierFor 

func (st StrongTiers) StrongTierFor(vendor string) StrongTier

StrongTierFor resolves a tier from a vendor name. The empty string falls back to the deployment default (SetDefaultVendor); an unknown or unkeyed vendor falls back the same way, then to any configured tier as a last resort. Used by the side channels (websearch/webfetch), which have no per-turn model context.

type WebFetcher 

type WebFetcher interface {
	WebFetch(ctx context.Context, url string) (string, wire.Response, error)
}

webSearchToolName is the CLI's web_search FUNCTION tool. A strong provider with a native server-side search swaps this function def for its built-in search at request build time, so the SERVING model searches in-request — results land in the turn's own (cached) context instead of bouncing through a cold side-channel model call. Who does what: OpenAI → the Responses built-in web_search tool; Anthropic → web_search_20250305; Grok → xAI Live Search (a request-level search_parameters field, not a tool). The cheap lane (Fireworks) keeps the function def — no native search there, the CLI executes it via the gateway's /v1/websearch side channel — and so does Gemini, whose google_search grounding cannot coexist with function declarations in one request. WebFetcher is an optional capability: a provider that can fetch a specific URL server-side (text/PDF; not JS-rendered pages) and return its readable content, plus the usage billed (see WebSearcher).

type WebSearcher 

type WebSearcher interface {
	WebSearch(ctx context.Context, query string) (string, wire.Response, error)
}

WebSearcher is an optional capability: a provider that can answer a query using server-side web search. Returns the synthesized text AND the usage the call billed (model + token counts) so the gateway can meter side-channel spend — these calls moved real money invisibly when they returned text alone.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.