Documentation
¶
Overview ¶
Package config loads kram-gateway's YAML configuration: which upstream providers exist, how they're grouped into fallback combos, and which routing strategy each combo uses.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type ComboConfig ¶
type ComboConfig struct {
ID string `yaml:"id"`
Strategy string `yaml:"strategy"` // "round-robin" in v0; see internal/router for the full set
Providers []string `yaml:"providers"`
StrategyOptions StrategyOptions `yaml:"strategy_options,omitempty"`
Response ResponseGateConfig `yaml:"response,omitempty"`
}
ComboConfig is a named, ordered fallback chain of providers plus the strategy used to pick among the healthy ones.
type Config ¶
type Config struct {
Host string `yaml:"host"`
Port int `yaml:"port"`
Providers []ProviderConfig `yaml:"providers"`
Combos []ComboConfig `yaml:"combos"`
// DefaultCombo is used when a request's "model" doesn't match a combo ID.
DefaultCombo string `yaml:"default_combo"`
// Tunables overrides the operational timeouts and breaker thresholds
// that were previously compiled-in constants. Optional — an absent
// block reproduces the old behavior exactly. See tunables.go.
Tunables Tunables `yaml:"tunables,omitempty"`
}
Config is the top-level gateway configuration.
func (*Config) ComboContextWindow ¶ added in v0.4.0
ComboContextWindow returns the effective context-window token budget for the named combo: the minimum ContextWindow across its providers, ignoring providers whose window is unknown (0). Returns 0 if the combo is unknown or none of its providers declare a window — the caller then falls back to the compiled-in default. The minimum is the right aggregate: a fallback chain can route to any provider in the combo, so the budget must fit the smallest window, or a fallback to that provider would overflow it.
func (*Config) ComboModels ¶ added in v0.7.0
ComboModels returns the configured upstream model names of the named combo's providers, in combo order. Providers with no pinned Model contribute an empty string — the caller decides what "unknown model" means (agent.ProfileForModels treats it as not-frontier). Returns nil for an unknown combo.
func (*Config) MaxComboLength ¶ added in v0.4.0
MaxComboLength returns the provider count of the longest combo — the worst-case number of upstreams a single gateway call might try before giving up, which sets the coherent lower bound for the gateway-client timeout.
type Duration ¶ added in v0.4.0
Duration is a time.Duration that marshals to / from a Go duration string ("120s", "2m", "1m30s") in YAML. yaml.v3 would otherwise decode a bare number as nanoseconds, which is a footgun for a human-edited config file — "120" meaning 120ns is never what someone means by a timeout. The zero value marshals to nothing (omitempty-friendly) and is read as "unset, use the default" everywhere it's consumed.
func (Duration) MarshalYAML ¶ added in v0.4.0
MarshalYAML writes the duration as a string, or nothing when it's the zero value, so a Save round-trip never litters the file with "0s" lines for tunables the user never set.
type ProviderConfig ¶
type ProviderConfig struct {
ID string `yaml:"id"`
// Kind selects the adapter: "anthropic", "gemini", "openai-compat", or
// "openai-responses" (the ChatGPT-login-only Codex backend — see
// internal/provider/openai_responses.go).
Kind string `yaml:"kind"`
// BaseURL is the upstream API root. Optional for kinds with a
// well-known default (anthropic, gemini, openai-responses).
BaseURL string `yaml:"base_url,omitempty"`
// APIKeyEnv names the environment variable holding the credential —
// for AuthMode "oauth" it's used purely as the lookup key into the
// credentials store's OAuth token map, not as a real env var.
APIKeyEnv string `yaml:"api_key_env"`
// AuthMode selects how the gateway resolves this provider's
// credential. Empty (the default) is today's behavior: read
// APIKeyEnv from the process environment once at startup and use it
// for the provider's lifetime. "oauth" means the credential is a
// refreshable token in internal/credentials' OAuth store, resolved
// live on every request (see internal/gateway.Run) since it can
// expire mid-session — see internal/credentials.Store.Resolve.
AuthMode string `yaml:"auth_mode,omitempty"`
// KeyOptional means APIKey should return an empty string instead of an
// error when APIKeyEnv isn't set — true only for
// internal/customprovider entries, most of which point at a local/LAN
// server with no auth at all. Every other provider keeps the strict
// default: an unset env var is a real misconfiguration worth failing
// loudly on (a clear startup error beats a confusing 401 later).
KeyOptional bool `yaml:"key_optional,omitempty"`
// Model is the upstream model ID to request, if it should be pinned
// regardless of what the client asked for. Empty means passthrough.
Model string `yaml:"model,omitempty"`
// Temperature pins a fixed sampling temperature for this provider,
// overriding whatever the client's request carries (today, that's
// always nothing — openai.ChatCompletionRequest.Temperature exists on
// the wire type but nothing in Kram ever populates it, so every
// request defers entirely to the upstream server's own default
// unless this is set). A pointer so "never set" (the common case) is
// distinguishable from "explicitly pinned to 0.0" — a real,
// maximally-deterministic value someone might genuinely want, not
// the same as leaving it alone. Currently only honored by the
// openai-compat adapter (internal/provider/openai_compat.go) — see
// its own doc comment for why the other three kinds aren't wired up
// yet.
Temperature *float64 `yaml:"temperature,omitempty"`
// SupportsImages declares whether this provider's model accepts image
// input. Callers must check this (via /admin/status) before attaching
// images — the gateway never guesses.
SupportsImages bool `yaml:"supports_images,omitempty"`
// SupportsTools declares whether this provider's model can be sent
// tool/function definitions. Kram's agent loop skips providers that
// don't when a request requires tool calling.
SupportsTools bool `yaml:"supports_tools,omitempty"`
// ContextWindow is the model's total token window, used to size the
// compaction budget (see Config.ComboContextWindow). 0 means "unknown"
// and is ignored when taking a combo's minimum window. Populated from
// the provider catalog / custom-provider registration, and overridable
// by editing this field in config.yaml.
ContextWindow int `yaml:"context_window,omitempty"`
// QualityHint is an optional, explicitly user-configured 0..1 signal
// feeding the "quality" scoring factor (see internal/router) —
// deliberately never inferred or fabricated by Kram itself, since
// there is no real per-provider quality measurement to derive it
// from. Zero means "no opinion", which the scorer treats as neutral
// (0.5), not "worst possible" — see DECISIONS.md.
QualityHint float64 `yaml:"quality_hint,omitempty"`
}
ProviderConfig describes one upstream LLM backend.
func (ProviderConfig) APIKey ¶
func (p ProviderConfig) APIKey() (string, error)
APIKey resolves the provider's credential from its configured env var.
type ResponseGateConfig ¶
type ResponseGateConfig struct {
// RejectEmpty rejects a response with neither text content nor tool
// calls.
RejectEmpty bool `yaml:"reject_empty,omitempty"`
// RequireTerminal rejects a stream that never produced a proper
// finish signal (e.g. a connection that was cut mid-stream) even if
// it carried some content.
RequireTerminal bool `yaml:"require_terminal,omitempty"`
// MinContentLength rejects a text response shorter than this many
// characters (tool-call-only responses are exempt — a short "ok,
// calling the tool now" isn't what this guards against).
MinContentLength int `yaml:"min_content_length,omitempty"`
// ForbiddenSubstrings rejects a response whose content contains any
// of these — for masked upstream errors that come back as HTTP 200
// with an error message as the body, not for filtering legitimate
// model output.
ForbiddenSubstrings []string `yaml:"forbidden_substrings,omitempty"`
}
ResponseGateConfig configures deterministic, technical validation of an otherwise-successful response before it's allowed to end the fallback chain — see router.ResponseGate. Every field is opt-in; an absent response block disables gating entirely (the v0 behavior: any technically-successful response is accepted).
type StrategyOptions ¶
type StrategyOptions struct {
// Sticky pins a run (see router.RunKey) to its winning provider
// across tool round-trips, only used by the weighted (smart/quality/
// fast/cheap/reliable/custom) strategy family. Defaults to true for
// that family if unset — see DECISIONS.md, "Sticky by default".
Sticky *bool `yaml:"sticky,omitempty"`
// LKGPBoost is the additive score bonus (0..1) a candidate gets when
// it's the combo's current last-known-good provider, on top of its
// own weighted score. 0 (the zero value) if unset would disable the
// boost entirely, so an unset field falls back to a small nonzero
// default instead — see router.defaultLKGPBoost.
LKGPBoost *float64 `yaml:"lkgp_boost,omitempty"`
// Exploration is the probability (0..1) of picking a uniformly random
// eligible candidate as the leader instead of the top-ranked one, so
// non-winning candidates still occasionally get real telemetry — see
// DECISIONS.md, "Exploration". Small and conservative by default.
Exploration *float64 `yaml:"exploration,omitempty"`
// Weights overrides the weighted strategy's per-factor weights
// (health, reliability, latency, quality, cache_affinity, priority).
// Any factor not mentioned keeps its preset's weight; values are
// normalized automatically, so they don't need to sum to any
// particular total.
Weights map[string]float64 `yaml:"weights,omitempty"`
}
StrategyOptions tunes a combo's routing strategy — every field is optional, and an absent strategy_options block (the whole existing v0 config shape) means "use the strategy's own defaults", never a required section. Pointers distinguish "not set, use the default" from "set to the zero value", which matters for Sticky in particular (explicitly disabling stickiness is a real, different choice from never mentioning it).
type Tunables ¶ added in v0.4.0
type Tunables struct {
// ProviderTimeout caps a single upstream request (dial + headers +
// reading the whole body) per provider. Default 120s. Generous values
// are the point for slow local models; a genuinely dead provider is
// still cut eventually.
ProviderTimeout Duration `yaml:"provider_timeout,omitempty"`
// GatewayClientTimeout caps the daemon's whole call to the gateway,
// which may itself walk a fallback chain of several providers. Left
// unset it's *derived* to stay coherent with the chain (see
// ResolvedGatewayClientTimeout) rather than defaulting to a fixed value
// that a legitimate multi-candidate round could exceed.
GatewayClientTimeout Duration `yaml:"gateway_client_timeout,omitempty"`
// BreakerFailureThreshold is how many consecutive failures trip a
// provider's circuit breaker open. Default 3.
BreakerFailureThreshold int `yaml:"breaker_failure_threshold,omitempty"`
// BreakerCooldown is how long a tripped provider stays open before a
// half-open trial request. Default 30s.
BreakerCooldown Duration `yaml:"breaker_cooldown,omitempty"`
}
Tunables holds operational timeouts and circuit-breaker thresholds that used to be compiled-in constants. Every field is optional; a zero value means "use the built-in default", so an existing config.yaml with no tunables block behaves exactly as before. This exists because Kram targets local models, where latency varies from seconds to minutes with the model and cold-load — fixed timeouts calibrated for fast hosted APIs cut off healthy-but-slow responses and force fallback to worse candidates. See DECISIONS.md.
func (Tunables) ResolvedBreakerCooldown ¶ added in v0.4.0
ResolvedBreakerCooldown is the breaker cooldown, defaulted.
func (Tunables) ResolvedBreakerFailureThreshold ¶ added in v0.4.0
ResolvedBreakerFailureThreshold is the breaker trip threshold, defaulted.
func (Tunables) ResolvedGatewayClientTimeout ¶ added in v0.4.0
ResolvedGatewayClientTimeout returns the timeout the daemon's gateway client should use, given the longest fallback chain it might drive.
If the user pinned a value, that's honored verbatim (their call). Left unset, it's *derived* so the client never cuts off a legitimate fallback round: the gateway may try every provider in the longest combo back to back, so the client must allow at least maxComboLen × providerTimeout, plus a small margin. This is the incoherence the audit flagged — a fixed 180s client timeout is smaller than 2×120s, so a healthy two-candidate fallback could be killed by the client before the chain was exhausted. The result is floored at defaultGatewayClientFloor so single-provider setups keep today's generous ceiling.
func (Tunables) ResolvedProviderTimeout ¶ added in v0.4.0
ResolvedProviderTimeout is the per-provider request timeout to use, substituting the default when unset.
Source Files