config

package
v0.1.58 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 22, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package config provides configuration management for the application.

Index

Constants

View Source 
const (
	MCPTransportHTTP  = "http"
	MCPTransportSSE   = "sse"
	MCPTransportStdio = "stdio"
)

MCP transport names accepted in MCPServerConfig.Transport.

View Source 
const (
	DefaultBodySizeLimit int64 = 10 * 1024 * 1024  // 10MB
	MinBodySizeLimit     int64 = 1 * 1024          // 1KB
	MaxBodySizeLimit     int64 = 100 * 1024 * 1024 // 100MB
)

Body size limit constants

View Source 
const DefaultMCPToolTimeout = 30 * time.Second

DefaultMCPToolTimeout bounds a single upstream tools/call when the server declares no timeout of its own.

View Source 
const DefaultTaggingDelimiter = ","

DefaultTaggingDelimiter separates multiple labels inside one header value.

Variables

This section is empty.

Functions

func DeriveMCPServerSlug 

func DeriveMCPServerSlug(name string) string

DeriveMCPServerSlug creates a conservative default slug from a display name. Callers may let users edit it before creation; once persisted it is a stable identity and should not change when the display name changes.

func JoinBasePath 

func JoinBasePath(basePath, urlPath string) string

JoinBasePath prefixes urlPath with the normalized public mount path.

func MCPServerEnabled 

func MCPServerEnabled(server MCPServerConfig) bool

MCPServerEnabled reports the effective enabled state (default true).

func NormalizeBasePath 

func NormalizeBasePath(value string) string

NormalizeBasePath canonicalizes the public mount path for the HTTP server. Empty, whitespace-only, and "/" all resolve to root.

func NormalizeHeaderName 

func NormalizeHeaderName(value, fallback string) (string, error)

NormalizeHeaderName canonicalizes an HTTP header field name. Empty values fall back to fallback.

func ParseBodySizeLimitBytes 

func ParseBodySizeLimitBytes(s string) (int64, error)

ParseBodySizeLimitBytes parses a configured body size limit into bytes. Accepts formats like: "10M", "10MB", "1024K", "1024KB", "104857600". Returns an error if the format is invalid or value is outside bounds (1KB - 100MB).

func ProviderModelIDs 

func ProviderModelIDs(models []RawProviderModel) []string

ProviderModelIDs returns the ID of each model entry, preserving order and dropping entries with empty IDs.

func ProviderModelMetadataOverrides 

func ProviderModelMetadataOverrides(models []RawProviderModel) map[string]*core.ModelMetadata

ProviderModelMetadataOverrides returns id -> metadata for entries with non-nil Metadata. Returns nil if no entries declare metadata.

func SemanticCacheActive 

func SemanticCacheActive(sem *SemanticCacheConfig) bool

SemanticCacheActive reports whether the semantic response cache should be validated and constructed. The semantic block must be present (YAML or SEMANTIC_CACHE_ENABLED=true); omitted enabled means true.

func SimpleCacheEnabled 

func SimpleCacheEnabled(s *SimpleCacheConfig) bool

SimpleCacheEnabled reports whether the exact-match response cache layer is allowed to run for a non-nil simple config. Omitted enabled means true.

func ValidateBodySizeLimit 

func ValidateBodySizeLimit(s string) error

ValidateBodySizeLimit validates a body size limit string. Accepts formats like: "10M", "10MB", "1024K", "1024KB", "104857600" Returns an error if the format is invalid or value is outside bounds (1KB - 100MB).

func ValidateCacheConfig 

func ValidateCacheConfig(c *CacheConfig) error

ValidateCacheConfig validates the cache configuration in c. For the model cache, exactly one backend (Local or Redis) must be configured; having both or neither is an error. When Redis is selected, its URL must be non-empty. Returns a descriptive error if any constraint is violated, or nil if the configuration is valid.

func ValidateMCPServerConfig 

func ValidateMCPServerConfig(server *MCPServerConfig) error

ValidateMCPServerConfig validates one server definition and applies defaults in place. It is shared by config loading and the admin API.

func ValidateMCPServerName 

func ValidateMCPServerName(name string) error

ValidateMCPServerName accepts a human-facing Unicode display name. Machine constraints belong to the separate immutable slug.

func ValidateMCPServerSlug 

func ValidateMCPServerSlug(slug string) error

ValidateMCPServerSlug validates the stable ASCII identity used in routes, scope headers, and aggregated tool/prompt names.

Types

type AdminConfig 

type AdminConfig struct {
	// EndpointsEnabled controls whether the admin REST API is active
	// Default: true
	EndpointsEnabled bool `yaml:"endpoints_enabled" env:"ADMIN_ENDPOINTS_ENABLED"`

	// UIEnabled controls whether the admin dashboard UI is active
	// Requires EndpointsEnabled — if endpoints are disabled and UI is enabled,
	// a warning is logged and UI is forced to false.
	// Default: true
	UIEnabled bool `yaml:"ui_enabled" env:"ADMIN_UI_ENABLED"`

	// LiveLogsEnabled controls whether the dashboard opens a realtime log stream.
	// Default: true
	LiveLogsEnabled bool `yaml:"live_logs_enabled" env:"DASHBOARD_LIVE_LOGS_ENABLED"`

	// LiveLogsBufferSize is the in-memory replay window for dashboard live log events.
	// Default: 10000
	LiveLogsBufferSize int `yaml:"live_logs_buffer_size" env:"DASHBOARD_LIVE_LOGS_BUFFER_SIZE"`

	// LiveLogsReplayLimit caps events replayed to one reconnecting dashboard client.
	// Default: 1000
	LiveLogsReplayLimit int `yaml:"live_logs_replay_limit" env:"DASHBOARD_LIVE_LOGS_REPLAY_LIMIT"`

	// LiveLogsHeartbeatSeconds keeps idle stream connections and proxies active.
	// Default: 15
	LiveLogsHeartbeatSeconds int `yaml:"live_logs_heartbeat_seconds" env:"DASHBOARD_LIVE_LOGS_HEARTBEAT_SECONDS"`
}

AdminConfig holds configuration for the admin API and dashboard UI.

type BudgetLimitConfig 

type BudgetLimitConfig struct {
	// Period accepts hourly, daily, weekly, or monthly. The resolved period is
	// persisted as PeriodSeconds in the database.
	Period string `yaml:"period" json:"period"`

	// PeriodSeconds can be set directly instead of Period. Standard values are
	// 3600, 86400, 604800, and 2592000.
	PeriodSeconds int64 `yaml:"period_seconds" json:"period_seconds"`

	// Amount is the maximum allowed tracked provider spend for the period.
	Amount float64 `yaml:"amount" json:"amount"`
}

BudgetLimitConfig declares one spend limit for a reset period. The json tags support the JSON-array form of SET_BUDGET_* env values.

type BudgetUserPathConfig 

type BudgetUserPathConfig struct {
	Path   string              `yaml:"path"`
	Limits []BudgetLimitConfig `yaml:"limits"`
}

BudgetUserPathConfig declares one or more budget limits for a user path.

type BudgetsConfig 

type BudgetsConfig struct {
	// Enabled controls whether budget checks are active.
	// Default: true. Requires usage tracking because spend limits are evaluated
	// from usage cost records.
	Enabled bool `yaml:"enabled" env:"BUDGETS_ENABLED"`

	// UserPaths declares budget limits by tracked user path.
	UserPaths []BudgetUserPathConfig `yaml:"user_paths"`
}

BudgetsConfig holds per-user-path spend limits.

type CacheConfig 

type CacheConfig struct {
	Model    ModelCacheConfig    `yaml:"model"`
	Response ResponseCacheConfig `yaml:"response"`
}

CacheConfig holds model and response cache configuration.

type CircuitBreakerConfig 

type CircuitBreakerConfig struct {
	FailureThreshold int           `yaml:"failure_threshold" env:"CIRCUIT_BREAKER_FAILURE_THRESHOLD"`
	SuccessThreshold int           `yaml:"success_threshold" env:"CIRCUIT_BREAKER_SUCCESS_THRESHOLD"`
	Timeout          time.Duration `yaml:"timeout"           env:"CIRCUIT_BREAKER_TIMEOUT"`
}

CircuitBreakerConfig holds resolved circuit breaker settings. This is the canonical type shared between config and llmclient.

func DefaultCircuitBreakerConfig 

func DefaultCircuitBreakerConfig() CircuitBreakerConfig

DefaultCircuitBreakerConfig returns the default circuit breaker settings.

type Config 

type Config struct {
	Server     ServerConfig     `yaml:"server"`
	Models     ModelsConfig     `yaml:"models"`
	Cache      CacheConfig      `yaml:"cache"`
	Storage    StorageConfig    `yaml:"storage"`
	Logging    LogConfig        `yaml:"logging"`
	Usage      UsageConfig      `yaml:"usage"`
	Budgets    BudgetsConfig    `yaml:"budgets"`
	RateLimits RateLimitsConfig `yaml:"rate_limits"`
	Metrics    MetricsConfig    `yaml:"metrics"`
	HTTP       HTTPConfig       `yaml:"http"`
	Admin      AdminConfig      `yaml:"admin"`
	Guardrails GuardrailsConfig `yaml:"guardrails"`
	Failover   FailoverConfig   `yaml:"failover"`
	Workflows  WorkflowsConfig  `yaml:"workflows"`
	Resilience ResilienceConfig `yaml:"resilience"`
	Tagging    TaggingConfig    `yaml:"tagging"`
	MCP        MCPConfig        `yaml:"mcp"`

	// VirtualModels declares redirects, load balancers, and access policies as
	// infrastructure-as-code. They override admin-store rows of the same source.
	VirtualModels []VirtualModelConfig `yaml:"virtual_models"`
}

Config holds the application configuration.

type ConfiguredProviderModelsMode 

type ConfiguredProviderModelsMode string

ConfiguredProviderModelsMode controls how explicitly configured provider model lists are applied to the discovered model inventory. 

const (
	ConfiguredProviderModelsModeFallback  ConfiguredProviderModelsMode = "fallback"
	ConfiguredProviderModelsModeAllowlist ConfiguredProviderModelsMode = "allowlist"
)

func NormalizeConfiguredProviderModelsMode 

func NormalizeConfiguredProviderModelsMode(mode ConfiguredProviderModelsMode) ConfiguredProviderModelsMode

NormalizeConfiguredProviderModelsMode canonicalizes a configured provider models mode.

func ResolveConfiguredProviderModelsMode 

func ResolveConfiguredProviderModelsMode(mode ConfiguredProviderModelsMode) ConfiguredProviderModelsMode

ResolveConfiguredProviderModelsMode canonicalizes mode and applies the process default.

func (ConfiguredProviderModelsMode) Valid 

func (m ConfiguredProviderModelsMode) Valid() bool

Valid reports whether mode is one of the supported configured-provider-models modes.

type EmbedderConfig 

type EmbedderConfig struct {
	Provider string `yaml:"provider"`
	Model    string `yaml:"model"`
}

EmbedderConfig selects how embeddings are generated. Provider must match a key in the top-level providers map when semantic caching is active; that provider's api_key and base_url are reused for POST /v1/embeddings. There is no default provider.

type FailoverConfig 

type FailoverConfig struct {
	// Enabled controls failover globally. It defaults to true; configured rules
	// and workflow policy decide whether any request has failover candidates.
	Enabled bool `yaml:"enabled" env:"FAILOVER_ENABLED"`

	// DefaultMode is a deprecated compatibility field. It is accepted from old
	// config files and FAILOVER_MODE, but runtime failover is manual-only.
	DefaultMode FailoverMode `yaml:"default_mode" env:"FAILOVER_MODE"`

	// ManualRulesPath points to a JSON file that maps source model selectors to
	// ordered failover model selector lists. Empty disables manual rules.
	ManualRulesPath string `yaml:"manual_rules_path" env:"FAILOVER_MANUAL_RULES_PATH"`

	// Rules defines manual failover rules inline in config.yaml.
	Rules map[string][]string `yaml:"rules"`

	// RulesJSON defines manual failover rules inline from env.
	RulesJSON string `yaml:"rules_json" env:"FAILOVER_RULES_JSON"`

	// DisabledModels disables failover for matching source selectors.
	DisabledModels []string `yaml:"disabled_models" env:"FAILOVER_DISABLED_MODELS"`

	// DisabledModelsJSON disables failover for matching source selectors from
	// env. It accepts either a JSON string array or object with boolean values.
	DisabledModelsJSON string `yaml:"disabled_models_json" env:"FAILOVER_DISABLED_MODELS_JSON"`

	// Overrides is a removed compatibility field. Per-model failover modes are gone;
	// operators migrate to DisabledModels. It is still parsed — and ignored, with a
	// warning — so an old config file keeps booting under strict YAML validation.
	Overrides map[string]any `yaml:"overrides"`

	// Manual holds the parsed manual failover lists loaded from ManualRulesPath.
	Manual map[string][]string `yaml:"-"`

	// Disabled holds normalized per-model failover disables.
	Disabled map[string]bool `yaml:"-"`
}

FailoverConfig holds translated-route model failover policy.

type FailoverMode 

type FailoverMode string
const (
	FailoverModeOff    FailoverMode = "off"
	FailoverModeManual FailoverMode = "manual"
	FailoverModeAuto   FailoverMode = "auto"
)

func ResolveFailoverDefaultMode 

func ResolveFailoverDefaultMode(mode FailoverMode) FailoverMode

ResolveFailoverDefaultMode canonicalizes the global failover default mode and applies the process default when unset.

func (FailoverMode) Valid 

func (m FailoverMode) Valid() bool

Valid reports whether mode is one of the supported failover modes.

type GuardrailRuleConfig 

type GuardrailRuleConfig struct {
	// Name is a unique identifier for this guardrail instance (used in logs and errors)
	Name string `yaml:"name"`

	// Type selects the guardrail implementation: "system_prompt" or "llm_based_altering"
	Type string `yaml:"type"`

	// UserPath scopes internal auxiliary guardrail requests for workflow
	// selection and audit logging. When empty, the caller user path is used.
	UserPath string `yaml:"user_path"`

	// Order controls execution ordering relative to other guardrails.
	// Guardrails with the same order run in parallel; different orders run sequentially.
	// Default: 0
	Order int `yaml:"order"`

	// SystemPrompt holds settings when Type is "system_prompt"
	SystemPrompt SystemPromptSettings `yaml:"system_prompt"`

	// LLMBasedAltering holds settings when Type is "llm_based_altering"
	LLMBasedAltering LLMBasedAlteringSettings `yaml:"llm_based_altering"`
}

GuardrailRuleConfig defines a single guardrail instance.

type GuardrailsConfig 

type GuardrailsConfig struct {
	// Enabled controls whether guardrails are active
	// Default: false
	Enabled bool `yaml:"enabled" env:"GUARDRAILS_ENABLED"`

	// EnableForBatchProcessing controls whether guardrails are applied to inline
	// batch items for /v1/batches requests.
	// Default: false
	EnableForBatchProcessing bool `yaml:"enable_for_batch_processing" env:"ENABLE_GUARDRAILS_FOR_BATCH_PROCESSING"`

	// Rules is a list of guardrail instances. Each entry defines one guardrail
	// with its own name, type, order, and type-specific settings. Multiple
	// instances of the same type are allowed (e.g. two system_prompt guardrails
	// with different content).
	Rules []GuardrailRuleConfig `yaml:"rules"`
}

GuardrailsConfig holds configuration for the request guardrails pipeline.

type HTTPConfig 

type HTTPConfig struct {
	// Timeout is the overall HTTP request timeout in seconds (default: 600)
	Timeout int `yaml:"timeout" env:"HTTP_TIMEOUT"`

	// ResponseHeaderTimeout is the time to wait for response headers in seconds (default: 600)
	ResponseHeaderTimeout int `yaml:"response_header_timeout" env:"HTTP_RESPONSE_HEADER_TIMEOUT"`
}

HTTPConfig holds HTTP client configuration for upstream API requests. App startup installs these values into internal/httpclient before providers are constructed; the HTTP_TIMEOUT and HTTP_RESPONSE_HEADER_TIMEOUT env vars take precedence over the YAML values.

type LLMBasedAlteringSettings 

type LLMBasedAlteringSettings struct {
	// Model is the model selector used for the auxiliary rewrite call.
	// This can be a concrete model name, provider-qualified selector, or alias.
	Model string `yaml:"model"`

	// Provider is an optional routing hint for Model.
	Provider string `yaml:"provider"`

	// Prompt is the system prompt used to rewrite targeted messages.
	// When empty, the built-in LiteLLM-derived anonymization prompt is used.
	Prompt string `yaml:"prompt"`

	// Roles selects which message roles are rewritten.
	// Default: ["user"]
	Roles []string `yaml:"roles"`

	// SkipContentPrefix skips rewriting for messages whose trimmed text begins with this prefix.
	SkipContentPrefix string `yaml:"skip_content_prefix"`

	// MaxTokens limits the auxiliary rewrite completion.
	// Default: 4096
	MaxTokens int `yaml:"max_tokens"`
}

LLMBasedAlteringSettings holds the type-specific settings for an llm_based_altering guardrail.

type LoadResult 

type LoadResult struct {
	Config       *Config
	RawProviders map[string]RawProviderConfig
}

LoadResult is returned by Load and bundles the application config with the raw provider map parsed from YAML. Provider env vars and resolution are handled by the providers package.

func Load 

func Load() (*LoadResult, error)

Load reads configuration from file and environment using a three-layer pipeline: 

defaults (code) → config.yaml (optional overlay) → env vars (always win)

The returned LoadResult contains the resolved application Config and the raw provider map parsed from YAML. Provider env var discovery, credential filtering, and resilience merging are handled by the providers package.

type LocalCacheConfig 

type LocalCacheConfig struct {
	CacheDir string `yaml:"cache_dir" env:"GOMODEL_CACHE_DIR"`
}

LocalCacheConfig holds local file cache configuration.

type LogConfig 

type LogConfig struct {
	// Enabled controls whether audit logging is active
	// Default: false
	Enabled bool `yaml:"enabled" env:"LOGGING_ENABLED"`

	// LogBodies enables logging of full request/response bodies
	// WARNING: May contain sensitive data (PII, API keys in prompts)
	// Default: true
	LogBodies bool `yaml:"log_bodies" env:"LOGGING_LOG_BODIES"`

	// LogAudioBodies refines LogBodies for audio endpoints: when both are
	// enabled, the /v1/audio/speech JSON input and binary audio output are
	// stored (audio as base64 for playback) and /v1/audio/transcriptions upload
	// metadata is recorded. Requires LogBodies (the master body-logging switch);
	// when LogBodies is on but this is off, audio responses are recorded as a
	// lightweight placeholder instead of the full bytes.
	// WARNING: stores full audio in the audit log; grows storage quickly.
	// Default: false
	LogAudioBodies bool `yaml:"log_audio_bodies" env:"LOGGING_LOG_AUDIO_BODIES"`

	// LogHeaders enables logging of request/response headers
	// Sensitive headers (Authorization, Cookie, etc.) are auto-redacted
	// Default: true
	LogHeaders bool `yaml:"log_headers" env:"LOGGING_LOG_HEADERS"`

	// BufferSize is the number of log entries to buffer before flushing
	// Default: 1000
	BufferSize int `yaml:"buffer_size" env:"LOGGING_BUFFER_SIZE"`

	// FlushInterval is how often to flush buffered logs (in seconds)
	// Default: 5
	FlushInterval int `yaml:"flush_interval" env:"LOGGING_FLUSH_INTERVAL"`

	// RetentionDays is how long to keep logs (0 = forever)
	// Default: 30
	RetentionDays int `yaml:"retention_days" env:"LOGGING_RETENTION_DAYS"`

	// OnlyModelInteractions limits audit logging to AI model endpoints only
	// When true, only /v1/chat/completions, /v1/responses, /v1/embeddings, /v1/files, and /v1/batches are logged
	// Endpoints like /health, /metrics, /admin, /v1/models are skipped
	// Default: true
	OnlyModelInteractions bool `yaml:"only_model_interactions" env:"LOGGING_ONLY_MODEL_INTERACTIONS"`
}

LogConfig holds audit logging configuration

type MCPConfig 

type MCPConfig struct {
	// Enabled gates the /mcp routes. Default: true (a no-op without servers).
	Enabled bool `yaml:"enabled" env:"MCP_ENABLED"`

	// Servers maps stable server slugs to upstream definitions. Slugs become
	// tool namespaces and URL segments, so they are restricted to [a-z0-9_-].
	Servers map[string]MCPServerConfig `yaml:"servers"`
}

MCPConfig declares the MCP gateway: upstream MCP servers aggregated behind the authenticated /mcp endpoint. Declarative entries override admin-store rows with the same name and are read-only in the dashboard.

type MCPServerConfig 

type MCPServerConfig struct {
	// URL is the upstream MCP endpoint for http/sse transports.
	URL string `yaml:"url,omitempty" json:"url,omitempty"`

	// Transport selects the upstream transport: "http" (streamable HTTP,
	// default), "sse" (legacy HTTP+SSE), or "stdio" (spawned subprocess).
	Transport string `yaml:"transport,omitempty" json:"transport,omitempty"`

	// Headers are sent verbatim on every upstream request (http/sse). Values
	// support ${ENV} expansion via the standard config pipeline. This is the
	// credential boundary: client bearer tokens are never forwarded upstream.
	Headers map[string]string `yaml:"headers,omitempty" json:"headers,omitempty"`

	// Command, Args, and Env launch a stdio server as a subprocess. Stdio
	// servers are deliberately declarative-only: the admin API and dashboard
	// reject them, because registering subprocesses at runtime is a remote
	// code execution vector.
	Command string            `yaml:"command,omitempty" json:"command,omitempty"`
	Args    []string          `yaml:"args,omitempty" json:"args,omitempty"`
	Env     map[string]string `yaml:"env,omitempty" json:"env,omitempty"`

	// Description is an optional human-readable note.
	Description string `yaml:"description,omitempty" json:"description,omitempty"`

	// Enabled toggles the entry. It defaults to true when omitted.
	Enabled *bool `yaml:"enabled,omitempty" json:"enabled,omitempty"`

	// AllowedTools restricts the tools exposed from this server (original,
	// un-prefixed names). Empty means all tools.
	AllowedTools []string `yaml:"allowed_tools,omitempty" json:"allowed_tools,omitempty"`

	// DisallowedTools hides specific tools; applied after AllowedTools.
	DisallowedTools []string `yaml:"disallowed_tools,omitempty" json:"disallowed_tools,omitempty"`

	// UserPaths scopes server visibility to specific request user paths
	// (subtree match, same semantics as virtual models). Empty means all.
	UserPaths []string `yaml:"user_paths,omitempty" json:"user_paths,omitempty"`

	// ToolTimeout bounds a single tools/call against this server.
	// Default: 30s.
	ToolTimeout time.Duration `yaml:"tool_timeout,omitempty" json:"tool_timeout,omitempty"`
}

MCPServerConfig declares one upstream MCP server.

type MetricsConfig 

type MetricsConfig struct {
	// Enabled controls whether Prometheus metrics are collected and exposed
	// Default: false
	Enabled bool `yaml:"enabled" env:"METRICS_ENABLED"`

	// Endpoint is the HTTP path where metrics are exposed
	// Default: "/metrics"
	Endpoint string `yaml:"endpoint" env:"METRICS_ENDPOINT"`
}

MetricsConfig holds observability configuration for Prometheus metrics

type ModelCacheConfig 

type ModelCacheConfig struct {
	RefreshInterval int `yaml:"refresh_interval" env:"CACHE_REFRESH_INTERVAL"`
	// RecheckInterval is how often (seconds) providers whose latest refresh
	// failed are re-checked, so outage recovery is detected without waiting
	// for the next full refresh. Zero or negative disables the fast recheck.
	RecheckInterval int               `yaml:"recheck_interval" env:"PROVIDER_RECHECK_INTERVAL"`
	ModelList       ModelListConfig   `yaml:"model_list"`
	Local           *LocalCacheConfig `yaml:"local"`
	Redis           *RedisModelConfig `yaml:"redis"`
}

ModelCacheConfig holds cache configuration for model registry. Exactly one of Local or Redis must be non-nil.

type ModelListConfig 

type ModelListConfig struct {
	// URL is the HTTP(S) URL to fetch models.json from (empty = disabled)
	URL string `yaml:"url" env:"MODEL_LIST_URL"`
}

ModelListConfig holds configuration for fetching the external model metadata registry.

type ModelsConfig 

type ModelsConfig struct {
	// EnabledByDefault controls whether provider models are available
	// when no persisted user-path access override exists.
	// Default: true.
	EnabledByDefault bool `yaml:"enabled_by_default" env:"MODELS_ENABLED_BY_DEFAULT"`

	// KeepOnlyAliasesAtModelsEndpoint controls whether GET /v1/models hides
	// provider models and returns only alias-projected model entries.
	// Default: false.
	KeepOnlyAliasesAtModelsEndpoint bool `yaml:"keep_only_aliases_at_models_endpoint" env:"KEEP_ONLY_ALIASES_AT_MODELS_ENDPOINT"`

	// ConfiguredProviderModelsMode controls how providers.<name>.models and
	// provider *_MODELS env vars affect the provider model inventory.
	// Supported values: "fallback", "allowlist". Default: "fallback".
	ConfiguredProviderModelsMode ConfiguredProviderModelsMode `yaml:"configured_provider_models_mode" env:"CONFIGURED_PROVIDER_MODELS_MODE"`
}

ModelsConfig holds global model access defaults.

type MongoDBStorageConfig 

type MongoDBStorageConfig struct {
	// URL is the connection string; a database named in its path is honored
	// (e.g., mongodb://localhost:27017/gomodel)
	URL string `yaml:"url" env:"MONGODB_URL"`
	// Database overrides the database named in the URL (default: gomodel)
	Database string `yaml:"database" env:"MONGODB_DATABASE"`
}

MongoDBStorageConfig holds MongoDB-specific storage configuration

type PGVectorConfig 

type PGVectorConfig struct {
	URL       string `yaml:"url"`
	Table     string `yaml:"table"`
	Dimension int    `yaml:"dimension"`
}

PGVectorConfig holds connection configuration for the pgvector vector store.

type PineconeConfig 

type PineconeConfig struct {
	Host      string `yaml:"host"`
	APIKey    string `yaml:"api_key"`
	Namespace string `yaml:"namespace"`
	Dimension int    `yaml:"dimension"`
}

PineconeConfig holds connection configuration for Pinecone (data-plane HTTP API).

type PostgreSQLStorageConfig 

type PostgreSQLStorageConfig struct {
	// URL is the connection string (e.g., postgres://user:pass@localhost/dbname)
	URL string `yaml:"url" env:"POSTGRES_URL"`
	// MaxConns is the maximum connection pool size (default: 10)
	MaxConns int `yaml:"max_conns" env:"POSTGRES_MAX_CONNS"`
}

PostgreSQLStorageConfig holds PostgreSQL-specific storage configuration

type QdrantConfig 

type QdrantConfig struct {
	URL        string `yaml:"url"`
	Collection string `yaml:"collection"`
	APIKey     string `yaml:"api_key"`
}

QdrantConfig holds connection configuration for the Qdrant vector store.

type RateLimitModelConfig 

type RateLimitModelConfig struct {
	Model  string                `yaml:"model"`
	Limits []RateLimitRuleConfig `yaml:"limits"`
}

RateLimitModelConfig declares one or more rate limit rules for a model.

type RateLimitProviderConfig 

type RateLimitProviderConfig struct {
	Name   string                `yaml:"name"`
	Limits []RateLimitRuleConfig `yaml:"limits"`
}

RateLimitProviderConfig declares one or more rate limit rules for a provider.

type RateLimitRuleConfig 

type RateLimitRuleConfig struct {
	// Period accepts minute, hour, day, or concurrent. The resolved period is
	// persisted as PeriodSeconds in the database.
	Period string `yaml:"period" json:"period"`

	// PeriodSeconds can be set directly instead of Period for custom windows.
	// 0 means the concurrent (in-flight) limit.
	PeriodSeconds *int64 `yaml:"period_seconds" json:"period_seconds"`

	// MaxRequests caps requests per period, or in-flight requests for the
	// concurrent period.
	MaxRequests *int64 `yaml:"max_requests" json:"max_requests"`

	// MaxTokens caps total tokens per period. Not valid for the concurrent
	// period. Requires usage tracking to be enforced.
	MaxTokens *int64 `yaml:"max_tokens" json:"max_tokens"`
}

RateLimitRuleConfig declares the limits for one period. The json tags support the JSON-array form of SET_RATE_LIMIT_* env values.

type RateLimitUserPathConfig 

type RateLimitUserPathConfig struct {
	Path   string                `yaml:"path"`
	Limits []RateLimitRuleConfig `yaml:"limits"`
}

RateLimitUserPathConfig declares one or more rate limit rules for a user path.

type RateLimitsConfig 

type RateLimitsConfig struct {
	// Enabled controls whether rate limit checks are active.
	// Default: true. With no rules configured the check is a no-op.
	Enabled bool `yaml:"enabled" env:"RATE_LIMITS_ENABLED"`

	// UserPaths declares rate limit rules by tracked user path.
	UserPaths []RateLimitUserPathConfig `yaml:"user_paths"`

	// Providers declares rate limit rules by configured provider name.
	// Provider rules cap all traffic routed to that provider instance; load
	// balancing and failover skip a saturated provider while capacity exists
	// elsewhere.
	Providers []RateLimitProviderConfig `yaml:"providers"`

	// Models declares rate limit rules by model. A provider-qualified subject
	// ("openai/gpt-4o") caps one provider's model; a bare id ("gpt-4o") caps
	// the model across every provider.
	Models []RateLimitModelConfig `yaml:"models"`
}

RateLimitsConfig holds request, token, and concurrency limits scoped to user paths (consumers), providers, and models.

type RawCircuitBreakerConfig 

type RawCircuitBreakerConfig struct {
	FailureThreshold *int           `yaml:"failure_threshold"`
	SuccessThreshold *int           `yaml:"success_threshold"`
	Timeout          *time.Duration `yaml:"timeout"`
}

RawCircuitBreakerConfig holds optional per-provider circuit breaker overrides from YAML. Nil fields inherit from the global CircuitBreakerConfig.

type RawProviderConfig 

type RawProviderConfig struct {
	Type   string `yaml:"type"`
	APIKey string `yaml:"api_key"`
	// APIKeys lists additional API keys for this provider. When more than one
	// key is resolved (counting APIKey), requests rotate across them round
	// robin. Set it via `api_keys:` or the `<PROVIDER>_API_KEY_<n>` env vars.
	APIKeys                  []string             `yaml:"api_keys"`
	BaseURL                  string               `yaml:"base_url"`
	APIVersion               string               `yaml:"api_version"`
	Backend                  string               `yaml:"backend"`
	AuthType                 string               `yaml:"auth_type"`
	APIMode                  string               `yaml:"api_mode"`
	VertexProject            string               `yaml:"vertex_project"`
	VertexLocation           string               `yaml:"vertex_location"`
	ServiceAccountFile       string               `yaml:"service_account_file"`
	ServiceAccountJSON       string               `yaml:"service_account_json"`
	ServiceAccountJSONBase64 string               `yaml:"service_account_json_base64"`
	GCPScope                 string               `yaml:"gcp_scope"`
	Models                   []RawProviderModel   `yaml:"models"`
	Resilience               *RawResilienceConfig `yaml:"resilience"`
}

RawProviderConfig is the YAML-sourced provider configuration before env var overrides, credential filtering, or resilience merging. Exported so the providers package can resolve it into a fully-configured ProviderConfig.

type RawProviderModel 

type RawProviderModel struct {
	ID       string              `yaml:"id"`
	Metadata *core.ModelMetadata `yaml:"metadata,omitempty"`
}

RawProviderModel is a single entry under providers.<name>.models. It supports two YAML shapes so operators can opt into rich metadata without churning simple configs: 

models:
  - some-model-id                    # bare string
  - id: local-model                  # mapping with optional metadata
    metadata:
      context_window: 131072
      capabilities:
        tools: true

Metadata is merged onto whatever the remote model registry supplies, with config-declared fields taking precedence. This lets local providers (Ollama, custom OpenAI-compatible endpoints) advertise their context windows, pricing, and capabilities via /v1/models even when the remote registry has no entry.

func (*RawProviderModel) UnmarshalYAML 

func (m *RawProviderModel) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML accepts either a bare string (model ID) or a mapping with id and metadata.

type RawResilienceConfig 

type RawResilienceConfig struct {
	Retry          *RawRetryConfig          `yaml:"retry"`
	CircuitBreaker *RawCircuitBreakerConfig `yaml:"circuit_breaker"`
}

RawResilienceConfig holds optional per-provider resilience overrides from YAML. Nil fields inherit from the global ResilienceConfig.

type RawRetryConfig 

type RawRetryConfig struct {
	MaxRetries     *int           `yaml:"max_retries"`
	InitialBackoff *time.Duration `yaml:"initial_backoff"`
	MaxBackoff     *time.Duration `yaml:"max_backoff"`
	BackoffFactor  *float64       `yaml:"backoff_factor"`
	JitterFactor   *float64       `yaml:"jitter_factor"`
}

RawRetryConfig holds optional per-provider retry overrides from YAML. Nil fields inherit from the global RetryConfig.

type RedisModelConfig 

type RedisModelConfig struct {
	URL string `yaml:"url" env:"REDIS_URL"`
	Key string `yaml:"key" env:"REDIS_KEY_MODELS"`
	TTL int    `yaml:"ttl" env:"REDIS_TTL_MODELS"`
}

RedisModelConfig holds Redis connection configuration for the model registry cache.

type RedisResponseConfig 

type RedisResponseConfig struct {
	URL string `yaml:"url"`
	Key string `yaml:"key"`
	TTL int    `yaml:"ttl"`
}

RedisResponseConfig holds Redis connection configuration for the response cache. Uses separate env vars from RedisModelConfig for key and TTL to allow independent configuration. The URL is shared via REDIS_URL to simplify single-Redis deployments; use YAML config if different Redis instances are needed for model and response caches. Env vars are applied in Load via applyResponseSimpleEnv, only when cache.response.simple is present (see RESPONSE_CACHE_SIMPLE_ENABLED for env-only opt-in without YAML).

type ResilienceConfig 

type ResilienceConfig struct {
	Retry          RetryConfig          `yaml:"retry"`
	CircuitBreaker CircuitBreakerConfig `yaml:"circuit_breaker"`
}

ResilienceConfig holds resolved resilience settings (retry and circuit breaker).

type ResponseCacheConfig 

type ResponseCacheConfig struct {
	Simple   *SimpleCacheConfig   `yaml:"simple"`
	Semantic *SemanticCacheConfig `yaml:"semantic"`
}

ResponseCacheConfig holds configuration for response cache middleware.

type RetryConfig 

type RetryConfig struct {
	MaxRetries     int           `yaml:"max_retries"     env:"RETRY_MAX_RETRIES"`
	InitialBackoff time.Duration `yaml:"initial_backoff" env:"RETRY_INITIAL_BACKOFF"`
	MaxBackoff     time.Duration `yaml:"max_backoff"     env:"RETRY_MAX_BACKOFF"`
	BackoffFactor  float64       `yaml:"backoff_factor"  env:"RETRY_BACKOFF_FACTOR"`
	JitterFactor   float64       `yaml:"jitter_factor"   env:"RETRY_JITTER_FACTOR"`
}

RetryConfig holds resolved retry settings for an LLM client. This is the canonical type shared between config and llmclient.

func DefaultRetryConfig 

func DefaultRetryConfig() RetryConfig

DefaultRetryConfig returns the default retry settings.

type SQLiteStorageConfig 

type SQLiteStorageConfig struct {
	// Path is the database file path. Default: ./data/gomodel.db when a
	// ./data directory exists, otherwise the OS per-user data directory
	// (e.g. ~/.local/share/gomodel/gomodel.db).
	Path string `yaml:"path" env:"SQLITE_PATH"`
}

SQLiteStorageConfig holds SQLite-specific storage configuration

type SemanticCacheConfig 

type SemanticCacheConfig struct {
	Enabled                 *bool             `yaml:"enabled"`
	SimilarityThreshold     float64           `yaml:"similarity_threshold"`
	TTL                     *int              `yaml:"ttl"`
	MaxConversationMessages *int              `yaml:"max_conversation_messages"`
	ExcludeSystemPrompt     bool              `yaml:"exclude_system_prompt"`
	Embedder                EmbedderConfig    `yaml:"embedder"`
	VectorStore             VectorStoreConfig `yaml:"vector_store"`
}

SemanticCacheConfig holds configuration for the semantic (vector-similarity) response cache. When the semantic block is omitted from config.yaml, this layer stays off unless SEMANTIC_CACHE_ENABLED=true is set. Omitted enabled (nil) means true whenever the semantic block exists. Tuning env vars are applied in Load via applyResponseSemanticEnv when this block exists.

type ServerConfig 

type ServerConfig struct {
	Port           string `yaml:"port" env:"PORT"`
	BasePath       string `yaml:"base_path" env:"BASE_PATH"`             // URL path prefix where the app is mounted (e.g., "/g")
	MasterKey      string `yaml:"master_key" env:"GOMODEL_MASTER_KEY"`   // Optional: Master key for authentication
	BodySizeLimit  string `yaml:"body_size_limit" env:"BODY_SIZE_LIMIT"` // Max request body size (e.g., "10M", "1024K")
	SwaggerEnabled bool   `yaml:"swagger_enabled" env:"SWAGGER_ENABLED"` // Whether to expose the Swagger UI at /swagger/index.html
	PprofEnabled   bool   `yaml:"pprof_enabled" env:"PPROF_ENABLED"`     // Whether to expose debug profiling routes at /debug/pprof/*
	// EnablePassthroughRoutes exposes provider-native passthrough endpoints under
	// /p/{provider}/{endpoint}. Default: true.
	EnablePassthroughRoutes bool `yaml:"enable_passthrough_routes" env:"ENABLE_PASSTHROUGH_ROUTES"`
	// AllowPassthroughV1Alias allows /p/{provider}/v1/... style passthrough routes
	// while keeping /p/{provider}/... as the canonical form. Default: true.
	AllowPassthroughV1Alias bool `yaml:"allow_passthrough_v1_alias" env:"ALLOW_PASSTHROUGH_V1_ALIAS"`
	// UserPathHeader is the inbound HTTP header used to read/write user paths.
	// Default: X-GoModel-User-Path.
	UserPathHeader string `yaml:"user_path_header" env:"USER_PATH_HEADER"`
	// EnabledPassthroughProviders lists the provider types enabled on
	// /p/{provider}/... passthrough routes. Default:
	// ["openai", "anthropic", "openrouter", "kilo", "zai", "vllm", "deepseek"].
	EnabledPassthroughProviders []string `yaml:"enabled_passthrough_providers" env:"ENABLED_PASSTHROUGH_PROVIDERS"`
	// RealtimeEnabled exposes the realtime (speech-to-speech) websocket endpoint
	// at /v1/realtime and the /p/{provider}/v1/realtime passthrough upgrade.
	// Default: true. Only providers implementing realtime accept sessions.
	RealtimeEnabled bool `yaml:"realtime_enabled" env:"REALTIME_ENABLED"`
}

ServerConfig holds HTTP server configuration

type SimpleCacheConfig 

type SimpleCacheConfig struct {
	Enabled *bool                `yaml:"enabled"`
	Redis   *RedisResponseConfig `yaml:"redis"`
}

SimpleCacheConfig holds configuration for exact-match response caching. When the simple block is omitted from config.yaml, this layer stays off unless RESPONSE_CACHE_SIMPLE_ENABLED=true is set (e.g. Helm without a response-cache YAML fragment). Omitted enabled (nil) means true whenever the simple block exists.

type StorageConfig 

type StorageConfig struct {
	// Type specifies the storage backend: "sqlite" (default), "postgresql", or "mongodb"
	Type string `yaml:"type" env:"STORAGE_TYPE"`

	// SQLite configuration
	SQLite SQLiteStorageConfig `yaml:"sqlite"`

	// PostgreSQL configuration
	PostgreSQL PostgreSQLStorageConfig `yaml:"postgresql"`

	// MongoDB configuration
	MongoDB MongoDBStorageConfig `yaml:"mongodb"`
}

StorageConfig holds database storage configuration (used by audit logging, usage tracking, future IAM, etc.)

func (StorageConfig) BackendConfig 

func (c StorageConfig) BackendConfig() storage.Config

BackendConfig converts the application storage config into the internal storage config.

type SystemPromptSettings 

type SystemPromptSettings struct {
	// Mode controls how the system prompt is applied: "inject", "override", or "decorator"
	//   - inject: adds a system message only if none exists
	//   - override: replaces all existing system messages
	//   - decorator: prepends to the first existing system message
	// Default: "inject"
	Mode string `yaml:"mode"`

	// Content is the system prompt text to apply
	Content string `yaml:"content"`
}

SystemPromptSettings holds the type-specific settings for a system_prompt guardrail.

type TaggingConfig 

type TaggingConfig struct {
	Headers []TaggingHeaderConfig `yaml:"headers"`
}

TaggingConfig declares request labelling based on HTTP headers. Headers listed here are read on every request; their values become request labels. Declarative entries override admin-store rows with the same header name and are read-only in the dashboard.

type TaggingHeaderConfig 

type TaggingHeaderConfig struct {
	// Header is the HTTP header name to read labels from.
	Header string `yaml:"header" json:"header"`

	// Prefix is optionally trimmed from the front of each label. Trimming only
	// affects the extracted label, never the forwarded header value.
	Prefix string `yaml:"prefix,omitempty" json:"prefix,omitempty"`

	// DoNotPass strips the header before forwarding the request upstream.
	// Default: false (headers are passed through as-is).
	DoNotPass bool `yaml:"do_not_pass,omitempty" json:"do_not_pass,omitempty"`

	// Delimiter splits one header value into multiple labels. Default: ",".
	Delimiter string `yaml:"delimiter,omitempty" json:"delimiter,omitempty"`
}

TaggingHeaderConfig declares one header to extract labels from.

type UsageConfig 

type UsageConfig struct {
	// Enabled controls whether usage tracking is active
	// Default: true
	Enabled bool `yaml:"enabled" env:"USAGE_ENABLED"`

	// EnforceReturningUsageData controls whether to ask streaming providers to return usage data when possible.
	// When true, stream_options: {"include_usage": true} is added for provider paths that support it.
	// Default: true
	EnforceReturningUsageData bool `yaml:"enforce_returning_usage_data" env:"ENFORCE_RETURNING_USAGE_DATA"`

	// PricingRecalculationEnabled controls whether the admin pricing recalculation action is available.
	// Storage and pricing metadata support are still required; false always disables the feature.
	// Default: true
	PricingRecalculationEnabled bool `yaml:"pricing_recalculation_enabled" env:"USAGE_PRICING_RECALCULATION_ENABLED"`

	// BufferSize is the number of usage entries to buffer before flushing
	// Default: 1000
	BufferSize int `yaml:"buffer_size" env:"USAGE_BUFFER_SIZE"`

	// FlushInterval is how often to flush buffered usage entries (in seconds)
	// Default: 5
	FlushInterval int `yaml:"flush_interval" env:"USAGE_FLUSH_INTERVAL"`

	// RetentionDays is how long to keep usage data (0 = forever)
	// Default: 90
	RetentionDays int `yaml:"retention_days" env:"USAGE_RETENTION_DAYS"`
}

UsageConfig holds token usage tracking configuration

type VectorStoreConfig 

type VectorStoreConfig struct {
	Type     string         `yaml:"type"`
	Qdrant   QdrantConfig   `yaml:"qdrant"`
	PGVector PGVectorConfig `yaml:"pgvector"`
	Pinecone PineconeConfig `yaml:"pinecone"`
	Weaviate WeaviateConfig `yaml:"weaviate"`
}

VectorStoreConfig selects the vector DB backend. Type must be set when semantic caching is enabled: qdrant, pgvector, pinecone, weaviate.

type VirtualModelConfig 

type VirtualModelConfig struct {
	// Source is the addressable virtual model name (for a redirect/load balancer)
	// or the access selector (for an access policy).
	Source string `yaml:"source" json:"source"`

	// Strategy selects load balancing across multiple targets: "round_robin"
	// (default) or "cost". Ignored for single-target aliases and access policies.
	Strategy string `yaml:"strategy,omitempty" json:"strategy,omitempty"`

	// Target is shorthand for a single-target alias, e.g. "openai/gpt-4o". Use
	// Targets instead to load balance across several models.
	Target string `yaml:"target,omitempty" json:"target,omitempty"`

	// Targets are the redirect destinations. One target is a plain alias; several
	// are load balanced across by Strategy.
	Targets []VirtualModelTargetConfig `yaml:"targets,omitempty" json:"targets,omitempty"`

	// UserPaths scopes the entry to specific request user paths. Empty means all.
	UserPaths []string `yaml:"user_paths,omitempty" json:"user_paths,omitempty"`

	// Description is an optional human-readable note.
	Description string `yaml:"description,omitempty" json:"description,omitempty"`

	// Enabled toggles the entry. It defaults to true when omitted.
	Enabled *bool `yaml:"enabled,omitempty" json:"enabled,omitempty"`
}

VirtualModelConfig declares one virtual model in config.yaml or the VIRTUAL_MODELS env var, so operators can manage redirects, load balancers, and access policies as infrastructure-as-code. Declarative virtual models override admin-store rows of the same source and are read-only in the dashboard.

type VirtualModelTargetConfig 

type VirtualModelTargetConfig struct {
	Provider string  `yaml:"provider,omitempty" json:"provider,omitempty"`
	Model    string  `yaml:"model" json:"model"`
	Weight   float64 `yaml:"weight,omitempty" json:"weight,omitempty"`
}

VirtualModelTargetConfig is one load-balancing destination. Model may be a bare id (with Provider set) or a "provider/model" selector. Weight biases the round_robin strategy and defaults to 1.

type WeaviateConfig 

type WeaviateConfig struct {
	URL    string `yaml:"url"`
	Class  string `yaml:"class"`
	APIKey string `yaml:"api_key"`
}

WeaviateConfig holds connection configuration for Weaviate.

type WorkflowsConfig 

type WorkflowsConfig struct {
	// RefreshInterval controls how often the in-memory workflow snapshot
	// is refreshed from storage. Default: 1m.
	RefreshInterval time.Duration `yaml:"refresh_interval" env:"WORKFLOW_REFRESH_INTERVAL"`
}

WorkflowsConfig holds runtime refresh behavior for persisted workflows.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.