aigateway

README

Ferro Labs AI Gateway

Open-source, OpenAI-compatible AI gateway built in Go.
Route, govern, and observe LLM traffic across multiple providers through one API.

---

Ferro Labs AI Gateway is a lightweight control plane that sits between your app and model providers. It exposes OpenAI-style endpoints (/v1/chat/completions, /v1/models, /v1/embeddings, /v1/images/generations) while handling routing, retries, OSS plugins, logging, admin controls, and provider auth centrally.

Why Ferro

• OpenAI-compatible API surface for easier migration and standard client support.
• 29 built-in providers with one canonical provider registry and OpenAI-style routing surface.
• Multi-provider routing with 8 strategy modes: single, fallback, loadbalance, conditional, least-latency, cost-optimized, content-based, ab-test.
• Built-in resilience via per-target retry controls and circuit breakers.
• Built-in governance hooks via plugin lifecycle stages (before_request, after_request, on_error).
• Operational visibility through structured logs, /metrics, deep /health, admin APIs, and dashboard UI.
• Production-friendly storage options for runtime config, API keys, and request logs (memory, sqlite, postgres).
• MCP tool server integration — attach any MCP 2025-11-25 Streamable HTTP server and let the LLM drive an agentic tool-call loop without changing client code.

What Ships In v1.0.0-rc.1

• One API across 29 providers — route OpenAI-compatible traffic to OpenAI, Anthropic, Gemini, Groq, Bedrock, Vertex AI, Hugging Face, OpenRouter, Cloudflare, Qwen, Moonshot, and more.
• Smart routing built in — use fallback, weighted load balancing, latency-aware routing, cost-aware routing, content-based routing, and A/B traffic splits without changing your client API.
• Focused OSS plugin surface — ship guardrails, caching, logging, rate limiting, and budget controls with a small, understandable built-in plugin set.
• Operations included — expose /health, /metrics, admin APIs, persistent config/key storage, request logs, and a built-in dashboard.
• Agent workflows supported — connect MCP tool servers and let the gateway manage tool discovery and loop execution.

Performance work

Recent gateway-overhead work has focused on making the Go implementation pay off in the real hot path, not just in theory. The current optimization passes include shared transport reuse, precomputed model/provider indexes, faster proxy model extraction, lower-allocation events/SSE writes, local pprof profiling support, and a lighter non-stream OpenAI completion path.

The benchmarking repo in ai-gateway-performance-benchmarks now supports rebuilding FerroGateway from the local workspace for iterative profiling. Recent isolated local runs improved the short baseline scenario from roughly 10.6k rps to about 14.8k rps after profile-guided fixes. Final cross-gateway publication numbers should still come from identical gateway-in-path comparison runs, not from these local optimization snapshots.

See CHANGELOG.md for full release notes.

Quick Start

Run with Docker

docker run --rm -p 8080:8080 \
  -e OPENAI_API_KEY=sk-your-key \
  ghcr.io/ferro-labs/ai-gateway:latest

Build from source

git clone https://github.com/ferro-labs/ai-gateway.git
cd ai-gateway

export OPENAI_API_KEY=sk-your-key
make run

Verify gateway is running

curl -s http://localhost:8080/health | jq
curl -s http://localhost:8080/v1/models | jq '.data | length'

Send your first request

curl -s http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer sk-ferro-or-upstream-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role":"user","content":"Say hello from Ferro Gateway"}]
  }' | jq

Core Endpoints

Public/API routes

• GET /health — deep provider health summary.
• GET /metrics — Prometheus metrics.
• GET /v1/models — OpenAI-style model list, enriched from catalog when available.
• POST /v1/chat/completions — chat (streaming + non-streaming).
• POST /v1/completions — legacy text completions.
• POST /v1/embeddings — embeddings.
• POST /v1/images/generations — image generation.
• GET /dashboard — built-in admin dashboard page.
• GET /v1/* (other) — transparent proxy pass-through to provider.

Admin routes (Bearer auth required)

Read scope (read_only or admin):

• GET /admin/dashboard
• GET /admin/keys, GET /admin/keys/{id}, GET /admin/keys/usage
• GET /admin/providers, GET /admin/health
• GET /admin/config, GET /admin/config/history
• GET /admin/logs, GET /admin/logs/stats

Admin scope (admin only):

• POST /admin/keys, PUT /admin/keys/{id}, DELETE /admin/keys/{id}
• POST /admin/keys/{id}/revoke, POST /admin/keys/{id}/rotate
• POST /admin/config, PUT /admin/config, DELETE /admin/config
• POST /admin/config/rollback/{version}
• DELETE /admin/logs

MCP tool server integration

Attach any Model Context Protocol 2025-11-25 Streamable HTTP server. The gateway initialises all configured servers at startup, injects the discovered tools into every LLM request, and drives the agentic tool_calls loop automatically up to max_call_depth iterations.

mcp_servers:
  - name: my-tools
    url: https://mcp.example.com/mcp
    headers:
      Authorization: Bearer ${MY_TOOLS_TOKEN}
    allowed_tools: [search, get_weather]   # omit to allow all
    max_call_depth: 5
    timeout_seconds: 30

Multiple servers can be registered. Tools are deduplicated by name across servers.

Routing strategy modes

Set strategy.mode in your config:

• single — fixed provider target.
• fallback — try targets in order, with per-target retry.
• loadbalance — weighted selection across targets.
• conditional — route by request metadata conditions.
• least-latency — pick lowest observed latency provider.
• cost-optimized — pick cheapest compatible provider from model catalog pricing.
• content-based — route by prompt content using contains, not-contains, or regex rules; first matching rule wins, falls back to the first target.
• ab-test — weighted random traffic split across named variants; zero-weight variants are treated as equal weight.

Configuration

Use GATEWAY_CONFIG to load YAML/JSON config at startup.

export GATEWAY_CONFIG=./config.yaml

Minimal production-style example:

strategy:
  mode: fallback

targets:
  - virtual_key: openai
    retry:
      attempts: 3
      on_status_codes: [429, 502, 503]
      initial_backoff_ms: 100

  - virtual_key: anthropic
    retry:
      attempts: 2

aliases:
  fast: gpt-4o-mini
  smart: claude-3-5-sonnet-20241022

plugins:
  - name: request-logger
    type: logging
    stage: before_request
    enabled: true
    config:
      level: info
      persist: true
      backend: sqlite
      dsn: ferrogw-requests.db

See config.example.yaml and config.example.json for a full template.

Built-In Providers

Canonical provider keys used in config (targets[].virtual_key):

Virtual Key	Provider	Enablement Env
ai21	AI21	AI21_API_KEY
anthropic	Anthropic	ANTHROPIC_API_KEY
azure-foundry	Azure Foundry	AZURE_FOUNDRY_API_KEY + AZURE_FOUNDRY_ENDPOINT
azure-openai	Azure OpenAI	AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT + AZURE_OPENAI_DEPLOYMENT
bedrock	AWS Bedrock	AWS_REGION or AWS_ACCESS_KEY_ID
cerebras	Cerebras	CEREBRAS_API_KEY
cloudflare	Cloudflare Workers AI	CLOUDFLARE_API_KEY + CLOUDFLARE_ACCOUNT_ID
cohere	Cohere	COHERE_API_KEY
databricks	Databricks	DATABRICKS_TOKEN + DATABRICKS_HOST
deepinfra	DeepInfra	DEEPINFRA_API_KEY
deepseek	DeepSeek	DEEPSEEK_API_KEY
fireworks	Fireworks	FIREWORKS_API_KEY
gemini	Google Gemini	GEMINI_API_KEY
groq	Groq	GROQ_API_KEY
hugging-face	Hugging Face	HUGGING_FACE_API_KEY
mistral	Mistral	MISTRAL_API_KEY
moonshot	Moonshot AI / Kimi	MOONSHOT_API_KEY
novita	Novita AI	NOVITA_API_KEY
nvidia-nim	NVIDIA NIM	NVIDIA_NIM_API_KEY
ollama	Ollama	OLLAMA_HOST
openai	OpenAI	OPENAI_API_KEY
openrouter	OpenRouter	OPENROUTER_API_KEY
perplexity	Perplexity	PERPLEXITY_API_KEY
qwen	Qwen / DashScope	QWEN_API_KEY
replicate	Replicate	REPLICATE_API_TOKEN
sambanova	SambaNova	SAMBANOVA_API_KEY
together	Together AI	TOGETHER_API_KEY
vertex-ai	Vertex AI	VERTEX_AI_PROJECT_ID + VERTEX_AI_REGION + (VERTEX_AI_API_KEY or VERTEX_AI_SERVICE_ACCOUNT_JSON)
xai	xAI	XAI_API_KEY

Built-In Plugins

Registered OSS plugin set:

• Guardrails: word-filter, max-token
• Transform: response-cache
• Logging: request-logger
• Rate limiting: rate-limit — global RPM/RPS plus optional key_rpm (per-API-key) and user_rpm (per-user) limits.
• Budget control: budget — per-API-key USD spend tracking and enforcement with configurable input/output token pricing.

Inspect available plugins with:

make build-cli
./bin/ferrogw-cli plugins

Persistence Backends

Configure state backends with environment variables:

Area	Backend Env	DSN Env	Values
Runtime config	CONFIG_STORE_BACKEND	CONFIG_STORE_DSN	memory (default), sqlite, postgres
API keys	API_KEY_STORE_BACKEND	API_KEY_STORE_DSN	memory (default), sqlite, postgres
Request logs	REQUEST_LOG_STORE_BACKEND	REQUEST_LOG_STORE_DSN	sqlite, postgres (unset = disabled)

SQLite local example:

export CONFIG_STORE_BACKEND=sqlite
export CONFIG_STORE_DSN=./ferrogw-config.db

export API_KEY_STORE_BACKEND=sqlite
export API_KEY_STORE_DSN=./ferrogw-keys.db

export REQUEST_LOG_STORE_BACKEND=sqlite
export REQUEST_LOG_STORE_DSN=./ferrogw-requests.db

PostgreSQL example:

export CONFIG_STORE_BACKEND=postgres
export CONFIG_STORE_DSN='postgresql://user:pass@db:5432/ferrogw?sslmode=require'

export API_KEY_STORE_BACKEND=postgres
export API_KEY_STORE_DSN='postgresql://user:pass@db:5432/ferrogw?sslmode=require'

export REQUEST_LOG_STORE_BACKEND=postgres
export REQUEST_LOG_STORE_DSN='postgresql://user:pass@db:5432/ferrogw?sslmode=require'

Production Notes

• CORS defaults to wildcard when CORS_ORIGINS is unset/empty. Set explicit origins in production.
• Bootstrap keys (ADMIN_BOOTSTRAP_KEY, ADMIN_BOOTSTRAP_READ_ONLY_KEY) are for first-run setup only.
• Prefer TLS-backed Postgres DSNs (sslmode=require or stronger).
• Use scoped admin keys and rotate periodically.

Bootstrap key quick setup:

export ADMIN_BOOTSTRAP_ENABLED=true
export ADMIN_BOOTSTRAP_KEY='change-me-admin'
export ADMIN_BOOTSTRAP_READ_ONLY_KEY='change-me-readonly'

CLI

ferrogw-cli manages and inspects a running gateway:

make build-cli

./bin/ferrogw-cli validate config.example.yaml
./bin/ferrogw-cli plugins
./bin/ferrogw-cli admin providers list --api-key "$FERROGW_API_KEY"
./bin/ferrogw-cli admin config history --api-key "$FERROGW_API_KEY"

Persistent CLI flags:

• --gateway-url (env: FERROGW_URL, default: http://localhost:8080)
• --api-key (env: FERROGW_API_KEY)
• --format (table, json, yaml)

Development

Common Make targets:

make deps
make fmt
make lint
make test
make test-coverage
make test-integration
make bench

Run release validation locally:

make release-check
make release-dry-run

OpenAI SDK Migration

Point existing OpenAI SDK clients to Ferro Gateway by changing only the base URL.

Python

from openai import OpenAI

client = OpenAI(
    api_key="sk-ferro-...",
    base_url="http://localhost:8080/v1",
)

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "sk-ferro-...",
  baseURL: "http://localhost:8080/v1",
});

Examples

Runnable examples now live in the dedicated examples repository: ferro-labs/ai-gateway-examples.

Use that repo for migration samples, streaming demos, MCP flows, and integration examples.

Roadmap

The current release roadmap is maintained in ROADMAP.md.

Contributing

Contributions are welcome. Please read CONTRIBUTING.md and follow CODE_OF_CONDUCT.md.

License

Licensed under Apache 2.0.

Documentation

Overview

Package aigateway provides a high-performance, zero-dependency AI gateway for routing requests to large language model (LLM) providers.

The Gateway type is the main entry point: create one with New, register providers with RegisterProvider, load plugins from config with LoadPlugins, and route requests with Route or RouteStream.

Plugins and routing strategies (single, fallback, load-balance, conditional, content-based, ab-test) are configured via Config which can be loaded from a YAML or JSON file using LoadConfig.

Index

• Constants
• func ValidateConfig(cfg Config) error
• type ABVariantConfig
• type CircuitBreakerConfig
• type Condition
• type Config
  • func LoadConfig(path string) (*Config, error)
• type ContentCondition
• type EventHookFunc
• type Gateway
  • func New(cfg Config) (*Gateway, error)
  • func (g *Gateway) AddHook(fn EventHookFunc)
  • func (g *Gateway) AllModels() []providers.ModelInfo
  • func (g *Gateway) Catalog() models.Catalog
  • func (g *Gateway) Close() error
  • func (g *Gateway) Embed(ctx context.Context, req providers.EmbeddingRequest) (*providers.EmbeddingResponse, error)
  • func (g *Gateway) FindByModel(model string) (providers.Provider, bool)
  • func (g *Gateway) FindStreamingByModel(model string) (providers.StreamProvider, bool)
  • func (g *Gateway) GenerateImage(ctx context.Context, req providers.ImageRequest) (*providers.ImageResponse, error)
  • func (g *Gateway) Get(name string) (providers.Provider, bool)
  • func (g *Gateway) GetConfig() Config
  • func (g *Gateway) GetProvider(name string) (providers.Provider, bool)
  • func (g *Gateway) List() []string
  • func (g *Gateway) ListProviders() []string
  • func (g *Gateway) LoadPlugins() error
  • func (g *Gateway) MCPInitDone() <-chan struct{}
  • func (g *Gateway) RegisterPlugin(stage plugin.Stage, p plugin.Plugin) error
  • func (g *Gateway) RegisterProvider(p providers.Provider)
  • func (g *Gateway) ReloadConfig(cfg Config) error
  • func (g *Gateway) Route(ctx context.Context, req providers.Request) (*providers.Response, error)
  • func (g *Gateway) RouteStream(ctx context.Context, req providers.Request) (<-chan providers.StreamChunk, error)
  • func (g *Gateway) StartDiscovery(ctx context.Context, interval time.Duration) error
• type PluginConfig
• type RetryConfig
• type StrategyConfig
• type StrategyMode
• type Target

Constants

const (
	SubjectRequestCompleted = "gateway.request.completed"
	SubjectRequestFailed    = "gateway.request.failed"
)

Event subject constants used when invoking gateway hooks.

Functions

func ValidateConfig

func ValidateConfig(cfg Config) error

ValidateConfig validates a Config for correctness.

Types

type ABVariantConfig

type ABVariantConfig struct {
	// TargetKey is the virtual_key of the provider for this variant.
	TargetKey string `json:"target_key" yaml:"target_key"`
	// Weight is the relative traffic share for this variant.
	// All weights are summed; each variant's fraction is Weight/Total.
	// Zero is treated as 1 (equal distribution).
	Weight float64 `json:"weight" yaml:"weight"`
	// Label is a short human-readable identifier (e.g. "control", "challenger").
	// It is logged with every routed request for observability.
	Label string `json:"label" yaml:"label"`
}

ABVariantConfig defines a single traffic variant for the "ab-test" strategy.

type CircuitBreakerConfig

type CircuitBreakerConfig struct {
	// FailureThreshold is the number of consecutive failures before the circuit
	// opens. Defaults to 5.
	FailureThreshold int `json:"failure_threshold" yaml:"failure_threshold"`
	// SuccessThreshold is the number of consecutive successes in half-open state
	// required to close the circuit. Defaults to 1.
	SuccessThreshold int `json:"success_threshold" yaml:"success_threshold"`
	// Timeout is the duration the circuit stays open before transitioning to
	// half-open (e.g. "30s"). Defaults to "30s".
	Timeout string `json:"timeout" yaml:"timeout"`
}

CircuitBreakerConfig configures the per-provider circuit breaker.

type Condition

type Condition struct {
	Key       string `json:"key" yaml:"key"`
	Value     string `json:"value" yaml:"value"`
	TargetKey string `json:"target_key" yaml:"target_key"`
}

Condition represents a condition for conditional routing.

type Config

type Config struct {
	// Strategy defines how requests are routed (e.g., single, fallback, loadbalance).
	Strategy StrategyConfig `json:"strategy" yaml:"strategy"`
	// Targets is a list of provider targets to route requests to.
	Targets []Target `json:"targets" yaml:"targets"`
	// Plugins configuration (optional).
	Plugins []PluginConfig `json:"plugins,omitempty" yaml:"plugins,omitempty"`
	// Aliases maps friendly model names (e.g. "fast", "smart") to concrete model IDs.
	// Aliases are resolved before routing — they must not reference other aliases.
	Aliases map[string]string `json:"aliases,omitempty" yaml:"aliases,omitempty"`
	// MCPServers configures external MCP tool servers for agentic tool calling.
	// When set, the gateway injects discovered tools into every chat completion
	// request and executes an agentic loop when the LLM returns tool_calls.
	// FerroCloud populates this field from the tenant's mcp_servers table at
	// gateway.New() time — no separate MCPRegistry() public method is exposed.
	MCPServers []mcp.ServerConfig `json:"mcp_servers,omitempty" yaml:"mcp_servers,omitempty"`
	// MCPToolCallAuditFn, if non-nil, is called after every MCP tool invocation.
	// This field cannot be set via JSON or YAML — set it programmatically before
	// calling New. FerroCloud uses it to write async audit entries to the
	// mcp_tool_call_logs table.
	MCPToolCallAuditFn mcp.ToolCallAuditFn `json:"-" yaml:"-"`
}

Config holds the configuration for the AI Gateway.

func LoadConfig

func LoadConfig(path string) (*Config, error)

LoadConfig reads and parses a config file from the given path. Supported formats: JSON (.json), YAML (.yaml, .yml).

type ContentCondition

type ContentCondition struct {
	// Type is the matching rule type.
	Type string `json:"type" yaml:"type"`
	// Value is the substring or regex pattern to match against.
	Value string `json:"value" yaml:"value"`
	// TargetKey is the virtual_key of the provider to route to when this rule matches.
	TargetKey string `json:"target_key" yaml:"target_key"`
}

ContentCondition maps a prompt-content matching rule to a routing target. Used with the "content-based" strategy mode.

Supported types:

• "prompt_contains" — case-insensitive substring match on user messages
• "prompt_not_contains" — true when NO user message contains the value
• "prompt_regex" — Go regular expression match on user messages

type EventHookFunc

type EventHookFunc func(ctx context.Context, subject string, data map[string]interface{})

EventHookFunc is called asynchronously after a gateway event (request completed or failed). It replaces the old EventPublisher interface with a simpler function-based hook pattern.

type Gateway

type Gateway struct {
	// contains filtered or unexported fields
}

Gateway is the main entry point for routing LLM requests.

func New

func New(cfg Config) (*Gateway, error)

New creates a new Gateway instance with the given configuration.

func (*Gateway) AddHook

func (g *Gateway) AddHook(fn EventHookFunc)

AddHook registers an EventHookFunc that is called asynchronously on each completed or failed request. Multiple hooks may be registered; all are invoked for every event on the shared bounded hook worker pool, so hook implementations should return promptly and avoid indefinite blocking.

func (*Gateway) AllModels

func (g *Gateway) AllModels() []providers.ModelInfo

AllModels returns ModelInfo from all registered providers. If auto-discovery has run for a provider, discovered models take precedence over the provider's static model list.

func (*Gateway) Catalog

func (g *Gateway) Catalog() models.Catalog

Catalog returns a shallow copy of the loaded model catalog. A copy is returned so callers cannot mutate the gateway's internal catalog.

func (*Gateway) Close

func (g *Gateway) Close() error

Close cleans up resources.

func (*Gateway) Embed

func (g *Gateway) Embed(ctx context.Context, req providers.EmbeddingRequest) (*providers.EmbeddingResponse, error)

Embed routes an embedding request to the first registered EmbeddingProvider that supports the requested model.

func (*Gateway) FindByModel

func (g *Gateway) FindByModel(model string) (providers.Provider, bool)

FindByModel returns the first registered provider that supports the given model.

func (*Gateway) FindStreamingByModel

func (g *Gateway) FindStreamingByModel(model string) (providers.StreamProvider, bool)

FindStreamingByModel returns the first registered streaming-capable provider that supports the given model.

func (*Gateway) GenerateImage

func (g *Gateway) GenerateImage(ctx context.Context, req providers.ImageRequest) (*providers.ImageResponse, error)

GenerateImage routes an image generation request to the first registered ImageProvider that supports the requested model.

func (*Gateway) Get

func (g *Gateway) Get(name string) (providers.Provider, bool)

Get satisfies providers.ProviderSource (alias for GetProvider).

func (*Gateway) GetConfig

func (g *Gateway) GetConfig() Config

GetConfig returns a copy of the current configuration.

func (*Gateway) GetProvider

func (g *Gateway) GetProvider(name string) (providers.Provider, bool)

GetProvider returns a registered provider by name.

func (*Gateway) List

func (g *Gateway) List() []string

List satisfies providers.ProviderSource (alias for ListProviders).

func (*Gateway) ListProviders

func (g *Gateway) ListProviders() []string

ListProviders returns the names of all registered providers.

func (*Gateway) LoadPlugins

func (g *Gateway) LoadPlugins() error

LoadPlugins initializes and registers plugins from the gateway configuration.

func (*Gateway) MCPInitDone

func (g *Gateway) MCPInitDone() <-chan struct{}

MCPInitDone returns a channel that is closed once all MCP servers have completed their initialization handshake. The channel is pre-closed when no MCP servers are configured.

func (*Gateway) RegisterPlugin

func (g *Gateway) RegisterPlugin(stage plugin.Stage, p plugin.Plugin) error

RegisterPlugin registers a plugin at the given lifecycle stage.

func (*Gateway) RegisterProvider

func (g *Gateway) RegisterProvider(p providers.Provider)

RegisterProvider registers a provider with the gateway.

func (*Gateway) ReloadConfig

func (g *Gateway) ReloadConfig(cfg Config) error

ReloadConfig validates and applies a new configuration, forcing strategy rebuild on next request.

func (*Gateway) Route

func (g *Gateway) Route(ctx context.Context, req providers.Request) (*providers.Response, error)

Route routes a request to the appropriate provider based on the configuration.

func (*Gateway) RouteStream

func (g *Gateway) RouteStream(ctx context.Context, req providers.Request) (<-chan providers.StreamChunk, error)

RouteStream runs before-request plugins then returns a metered streaming response channel. Provider resolution follows the configured strategy mode, then falls back to any registered provider that supports the requested model and streaming. Prometheus metrics and event hooks are emitted when the returned channel drains (matching the behaviour of Route for non-streaming).

When MCP servers are configured the request is routed through Route instead so that the full agentic tool-call loop can run. The final response is wrapped into a single-chunk stream and returned to the caller (Phase 1 behaviour — true final-response streaming is Phase 1.5).

func (*Gateway) StartDiscovery

func (g *Gateway) StartDiscovery(ctx context.Context, interval time.Duration) error

StartDiscovery periodically refreshes model lists from providers that implement DiscoveryProvider. It runs in a background goroutine until ctx is cancelled. interval must be greater than zero; an error is returned otherwise.

type PluginConfig

type PluginConfig struct {
	Name    string                 `json:"name" yaml:"name"`
	Type    string                 `json:"type" yaml:"type"`
	Stage   string                 `json:"stage" yaml:"stage"`
	Enabled bool                   `json:"enabled" yaml:"enabled"`
	Config  map[string]interface{} `json:"config" yaml:"config"`
}

PluginConfig holds plugin configuration.

type RetryConfig

type RetryConfig struct {
	// Attempts is the maximum number of attempts per target (1 = no retries).
	Attempts int `json:"attempts" yaml:"attempts"`
	// OnStatusCodes, when non-empty, limits retries to the listed HTTP status
	// codes. A retry is skipped when the provider returns a code not in the
	// list, and the strategy moves on to the next target immediately.
	// Leave empty to retry on any error (default behaviour).
	// Example: [429, 502, 503]
	OnStatusCodes []int `json:"on_status_codes,omitempty" yaml:"on_status_codes,omitempty"`
	// InitialBackoffMs is the base backoff in milliseconds for the exponential
	// back-off formula: delay = InitialBackoffMs * 2^(attempt-1).
	// Defaults to 100 ms when unset or zero.
	InitialBackoffMs int `json:"initial_backoff_ms,omitempty" yaml:"initial_backoff_ms,omitempty"`
}

RetryConfig defines retry behavior for the fallback strategy.

type StrategyConfig

type StrategyConfig struct {
	Mode       StrategyMode `json:"mode" yaml:"mode"`
	Conditions []Condition  `json:"conditions,omitempty" yaml:"conditions,omitempty"` // For conditional routing
	// ContentConditions defines rules for the content-based routing strategy.
	// Rules are evaluated in order; the first match wins.
	ContentConditions []ContentCondition `json:"content_conditions,omitempty" yaml:"content_conditions,omitempty"`
	// ABVariants defines the weighted variants for the ab-test strategy.
	ABVariants []ABVariantConfig `json:"ab_variants,omitempty" yaml:"ab_variants,omitempty"`
}

StrategyConfig defines the routing strategy.

type StrategyMode

type StrategyMode string

StrategyMode represents the routing strategy mode.

const (
	ModeSingle        StrategyMode = "single"
	ModeFallback      StrategyMode = "fallback"
	ModeLoadBalance   StrategyMode = "loadbalance"
	ModeConditional   StrategyMode = "conditional"
	ModeLatency       StrategyMode = "least-latency"
	ModeCostOptimized StrategyMode = "cost-optimized"
	ModeContentBased  StrategyMode = "content-based"
	ModeABTest        StrategyMode = "ab-test"
)

StrategyMode constants define the supported routing strategies.

type Target

type Target struct {
	// VirtualKey is the unique identifier for the provider (or a virtual key in the vault).
	VirtualKey string `json:"virtual_key" yaml:"virtual_key"`
	// Weight is used for load balancing.
	Weight float64 `json:"weight,omitempty" yaml:"weight,omitempty"`
	// Retry configuration for this target.
	Retry *RetryConfig `json:"retry,omitempty" yaml:"retry,omitempty"`
	// CircuitBreaker configuration for this target (optional).
	CircuitBreaker *CircuitBreakerConfig `json:"circuit_breaker,omitempty" yaml:"circuit_breaker,omitempty"`
}

Target represents a specific provider target.