router

package
v0.13.0-go Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 11, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package router provides smart LLM request routing with 15-dimension weighted scoring and sigmoid confidence calibration.

Entry point: Route() classifies a request and returns which model to use.

Package router implements a smart LLM request router with 15-dimension weighted scoring, sigmoid confidence calibration, and pluggable strategies.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CalculateModelCost 

func CalculateModelCost(
	model string,
	modelPricing map[string]ModelPricing,
	estimatedInputTokens int,
	maxOutputTokens int,
	routingProfile string,
) (costEstimate, baselineCost, savings float64)

CalculateModelCost computes cost for a specific model (used for fallback models).

func FilterByExcludeList 

func FilterByExcludeList(models []string, excludeList map[string]bool) []string

FilterByExcludeList removes user-excluded models from the list.

func FilterByToolCalling 

func FilterByToolCalling(models []string, hasTools bool, supportsToolCalling func(string) bool) []string

FilterByToolCalling filters models to those supporting tool calling.

func FilterByVision 

func FilterByVision(models []string, hasVision bool, supportsVision func(string) bool) []string

FilterByVision filters models to those supporting vision inputs.

func GetFallbackChain 

func GetFallbackChain(tier Tier, tierConfigs map[Tier]TierConfig) []string

GetFallbackChain returns [primary, ...fallbacks] for a tier.

func GetFallbackChainFiltered 

func GetFallbackChainFiltered(
	tier Tier,
	tierConfigs map[Tier]TierConfig,
	estimatedTotalTokens int,
	getContextWindow func(modelID string) (int, bool),
) []string

GetFallbackChainFiltered returns models that can handle the estimated context.

func RegisterStrategy 

func RegisterStrategy(s RouterStrategy)

RegisterStrategy adds a custom routing strategy to the registry.

func TierRank 

func TierRank(t Tier) int

TierRank returns the numeric rank of a tier for comparison.

Types

type ClassifierConfig 

type ClassifierConfig struct {
	LLMModel              string  `json:"llmModel"`
	LLMMaxTokens          int     `json:"llmMaxTokens"`
	LLMTemperature        float64 `json:"llmTemperature"`
	PromptTruncationChars int     `json:"promptTruncationChars"`
	CacheTTLMs            int64   `json:"cacheTtlMs"`
}

ClassifierConfig controls the LLM fallback classifier.

type DimensionScore 

type DimensionScore struct {
	Name   string  // dimension identifier
	Score  float64 // score in [-1, 1]
	Signal string  // human-readable signal, empty if no signal
}

DimensionScore is a single dimension's contribution to the overall score.

type LLMClassifierConfig 

type LLMClassifierConfig struct {
	Model           string
	MaxTokens       int
	Temperature     float64
	TruncationChars int
	CacheTTLMs      int64
}

LLMClassifierConfig controls the LLM fallback classifier behavior.

type ModelPricing 

type ModelPricing struct {
	InputPrice  float64  // per 1M tokens
	OutputPrice float64  // per 1M tokens
	FlatPrice   *float64 // promo flat price per request (overrides token pricing)
}

ModelPricing holds per-model pricing info.

type OverridesConfig 

type OverridesConfig struct {
	MaxTokensForceComplex   int   `json:"maxTokensForceComplex"`
	StructuredOutputMinTier Tier  `json:"structuredOutputMinTier"`
	AmbiguousDefaultTier    Tier  `json:"ambiguousDefaultTier"`
	AgenticMode             *bool `json:"agenticMode,omitempty"`
}

OverridesConfig holds override rules.

type PartialTierConfig 

type PartialTierConfig struct {
	Primary  string   `json:"primary,omitempty"`
	Fallback []string `json:"fallback,omitempty"`
}

PartialTierConfig allows partial override of a TierConfig.

type Promotion 

type Promotion struct {
	Name          string                     `json:"name"`
	StartDate     string                     `json:"startDate"`
	EndDate       string                     `json:"endDate"`
	TierOverrides map[Tier]PartialTierConfig `json:"tierOverrides"`
	Profiles      []string                   `json:"profiles,omitempty"`
}

Promotion is a time-windowed tier override.

type RouterOptions 

type RouterOptions struct {
	Config         RoutingConfig
	ModelPricing   map[string]ModelPricing
	RoutingProfile string // "eco", "auto", "premium"
	HasTools       bool
	Now            *int64 // unix millis override for promotion window checks (testing)
}

RouterOptions carries configuration for a single routing call.

type RouterStrategy 

type RouterStrategy interface {
	Name() string
	Route(prompt string, systemPrompt string, maxOutputTokens int, options RouterOptions) RoutingDecision
}

RouterStrategy is the interface for pluggable routing strategies.

func GetStrategy 

func GetStrategy(name string) (RouterStrategy, error)

GetStrategy returns a registered routing strategy by name.

type RoutingConfig 

type RoutingConfig struct {
	Version      string              `json:"version"`
	Classifier   ClassifierConfig    `json:"classifier"`
	Scoring      ScoringConfig       `json:"scoring"`
	Tiers        map[Tier]TierConfig `json:"tiers"`
	AgenticTiers map[Tier]TierConfig `json:"agenticTiers,omitempty"`
	EcoTiers     map[Tier]TierConfig `json:"ecoTiers,omitempty"`
	PremiumTiers map[Tier]TierConfig `json:"premiumTiers,omitempty"`
	Promotions   []Promotion         `json:"promotions,omitempty"`
	Overrides    OverridesConfig     `json:"overrides"`
}

RoutingConfig is the top-level configuration.

func DefaultRoutingConfig 

func DefaultRoutingConfig() RoutingConfig

DefaultRoutingConfig returns the default routing configuration with all multilingual keyword lists (EN, ZH, JA, RU, DE, ES, PT, KO, AR), dimension weights, tier boundaries, and tier model configs.

type RoutingDecision 

type RoutingDecision struct {
	Model        string              // selected model ID
	Tier         Tier                // classified tier
	Confidence   float64             // confidence in classification
	Method       string              // "rules" or "llm"
	Reasoning    string              // human-readable reasoning
	CostEstimate float64             // estimated cost in USD
	BaselineCost float64             // what Claude Opus would cost
	Savings      float64             // 0-1 percentage saved vs baseline
	AgenticScore float64             // 0-1 agentic task score
	TierConfigs  map[Tier]TierConfig // tier configs used for this decision
	Profile      string              // "auto", "eco", "premium", "agentic"
}

RoutingDecision is the final output: which model to use and why.

func Route 

func Route(prompt string, systemPrompt string, maxOutputTokens int, options RouterOptions) (RoutingDecision, error)

Route classifies a request and returns the cheapest capable model. Delegates to the registered "rules" strategy by default.

func SelectModel 

func SelectModel(
	tier Tier,
	confidence float64,
	method string,
	reasoning string,
	tierConfigs map[Tier]TierConfig,
	modelPricing map[string]ModelPricing,
	estimatedInputTokens int,
	maxOutputTokens int,
	routingProfile string,
	agenticScore float64,
) RoutingDecision

SelectModel picks the primary model for a tier and builds a RoutingDecision.

type RulesStrategy 

type RulesStrategy struct{}

RulesStrategy implements RouterStrategy using the 15-dimension rule-based classifier.

func (*RulesStrategy) Name 

func (s *RulesStrategy) Name() string

func (*RulesStrategy) Route 

func (s *RulesStrategy) Route(prompt string, systemPrompt string, maxOutputTokens int, options RouterOptions) RoutingDecision

type ScoringConfig 

type ScoringConfig struct {
	TokenCountThresholds struct {
		Simple  int `json:"simple"`
		Complex int `json:"complex"`
	} `json:"tokenCountThresholds"`

	CodeKeywords           []string `json:"codeKeywords"`
	ReasoningKeywords      []string `json:"reasoningKeywords"`
	SimpleKeywords         []string `json:"simpleKeywords"`
	TechnicalKeywords      []string `json:"technicalKeywords"`
	CreativeKeywords       []string `json:"creativeKeywords"`
	ImperativeVerbs        []string `json:"imperativeVerbs"`
	ConstraintIndicators   []string `json:"constraintIndicators"`
	OutputFormatKeywords   []string `json:"outputFormatKeywords"`
	ReferenceKeywords      []string `json:"referenceKeywords"`
	NegationKeywords       []string `json:"negationKeywords"`
	DomainSpecificKeywords []string `json:"domainSpecificKeywords"`
	AgenticTaskKeywords    []string `json:"agenticTaskKeywords"`

	DimensionWeights map[string]float64 `json:"dimensionWeights"`

	TierBoundaries struct {
		SimpleMedium     float64 `json:"simpleMedium"`
		MediumComplex    float64 `json:"mediumComplex"`
		ComplexReasoning float64 `json:"complexReasoning"`
	} `json:"tierBoundaries"`

	ConfidenceSteepness float64 `json:"confidenceSteepness"`
	ConfidenceThreshold float64 `json:"confidenceThreshold"`
}

ScoringConfig controls the weighted dimension scorer.

type ScoringResult 

type ScoringResult struct {
	Score        float64          // weighted float (roughly [-0.3, 0.4])
	Tier         *Tier            // nil = ambiguous, needs fallback classifier
	Confidence   float64          // sigmoid-calibrated [0, 1]
	Signals      []string         // human-readable signal descriptions
	AgenticScore float64          // 0-1 agentic task score
	Dimensions   []DimensionScore // per-dimension breakdown
}

ScoringResult holds the output of the rule-based classifier.

func ClassifyByRules 

func ClassifyByRules(prompt, systemPrompt string, estimatedTokens int, config ScoringConfig) ScoringResult

ClassifyByRules scores a request across 15 weighted dimensions and maps the aggregate score to a tier using configurable boundaries. Confidence is calibrated via sigmoid - low confidence triggers the fallback classifier.

Handles 70-80% of requests in < 1ms with zero cost.

type Tier 

type Tier string

Tier represents the complexity classification of a request. REASONING is distinct from COMPLEX because reasoning tasks need different models (o3, gemini-pro) than general complex tasks (gpt-4o, sonnet-4). 

const (
	TierSimple    Tier = "SIMPLE"
	TierMedium    Tier = "MEDIUM"
	TierComplex   Tier = "COMPLEX"
	TierReasoning Tier = "REASONING"
)

func ClassifyByLLM 

func ClassifyByLLM(prompt string, config LLMClassifierConfig, apiBase string, httpClient *http.Client) (Tier, float64)

ClassifyByLLM classifies a prompt using a cheap LLM. Returns tier and confidence. Defaults to MEDIUM on any failure.

type TierConfig 

type TierConfig struct {
	Primary  string   `json:"primary"`
	Fallback []string `json:"fallback"`
}

TierConfig maps a tier to its primary model and fallback chain.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.