provider

Documentation

Overview

Package provider defines the model-backend abstraction and a registry mapping a provider "kind" to a factory. Concrete implementations live in subpackages (e.g. provider/openai) and self-register via init(). The core resolves providers by kind from config and never hardcodes a specific model.

Index

• func Kinds() []string
• func Register(kind string, f Factory)
• type AuthError
  • func (e *AuthError) Error() string
• type Chunk
• type ChunkType
• type Config
• type Factory
• type FallbackProvider
  • func (f *FallbackProvider) Name() string
  • func (f *FallbackProvider) Stream(ctx context.Context, req Request) (<-chan Chunk, error)
• type Message
• type Pricing
  • func (p *Pricing) Cost(u *Usage) float64
  • func (p *Pricing) Symbol() string
• type Provider
  • func New(kind string, cfg Config) (Provider, error)
• type ProviderEntry
• type ProviderRegistry
  • func NewProviderRegistry(entries []ProviderEntry) *ProviderRegistry
  • func (r *ProviderRegistry) All() []ProviderEntry
  • func (r *ProviderRegistry) Select(want Tier) (Provider, bool)
• type Request
• type Role
• type Tier
  • func (t Tier) String() string
• type ToolCall
• type ToolSchema
• type Usage

Functions

func Kinds

func Kinds() []string

Kinds returns the registered kinds, sorted.

func Register

func Register(kind string, f Factory)

Register adds a factory under a kind (e.g. "openai"). Intended for init(). A duplicate kind is logged to stderr and skipped — a duplicate is a compile-time wiring mistake, but crashing the binary is worse for the user.

Types

type AuthError

type AuthError struct {
	Provider string // the provider instance name, e.g. "deepseek"
	KeyEnv   string // the api_key_env the key is read from, when known
	Status   int    // the HTTP status (401 or 403)
}

AuthError reports that a provider rejected the API key (HTTP 401/403). Its message is already user-facing and actionable — it names the provider and, when known, the environment variable the key comes from — so the CLI can surface it verbatim instead of dumping a raw status body. Providers should return this (rather than a generic status error) for auth failures.

func (*AuthError) Error

func (e *AuthError) Error() string

type Chunk

type Chunk struct {
	Type     ChunkType
	Text     string    // ChunkText, ChunkReasoning
	ToolCall *ToolCall // ChunkToolCallStart (ID+Name only), ChunkToolCall (complete)
	Usage    *Usage    // ChunkUsage
	Err      error     // ChunkError
}

Chunk is a single streamed event. Read the field matching Type.

type ChunkType

type ChunkType int

ChunkType identifies the kind of a streamed increment.

const (
	ChunkText          ChunkType = iota // text delta
	ChunkReasoning                      // thinking-mode reasoning delta (before the visible answer)
	ChunkToolCallStart                  // a tool call has begun (ToolCall: ID+Name; args still streaming)
	ChunkToolCall                       // one complete tool call
	ChunkUsage                          // token usage for the completion
	ChunkDone                           // completion finished normally
	ChunkError                          // an error occurred
)

type Config

type Config struct {
	Name      string         // instance name, e.g. "deepseek"
	BaseURL   string         // OpenAI-compatible endpoint
	Model     string         // model id
	APIKey    string         // resolved from api_key_env
	APIKeyEnv string         // the env var name the key comes from, for actionable auth errors
	Extra     map[string]any // kind-specific options
}

Config is a resolved provider instance configuration.

type Factory

type Factory func(cfg Config) (Provider, error)

Factory builds a Provider from a resolved Config.

type FallbackProvider

type FallbackProvider struct {
	Primary   Provider
	Secondary Provider
}

FallbackProvider wraps a primary and secondary provider. When the primary fails (Stream returns error), it transparently retries with the secondary. This enables offline fallback: primary = cloud API, secondary = local Ollama.

func (*FallbackProvider) Name

func (f *FallbackProvider) Name() string

func (*FallbackProvider) Stream

func (f *FallbackProvider) Stream(ctx context.Context, req Request) (<-chan Chunk, error)

type Message

type Message struct {
	Role             Role       `json:"role"`
	Content          string     `json:"content,omitempty"`
	ReasoningContent string     `json:"reasoning_content,omitempty"` // assistant: thinking-mode chain-of-thought, round-tripped on multi-turn
	ToolCalls        []ToolCall `json:"tool_calls,omitempty"`        // set by assistant
	ToolCallID       string     `json:"tool_call_id,omitempty"`      // links a tool result to its call
	Name             string     `json:"name,omitempty"`              // tool message: tool name
}

Message is a single conversation message.

type Pricing

type Pricing struct {
	CacheHit float64 `toml:"cache_hit"` // per 1M cached prompt tokens
	Input    float64 `toml:"input"`     // per 1M uncached prompt tokens
	Output   float64 `toml:"output"`    // per 1M completion tokens
	Currency string  `toml:"currency"`
}

Pricing is a provider's per-1M-token rates, used to estimate spend. Currency is just a display symbol (default "¥"). toml tags let config decode it.

func (*Pricing) Cost

func (p *Pricing) Cost(u *Usage) float64

Cost estimates the spend for a usage record.

func (*Pricing) Symbol

func (p *Pricing) Symbol() string

Symbol returns the currency display symbol, defaulting to "¥".

type Provider

type Provider interface {
	// Name returns the provider instance name, e.g. "deepseek" / "mimo".
	Name() string
	// Stream starts a streaming completion, pushing increments on the channel.
	// Canceling ctx must abort the underlying request; a closed channel marks
	// the end of the completion.
	Stream(ctx context.Context, req Request) (<-chan Chunk, error)
}

Provider is a chat-capable model backend.

func New

func New(kind string, cfg Config) (Provider, error)

New instantiates the provider of the given kind.

type ProviderEntry

type ProviderEntry struct {
	Provider Provider
	Tier     Tier
	Name     string
}

ProviderEntry pairs a Provider with its tier.

type ProviderRegistry

type ProviderRegistry struct {
	// contains filtered or unexported fields
}

ProviderRegistry holds multiple providers with tier metadata, supporting automatic selection and fallback.

func NewProviderRegistry

func NewProviderRegistry(entries []ProviderEntry) *ProviderRegistry

NewProviderRegistry creates a registry from a slice of providers + tiers.

func (*ProviderRegistry) All

func (r *ProviderRegistry) All() []ProviderEntry

All returns all entries.

func (*ProviderRegistry) Select

func (r *ProviderRegistry) Select(want Tier) (Provider, bool)

Select returns the first provider matching the requested tier, falling back to TierDefault then TierCheap then the first available.

type Request

type Request struct {
	Messages    []Message
	Tools       []ToolSchema
	Temperature float64
	MaxTokens   int
}

Request is a single completion request.

type Role

type Role string

Role is the role of a message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type Tier

type Tier int

Tier categorises a provider by capability/cost.

const (
	TierCheap Tier = iota
	TierDefault
	TierStrong
)

func (Tier) String

func (t Tier) String() string

type ToolCall

type ToolCall struct {
	ID        string `json:"id"`
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

ToolCall is a tool invocation requested by the model. Arguments is raw JSON.

type ToolSchema

type ToolSchema struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Parameters  json.RawMessage `json:"parameters"`
}

ToolSchema is a tool definition exposed to the model. Parameters is JSON Schema.

type Usage

type Usage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
	CacheHitTokens   int    // prompt tokens served from cache
	CacheMissTokens  int    // prompt tokens not cached
	ReasoningTokens  int    // subset of CompletionTokens spent on chain-of-thought
	FinishReason     string // "stop", "tool_calls", "length", "content_filter", "repetition_truncation", …
}

Usage reports token accounting for a completion. Cache hit/miss come from either DeepSeek's top-level prompt_cache_{hit,miss}_tokens or the OpenAI/MiMo standard prompt_tokens_details.cached_tokens — the openai provider normalises both shapes into these fields. ReasoningTokens is the thinking-mode subset of CompletionTokens reported by thinking-capable models. FinishReason carries the model's last reported choices[0].finish_reason so the agent can surface abnormal terminations ("length", "content_filter", "repetition_truncation").