provider

package
v0.3.54 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 31, 2026 License: MIT Imports: 37 Imported by: 0

Documentation

Overview

Package provider defines the LLM client abstraction used by zot.

It supports exactly two providers: Anthropic (Messages API) and OpenAI (Chat Completions API). Everything above this package operates on the types declared here and does not know about HTTP or SSE.

Index

Constants

View Source 
const (
	// APICompletions identifies the OpenAI Chat Completions wire API.
	APICompletions = "openai-completions"
	// APIResponses identifies the OpenAI Responses wire API.
	APIResponses = "openai-responses"
)
View Source 
const CacheTTL = 6 * time.Hour

CacheTTL is how long a discovered list is considered fresh.

View Source 
const (
	LlamaCPPProviderID = "llama.cpp"
)

Variables

View Source 
var Catalog = []Model{

	{
		Provider: "anthropic", ID: "claude-sonnet-4-5", DisplayName: "Claude Sonnet 4.5 (latest)",
		ContextWindow: 200000, MaxOutput: 64000, Reasoning: true,
		PriceInput: 3, PriceOutput: 15, PriceCacheRead: 0.3, PriceCacheWrite: 3.75,
	},
	{
		Provider: "anthropic", ID: "claude-opus-4-1", DisplayName: "Claude Opus 4.1 (latest)",
		ContextWindow: 200000, MaxOutput: 32000, Reasoning: true,
		PriceInput: 15, PriceOutput: 75, PriceCacheRead: 1.5, PriceCacheWrite: 18.75,
	},
	{
		Provider: "anthropic", ID: "claude-opus-4-0", DisplayName: "Claude Opus 4 (latest)",
		ContextWindow: 200000, MaxOutput: 32000, Reasoning: true,
		PriceInput: 15, PriceOutput: 75, PriceCacheRead: 1.5, PriceCacheWrite: 18.75,
	},
	{
		Provider: "anthropic", ID: "claude-sonnet-4-0", DisplayName: "Claude Sonnet 4 (latest)",
		ContextWindow: 200000, MaxOutput: 64000, Reasoning: true,
		PriceInput: 3, PriceOutput: 15, PriceCacheRead: 0.3, PriceCacheWrite: 3.75,
	},
	{
		Provider: "anthropic", ID: "claude-haiku-4-5", DisplayName: "Claude Haiku 4.5 (latest)",
		ContextWindow: 200000, MaxOutput: 64000, Reasoning: true,
		PriceInput: 1, PriceOutput: 5, PriceCacheRead: 0.1, PriceCacheWrite: 1.25,
	},

	{
		Provider: "anthropic", ID: "claude-3-7-sonnet-20250219", DisplayName: "Claude Sonnet 3.7",
		ContextWindow: 200000, MaxOutput: 64000, Reasoning: true,
		PriceInput: 3, PriceOutput: 15, PriceCacheRead: 0.3, PriceCacheWrite: 3.75,
	},
	{
		Provider: "anthropic", ID: "claude-3-5-sonnet-20241022", DisplayName: "Claude Sonnet 3.5 v2",
		ContextWindow: 200000, MaxOutput: 8192, Reasoning: false,
		PriceInput: 3, PriceOutput: 15, PriceCacheRead: 0.3, PriceCacheWrite: 3.75,
	},
	{
		Provider: "anthropic", ID: "claude-3-5-haiku-latest", DisplayName: "Claude Haiku 3.5 (latest)",
		ContextWindow: 200000, MaxOutput: 8192, Reasoning: false,
		PriceInput: 0.8, PriceOutput: 4, PriceCacheRead: 0.08, PriceCacheWrite: 1,
	},
	{
		Provider: "anthropic", ID: "claude-3-opus-20240229", DisplayName: "Claude Opus 3",
		ContextWindow: 200000, MaxOutput: 4096, Reasoning: false,
		PriceInput: 15, PriceOutput: 75, PriceCacheRead: 1.5, PriceCacheWrite: 18.75,
	},

	{
		Provider: "deepseek", ID: "deepseek-v4-pro", DisplayName: "DeepSeek V4 Pro",
		ContextWindow: 1000000, MaxOutput: 384000, Reasoning: true,
		PriceInput: 0.435, PriceOutput: 0.87, PriceCacheRead: 0.003625,
		BaseURL: "https://api.deepseek.com",
	},
	{
		Provider: "deepseek", ID: "deepseek-v4-flash", DisplayName: "DeepSeek V4 Flash",
		ContextWindow: 1000000, MaxOutput: 384000, Reasoning: true,
		PriceInput: 0.14, PriceOutput: 0.28, PriceCacheRead: 0.0028,
		BaseURL: "https://api.deepseek.com",
	},

	{
		Provider: "kimi", ID: "kimi-for-coding", DisplayName: "Kimi For Coding",
		ContextWindow: 262144, MaxOutput: 32768, Reasoning: true,
		PriceInput: 0, PriceOutput: 0, PriceCacheRead: 0,
		BaseURL: "https://api.kimi.com/coding",
	},

	{
		Provider: "openai", ID: "gpt-5", DisplayName: "GPT-5",
		ContextWindow: 400000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 1.25, PriceOutput: 10, PriceCacheRead: 0.125,
	},
	{
		Provider: "openai", ID: "gpt-5-mini", DisplayName: "GPT-5 Mini",
		ContextWindow: 400000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 0.25, PriceOutput: 2, PriceCacheRead: 0.025,
	},
	{
		Provider: "openai", ID: "gpt-5-nano", DisplayName: "GPT-5 Nano",
		ContextWindow: 400000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 0.05, PriceOutput: 0.4, PriceCacheRead: 0.005,
	},

	{
		Provider: "openai", ID: "gpt-4.1", DisplayName: "GPT-4.1",
		ContextWindow: 1047576, MaxOutput: 32768, Reasoning: false,
		PriceInput: 2, PriceOutput: 8, PriceCacheRead: 0.5,
	},
	{
		Provider: "openai", ID: "gpt-4.1-mini", DisplayName: "GPT-4.1 mini",
		ContextWindow: 1047576, MaxOutput: 32768, Reasoning: false,
		PriceInput: 0.4, PriceOutput: 1.6, PriceCacheRead: 0.1,
	},
	{
		Provider: "openai", ID: "gpt-4.1-nano", DisplayName: "GPT-4.1 nano",
		ContextWindow: 1047576, MaxOutput: 32768, Reasoning: false,
		PriceInput: 0.1, PriceOutput: 0.4, PriceCacheRead: 0.03,
	},

	{
		Provider: "openai", ID: "gpt-4o", DisplayName: "GPT-4o",
		ContextWindow: 128000, MaxOutput: 16384, Reasoning: false,
		PriceInput: 2.5, PriceOutput: 10, PriceCacheRead: 1.25,
	},
	{
		Provider: "openai", ID: "gpt-4o-mini", DisplayName: "GPT-4o mini",
		ContextWindow: 128000, MaxOutput: 16384, Reasoning: false,
		PriceInput: 0.15, PriceOutput: 0.6, PriceCacheRead: 0.08,
	},

	{
		Provider: "openai", ID: "o4-mini", DisplayName: "o4-mini",
		ContextWindow: 200000, MaxOutput: 100000, Reasoning: true,
		PriceInput: 1.1, PriceOutput: 4.4, PriceCacheRead: 0.28,
	},
	{
		Provider: "openai", ID: "o3", DisplayName: "o3",
		ContextWindow: 200000, MaxOutput: 100000, Reasoning: true,
		PriceInput: 2, PriceOutput: 8, PriceCacheRead: 0.5,
	},
	{
		Provider: "openai", ID: "o3-mini", DisplayName: "o3-mini",
		ContextWindow: 200000, MaxOutput: 100000, Reasoning: true,
		PriceInput: 1.1, PriceOutput: 4.4, PriceCacheRead: 0.55,
	},
	{
		Provider: "openai", ID: "o1", DisplayName: "o1",
		ContextWindow: 200000, MaxOutput: 100000, Reasoning: true,
		PriceInput: 15, PriceOutput: 60, PriceCacheRead: 7.5,
	},

	{
		Provider: "google", ID: "gemini-2.5-pro", DisplayName: "Gemini 2.5 Pro",
		ContextWindow: 1048576, MaxOutput: 65536, Reasoning: true,
		PriceInput: 1.25, PriceOutput: 10, PriceCacheRead: 0.125,
	},
	{
		Provider: "google", ID: "gemini-2.5-flash", DisplayName: "Gemini 2.5 Flash",
		ContextWindow: 1048576, MaxOutput: 65536, Reasoning: true,
		PriceInput: 0.3, PriceOutput: 2.5, PriceCacheRead: 0.03,
	},
	{
		Provider: "google", ID: "gemini-2.5-flash-lite", DisplayName: "Gemini 2.5 Flash-Lite",
		ContextWindow: 1048576, MaxOutput: 65536, Reasoning: true,
		PriceInput: 0.1, PriceOutput: 0.4, PriceCacheRead: 0.01,
	},
	{
		Provider: "google", ID: "gemini-2.0-flash", DisplayName: "Gemini 2.0 Flash",
		ContextWindow: 1048576, MaxOutput: 8192, Reasoning: false,
		PriceInput: 0.1, PriceOutput: 0.4, PriceCacheRead: 0.025,
	},
	{
		Provider: "google", ID: "gemini-2.0-flash-lite", DisplayName: "Gemini 2.0 Flash-Lite",
		ContextWindow: 1048576, MaxOutput: 8192, Reasoning: false,
		PriceInput: 0.075, PriceOutput: 0.3, PriceCacheRead: 0,
	},

	{
		Provider: "openrouter", ID: "anthropic/claude-sonnet-4.5", DisplayName: "Claude Sonnet 4.5 (OpenRouter)",
		ContextWindow: 1000000, MaxOutput: 64000, Reasoning: true,
		PriceInput: 3, PriceOutput: 15, PriceCacheRead: 0.3, PriceCacheWrite: 3.75,
		BaseURL: openrouterDefaultBaseURL,
	},
	{
		Provider: "openrouter", ID: "anthropic/claude-sonnet-5", DisplayName: "Claude Sonnet 5 (OpenRouter)",
		ContextWindow: 1000000, MaxOutput: 128000, Reasoning: true, AdaptiveThinking: true,
		PriceInput: 2, PriceOutput: 10, PriceCacheRead: 0.2, PriceCacheWrite: 2.5,
		BaseURL: openrouterDefaultBaseURL,
	},

	{
		Provider: "gondola", ID: "kimi-k3", DisplayName: "Kimi K3 (Gondola)",
		ContextWindow: 1000000, MaxOutput: 131072, Reasoning: true,
		PriceInput: 1.1475, PriceOutput: 5.7375, PriceCacheRead: 0.11475,
		BaseURL: gondolaDefaultBaseURL,
	},
	{
		Provider: "gondola", ID: "claude-opus-5", DisplayName: "Claude Opus 5 (Gondola)",
		ContextWindow: 1000000, MaxOutput: 128000, Reasoning: true, AdaptiveThinking: true,
		PriceInput: 1.836, PriceOutput: 9.18, PriceCacheRead: 0.1836, PriceCacheWrite: 2.295,
		BaseURL: gondolaDefaultBaseURL,
	},

	{
		Provider: "anthropic", ID: "claude-opus-4-5", DisplayName: "Claude Opus 4.5 (latest)",
		ContextWindow: 200000, MaxOutput: 64000, Reasoning: true,
		PriceInput: 5, PriceOutput: 25, PriceCacheRead: 0.5, PriceCacheWrite: 6.25,
		Speculative: true,
	},
	{
		Provider: "anthropic", ID: "claude-opus-4-6", DisplayName: "Claude Opus 4.6",
		ContextWindow: 1000000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 5, PriceOutput: 25, PriceCacheRead: 0.5, PriceCacheWrite: 6.25,
		Speculative: true,
	},
	{
		Provider: "anthropic", ID: "claude-opus-4-7", DisplayName: "Claude Opus 4.7",
		ContextWindow: 1000000, MaxOutput: 128000, Reasoning: true, AdaptiveThinking: true,
		PriceInput: 5, PriceOutput: 25, PriceCacheRead: 0.5, PriceCacheWrite: 6.25,
		Speculative: true,
	},
	{
		Provider: "anthropic", ID: "claude-opus-4-8", DisplayName: "Claude Opus 4.8",
		ContextWindow: 1000000, MaxOutput: 128000, Reasoning: true, AdaptiveThinking: true,
		PriceInput: 5, PriceOutput: 25, PriceCacheRead: 0.5, PriceCacheWrite: 6.25,
		Speculative: true,
	},
	{
		Provider: "anthropic", ID: "claude-opus-5", DisplayName: "Claude Opus 5",
		ContextWindow: 1000000, MaxOutput: 128000, Reasoning: true, AdaptiveThinking: true,
		PriceInput: 5, PriceOutput: 25, PriceCacheRead: 0.5, PriceCacheWrite: 6.25,
	},
	{
		Provider: "anthropic", ID: "claude-sonnet-4-6", DisplayName: "Claude Sonnet 4.6",
		ContextWindow: 1000000, MaxOutput: 64000, Reasoning: true,
		PriceInput: 3, PriceOutput: 15, PriceCacheRead: 0.3, PriceCacheWrite: 3.75,
		Speculative: true,
	},
	{
		Provider: "anthropic", ID: "claude-sonnet-5", DisplayName: "Claude Sonnet 5",
		ContextWindow: 1000000, MaxOutput: 128000, Reasoning: true, AdaptiveThinking: true,
		PriceInput: 2, PriceOutput: 10, PriceCacheRead: 0.2, PriceCacheWrite: 2.5,
	},
	{
		Provider: "anthropic", ID: "claude-fable-5", DisplayName: "Claude Fable 5",
		ContextWindow: 1000000, MaxOutput: 128000, Reasoning: true, AdaptiveThinking: true,
		PriceInput: 10, PriceOutput: 50, PriceCacheRead: 0.5, PriceCacheWrite: 6.25,
		Speculative: true,
	},

	{
		Provider: "openai", ID: "gpt-5.1", DisplayName: "GPT-5.1",
		ContextWindow: 400000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 1.25, PriceOutput: 10, PriceCacheRead: 0.13,
		Speculative: true,
	},
	{
		Provider: "openai", ID: "gpt-5.2", DisplayName: "GPT-5.2",
		ContextWindow: 400000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 1.75, PriceOutput: 14, PriceCacheRead: 0.175,
		Speculative: true,
	},
	{
		Provider: "openai", ID: "gpt-5.4", DisplayName: "GPT-5.4",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 2.5, PriceOutput: 15, PriceCacheRead: 0.25,
		Speculative: true,
	},
	{
		Provider: "openai", ID: "gpt-5.4-mini", DisplayName: "GPT-5.4 mini",
		ContextWindow: 400000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 0.75, PriceOutput: 4.5, PriceCacheRead: 0.075,
		Speculative: true,
	},
	{
		Provider: "openai", ID: "gpt-5.5", DisplayName: "GPT-5.5",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 5, PriceOutput: 30, PriceCacheRead: 0.5,
		Speculative: true,
	},
	{
		Provider: "openai", ID: "gpt-5.6-luna", DisplayName: "GPT-5.6 Luna", API: APIResponses,
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 1, PriceOutput: 6, PriceCacheRead: 0.1, PriceCacheWrite: 1.25,
		Speculative: true,
	},
	{
		Provider: "openai", ID: "gpt-5.6-sol", DisplayName: "GPT-5.6 Sol", API: APIResponses,
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 5, PriceOutput: 30, PriceCacheRead: 0.5, PriceCacheWrite: 6.25,
		Speculative: true,
	},
	{
		Provider: "openai", ID: "gpt-5.6-terra", DisplayName: "GPT-5.6 Terra", API: APIResponses,
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 2.5, PriceOutput: 15, PriceCacheRead: 0.25, PriceCacheWrite: 3.125,
		Speculative: true,
	},

	{
		Provider: "openai-codex", ID: "gpt-5.3-codex-spark", DisplayName: "GPT-5.3 Codex Spark",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 1.75, PriceOutput: 14, PriceCacheRead: 0.175,
		Speculative: true,
	},
	{
		Provider: "openai-codex", ID: "gpt-5.4", DisplayName: "GPT-5.4",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 2.5, PriceOutput: 15, PriceCacheRead: 0.25,
		Speculative: true,
	},
	{
		Provider: "openai-codex", ID: "gpt-5.4-mini", DisplayName: "GPT-5.4 mini",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 0.75, PriceOutput: 4.5, PriceCacheRead: 0.075,
		Speculative: true,
	},
	{
		Provider: "openai-codex", ID: "gpt-5.5", DisplayName: "GPT-5.5",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 5, PriceOutput: 30, PriceCacheRead: 0.5,
		Speculative: true,
	},
	{
		Provider: "openai-codex", ID: "gpt-5.6-luna", DisplayName: "GPT-5.6 Luna",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 1, PriceOutput: 6, PriceCacheRead: 0.1, PriceCacheWrite: 1.25,
		Speculative: true,
	},
	{
		Provider: "openai-codex", ID: "gpt-5.6-sol", DisplayName: "GPT-5.6 Sol",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 5, PriceOutput: 30, PriceCacheRead: 0.5, PriceCacheWrite: 6.25,
		Speculative: true,
	},
	{
		Provider: "openai-codex", ID: "gpt-5.6-terra", DisplayName: "GPT-5.6 Terra",
		ContextWindow: 272000, MaxOutput: 128000, Reasoning: true,
		PriceInput: 2.5, PriceOutput: 15, PriceCacheRead: 0.25, PriceCacheWrite: 3.125,
		Speculative: true,
	},
}

Catalog is the hardcoded, read-only list of supported models. Prices are USD per 1M tokens. The list is curated to what zot's clients (Anthropic Messages + OpenAI Chat Completions) can actually talk to; models that are only reachable through the OpenAI Responses API (o1-pro, o3-pro, gpt-5-pro) are omitted.

View Source 
var DefaultModel = Catalog[0] // claude-sonnet-4-5

DefaultModel is used when the user does not specify one.

Functions

func AnthropicAdaptiveEffort added in v0.2.3 

func AnthropicAdaptiveEffort(level string) string

AnthropicAdaptiveEffort maps zot's user-facing reasoning levels onto the effort enum used by adaptive-thinking models. These models reject explicit thinking budgets; reasoning depth is controlled by output_config.effort.

func AvailableReasoningLevels added in v0.3.32 

func AvailableReasoningLevels(model Model) []string

AvailableReasoningLevels returns the distinct reasoning levels supported by a model. Optional per-model overrides can remove, remap, or extend protocol defaults. The empty string represents off.

func ClampReasoningForModel added in v0.3.32 

func ClampReasoningForModel(model Model, level string) string

ClampReasoningForModel maps a configured level to the nearest level exposed for the active model. Ties prefer the higher level.

func ComputeCost 

func ComputeCost(m Model, u Usage) float64

ComputeCost returns the USD cost for the given usage on model m.

func CustomProviders added in v0.2.35 

func CustomProviders() map[string]CustomProviderConfig

CustomProviders returns the set of user-defined providers loaded from models.json. Keys are provider names; values carry the base URL and wire-format hint.

func FindHuggingFaceToken added in v0.3.1 

func FindHuggingFaceToken() string

FindHuggingFaceToken follows the standard Hugging Face token locations.

func FormatBytes added in v0.3.1 

func FormatBytes(bytes int64) string

FormatBytes renders byte counts using binary units.

func LlamaCPPInferenceURL added in v0.3.1 

func LlamaCPPInferenceURL(serverURL string) (string, error)

LlamaCPPInferenceURL returns the OpenAI-compatible inference base URL.

func NewHTTPClient added in v0.2.34 

func NewHTTPClient(insecureTLS bool) *http.Client

NewHTTPClient returns a provider HTTP client. When insecureTLS is true, only this client skips TLS certificate verification. The process-wide default transport is left untouched so auth, discovery, and other providers keep normal certificate validation.

func NormalizeLlamaCPPURL added in v0.3.1 

func NormalizeLlamaCPPURL(value string) (string, error)

NormalizeLlamaCPPURL validates a router URL and removes a trailing /v1, which belongs to the inference API rather than the management API.

func NormalizeReasoning 

func NormalizeReasoning(level string) string

NormalizeReasoning canonicalizes zot's user-facing reasoning levels. Empty string means reasoning is disabled. "maximum" remains an alias for xhigh; "max" is the separate opt-in tier above it.

func OpenAICodexReasoningEffort 

func OpenAICodexReasoningEffort(level, model string) string

OpenAICodexReasoningEffort maps zot levels onto the Responses API effort enum. GPT-5.6 supports native max; other models clamp max to xhigh.

func OpenAICompatAnthropicEffort added in v0.2.3 

func OpenAICompatAnthropicEffort(level string) string

OpenAICompatAnthropicEffort maps zot's thinking setting when an adaptive Anthropic model is served over an OpenAI-compatible chat-completions wire. Adaptive models accept native xhigh and max effort values.

func OpenAIReasoningEffort 

func OpenAIReasoningEffort(level string) string

OpenAIReasoningEffort maps zot's thinking setting onto the effort enum accepted by generic OpenAI-compatible chat-completions endpoints.

func ProviderLabel 

func ProviderLabel(id string) string

ProviderLabel returns the user-facing label for a provider id.

func ReasoningBudget 

func ReasoningBudget(level string) int

ReasoningBudget returns zot's approximate token budget for reasoning-capable providers that accept explicit budgets.

func SaveCache 

func SaveCache(path string, c ModelCache) error

SaveCache writes the cache atomically.

func SetLiveModels 

func SetLiveModels(live []Model)

SetLiveModels replaces the "live" overlay used by the active catalog. Typically called after a successful /v1/models discovery or on load from the on-disk cache.

func SetManagedModels added in v0.3.1 

func SetManagedModels(models []Model)

SetManagedModels replaces the ephemeral catalog entries supplied by local model managers. Unlike SetLiveModels, this does not disturb provider model discovery or its on-disk cache.

func SetUserModels 

func SetUserModels(models []Model)

SetUserModels merges user-defined models into the active catalog. User models take precedence over both the baked-in catalog and live-discovered models.

Types

type Client 

type Client interface {
	// Name returns "anthropic" or "openai".
	Name() string
	// Stream starts a request. The returned channel delivers events
	// and is closed after EventDone. Errors during request setup are
	// returned directly; runtime errors arrive as EventDone{Err: ...}.
	Stream(ctx context.Context, req Request) (<-chan Event, error)
}

Client is an LLM streaming client.

func NewAnthropic 

func NewAnthropic(apiKey, baseURL string) Client

NewAnthropic creates an Anthropic client using an API key. baseURL may be empty.

func NewAnthropicCompat 

func NewAnthropicCompat(name, apiKey, baseURL string) Client

NewAnthropicCompat returns an anthropicClient pinned to a non-default base URL and identifying as `name` for cost / logging purposes. Auth is API key (x-api-key header). For OAuth-fronted compatibles (rare) use NewAnthropicOAuth and rename via NameClient.

func NewAnthropicOAuth 

func NewAnthropicOAuth(accessToken, baseURL string) Client

NewAnthropicOAuth creates an Anthropic client using a subscription OAuth access token.

func NewAzureOpenAI 

func NewAzureOpenAI(apiKey, baseURL string) Client

NewAzureOpenAI returns an Azure OpenAI client.

baseURL examples (any is fine; we normalise trailing slashes): 

https://my-resource.openai.azure.com
https://my-resource.openai.azure.com/openai/v1

apiKey is the Azure resource key. The api-version comes from AZURE_OPENAI_API_VERSION env var (default "2024-10-21").

func NewAzureOpenAIResponses 

func NewAzureOpenAIResponses(apiKey, baseURL string) Client

NewAzureOpenAIResponses creates an Azure OpenAI Responses API client. The configured model id is used as the deployment name unless overridden through AZURE_OPENAI_DEPLOYMENT_NAME_MAP (model=deployment pairs).

func NewBedrock 

func NewBedrock(apiKey, baseURL string) Client

NewBedrock returns an AWS Bedrock client. See amazon_bedrock.go for the hand-rolled Converse-Stream wire-format parser. Auth is via AWS_BEARER_TOKEN_BEDROCK (the modern Bedrock API key flow); SigV4 signing for IAM access-key + secret credentials is not yet wired.

func NewBedrockClient 

func NewBedrockClient(apiKey, baseURL string) Client

NewBedrockClient returns a Bedrock client.

Auth resolution (first match wins):

  1. apiKey == real bearer-ish string (not "<aws>") -> bearer route.
  2. AWS_BEARER_TOKEN_BEDROCK env var -> bearer route.
  3. AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY (+ optional AWS_SESSION_TOKEN) -> SigV4 route.
  4. AWS_PROFILE -> read ~/.aws/credentials, take that profile's keys.

region defaults to us-east-1 unless AWS_REGION / AWS_DEFAULT_REGION is set or the baseURL embeds a region.

func NewCerebras 

func NewCerebras(apiKey, baseURL string) Client

NewCerebras: ultra-fast inference (Llama/Qwen/GPT-OSS/GLM).

func NewCloudflareAIGateway 

func NewCloudflareAIGateway(apiKey, baseURL string) Client

NewCloudflareAIGateway returns the AI Gateway client (OpenAI compat route). Sends `cf-aig-authorization` instead of `Authorization` so the gateway authenticates the caller (downstream-provider auth is configured per-gateway in the Cloudflare dashboard).

func NewCloudflareWorkersAI 

func NewCloudflareWorkersAI(apiKey, baseURL string) Client

NewCloudflareWorkersAI returns an OpenAI-compatible client pinned to the Workers AI base URL with {CLOUDFLARE_ACCOUNT_ID} substituted. Returns an erroring client (deferred error on first Stream call) if the env var is missing, so the constructor signature stays the same.

func NewDeepSeek 

func NewDeepSeek(apiKey, baseURL string) Client

NewDeepSeek creates a DeepSeek client. DeepSeek's chat API is OpenAI-compatible at https://api.deepseek.com/v1.

func NewFireworksAnthropic 

func NewFireworksAnthropic(apiKey, baseURL string) Client

NewFireworksAnthropic is the main Fireworks route. The anthropic-messages-compatible endpoint at api.fireworks.ai/inference expects Anthropic-style request bodies; use this rather than the OpenAI flavor.

func NewFireworksOpenAI 

func NewFireworksOpenAI(apiKey, baseURL string) Client

NewFireworksOpenAI is the OpenAI-completions flavor of Fireworks. The main route uses the anthropic-messages variant on api.fireworks.ai/inference; see NewFireworksAnthropic.

func NewGemini 

func NewGemini(apiKey, baseURL string) Client

NewGemini creates a Gemini client using an AI Studio API key. baseURL may be empty; defaults to https://generativelanguage.googleapis.com.

func NewGithubCopilot 

func NewGithubCopilot(apiKey, _ string) Client

NewGithubCopilot returns a GitHub Copilot client. The provided credential must be a GitHub Personal Access Token (PAT) with Copilot access enabled; zot trades it for a short-lived Copilot token on every inference request (cached in memory until ~5min before expiry).

Wire format: OpenAI Chat Completions. Copilot-specific headers (X-Initiator, Openai-Intent, Editor-Version, Editor-Plugin-Version, Copilot-Integration-Id, User-Agent) are added by the refresh transport. The model id passes through unchanged.

baseURL is ignored: the canonical host is read from `proxy-ep=...` in the short-lived token's value.

func NewGithubCopilotClient 

func NewGithubCopilotClient(pat string) Client

NewGithubCopilotClient returns a Copilot-pinned OpenAI-compat client. The pat must be a GitHub Personal Access Token with Copilot access.

Copilot exposes two wire protocols: most models use Chat Completions (/chat/completions), but the newer GPT-5.6 family (sol/terra/luna) is only served through the Responses API (/responses) and returns an "chat/completions is not available for gpt models" error otherwise. A model router dispatches each request to the matching wire client based on the model's catalog API tag.

func NewGondola added in v0.3.43 

func NewGondola(apiKey, baseURL string) Client

NewGondola: Gondola, a USDC-paid marketplace gateway for Venice AI inference. OpenAI Chat Completions-compatible.

func NewGoogleVertex 

func NewGoogleVertex(apiKey, baseURL string) Client

NewGoogleVertex returns a Vertex AI client. See google_vertex.go for the full auth + URL-rewrite implementation. Requires GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION, and either GOOGLE_CLOUD_API_KEY or a service- account JSON file pointed to by GOOGLE_APPLICATION_CREDENTIALS (or the default ADC location ~/.config/gcloud/application_default_credentials.json).

func NewGroq 

func NewGroq(apiKey, baseURL string) Client

NewGroq: LPU inference (Llama/Kimi/Qwen/GPT-OSS).

func NewHuggingFace 

func NewHuggingFace(apiKey, baseURL string) Client

NewHuggingFace: HF inference router.

func NewKimi 

func NewKimi(apiKey, baseURL string) Client

NewKimi creates a Kimi/Moonshot client. Kimi's chat API is OpenAI-compatible.

func NewKimiCoding 

func NewKimiCoding(apiKey, baseURL string) Client

NewKimiCoding is the Kimi Code client: Kimi behind the Anthropic Messages API at https://api.kimi.com/coding. Replaces the older OpenAI-completions-on-/coding/v1 wiring.

func NewKimiCodingWithHeaders 

func NewKimiCodingWithHeaders(apiKey, baseURL string, headers map[string]string) Client

NewKimiCodingWithHeaders is the headered variant used by OAuth.

func NewKimiWithHeaders 

func NewKimiWithHeaders(apiKey, baseURL string, headers map[string]string) Client

NewKimiWithHeaders creates a Kimi/Moonshot client with extra headers. Subscription tokens from Kimi Code need the official CLI's X-Msh-* headers.

func NewMinimaxAnthropic 

func NewMinimaxAnthropic(apiKey, baseURL string) Client

NewMinimaxAnthropic is the anthropic-messages flavor on api.minimax.io/anthropic, catalogued under provider=minimax.

func NewMinimaxCNAnthropic 

func NewMinimaxCNAnthropic(apiKey, baseURL string) Client

NewMinimaxCNAnthropic is the CN-region MiniMax (anthropic-messages).

func NewMinimaxCNOpenAI 

func NewMinimaxCNOpenAI(apiKey, baseURL string) Client

NewMinimaxCNOpenAI is the CN-region MiniMax (openai-completions).

func NewMinimaxOpenAI 

func NewMinimaxOpenAI(apiKey, baseURL string) Client

NewMinimaxOpenAI is the OpenAI-completions flavor of MiniMax, in case downstream models switch from anthropic-messages. The main MiniMax route uses anthropic-messages; see NewMinimaxAnthropic below.

func NewMistral 

func NewMistral(apiKey, baseURL string) Client

NewMistral returns a Mistral client using their OpenAI-compatible Chat Completions endpoint at https://api.mistral.ai/v1. Mistral also offers a bespoke "Conversations" API, but the OpenAI-compat endpoint supports the same models with tool calling and streaming, so we use that for simplicity (no extra wire format to maintain).

func NewModelRouter added in v0.2.90 

func NewModelRouter(name string, fallback Client, byAPI map[string]Client) Client

NewModelRouter creates a client that dispatches requests using Model.API. Models with no API override, and models absent from the catalog, use fallback.

func NewMoonshot 

func NewMoonshot(apiKey, baseURL string) Client

NewMoonshot is the global Moonshot AI endpoint (Kimi-K2 family by id). Provider id is `moonshotai`.

func NewMoonshotCN 

func NewMoonshotCN(apiKey, baseURL string) Client

NewMoonshotCN is the China-region Moonshot endpoint. Same model ids as the global flavor, different base URL.

func NewOpenAI 

func NewOpenAI(apiKey, baseURL string) Client

NewOpenAI creates an OpenAI client using an API key. baseURL may be empty.

func NewOpenAICodex 

func NewOpenAICodex(token, accountID, baseURL string) Client

NewOpenAICodex creates a client that talks to ChatGPT's Codex endpoint using a subscription OAuth access token and the user's ChatGPT account id. baseURL may be empty to use the default.

func NewOpenAICompat added in v0.2.35 

func NewOpenAICompat(name, apiKey, baseURL, fallbackBaseURL string) Client

NewOpenAICompat is the shared constructor for OpenAI-completions clones. Exported so build.go can instantiate clients for user-defined providers.

func NewOpenAIOAuth 

func NewOpenAIOAuth(accessToken, baseURL string) Client

NewOpenAIOAuth creates an OpenAI client using a subscription OAuth access token. The token is sent as an HTTP Bearer credential on the standard chat/completions endpoint.

func NewOpenAIResponses 

func NewOpenAIResponses(apiKey, baseURL string) Client

NewOpenAIResponses returns an OpenAI Responses-API client (API-key flow). Uses the same wire format as the ChatGPT Codex backend but with the public api.openai.com endpoint and standard Bearer auth.

baseURL may be empty; defaults to https://api.openai.com/v1/responses.

func NewOpenAIResponsesNamed added in v0.2.71 

func NewOpenAIResponsesNamed(apiKey, baseURL, name string) Client

NewOpenAIResponsesNamed returns a public OpenAI Responses-API client reporting the supplied provider name. This lets the `openai` provider route Responses-only preview models without changing the visible provider.

func NewOpenCode 

func NewOpenCode(apiKey, baseURL string) Client

NewOpenCode is the opencode.ai Zen endpoint. Mixed APIs upstream; this constructor wires the openai-completions flavor only. Models that need the anthropic-messages flavor under the same provider should be built with NewAnthropicCompat against the same base URL.

func NewOpenCodeGo 

func NewOpenCodeGo(apiKey, baseURL string) Client

NewOpenCodeGo routes each model through the wire API exposed by OpenCode Go.

func NewOpenRouter 

func NewOpenRouter(apiKey, baseURL string) Client

NewOpenRouter: OpenRouter aggregator. Unlocks dozens of upstream models with one key.

func NewTogether 

func NewTogether(apiKey, baseURL string) Client

NewTogether: Together.ai aggregator.

func NewVercelGatewayAnthropic 

func NewVercelGatewayAnthropic(apiKey, baseURL string) Client

NewVercelGatewayAnthropic — Vercel AI Gateway anthropic-messages route.

func NewVercelGatewayOpenAI 

func NewVercelGatewayOpenAI(apiKey, baseURL string) Client

NewVercelGatewayOpenAI is Vercel AI Gateway's OpenAI-compat shim. The main route uses anthropic-messages; see NewVercelGatewayAnthropic.

func NewVertex 

func NewVertex(_ string, _ string) Client

NewVertex returns a Vertex AI client. The apiKey argument is ignored in favor of env-based config (GOOGLE_CLOUD_API_KEY or GOOGLE_APPLICATION_CREDENTIALS), since Vertex's auth model doesn't fit the "just paste a key" interface other providers use.

func NewXAI 

func NewXAI(apiKey, baseURL string) Client

NewXAI: xAI Grok.

func NewXiaomi 

func NewXiaomi(apiKey, baseURL string) Client

NewXiaomi: Xiaomi MiMo family (default endpoint).

func NewXiaomiTokenPlan 

func NewXiaomiTokenPlan(region, apiKey, baseURL string) Client

NewXiaomiTokenPlan creates a regional Xiaomi token-plan client. region must be "ams", "cn", or "sgp", matching the three `xiaomi-token-plan-*` provider ids.

func NewZAI 

func NewZAI(apiKey, baseURL string) Client

NewZAI: Z.AI GLM family.

func WithHTTPClient added in v0.2.34 

func WithHTTPClient(c Client, httpClient *http.Client) Client

WithHTTPClient scopes an HTTP client to a concrete provider client. Unsupported clients are returned unchanged.

type Content 

type Content interface {
	// contains filtered or unexported methods
}

Content is a block inside a Message. One of TextBlock, ImageBlock, ToolCallBlock, or ToolResultBlock.

type CustomProviderConfig added in v0.2.35 

type CustomProviderConfig struct {
	BaseURL string
	API     string // "openai", "openai-responses", or "anthropic"
}

CustomProviderConfig holds runtime config for a user-defined provider that isn't part of the built-in catalog.

type Event 

type Event interface {
	// contains filtered or unexported methods
}

Event is one item from a provider stream.

type EventDone 

type EventDone struct {
	Stop    StopReason
	Err     error
	Message Message // fully assembled assistant message
}

EventDone is always the final event on a stream.

type EventStart 

type EventStart struct {
	Model    string
	Provider string
}

type EventTextDelta 

type EventTextDelta struct {
	Delta string
}

type EventToolArgs 

type EventToolArgs struct {
	ID    string
	Delta string // partial JSON
}

type EventToolEnd 

type EventToolEnd struct {
	ID string
}

type EventToolStart 

type EventToolStart struct {
	ID   string
	Name string
}

type EventUsage 

type EventUsage struct {
	Usage Usage
}

type HuggingFaceClient added in v0.3.1 

type HuggingFaceClient struct {
	Token   string
	BaseURL string
	HTTP    *http.Client
}

func NewHuggingFaceClient added in v0.3.1 

func NewHuggingFaceClient(token string) *HuggingFaceClient

func (*HuggingFaceClient) Details added in v0.3.1 

func (c *HuggingFaceClient) Details(ctx context.Context, id string) (HuggingFaceModelDetails, error)

func (*HuggingFaceClient) Search added in v0.3.1 

func (c *HuggingFaceClient) Search(ctx context.Context, query string) ([]HuggingFaceModel, error)

type HuggingFaceModel added in v0.3.1 

type HuggingFaceModel struct {
	ID        string `json:"id"`
	Downloads int64  `json:"downloads"`
}

HuggingFaceModel is one GGUF repository search result.

type HuggingFaceModelDetails added in v0.3.1 

type HuggingFaceModelDetails struct {
	ID            string
	Gated         string
	Quantizations []HuggingFaceQuantization
}

type HuggingFaceQuantization added in v0.3.1 

type HuggingFaceQuantization struct {
	Name    string
	Size    int64
	HasSize bool
}

type ImageBlock 

type ImageBlock struct {
	MimeType         string `json:"mime_type"`
	Data             []byte `json:"data"` // raw bytes; encoded to base64 on the wire
	ThoughtSignature string `json:"thought_signature,omitempty"`
}

ImageBlock is an inline image (PNG/JPEG/GIF/WebP).

type LlamaCPPBytes added in v0.3.1 

type LlamaCPPBytes struct {
	Done  int64 `json:"done"`
	Total int64 `json:"total"`
}

LlamaCPPBytes describes byte progress for one downloaded file.

type LlamaCPPClient added in v0.3.1 

type LlamaCPPClient struct {
	ServerURL string
	APIKey    string
	HTTP      *http.Client
}

LlamaCPPClient talks to the model-management API exposed by llama-server when it is running in router mode.

func NewLlamaCPPClient added in v0.3.1 

func NewLlamaCPPClient(serverURL, apiKey string) (*LlamaCPPClient, error)

func (*LlamaCPPClient) Download added in v0.3.1 

func (c *LlamaCPPClient) Download(ctx context.Context, model string) error

func (*LlamaCPPClient) DownloadAndWait added in v0.3.1 

func (c *LlamaCPPClient) DownloadAndWait(ctx context.Context, model string, update func(LlamaCPPProgress)) ([]LlamaCPPModel, error)

func (*LlamaCPPClient) List added in v0.3.1 

func (c *LlamaCPPClient) List(ctx context.Context, reload bool) ([]LlamaCPPModel, error)

func (*LlamaCPPClient) Load added in v0.3.1 

func (c *LlamaCPPClient) Load(ctx context.Context, model string) error

func (*LlamaCPPClient) LoadAndWait added in v0.3.1 

func (c *LlamaCPPClient) LoadAndWait(ctx context.Context, model string, update func(LlamaCPPProgress)) (LlamaCPPModel, error)

func (*LlamaCPPClient) Remove added in v0.3.1 

func (c *LlamaCPPClient) Remove(ctx context.Context, model string) error

Remove deletes a router-managed cache model from disk. Models discovered from --models-dir or presets are not removable through the router API.

func (*LlamaCPPClient) Unload added in v0.3.1 

func (c *LlamaCPPClient) Unload(ctx context.Context, model string) error

func (*LlamaCPPClient) UnloadAndWait added in v0.3.1 

func (c *LlamaCPPClient) UnloadAndWait(ctx context.Context, model string) error

type LlamaCPPModel added in v0.3.1 

type LlamaCPPModel struct {
	ID        string   `json:"id"`
	Aliases   []string `json:"aliases,omitempty"`
	Source    string   `json:"source,omitempty"`
	CanRemove bool     `json:"can_remove,omitempty"`
	Status    struct {
		Value    string                   `json:"value"`
		Args     []string                 `json:"args,omitempty"`
		Failed   bool                     `json:"failed,omitempty"`
		ExitCode *int                     `json:"exit_code,omitempty"`
		Progress map[string]LlamaCPPBytes `json:"progress,omitempty"`
	} `json:"status"`
	Architecture struct {
		InputModalities []string `json:"input_modalities,omitempty"`
	} `json:"architecture,omitempty"`
	Meta struct {
		Context      int    `json:"n_ctx,omitempty"`
		TrainContext int    `json:"n_ctx_train,omitempty"`
		Size         int64  `json:"size,omitempty"`
		FileType     string `json:"ftype,omitempty"`
	} `json:"meta,omitempty"`
	// Progress is emitted at the model's top level by current llama.cpp
	// routers. Status.Progress supports versions that follow the documented
	// nested shape.
	Progress map[string]LlamaCPPBytes `json:"progress,omitempty"`
}

LlamaCPPModel is one entry returned by a llama.cpp router's /models endpoint.

type LlamaCPPProgress added in v0.3.1 

type LlamaCPPProgress struct {
	Message  string
	Ratio    float64
	HasRatio bool
	Detail   string
}

LlamaCPPProgress is a user-facing load or download progress update.

type Message 

type Message struct {
	Role           Role              `json:"role"`
	Content        []Content         `json:"content"`
	Time           time.Time         `json:"time"`
	Meta           map[string]string `json:"meta,omitempty"`
	AddedToolNames []string          `json:"added_tool_names,omitempty"`
}

Message is a single turn in the conversation.

func RepairOrphanedToolResults 

func RepairOrphanedToolResults(msgs []Message) []Message

RepairOrphanedToolResults removes invalid tool_result content blocks and entire messages that become empty. A result is invalid when its matching tool_use ID does not appear anywhere in the messages or when an earlier result already covers the same ID. Resume tails, compaction repair, and provider request builders all need this so the upstream API sees exactly one result for each referenced tool call.

type Model 

type Model struct {
	Provider          string // "anthropic" | "openai"
	ID                string // API id
	DisplayName       string
	API               string // wire API override for providers that support multiple protocols
	ContextWindow     int
	MaxOutput         int
	Reasoning         bool              // supports reasoning
	ReasoningLevelMap map[string]string // optional level overrides; empty values remove a level

	// AdaptiveThinking marks Anthropic models that only support the
	// adaptive thinking mode (Opus 4.7+). These reject explicit
	// thinking budgets (thinking:{type:"enabled",budget_tokens:N} -> 400)
	// and also reject non-default sampling params (temperature/top_p/
	// top_k). The Anthropic client sends thinking:{type:"adaptive"} plus
	// output_config.effort and omits temperature for these models.
	AdaptiveThinking bool

	// AdaptiveThinkingCompat marks Anthropic-compatible models that require
	// thinking:{type:"adaptive"} but still accept sampling parameters and do
	// not support Anthropic's output_config.effort extension.
	AdaptiveThinkingCompat bool

	// Prices are USD per 1M tokens. Price*Above fields apply to all token
	// classes when the total prompt size exceeds PriceTierInputTokens.
	PriceInput           float64
	PriceOutput          float64
	PriceCacheRead       float64
	PriceCacheWrite      float64
	PriceTierInputTokens int
	PriceInputAbove      float64
	PriceOutputAbove     float64
	PriceCacheReadAbove  float64
	PriceCacheWriteAbove float64

	// Speculative marks models whose ids are known from the upstream
	// vendor's CLI but not yet live on their public API. They'll 404
	// today but start working the moment the provider flips the switch.
	Speculative bool

	// BaseURL overrides the provider's default API endpoint for this
	// model. Optional; when empty the provider's default (or the
	// --base-url flag) is used. Useful for local models served by
	// ollama, vLLM, LM Studio, etc.
	BaseURL string

	// Source is where this model entry came from: "catalog" (baked in),
	// "live" (discovered via /v1/models), or "cache" (loaded from the
	// on-disk cache). Informational.
	Source string
}

Model describes a single LLM we know about.

func Active 

func Active() []Model

Active returns the current merged catalog.

When no live overlay has been set it returns the fully-assembled static Catalog. Reading Catalog here (rather than capturing it into a package-level var initializer) is load-bearing: the extended catalog in catalog_builtin.go / extra_models.go is appended from init() functions, which run AFTER package-level var initializers. Snapshotting Catalog at var-init time would freeze the picker to the curated seed list and drop every extra provider (openrouter, groq, xai, ...). Deferring the read to call time avoids that ordering trap.

func DiscoverAnthropic 

func DiscoverAnthropic(ctx context.Context, apiKey, baseURL string) ([]Model, error)

DiscoverAnthropic lists model ids visible to key on api.anthropic.com. The API returns a paginated list; we page through until has_more is false.

func DiscoverGondola added in v0.3.43 

func DiscoverGondola(ctx context.Context, baseURL string) ([]Model, error)

DiscoverGondola lists text models from Gondola's public catalog. Gondola reports prices directly in USD per 1M tokens, matching Model's units.

func DiscoverGoogle 

func DiscoverGoogle(ctx context.Context, apiKey, baseURL string) ([]Model, error)

DiscoverGoogle lists Gemini model ids visible to key on generativelanguage.googleapis.com. The API paginates with nextPageToken; we follow it until exhausted.

func DiscoverOpenAI 

func DiscoverOpenAI(ctx context.Context, apiKey, baseURL string) ([]Model, error)

DiscoverOpenAI lists model ids visible to key on api.openai.com.

func DiscoverOpenRouter added in v0.2.18 

func DiscoverOpenRouter(ctx context.Context, baseURL string) ([]Model, error)

DiscoverOpenRouter lists models from OpenRouter's public /models endpoint (no auth). Per-token USD prices are converted to USD per 1M tokens to match the rest of the catalog. baseURL defaults to the public endpoint.

func DiscoverOpenRouterPresets added in v0.3.54 

func DiscoverOpenRouterPresets(ctx context.Context, apiKey, baseURL string) ([]Model, error)

DiscoverOpenRouterPresets lists the authenticated user's OpenRouter presets and returns them as selectable models with ids of the form "@preset/{slug}". Requires an API key; /presets is not public.

func FindModel 

func FindModel(provider, id string) (Model, error)

FindModel returns a Model by id, optionally constrained by provider. If provider is empty, the first matching id is returned. Looks up against the merged active catalog.

func LlamaCPPModels added in v0.3.1 

func LlamaCPPModels(models []LlamaCPPModel, serverURL string) []Model

LlamaCPPModels converts loaded router entries into zot model metadata.

func LoadUserModels 

func LoadUserModels(path string) []Model

LoadUserModels reads a models.json file and returns the models converted to the internal Model type. Returns nil on any error (missing file, bad JSON, etc.) so the caller can treat it as optional without error handling.

func LoadUserModelsWithWarnings 

func LoadUserModelsWithWarnings(path string) ([]Model, []string)

LoadUserModelsWithWarnings is like LoadUserModels but also returns human-readable warnings about every recoverable issue it found in the file (unknown provider id, empty model id, malformed JSON for a single provider block, etc.). The caller is responsible for surfacing the warnings; the file is never rejected wholesale unless the top-level JSON itself fails to parse.

func MergeCatalog 

func MergeCatalog(live []Model) []Model

MergeCatalog returns the baked-in catalog overlaid with live entries. Precedence per id: live > catalog; speculative entries are promoted to non-speculative when a matching live id appears.

Unknown live ids (not in the static catalog) are appended at the end with placeholder prices so they still render in the picker. Prices can be populated later from a richer catalog source.

func ModelsForProvider 

func ModelsForProvider(provider string) []Model

ModelsForProvider returns all models for the given provider, from the merged active catalog.

type ModelCache 

type ModelCache struct {
	FetchedAt time.Time `json:"fetched_at"`
	Models    []Model   `json:"models"`
}

ModelCache is the on-disk shape for discovered models.

func LoadCache 

func LoadCache(path string) (ModelCache, error)

LoadCache reads the model cache from path. Returns an empty ModelCache (no error) if the file does not exist.

func (ModelCache) IsFresh 

func (c ModelCache) IsFresh() bool

IsFresh reports whether the cache was fetched within CacheTTL.

type ReasoningBlock 

type ReasoningBlock struct {
	ID        string `json:"reasoning_id,omitempty"`
	Summary   string `json:"summary,omitempty"`
	Encrypted string `json:"encrypted_content,omitempty"`
}

ReasoningBlock carries the assistant's chain-of-thought metadata so providers that require it on follow-up requests (OpenAI Codex with thinking enabled) can replay the same payload they emitted earlier. Summary is the human-readable reasoning summary (may be empty); the encrypted blob is opaque to zot. ID is the provider-issued reasoning item id.

type RefreshingClient 

type RefreshingClient struct {
	// contains filtered or unexported fields
}

RefreshingClient wraps a Client and calls a TokenRefresher before every Stream call. When the refresher returns a new token, a fresh underlying client is built via the factory function.

func NewRefreshingClient 

func NewRefreshingClient(inner Client, refreshFn TokenRefresher, factory func(token string) Client) *RefreshingClient

NewRefreshingClient wraps inner with automatic token refresh. refreshFn is called before each Stream; if it returns a non-empty token the factory rebuilds the underlying client with the new token.

func (*RefreshingClient) Name 

func (c *RefreshingClient) Name() string

func (*RefreshingClient) Stream 

func (c *RefreshingClient) Stream(ctx context.Context, req Request) (<-chan Event, error)

type Request 

type Request struct {
	Model       string
	System      string
	Messages    []Message
	Tools       []Tool
	MaxTokens   int
	Temperature *float32
	// Reasoning is "", "minimum", "low", "medium", "high", "xhigh", or
	// "max". Empty disables reasoning. The max tier is sent natively only to
	// models that support it and clamped elsewhere.
	Reasoning string
	// SessionID, when set, is the conversation id. Providers that support
	// sticky routing (OpenRouter session_id) forward it; others ignore it.
	SessionID string
	// MaxToolCalls, when set, caps OpenRouter's server-tool agent loop.
	// Zero omits the field.
	MaxToolCalls int
}

Request is a single LLM call.

type Role 

type Role string

Role is the speaker of a Message. 

const (
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type StopReason 

type StopReason string

StopReason describes why a turn ended. 

const (
	StopEnd     StopReason = "end"
	StopLength  StopReason = "length"
	StopToolUse StopReason = "tool_use"
	StopError   StopReason = "error"
	StopAborted StopReason = "aborted"
)

type TextBlock 

type TextBlock struct {
	Text             string `json:"text"`
	ThoughtSignature string `json:"thought_signature,omitempty"`
}

TextBlock is plain text content.

type TokenRefresher 

type TokenRefresher func(ctx context.Context) (newToken string, err error)

TokenRefresher is a callback that checks whether the current token is still valid and returns a fresh one if needed. The returned string is the new access token; if empty the old one is still fine. An error means refresh failed (network down, refresh token expired, etc.) — the caller should proceed with the stale token and let the API return 401 naturally.

type Tool 

type Tool struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Schema      json.RawMessage `json:"schema"` // JSON Schema for arguments
	// Deferred hides the definition until a tool result activates it.
	Deferred bool `json:"deferred,omitempty"`
}

Tool is a tool definition advertised to the LLM.

type ToolCallBlock 

type ToolCallBlock struct {
	ID               string          `json:"id"`
	Name             string          `json:"name"`
	Arguments        json.RawMessage `json:"arguments"`
	ThoughtSignature string          `json:"thought_signature,omitempty"`
	// Server is true when the provider executes this tool itself
	// (OpenRouter server tools). The agent must not run these locally
	// or send a client tool_result for them.
	Server bool `json:"server,omitempty"`
}

ToolCallBlock is an assistant-issued call to a tool.

type ToolResultBlock 

type ToolResultBlock struct {
	CallID  string    `json:"call_id"`
	Content []Content `json:"content"`
	IsError bool      `json:"is_error"`
}

ToolResultBlock is the result of a tool execution, attached to a Message with Role == RoleTool.

type Usage 

type Usage struct {
	InputTokens          int     `json:"input_tokens"`
	OutputTokens         int     `json:"output_tokens"`
	ReasoningTokens      int     `json:"reasoning_tokens"`
	ReasoningTokensKnown bool    `json:"reasoning_tokens_known,omitempty"`
	CacheReadTokens      int     `json:"cache_read_tokens"`
	CacheWriteTokens     int     `json:"cache_write_tokens"`
	CostUSD              float64 `json:"cost_usd"`
}

Usage aggregates token counts and cost for a turn.

func (Usage) Add 

func (u Usage) Add(v Usage) Usage

Add returns u plus v. A reasoning total is known only when both component usage reports include a separate reasoning-token count.

type UserModel 

type UserModel struct {
	ID                string            `json:"id"`
	Name              string            `json:"name"`
	Reasoning         bool              `json:"reasoning"`
	ReasoningLevelMap map[string]string `json:"reasoningLevelMap,omitempty"`
	ContextWindow     int               `json:"contextWindow"`
	MaxTokens         int               `json:"maxTokens"`
	PriceInput        float64           `json:"priceInput"`
	PriceOutput       float64           `json:"priceOutput"`
	PriceCacheRead    float64           `json:"priceCacheRead"`
	PriceCacheWrite   float64           `json:"priceCacheWrite"`
	BaseURL           string            `json:"baseUrl,omitempty"`
	Input             []string          `json:"input"` // informational only
	API               string            `json:"api"`   // informational only
}

UserModel is a single model entry in the user's models.json.

type UserModelsFile 

type UserModelsFile struct {
	Providers map[string]UserProvider `json:"providers"`
}

UserModelsFile is the JSON format for user-defined models. Place a models.json in $ZOT_HOME to add models that aren't in the baked-in catalog or to override catalog entries. Custom providers (not in the built-in set) may specify a baseUrl and api format at the provider level: 

{
  "providers": {
    "my-company": {
      "baseUrl": "https://llm.mycompany.com/v1",
      "api": "openai",
      "models": [
        {
          "id": "company-llm-v2",
          "name": "Company LLM v2",
          "contextWindow": 128000,
          "maxTokens": 32000
        }
      ]
    }
  }
}

type UserProvider 

type UserProvider struct {
	BaseURL string      `json:"baseUrl,omitempty"`
	API     string      `json:"api,omitempty"` // "openai" (default), "openai-responses", or "anthropic"
	Models  []UserModel `json:"models"`
}

UserProvider groups models under a provider key.

Source Files

View all Source files

Directories

Path Synopsis
Package auth handles credential storage and the login methods supported by zot: API keys and provider-specific subscription OAuth.
 Package auth handles credential storage and the login methods supported by zot: API keys and provider-specific subscription OAuth.
assets
Package assets holds static resources embedded in the zot binary.
 Package assets holds static resources embedded in the zot binary.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.