provider

package
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 29, 2026 License: Apache-2.0 Imports: 29 Imported by: 0

Documentation

Overview

Package provider implements the multi-provider system for sol. This matches opencode's provider architecture for supporting multiple LLM providers. Provider information is fetched dynamically from models.dev API.

Index

Constants

View Source 
const (
	CapText          = "text"
	CapVision        = "vision"
	CapImageGen      = "image_gen"
	CapSearch        = "search"
	CapEmbedding     = "embedding"
	CapSpeech        = "speech"
	CapTranscription = "transcription"
	CapReranking     = "reranking"
)

Capability constants. These are the strings exposed over the API and used as keys in the UI capability matrix. Keep them lowercase snake_case.

Two axes:

  • Kind-derived (CapEmbedding/CapSpeech/CapTranscription/CapReranking, plus CapText/CapImageGen which double as kinds). True when the provider has at least one model of that kind.
  • Modality-derived (CapVision). True when the provider has at least one language model that accepts the corresponding extra input modality.
View Source 
const (
	// ModelsDevURL is the URL for the models.dev API
	ModelsDevURL = "https://models.dev/api.json"

	// RefreshInterval is the cadence of the background catalog refresh
	// when StartPeriodicRefresh is in use. Also the freshness window
	// that the lazy LoadProviders() path uses to skip re-fetching.
	RefreshInterval = 12 * time.Hour
)

Variables

View Source 
var AttachmentOverlay = map[string]AttachmentPolicy{
	"openai": {
		SupportsURL:         true,
		SupportsFileURL:     false,
		MaxURLImages:        50,
		MaxURLBytesPerImage: 18 * 1024 * 1024,
		MaxInlineBytesTotal: 40 * 1024 * 1024,
	},
	"anthropic": {
		SupportsURL:         true,
		SupportsFileURL:     true,
		MaxURLImages:        50,
		MaxURLBytesPerImage: 4*1024*1024 + 512*1024,
		MaxInlineBytesTotal: 24 * 1024 * 1024,
	},
	"google": {
		SupportsURL:         true,
		SupportsFileURL:     true,
		MaxURLImages:        100,
		MaxURLBytesPerImage: 20 * 1024 * 1024,
		MaxInlineBytesTotal: 80 * 1024 * 1024,
	},
	"vertex": {
		SupportsURL:         true,
		SupportsFileURL:     true,
		MaxURLImages:        100,
		MaxURLBytesPerImage: 20 * 1024 * 1024,
		MaxInlineBytesTotal: 80 * 1024 * 1024,
	},
	"bedrock": {
		SupportsURL:         true,
		SupportsFileURL:     true,
		MaxURLImages:        50,
		MaxURLBytesPerImage: 4*1024*1024 + 512*1024,
		MaxInlineBytesTotal: 24 * 1024 * 1024,
	},
	"xai": {
		SupportsURL:         true,
		SupportsFileURL:     false,
		MaxURLImages:        50,
		MaxURLBytesPerImage: 18 * 1024 * 1024,
		MaxInlineBytesTotal: 40 * 1024 * 1024,
	},
	"openaicompat": {
		SupportsURL:         true,
		SupportsFileURL:     false,
		MaxURLImages:        5,
		MaxURLBytesPerImage: 3*1024*1024 + 512*1024,
		MaxInlineBytesTotal: 5 * 1024 * 1024,
	},
}

AttachmentOverlay is the per-provider override map. Keyed by provider_id (same as Overlay above). Values are applied as-is — not merged with DefaultAttachmentPolicy — so each entry must be complete.

Covered: every provider in goai whose message conversion detects the http(s) prefix on message.ImagePart.Image or reads message.FilePart.URL.

Sizing references (as of 2026-Q2 docs):

Our caps sit *below* each provider's hard limit to leave headroom for JSON framing, tool schemas, prompt bytes, etc. `MaxInlineBytesTotal` is the base64-encoded byte budget (roughly 1.33× the raw byte count).

View Source 
var DefaultAttachmentPolicy = AttachmentPolicy{
	MaxInlineBytesTotal: 5 * 1024 * 1024,
	SupportsURL:         false,
}

DefaultAttachmentPolicy is the fallback — inline-only, 5 MiB request cap.

View Source 
var Overlay = map[string]ProviderOverlay{
	"openai": {

		ExtraCapabilities: []string{"search"},
		SearchBackend:     "openai",
	},
	"xai": {
		ExtraCapabilities: []string{"search"},
		SearchBackend:     "grok",
	},
	"google": {
		ExtraCapabilities: []string{"search"},
		SearchBackend:     "gemini",
	},
	"moonshot": {
		ExtraCapabilities: []string{"search"},
		SearchBackend:     "kimi",
	},
	"perplexity": {
		ExtraCapabilities: []string{"search"},
		SearchBackend:     "perplexity",
	},
	"brave": {
		DisplayName:       "Brave Search",
		ExtraCapabilities: []string{"search"},
		SearchBackend:     "brave",
	},
}

Overlay is the hand-maintained overlay map. Keyed by the same provider_id used in models.dev (and by the providers table in Airlock).

Functions

func AllProviders 

func AllProviders() (map[string]*ModelsDevProvider, error)

AllProviders returns the full provider catalog: models.dev data merged with the hand-maintained Overlay AND goai's typed kind lists. Callers that want the live models.dev map unmodified should call LoadProviders directly.

Deprecated models (Status == "deprecated" per models.dev) are dropped before the goai merge so they don't surface in pickers / catalogs. Runtime lookups via GetModelInfo go through LoadProviders directly and so still resolve — agents configured on a now-deprecated model keep running, they're just hidden from the agent-create dropdown going forward. Overlay-supplied ExtraModels with Status="deprecated" are also filtered for consistency.

Merge semantics, in order:

  1. Start from models.dev's LoadProviders() result.
  2. Layer Overlay: providers in both sources merge ExtraModels into the Models map (overlay wins on key collisions). Providers only in Overlay (e.g. brave) get a synthesized stub.
  3. Drop deprecated entries from every provider in the merged set.
  4. Layer goai: for each provider goai has typed-list coverage for (see goaiProviderFactories), stamp ModelInfo.Kind on existing entries and synthesize new ModelInfo for goai-listed IDs that models.dev doesn't ship (e.g. openai's whisper-1, tts-1 — which is why we no longer carry them in Overlay.ExtraModels). Goai is authoritative for Kind; modalities, cost, and limits come from whichever source has them (models.dev wins, kind-derived defaults fall in for the rest).

The returned map's inner *ModelsDevProvider pointers are clones whenever we touched the provider, so callers can mutate top-level safely. Caches are unchanged.

func CreateEmbeddingModel 

func CreateEmbeddingModel(providerID, modelID string, opts Options) model.EmbeddingModel

CreateEmbeddingModel creates an embedding model from provider and model IDs.

func CreateImageModel 

func CreateImageModel(providerID, modelID string, opts Options) model.ImageModel

CreateImageModel creates an image generation model from provider and model IDs.

func CreateModel 

func CreateModel(providerID, modelID string, opts Options) stream.Model

CreateModel creates a language model from provider and model IDs. This is the main entry point for getting a model instance.

func CreateProxyModel 

func CreateProxyModel(fullModelID string, opts ProxyOptions) stream.Model

CreateProxyModel creates a model that proxies LLM calls through an Airlock-compatible endpoint. The full model string (e.g., "anthropic/claude-sonnet-4-20250514") is passed to the proxy which resolves credentials and forwards to the real provider.

func CreateSpeechModel 

func CreateSpeechModel(providerID, modelID string, opts Options) model.SpeechModel

CreateSpeechModel creates a text-to-speech model from provider and model IDs.

func CreateTranscriptionModel 

func CreateTranscriptionModel(providerID, modelID string, opts Options) model.TranscriptionModel

CreateTranscriptionModel creates a speech-to-text model from provider and model IDs.

func GetContextLimit 

func GetContextLimit(providerID, modelID string) int

GetContextLimit returns the context limit for a model. Returns 0 if the model is not found or has no limit defined.

func GetDisplayName 

func GetDisplayName(providerID string) string

GetDisplayName returns the display name for a provider. Uses dynamic data from models.dev when available.

func GetEnvVarName 

func GetEnvVarName(providerID string) string

GetEnvVarName returns the primary environment variable name for a provider's API key. Uses dynamic data from models.dev when available.

func GetOutputLimit 

func GetOutputLimit(providerID, modelID string) int

GetOutputLimit returns the output limit for a model. Returns 0 if the model is not found or has no limit defined.

func ListProviders 

func ListProviders() []string

ListProviders returns all available provider IDs

func LoadProviders 

func LoadProviders() (map[string]*ModelsDevProvider, error)

LoadProviders loads providers from cache or fetches from models.dev This is called lazily on first access

func MaxOutputTokens 

func MaxOutputTokens(modelID string) int

MaxOutputTokens returns the appropriate max output tokens for a model. This matches opencode's defaults for reasoning vs non-reasoning models.

func ParseModel 

func ParseModel(model string) (providerID, modelID string)

ParseModel parses a "provider/model" string into provider and model IDs. This matches opencode's parseModel function. Examples:

  • "openai/gpt-4o-mini" → ("openai", "gpt-4o-mini")
  • "anthropic/claude-3-5-sonnet" → ("anthropic", "claude-3-5-sonnet")
  • "gpt-4o-mini" → ("openai", "gpt-4o-mini") // defaults to openai

func ProviderOptions 

func ProviderOptions(providerID, modelID, sessionID string) map[string]any

ProviderOptions returns the provider-specific options for a model. This matches opencode's ProviderTransform.options() function.

func StartPeriodicRefresh added in v0.1.2 

func StartPeriodicRefresh(ctx context.Context)

StartPeriodicRefresh launches a background ticker that refreshes the models.dev catalog every RefreshInterval. Idempotent — calling more than once spawns at most one ticker (subsequent calls are no-ops).

Pre-flight steps before the ticker runs:

  1. LoadProviders() is invoked synchronously so state.providers is populated from disk cache (or builtin fallback) before the function returns. Callers can then serve catalog requests immediately, even while the network fetch below is in flight.

  2. A non-blocking refresh kicks off in a goroutine — picks up any models.dev changes since the on-disk cache was baked. Failures here are logged but non-fatal: the cache from step 1 stays authoritative.

The ticker honors ctx.Done() and exits cleanly on shutdown.

func SupportsInputModality 

func SupportsInputModality(providerID, modelID, modality string) bool

SupportsInputModality returns true if the model supports the given input type. Returns true if model info is unavailable (optimistic fallback).

Types

type AttachmentPolicy 

type AttachmentPolicy struct {
	// MaxInlineBytesTotal caps the total base64-encoded attachment bytes in
	// a single request body. Parts that don't fit are evicted (oldest first)
	// and replaced with a text placeholder that tells the LLM how to
	// re-attach.
	MaxInlineBytesTotal int

	// SupportsURL is true when the provider accepts http(s):// URLs for
	// image parts. OpenAI chat+responses detect the prefix on ImagePart.Image;
	// Anthropic + Google do the same and also accept it via FilePart.URL (see
	// SupportsFileURL).
	SupportsURL bool

	// SupportsFileURL is true when the provider accepts http(s):// URLs for
	// file parts via message.FilePart.URL. Anthropic + Google support PDFs
	// and other file types by URL; OpenAI currently does not on the chat
	// path.
	SupportsFileURL bool

	// MaxURLImages caps how many image/file parts per request can be sent
	// as URLs before we fall back to inline. Shared across both kinds to
	// keep the provider from rejecting oversized fan-out fetches.
	MaxURLImages int

	// MaxURLBytesPerImage is the per-object size ceiling when sending URLs.
	// Objects above this are fetched and inlined (or evicted) rather than
	// referenced.
	MaxURLBytesPerImage int
}

AttachmentPolicy describes how a given provider/model accepts image and file attachments. Used by airlock's attachref resolver to decide between URL mode (presigned S3 link) and inline mode (base64-encoded bytes), and to enforce per-request size caps.

Defaults are conservative: inline-only, 5 MiB per-request budget. Provider overrides (see AttachmentOverlay) relax these where the upstream API is known to support URL references.

func PolicyFor 

func PolicyFor(providerID, modelID string) AttachmentPolicy

PolicyFor returns the attachment policy for a given provider/model pair. modelID is accepted for future model-level overrides but currently unused.

type CapabilitySet 

type CapabilitySet struct {
	Text          bool
	Vision        bool
	ImageGen      bool
	Search        bool
	Embedding     bool
	Speech        bool
	Transcription bool
	Reranking     bool
}

CapabilitySet is the set of high-level capabilities a model or provider offers. Derived from models.dev modalities, goai-supplied kind, and the overlay's extras.

func CapabilitiesFromModel 

func CapabilitiesFromModel(m ModelInfo) CapabilitySet

CapabilitiesFromModel derives the capability set for a single model from its kind (goai-sourced) plus its modality list (models.dev-sourced). Search is never set at the model level — it's a provider capability (see ProviderCapabilities).

Kind-derived caps fire whenever Kind is set; modality-derived caps fire independently — a Kind=Language model with image input still gets Vision. Modality-only classification is the fallback for catalog entries from providers without goai typed-list coverage.

func ProviderCapabilities 

func ProviderCapabilities(p *ModelsDevProvider, extras []string) CapabilitySet

ProviderCapabilities unions ModelCapabilities across every model in the provider and then ORs in extras (capabilities that aren't derivable from any single model's modalities, such as "search"). Callers typically pass Overlay[providerID].ExtraCapabilities as extras.

It's valid for a provider to have no models — e.g. brave, which is only represented via Overlay. In that case the result is whatever the extras declare.

func (CapabilitySet) List 

func (c CapabilitySet) List() []string

List returns the set as a sorted slice of capability strings in the canonical UI order.

type ModelCapabilities 

type ModelCapabilities struct {
	// IsReasoningModel indicates if the model is a reasoning model
	IsReasoningModel bool

	// SystemMessageMode determines how system messages should be handled.
	// "system" - use standard system role
	// "developer" - convert to developer role (for reasoning models)
	// "remove" - remove system messages entirely
	SystemMessageMode string

	// DefaultReasoningEffort is the default reasoning effort level for this model.
	// Empty string means no default (non-reasoning model or special case like gpt-5-pro).
	DefaultReasoningEffort string

	// DefaultMaxOutputTokens is the recommended max output tokens for this model.
	// 0 means use the standard default.
	DefaultMaxOutputTokens int
}

ModelCapabilities describes capabilities of a model. This matches opencode's getResponsesModelConfig() in openai-responses-language-model.ts

func GetModelCapabilities 

func GetModelCapabilities(modelID string) ModelCapabilities

GetModelCapabilities returns capabilities for a given model ID. This matches opencode's getResponsesModelConfig() function.

type ModelCost 

type ModelCost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
}

ModelCost represents the cost structure for a model

type ModelInfo 

type ModelInfo struct {
	ID   string `json:"id"`
	Name string `json:"name"`
	// Kind classifies the model's primary purpose. Set by AllProviders()
	// from goai's typed lists (EmbeddingModels / ImageModels /
	// SpeechModels / TranscriptionModels / RerankingModels). Empty when
	// goai has no entry — clients treat empty as KindLanguage.
	Kind        ModelKind        `json:"kind,omitempty"`
	Family      string           `json:"family,omitempty"`
	ReleaseDate string           `json:"release_date,omitempty"`
	Attachment  bool             `json:"attachment,omitempty"`
	Reasoning   bool             `json:"reasoning,omitempty"`
	Temperature bool             `json:"temperature,omitempty"`
	ToolCall    bool             `json:"tool_call,omitempty"`
	Modalities  *ModelModalities `json:"modalities,omitempty"`
	Cost        *ModelCost       `json:"cost,omitempty"`
	Limit       *ModelLimit      `json:"limit,omitempty"`
	Status      string           `json:"status,omitempty"` // alpha, beta, deprecated
	// models.dev's `experimental` field was historically a bool but now
	// ships as an object (or absent) depending on the model. We don't
	// currently consume it; capturing it as RawMessage keeps the struct
	// schema-flexible — future consumers can json.Unmarshal it into
	// whatever typed shape they need without breaking the catalog parse
	// when upstream changes the wire shape again.
	Experimental json.RawMessage `json:"experimental,omitempty"`
}

ModelInfo represents a model from models.dev, optionally enriched with goai-supplied Kind classification by AllProviders().

func GetModelInfo 

func GetModelInfo(providerID, modelID string) (*ModelInfo, bool)

GetModelInfo returns model info for a specific provider and model

type ModelKind 

type ModelKind string

ModelKind classifies a model by its primary purpose. Sourced from goai's per-provider typed lists (Models / EmbeddingModels / ImageModels / SpeechModels / TranscriptionModels / RerankingModels). Empty means sol has no goai typed-list coverage for the provider — currently true for the openai-compat bucket (groq, xai, cerebras, fireworks, deepseek, perplexity, togetherai, etc.), all of which ship language models, so callers can safely treat empty as KindLanguage when filtering. 

const (
	KindLanguage      ModelKind = "language"
	KindEmbedding     ModelKind = "embedding"
	KindImage         ModelKind = "image"
	KindSpeech        ModelKind = "speech"        // text-to-speech
	KindTranscription ModelKind = "transcription" // speech-to-text
	KindReranking     ModelKind = "reranking"
)

type ModelLimit 

type ModelLimit struct {
	Context int `json:"context"`
	Input   int `json:"input,omitempty"`
	Output  int `json:"output"`
}

ModelLimit represents token limits for a model

type ModelModalities 

type ModelModalities struct {
	Input  []string `json:"input"`  // e.g. ["text", "image", "pdf", "audio", "video"]
	Output []string `json:"output"` // e.g. ["text"]
}

ModelModalities describes what input/output types a model supports.

func GetModalities 

func GetModalities(providerID, modelID string) *ModelModalities

GetModalities returns the model's input/output modalities. Returns nil if the model is not found or has no modalities defined.

func (*ModelModalities) SupportsInput 

func (m *ModelModalities) SupportsInput(modality string) bool

SupportsInput returns true if the model accepts the given input modality.

type ModelsDevProvider 

type ModelsDevProvider struct {
	ID     string               `json:"id"`
	Name   string               `json:"name"`
	API    string               `json:"api,omitempty"`
	NPM    string               `json:"npm,omitempty"`
	Env    []string             `json:"env"`
	Models map[string]ModelInfo `json:"models"`
}

ModelsDevProvider represents a provider from models.dev

func GetProviderInfo 

func GetProviderInfo(providerID string) (*ModelsDevProvider, bool)

GetProviderInfo returns provider info from the models.dev data

type Options 

type Options struct {
	APIKey  string
	BaseURL string
}

Options holds configuration for creating a provider.

type ProviderOverlay 

type ProviderOverlay struct {
	// DisplayName is used when the provider has no models.dev entry at all
	// (e.g. "brave"). For providers that exist in models.dev we keep the
	// upstream name.
	DisplayName string

	// ExtraModels are supplemental ModelInfo entries merged into the
	// provider's Models map after models.dev loads.
	ExtraModels []ModelInfo

	// ExtraCapabilities are capabilities not derivable from model
	// modalities. Currently only "search" — web search is a provider
	// feature, not a model one.
	ExtraCapabilities []string

	// SearchBackend is the sol/websearch client name used when this provider
	// supplies search. For LLM providers with native search, this differs
	// from the provider_id (xai→grok, google→gemini, moonshot→kimi) because
	// the search backend has its own historical name. For pure search
	// providers it's the same as the overlay key.
	//
	// Only set for providers that actually have a sol/websearch client
	// implementation — declaring a capability without a backend to serve
	// it would be a silent misconfiguration.
	SearchBackend string
}

ProviderOverlay carries hand-maintained data that models.dev either doesn't track or doesn't publish for a given provider. It fills two gaps:

  1. Extra models. models.dev doesn't currently list OpenAI's STT/TTS lineup (gpt-4o-transcribe, whisper-1, tts-1, ...), Deepgram's stt models, ElevenLabs' TTS voices, etc. We supplement them here so the capability matrix reflects what a user actually gets when they configure that provider.

  2. Non-modality capabilities. Web search is a provider-level tool, not a model modality. models.dev has no way to express "this provider has a web_search tool", so we flag it here.

type ProxyOptions 

type ProxyOptions struct {
	BaseURL string // Proxy server URL (e.g., "http://localhost:8080")
	Token   string // Bearer token for authentication
}

ProxyOptions holds configuration for creating a proxy-backed model.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.