providers

package
v0.17.21 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 29, 2026 License: MIT Imports: 28 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	BillingPayPerToken  = "pay_per_token" // default — real USD per token
	BillingSubscription = "subscription"  // flat-rate, quota/rate-limited
	BillingFree         = "free"          // self-hosted, zero marginal cost
)
View Source 
const (
	CerebrasClientType    = "cerebras"
	ChutesClientType      = "chutes"
	DeepinfraClientType   = "deepinfra"
	DeepseekClientType    = "deepseek"
	LmstudioClientType    = "lmstudio"
	MinimaxClientType     = "minimax"
	MistralClientType     = "mistral"
	OllamaCloudClientType = "ollama-cloud"
	OpenaiClientType      = "openai"
	OpenrouterClientType  = "openrouter"
	SproutLocalClientType = "sprout-local"
	ZaiClientType         = "zai"
	ZaiCodingClientType   = "zai-coding"
)

ClientType constants for all providers These are auto-generated from provider configs (as strings to avoid import cycles)

Variables

View Source 
var LocalActivityHook func()

LocalActivityHook is called after every successful local server interaction to reset the idle timer. Set by the agent package.

View Source 
var LocalServerHook func(providerID string) error

LocalServerHook is called when a local provider (sprout-local) gets a connection error on its first request. If set, the provider attempts to start the local LLM server and retries the request once. This decouples the agent_providers package (no build tags, compiles for WASM) from the localmodel package (which requires !js and platform-specific code).

The hook should:

  • Check if the server is already running (fast path)
  • Find the appropriate model for the machine
  • Spawn the server process if needed
  • Wait for health
  • Return nil on success, error if the server can't be started

Set this to nil (the default) to disable auto-start behavior entirely.

Functions

func AllProviderNames 

func AllProviderNames() []string

AllProviderNames returns all provider names as strings

func BuildOpenAIChatMessages 

func BuildOpenAIChatMessages(messages []api.Message, opts MessageConversionOptions) []map[string]interface{}

BuildOpenAIChatMessages converts agent messages into OpenAI/OpenRouter style chat message payloads, including multimodal content where necessary.

func BuildOpenAIStreamingMessages 

func BuildOpenAIStreamingMessages(messages []api.Message, opts MessageConversionOptions) []interface{}

BuildOpenAIStreamingMessages converts messages for streaming endpoints. The content is identical to the chat payload, but represented as []interface{} to match the JSON marshalling performed by providers.

func BuildOpenAIToolsPayload 

func BuildOpenAIToolsPayload(tools []api.Tool) []map[string]interface{}

BuildOpenAIToolsPayload normalises internal tool definitions to the OpenAI function-calling schema used by OpenRouter, DeepInfra, and other compatible providers.

func CalculateMaxTokens 

func CalculateMaxTokens(contextLimit int, messages []api.Message, tools []api.Tool) int

CalculateMaxTokens returns an appropriate max_tokens value given the context window and prompt size. The caller passes the effective context limit, making it easy to reuse across providers with custom limit lookups.

func CalculateMaxTokensWithLimits 

func CalculateMaxTokensWithLimits(contextLimit int, completionLimit int, messages []api.Message, tools []api.Tool) int

CalculateMaxTokensWithLimits computes a token budget from context and optional completion caps. Uses centralized token estimation for consistency across all providers.

func ClientTypeToString 

func ClientTypeToString(ct string) string

ClientTypeToString converts ClientType (as string) to string This replaces the hardcoded mapClientTypeToString function

func EstimateInputTokens 

func EstimateInputTokens(messages []api.Message, tools []api.Tool) int

EstimateInputTokens provides a quick upper bound for prompt tokens based on message lengths and attached tool metadata. Delegates to the centralized implementation in agent_api for consistency.

func KnownProviders 

func KnownProviders() []string

KnownProviders returns the list of known built-in provider names This replaces the hardcoded knownProviderNames list in api_keys.go

func NeutralizeSpecialTokens added in v0.17.18 

func NeutralizeSpecialTokens(s string) string

NeutralizeSpecialTokens replaces provider special-token literals in message content with inert look-alikes. It deliberately never touches tool-call arguments: arguments carry the work product (file contents the model is writing), and corrupting those corrupts the user's code. The model physically cannot emit the real token IDs in its own output (the serving stack renders them out or terminates on them), so arguments are not a re-priming vector.

func ProviderDisplayNames 

func ProviderDisplayNames() map[string]string

ProviderDisplayNames returns a map of provider names to display names. Values prefer the per-config display_name field (single source of truth in pkg/agent_providers/configs/*.json); the hardcoded switch in generate_providers.go::displayName is kept as a transitional fallback for configs that haven't been backfilled yet.

func ProviderEnvVar 

func ProviderEnvVar(name string) string

ProviderEnvVar returns the environment variable name for a provider's API key

func ProviderRequiresAPIKey 

func ProviderRequiresAPIKey(name string) bool

ProviderRequiresAPIKey returns whether a provider requires an API key

func StringToClientType 

func StringToClientType(name string) (string, error)

StringToClientType converts a string to ClientType (as string) This replaces the hardcoded ParseProviderName function

Types

type AuthConfig 

type AuthConfig struct {
	Type   string `json:"type"`    // "bearer", "api_key", "basic", "oauth"
	EnvVar string `json:"env_var"` // Environment variable containing the auth token
	Key    string `json:"-"`       // Runtime-only API key; injected at startup, never persisted
}

AuthConfig defines authentication configuration

type BackendType added in v0.17.16 

type BackendType string

BackendType identifies the serving backend for model discovery. 

const (
	BackendAuto     BackendType = "auto"
	BackendOpenAI   BackendType = "openai"
	BackendVLLM     BackendType = "vllm"
	BackendLlamaCPP BackendType = "llamacpp"
)

type CostConfig 

type CostConfig struct {
	InputTokenCost  float64 `json:"input_token_cost"`
	OutputTokenCost float64 `json:"output_token_cost"`
	Currency        string  `json:"currency"`
}

CostConfig defines cost tracking configuration

type GenericProvider 

type GenericProvider struct {
	// contains filtered or unexported fields
}

GenericProvider implements ClientInterface using JSON configuration

func NewGenericProvider 

func NewGenericProvider(config *ProviderConfig) (*GenericProvider, error)

NewGenericProvider creates a new generic provider from configuration

func (*GenericProvider) CheckConnection 

func (p *GenericProvider) CheckConnection() error

CheckConnection tests provider connection with current model

func (*GenericProvider) GetAverageTPS 

func (p *GenericProvider) GetAverageTPS() float64

func (*GenericProvider) GetEndpoint added in v0.17.7 

func (p *GenericProvider) GetEndpoint() string

GetEndpoint returns the API endpoint from the provider config.

func (*GenericProvider) GetHTTPClient 

func (p *GenericProvider) GetHTTPClient() *http.Client

GetHTTPClient returns the HTTP client for non-streaming requests (useful for WASM verification).

func (*GenericProvider) GetLastTPS 

func (p *GenericProvider) GetLastTPS() float64

TPS tracking methods (no-op placeholders)

func (*GenericProvider) GetModel 

func (p *GenericProvider) GetModel() string

GetModel returns the current model

func (*GenericProvider) GetModelContextLimit 

func (p *GenericProvider) GetModelContextLimit() (int, error)

GetModelContextLimit returns the context limit for the current model

func (*GenericProvider) GetProvider 

func (p *GenericProvider) GetProvider() string

GetProvider returns the provider name

func (*GenericProvider) GetStreamingClient 

func (p *GenericProvider) GetStreamingClient() *http.Client

GetStreamingClient returns the HTTP client for streaming requests (useful for WASM verification).

func (*GenericProvider) GetTPSStats 

func (p *GenericProvider) GetTPSStats() map[string]float64

func (*GenericProvider) GetVisionModel 

func (p *GenericProvider) GetVisionModel() string

GetVisionModel returns the vision model

func (*GenericProvider) ListModels 

func (p *GenericProvider) ListModels(ctx context.Context) ([]api.ModelInfo, error)

ListModels returns available models, dispatching to a backend-specific fetcher (openai, vllm, llamacpp, or auto). Results are cached; subsequent calls return the cached list without re-fetching.

func (*GenericProvider) RefreshAPIKey 

func (p *GenericProvider) RefreshAPIKey() error

RefreshAPIKey re-resolves the API key from the credential store for subsequent requests.

func (*GenericProvider) ResetTPSStats 

func (p *GenericProvider) ResetTPSStats()

func (*GenericProvider) SendChatRequest 

func (p *GenericProvider) SendChatRequest(ctx context.Context, messages []api.Message, tools []api.Tool, reasoning string, disableThinking bool) (*api.ChatResponse, error)

SendChatRequest sends a non-streaming chat request

func (*GenericProvider) SendChatRequestStream 

func (p *GenericProvider) SendChatRequestStream(ctx context.Context, messages []api.Message, tools []api.Tool, reasoning string, disableThinking bool, callback api.StreamCallback) (*api.ChatResponse, error)

SendChatRequestStream sends a streaming chat request

func (*GenericProvider) SendVisionRequest 

func (p *GenericProvider) SendVisionRequest(ctx context.Context, messages []api.Message, tools []api.Tool, reasoning string, disableThinking bool) (*api.ChatResponse, error)

SendVisionRequest sends a vision request (for providers that support it)

func (*GenericProvider) SetDebug 

func (p *GenericProvider) SetDebug(debug bool)

SetDebug enables or disables debug mode

func (*GenericProvider) SetHTTPClient 

func (p *GenericProvider) SetHTTPClient(c *http.Client)

SetHTTPClient sets the HTTP client used for non-streaming requests.

func (*GenericProvider) SetMaxTokensHint added in v0.17.14 

func (p *GenericProvider) SetMaxTokensHint(tokens int)

SetMaxTokensHint sets a pre-computed max_tokens override (0 to clear).

func (*GenericProvider) SetModel 

func (p *GenericProvider) SetModel(model string) error

SetModel sets the current model

func (*GenericProvider) SetStreamingClient 

func (p *GenericProvider) SetStreamingClient(c *http.Client)

SetStreamingClient sets the HTTP client used for streaming requests.

func (*GenericProvider) SupportsConversationalVision added in v0.16.19 

func (p *GenericProvider) SupportsConversationalVision() bool

SupportsConversationalVision returns whether the active model is suitable for inline multimodal chat messages. Currently equivalent to SupportsVision() because all GenericProvider entries that opt into vision are chat-format models; OCR-only clients (like OllamaLocalClient) override this method to return false for OCR-only tags.

func (*GenericProvider) SupportsVision 

func (p *GenericProvider) SupportsVision() bool

SupportsVision returns whether the current model can accept image input.

Resolution:

  1. If the provider config sets supports_vision: false, vision is off for the entire provider — return false immediately.
  2. If the current model is listed in model_info, the tag list is authoritative: return true only when a "vision" tag is present. This lets configs grant or withhold vision on a per-model basis when the model catalogue is populated.
  3. When supports_vision: true but the current model is NOT in model_info (no per-model data), trust the provider-level flag and return true. This is the common case — most configs set supports_vision: true with an empty model_info, meaning "all models from this provider accept images". Returning false here would silently break image embedding for models like gpt-5-mini that do support vision.

func (*GenericProvider) VisionCapabilities added in v0.16.20 

func (p *GenericProvider) VisionCapabilities() api.VisionCapabilities

VisionCapabilities returns per-provider vision limits (nil config returns safe defaults).

type MaxTokensHinter added in v0.17.14 

type MaxTokensHinter interface {
	SetMaxTokensHint(tokens int)
}

MaxTokensHinter lets callers pass a pre-computed max_tokens to the provider.

type MessageConversion 

type MessageConversion struct {
	IncludeToolCallID     bool   `json:"include_tool_call_id"`
	ConvertToolRoleToUser bool   `json:"convert_tool_role_to_user"`
	ReasoningContentField string `json:"reasoning_content_field"`
	// PreserveReasoningDetails replays the structured reasoning_details array
	// (OpenRouter unified reasoning blocks — encrypted/signed/summary) on
	// assistant history messages verbatim. Required for preserved thinking on
	// models whose reasoning cannot round-trip as a plain string (Anthropic).
	// Takes precedence over ReasoningContentField string replay when both exist.
	PreserveReasoningDetails bool `json:"preserve_reasoning_details,omitempty"`
	// UnifiedReasoningParam emits the unified top-level `reasoning` object
	// ({"effort": ...}) instead of provider-native thinking knobs. This is
	// OpenRouter's canonical control surface and the only way to steer
	// reasoning effort on Anthropic models through it.
	UnifiedReasoningParam bool `json:"unified_reasoning_param,omitempty"`
	// ChatTemplateKwargs are merged into the per-request
	// chat_template_kwargs object for template-driven local servers
	// (vLLM, llama.cpp, LM Studio). Only emitted for localhost endpoints —
	// hosted APIs reject unknown request fields.
	ChatTemplateKwargs       map[string]interface{} `json:"chat_template_kwargs,omitempty"`
	ArgumentsAsJSON          bool                   `json:"arguments_as_json"`
	SkipToolExecutionSummary bool                   `json:"skip_tool_execution_summary"` // For providers with strict role alternation
	ForceToolCallType        string                 `json:"force_tool_call_type"`        // Force tool call type to specific value (e.g., "function" for Mistral)
	// NeutralizeSpecialTokens replaces special-token literals (<|im_end|> etc.)
	// in message content with inert look-alikes before sending. Opt-in: enable
	// only for providers whose tokenizer treats those byte sequences as control
	// tokens (Qwen-family via vLLM, Llama). Trade-off: the model reads a
	// visually-near-identical but byte-different string, so sessions that must
	// round-trip the exact literal (e.g. writing tokenizer code) should not
	// enable it. Tool-call arguments are never touched (work product).
	NeutralizeSpecialTokens bool `json:"neutralize_special_tokens,omitempty"`
	// CacheControl enables provider prompt-prefix caching (Anthropic-style
	// cache_control: {type: "ephemeral"} markers). When true, cache breakpoints
	// are injected at three locations:
	//   1. The system message (static prefix)
	//   2. The last tool definition (static tool schema)
	//   3. The last conversation message (growing conversation prefix — highest impact)
	// Anthropic allows up to 4 breakpoints; we use 3, leaving headroom for future use.
	CacheControl bool `json:"cache_control,omitempty"`
	// FillMissingArrayItems walks every tool schema in the outgoing request
	// and fills any array-typed property that lacks an "items" key with a
	// permissive fallback ({"type": ArrayItemsFallback}). Opt-in per provider.
	// Required for Google Gemini 3.x strict function-calling validation, which
	// rejects the entire request (HTTP 400, e.g. via OpenRouter) when any
	// array property has no "items" — Gemini 2.5 tolerated them. Defense in
	// depth: native tool definitions already declare items, but the seed
	// registry's wire schema (and any third-party/MCP tool) may not, so the
	// shim guarantees a valid items on the wire for strict validators.
	FillMissingArrayItems bool `json:"fill_missing_array_items,omitempty"`
	// ArrayItemsFallback is the JSON-schema type used for the permissive items
	// fallback when filling missing array items. "object" (default, most
	// permissive) or "string". Only read when FillMissingArrayItems is true.
	ArrayItemsFallback string `json:"array_items_fallback,omitempty"`
}

MessageConversion defines how messages should be converted

type MessageConversionOptions 

type MessageConversionOptions struct {
	// Convert tool-role messages to user messages with a labeled prefix. Some
	// providers (DeepInfra) reject the "tool" role entirely.
	ConvertToolRoleToUser bool
	// Include tool_call_id when present. Required for OpenRouter when sending
	// tool execution results back to the model.
	IncludeToolCallID bool
	// Force tool call type to specific value (e.g., "function" for Mistral)
	ForceToolCallType string
}

MessageConversionOptions controls how agent messages are transformed into OpenAI-compatible payloads.

type ModelConfig 

type ModelConfig struct {
	DefaultContextLimit        int `json:"default_context_limit"`
	DefaultMaxCompletionTokens int `json:"default_max_completion_tokens,omitempty"`
	// DefaultModelPatterns defines preference patterns for auto-selecting a default model.
	// Patterns are tried in order; the first model whose ID contains all substrings in a pattern wins.
	// Example: []string{"deepseek.*instruct", "deepseek", "llama"}
	DefaultModelPatterns       []string          `json:"default_model_patterns,omitempty"`
	ModelOverrides             map[string]int    `json:"model_overrides"`
	MaxCompletionOverrides     map[string]int    `json:"max_completion_overrides,omitempty"`
	PatternOverrides           []PatternOverride `json:"pattern_overrides"`
	CompletionPatternOverrides []PatternOverride `json:"completion_pattern_overrides,omitempty"`
	// Config-based model definitions (fallback when endpoint fetch fails or lacks details)
	ModelInfo []ModelInfo `json:"model_info,omitempty"`
	// Legacy fields for backward compatibility
	ContextLimit    int      `json:"context_limit,omitempty"`
	SupportsVision  bool     `json:"supports_vision"`
	VisionModel     string   `json:"vision_model"`
	DefaultModel    string   `json:"default_model"`
	AvailableModels []string `json:"available_models"`
}

ModelConfig defines model-related configuration

type ModelInfo 

type ModelInfo struct {
	ID            string   `json:"id"`
	Name          string   `json:"name,omitempty"`
	Description   string   `json:"description,omitempty"`
	ContextLength int      `json:"context_length"`
	Tags          []string `json:"tags,omitempty"`
	// Pricing (USD per million tokens) — optional, used by enrich_registry
	// to estimate probe cost for models sourced from embedded configs.
	InputCost  float64 `json:"input_cost,omitempty"`
	OutputCost float64 `json:"output_cost,omitempty"`
	CachedCost float64 `json:"cached_input_cost,omitempty"`
}

ModelInfo represents information about a model (simplified version for config)

type PatternOverride 

type PatternOverride struct {
	Pattern      string `json:"pattern"`
	ContextLimit int    `json:"context_limit"`
}

PatternOverride defines context limit overrides for model patterns

type ProviderConfig 

type ProviderConfig struct {
	Name        string `json:"name"`
	BillingType string `json:"billing_type,omitempty"`
	// DisplayName is the user-facing label (e.g. "GLM Coding Plan").
	// Carried in the JSON config so onboarding menus, the env-var
	// credential sweep, and the model picker can label remote-only
	// providers (published to GitHub Pages but not embedded) without
	// a binary rebuild. Optional — callers should fall back to the
	// static knownProviderDisplayNames map and then the raw Name.
	DisplayName string            `json:"display_name,omitempty"`
	Endpoint    string            `json:"endpoint"`
	Auth        AuthConfig        `json:"auth"`
	Headers     map[string]string `json:"headers"`
	Defaults    RequestDefaults   `json:"defaults"`
	Conversion  MessageConversion `json:"message_conversion"`
	Streaming   StreamingConfig   `json:"streaming"`
	// Backend identifies the serving backend type. When set to "auto"
	// (default), the provider probes the endpoint to detect the backend.
	// Explicit values ("openai", "vllm", "llamacpp") skip detection.
	Backend string      `json:"backend,omitempty"`
	Models  ModelConfig `json:"models"`
	Retry   RetryConfig `json:"retry"`
	Cost    CostConfig  `json:"cost"`
}

ProviderConfig defines the configuration for a generic provider

func LoadProviderConfig 

func LoadProviderConfig(configPath string) (*ProviderConfig, error)

LoadProviderConfig loads a provider configuration from a JSON file

func (*ProviderConfig) BackendResolved added in v0.17.16 

func (c *ProviderConfig) BackendResolved() BackendType

BackendResolved returns the effective backend type, defaulting to "auto".

func (*ProviderConfig) BillingTypeResolved added in v0.16.19 

func (c *ProviderConfig) BillingTypeResolved() string

BillingTypeResolved returns the effective billing model for this provider. Explicit config value takes priority; otherwise heuristics apply:

  • localhost / 127.0.0.1 endpoints → free
  • zai-coding → subscription
  • everything else → pay_per_token (default)

func (*ProviderConfig) GetAuthToken 

func (c *ProviderConfig) GetAuthToken() (string, error)

GetAuthToken retrieves the authentication token based on the auth configuration.

For "bearer" and "api_key" auth types, token resolution follows this precedence:

  1. Auth.Key — runtime-resolved key injected by the provider factory via the unified credential path (credentials.ResolveProvider). This is the primary source because it checks env vars, keyring, and the encrypted file store.
  2. Auth.EnvVar — direct os.Getenv lookup as a fallback (only used when the factory did not pre-resolve credentials, e.g., in unit tests).

Auth.Key is runtime-only and must never be persisted to disk.

func (*ProviderConfig) GetContextLimit 

func (c *ProviderConfig) GetContextLimit(model string) int

GetContextLimit returns the context limit for a given model based on configuration Uses the following priority: 1. Exact model match in model_overrides 2. Pattern match in pattern_overrides 3. Lookup in model_info (catalog — source of truth for known models) 4. Provider default_context_limit (conservative fallback when catalog is absent) 5. Legacy context_limit field (for backward compatibility) 6. Conservative fallback (32000)

func (*ProviderConfig) GetFirstChunkTimeout added in v0.17.20 

func (c *ProviderConfig) GetFirstChunkTimeout() time.Duration

GetFirstChunkTimeout returns the deadline for the first chunk of a streaming response. Defaults far above the inter-chunk deadline because prompt prefill on large contexts (or slow/local models) can legitimately run many minutes before the first token. Must stay <= the HTTP client's streaming timeout or the transport kills the stream first.

func (*ProviderConfig) GetIdleChunkTimeout added in v0.17.20 

func (c *ProviderConfig) GetIdleChunkTimeout() time.Duration

GetIdleChunkTimeout returns the inter-chunk idle deadline applied after the first chunk has arrived. A stall mid-generation (proxy idle hole, dead upstream) is detected here and surfaced as a retryable network error. IdleChunkTimeoutMs takes precedence over the legacy ChunkTimeoutMs knob without shortening the HTTP client timeout.

func (*ProviderConfig) GetMaxCompletionLimit 

func (c *ProviderConfig) GetMaxCompletionLimit(model string) int

GetMaxCompletionLimit returns the completion-token limit for a given model. Uses the following priority: 1. Exact model match in max_completion_overrides 2. Pattern match in completion_pattern_overrides 3. Provider default_max_completion_tokens 4. 0 (unknown/unset)

func (*ProviderConfig) GetModelInfo 

func (c *ProviderConfig) GetModelInfo(modelID string) *ModelInfo

GetModelInfo returns model information from config if available

func (*ProviderConfig) GetStreamingTimeout 

func (c *ProviderConfig) GetStreamingTimeout() time.Duration

GetStreamingTimeout returns the configured streaming timeout duration

func (*ProviderConfig) GetTimeout 

func (c *ProviderConfig) GetTimeout() time.Duration

GetTimeout returns the configured timeout duration

func (*ProviderConfig) Validate 

func (c *ProviderConfig) Validate() error

Validate validates the provider configuration

type ProviderFactory 

type ProviderFactory struct {
	// contains filtered or unexported fields
}

ProviderFactory creates provider instances from JSON configurations

func GlobalFactory added in v0.16.2 

func GlobalFactory() *ProviderFactory

GlobalFactory returns the singleton ProviderFactory instance. It is initialized with embedded configs during package init.

func NewProviderFactory 

func NewProviderFactory() *ProviderFactory

NewProviderFactory creates a new provider factory

func (*ProviderFactory) CreateProvider 

func (f *ProviderFactory) CreateProvider(name string) (api.ClientInterface, error)

CreateProvider creates a provider instance by name

func (*ProviderFactory) CreateProviderWithModel 

func (f *ProviderFactory) CreateProviderWithModel(name, model string) (api.ClientInterface, error)

CreateProviderWithModel creates a provider instance with a specific model

func (*ProviderFactory) GetAvailableProviders 

func (f *ProviderFactory) GetAvailableProviders() []string

GetAvailableProviders returns a list of available provider names

func (*ProviderFactory) GetDefaultProvider 

func (f *ProviderFactory) GetDefaultProvider() string

GetDefaultProvider returns the default provider name

func (*ProviderFactory) GetProviderConfig 

func (f *ProviderFactory) GetProviderConfig(name string) (*ProviderConfig, error)

GetProviderConfig returns a copy of the configuration for a provider. A copy is returned (rather than a pointer to internal state) so that callers cannot mutate the factory's stored config after the RLock is released.

func (*ProviderFactory) GetRegistry 

func (f *ProviderFactory) GetRegistry() *ProviderRegistry

GetRegistry returns a deep copy of the provider registry. A copy is returned (rather than a pointer to internal state) so that callers cannot mutate the factory's stored registry after the RLock is released.

func (*ProviderFactory) ListProvidersWithModels 

func (f *ProviderFactory) ListProvidersWithModels() map[string][]string

ListProvidersWithModels returns all providers with their available models

func (*ProviderFactory) LoadConfigFromBytes 

func (f *ProviderFactory) LoadConfigFromBytes(data []byte) error

LoadConfigFromBytes loads a provider configuration from byte data

func (*ProviderFactory) LoadConfigFromFile 

func (f *ProviderFactory) LoadConfigFromFile(filename string) error

LoadConfigFromFile loads a single provider configuration from file

func (*ProviderFactory) LoadConfigsFromDirectory 

func (f *ProviderFactory) LoadConfigsFromDirectory(configDir string) error

LoadConfigsFromDirectory loads all provider configurations from a directory

func (*ProviderFactory) LoadEmbeddedConfigs 

func (f *ProviderFactory) LoadEmbeddedConfigs() error

LoadEmbeddedConfigs loads all provider configurations from the embedded filesystem

func (*ProviderFactory) ReloadConfig 

func (f *ProviderFactory) ReloadConfig(filename string) error

ReloadConfig reloads a provider configuration from file

func (*ProviderFactory) UpsertConfig added in v0.16.2 

func (f *ProviderFactory) UpsertConfig(name string, cfg *ProviderConfig) error

UpsertConfig inserts or updates a provider configuration in the factory. The provided config is deep-copied so external mutations have no effect. If cfg.Name differs from name, cfg.Name is overwritten to match name for consistency. The config is validated before insertion.

func (*ProviderFactory) ValidateProvider 

func (f *ProviderFactory) ValidateProvider(providerName, modelName string) error

ValidateProvider checks if a provider and model combination is valid

type ProviderRegistry 

type ProviderRegistry struct {
	DefaultProvider  string                    `json:"default_provider"`
	EnabledProviders []string                  `json:"enabled_providers"`
	ProviderConfigs  map[string]ProviderConfig `json:"provider_configs"`
}

ProviderRegistry holds all provider configurations

func LoadProviderRegistry 

func LoadProviderRegistry(registryPath string) (*ProviderRegistry, error)

LoadProviderRegistry loads the provider registry from a JSON file

type RequestDefaults 

type RequestDefaults struct {
	Model       string                 `json:"model"`
	Temperature *float64               `json:"temperature"`
	MaxTokens   *int                   `json:"max_tokens"`
	TopP        *float64               `json:"top_p"`
	Parameters  map[string]interface{} `json:"parameters,omitempty"` // Provider-specific parameters
}

RequestDefaults defines default request parameters

type RetryConfig 

type RetryConfig struct {
	MaxAttempts       int      `json:"max_attempts"`
	BaseDelayMs       int      `json:"base_delay_ms"`
	BackoffMultiplier float64  `json:"backoff_multiplier"`
	MaxDelayMs        int      `json:"max_delay_ms"`
	RetryableErrors   []string `json:"retryable_errors"`
}

RetryConfig defines retry behavior

type StreamingConfig 

type StreamingConfig struct {
	Format string `json:"format"` // "sse", "json_lines", "raw"
	// ChunkTimeoutMs overrides the streaming HTTP client timeout and the
	// inter-chunk idle deadline (legacy single knob; default 900s/120s).
	ChunkTimeoutMs int `json:"chunk_timeout_ms"`
	// FirstChunkTimeoutMs overrides the deadline for the FIRST chunk of a
	// stream (default 10m). Slow or locally-hosted models routinely spend
	// several minutes in prompt prefill before emitting the first token —
	// the inter-chunk deadline must not apply to that window or every
	// attempt is killed and retried in an invisible loop.
	FirstChunkTimeoutMs int `json:"first_chunk_timeout_ms"`
	// IdleChunkTimeoutMs overrides only the inter-chunk idle deadline after
	// the first chunk (default 120s). Takes precedence over ChunkTimeoutMs;
	// unlike ChunkTimeoutMs it does NOT shorten the HTTP client timeout,
	// which would kill legitimately long streams.
	IdleChunkTimeoutMs int    `json:"idle_chunk_timeout_ms"`
	DoneMarker         string `json:"done_marker"`
}

StreamingConfig defines streaming behavior

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.