llm

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	FinishStop          = "stop"
	FinishLength        = "length"
	FinishToolCalls     = "tool_calls"
	FinishContentFilter = "content_filter"
	FinishError         = "error"
	FinishOther         = "other"
)
View Source 
const (
	ToolChoiceAuto     = "auto"
	ToolChoiceNone     = "none"
	ToolChoiceRequired = "required"
	ToolChoiceNamed    = "named"
)

ToolChoice mode constants.

Variables

This section is empty.

Functions

func ErrorFromStatusCode 

func ErrorFromStatusCode(statusCode int, message, provider, errorCode string, raw json.RawMessage, retryAfter *float64) error

ErrorFromStatusCode maps an HTTP status code to the appropriate error type. For unknown status codes, it returns a ProviderError with Retryable=true as a conservative default (unknown errors are assumed transient).

func Float64Ptr 

func Float64Ptr(v float64) *float64

Float64Ptr returns a pointer to a float64 value.

func GenerateCallID 

func GenerateCallID() string

GenerateCallID produces a unique identifier for tool calls, prefixed with "call_". This is used for providers like Gemini that do not assign their own tool call IDs.

func IntPtr 

func IntPtr(v int) *int

IntPtr returns a pointer to an int value.

func Retry 

func Retry(ctx context.Context, policy RetryPolicy, fn func() error) error

Retry executes fn with the given retry policy. It retries on retryable errors up to MaxRetries times, using exponential backoff with optional jitter. If the error has a RetryAfter hint (e.g., from a RateLimitError), that value is used as the minimum delay. The context can be used to cancel retries early.

func SetDefaultClient 

func SetDefaultClient(c *Client)

SetDefaultClient sets the module-level default client. Pass nil to clear it.

Types

type AbortError 

type AbortError struct {
	SDKError
}

AbortError represents an intentionally aborted operation. Not retryable.

func (*AbortError) As 

func (e *AbortError) As(target any) bool

func (*AbortError) Error 

func (e *AbortError) Error() string

func (*AbortError) IsRetryable 

func (e *AbortError) IsRetryable() bool

func (*AbortError) Unwrap 

func (e *AbortError) Unwrap() error

type AccessDeniedError 

type AccessDeniedError struct {
	ProviderError
}

AccessDeniedError represents a 403 Forbidden response. Not retryable.

func (*AccessDeniedError) As 

func (e *AccessDeniedError) As(target any) bool

func (*AccessDeniedError) Error 

func (e *AccessDeniedError) Error() string

func (*AccessDeniedError) IsRetryable 

func (e *AccessDeniedError) IsRetryable() bool

func (*AccessDeniedError) Unwrap 

func (e *AccessDeniedError) Unwrap() error

type AdapterTimeout 

type AdapterTimeout struct {
	Connect    time.Duration `json:"connect"`
	Request    time.Duration `json:"request"`
	StreamRead time.Duration `json:"stream_read"`
}

AdapterTimeout specifies timeout durations at the adapter level.

func DefaultAdapterTimeout 

func DefaultAdapterTimeout() AdapterTimeout

DefaultAdapterTimeout returns sensible defaults for adapter timeouts.

type AnthropicAdapter 

type AnthropicAdapter struct {
	*BaseAdapter
	// contains filtered or unexported fields
}

AnthropicAdapter implements ProviderAdapter for the Anthropic Messages API.

func NewAnthropicAdapter deprecated 

func NewAnthropicAdapter(apiKey string, opts ...AnthropicOption) *AnthropicAdapter

NewAnthropicAdapter creates an AnthropicAdapter with the given API key and options. Authentication uses x-api-key header instead of Bearer token, so the API key is stored in DefaultHeaders rather than BaseAdapter.APIKey.

Deprecated: Use NewMuxAdapter with the appropriate mux/llm client instead. Retained as fallback for custom base URL configurations.

func (*AnthropicAdapter) Close 

func (a *AnthropicAdapter) Close() error

Close releases any resources held by the adapter.

func (*AnthropicAdapter) Complete 

func (a *AnthropicAdapter) Complete(ctx context.Context, req Request) (*Response, error)

Complete sends a synchronous completion request to the Anthropic Messages API.

func (*AnthropicAdapter) Name 

func (a *AnthropicAdapter) Name() string

Name returns the provider name "anthropic".

func (*AnthropicAdapter) Stream 

func (a *AnthropicAdapter) Stream(ctx context.Context, req Request) (<-chan StreamEvent, error)

Stream sends a streaming completion request to the Anthropic Messages API.

type AnthropicOption 

type AnthropicOption func(*AnthropicAdapter)

AnthropicOption is a functional option for configuring an AnthropicAdapter.

func WithAnthropicBaseURL deprecated 

func WithAnthropicBaseURL(url string) AnthropicOption

WithAnthropicBaseURL overrides the default Anthropic API base URL.

Deprecated: Use NewMuxAdapter with the appropriate mux/llm client instead. Retained as fallback for custom base URL configurations.

func WithAnthropicTimeout 

func WithAnthropicTimeout(timeout AdapterTimeout) AnthropicOption

WithAnthropicTimeout sets custom timeout values for the adapter.

func WithAnthropicVersion 

func WithAnthropicVersion(version string) AnthropicOption

WithAnthropicVersion sets the anthropic-version header value.

type AudioData 

type AudioData struct {
	URL       string `json:"url,omitempty"`
	Data      []byte `json:"data,omitempty"`
	MediaType string `json:"media_type,omitempty"`
}

AudioData holds audio content.

type AuthenticationError 

type AuthenticationError struct {
	ProviderError
}

AuthenticationError represents a 401 Unauthorized response. Not retryable.

func (*AuthenticationError) As 

func (e *AuthenticationError) As(target any) bool

func (*AuthenticationError) Error 

func (e *AuthenticationError) Error() string

func (*AuthenticationError) IsRetryable 

func (e *AuthenticationError) IsRetryable() bool

func (*AuthenticationError) Unwrap 

func (e *AuthenticationError) Unwrap() error

type BaseAdapter 

type BaseAdapter struct {
	APIKey         string
	BaseURL        string
	DefaultHeaders map[string]string
	Timeout        AdapterTimeout
	HTTPClient     *http.Client
}

BaseAdapter provides common HTTP functionality shared across all provider adapters. Provider-specific adapters embed BaseAdapter to reuse request building, header management, and rate limit parsing.

func NewBaseAdapter 

func NewBaseAdapter(apiKey, baseURL string, timeout AdapterTimeout) *BaseAdapter

NewBaseAdapter creates a BaseAdapter with the given API key, base URL, and timeout config. It initializes the HTTP client and default headers map.

func (*BaseAdapter) DoRequest 

func (b *BaseAdapter) DoRequest(ctx context.Context, method, path string, body any, headers map[string]string) (*http.Response, error)

DoRequest builds and executes an HTTP request against the provider's API. It JSON-encodes the body (if non-nil), sets authorization and content type headers, applies default headers, and then applies per-request header overrides. The request respects the provided context for timeout and cancellation.

func (*BaseAdapter) ParseRateLimitHeaders 

func (b *BaseAdapter) ParseRateLimitHeaders(headers http.Header) *RateLimitInfo

ParseRateLimitHeaders extracts rate limit information from provider response headers. It parses the standard x-ratelimit-* headers and the retry-after header. Returns nil if no rate limit headers are present.

type Catalog 

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog holds a collection of ModelInfo entries and supports lookup and filtering.

func DefaultCatalog 

func DefaultCatalog() *Catalog

DefaultCatalog returns a new Catalog pre-populated with built-in model definitions. Each call returns an independent copy so registrations on one catalog do not affect others.

func (*Catalog) GetLatestModel 

func (c *Catalog) GetLatestModel(provider string, capability string) *ModelInfo

GetLatestModel returns the first model in the catalog for the given provider, optionally filtered by a capability string ("reasoning", "vision", or "tools"). Returns nil if no model matches.

func (*Catalog) GetModelInfo 

func (c *Catalog) GetModelInfo(modelID string) *ModelInfo

GetModelInfo looks up a model by its canonical ID or any of its aliases. Returns nil if no matching model is found.

func (*Catalog) ListModels 

func (c *Catalog) ListModels(provider string) []ModelInfo

ListModels returns all models matching the given provider. If provider is empty, all models in the catalog are returned.

func (*Catalog) Register 

func (c *Catalog) Register(model ModelInfo)

Register adds a model to the catalog. If a model with the same ID already exists, it is replaced.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the primary entry point for making LLM API calls. It manages provider adapters, routes requests to the correct provider, and applies the middleware chain.

func FromEnv 

func FromEnv() (*Client, error)

FromEnv creates a Client by detecting API keys in the environment. It checks ANTHROPIC_API_KEY, OPENAI_API_KEY, and GEMINI_API_KEY. The first detected provider becomes the default. Provider-specific base URL env vars (ANTHROPIC_BASE_URL, OPENAI_BASE_URL, GEMINI_BASE_URL) are checked and used when present. Returns a ConfigurationError if no keys are found.

func GetDefaultClient 

func GetDefaultClient() *Client

GetDefaultClient returns the module-level default client. If no client has been set, it attempts lazy initialization via FromEnv. Returns nil if FromEnv fails (no API keys configured).

func NewClient 

func NewClient(opts ...ClientOption) *Client

NewClient creates a new Client with the given options applied.

func (*Client) Close 

func (c *Client) Close() error

Close shuts down all registered provider adapters. Errors from individual adapters are collected and returned as a combined error.

func (*Client) Complete 

func (c *Client) Complete(ctx context.Context, req Request) (*Response, error)

Complete sends a completion request through the middleware chain and then to the appropriate provider adapter. It routes based on req.Provider or the default provider.

func (*Client) RegisterProvider 

func (c *Client) RegisterProvider(name string, adapter ProviderAdapter)

RegisterProvider adds or replaces a provider adapter on the client. If no default provider is set, the newly registered provider becomes the default.

func (*Client) Stream 

func (c *Client) Stream(ctx context.Context, req Request) (<-chan StreamEvent, error)

Stream sends a streaming request to the appropriate provider adapter. It routes based on req.Provider or the default provider.

type ClientOption 

type ClientOption func(*Client)

ClientOption is a functional option for configuring a Client.

func WithDefaultProvider 

func WithDefaultProvider(name string) ClientOption

WithDefaultProvider sets the name of the provider used when a Request does not specify a Provider field.

func WithMiddleware 

func WithMiddleware(mw ...Middleware) ClientOption

WithMiddleware appends one or more middleware functions to the client's middleware chain. Middleware is executed in registration order for the request phase and reverse order for the response phase.

func WithProvider 

func WithProvider(name string, adapter ProviderAdapter) ClientOption

WithProvider registers a ProviderAdapter under the given name. If this is the first provider registered and no default has been set, it becomes the default provider.

type ConfigurationError 

type ConfigurationError struct {
	SDKError
}

ConfigurationError represents an SDK configuration problem (missing API key, etc.). Not retryable.

func (*ConfigurationError) As 

func (e *ConfigurationError) As(target any) bool

func (*ConfigurationError) Error 

func (e *ConfigurationError) Error() string

func (*ConfigurationError) IsRetryable 

func (e *ConfigurationError) IsRetryable() bool

func (*ConfigurationError) Unwrap 

func (e *ConfigurationError) Unwrap() error

type ContentFilterError 

type ContentFilterError struct {
	ProviderError
}

ContentFilterError represents a content moderation/filter rejection. Not retryable.

func (*ContentFilterError) As 

func (e *ContentFilterError) As(target any) bool

func (*ContentFilterError) Error 

func (e *ContentFilterError) Error() string

func (*ContentFilterError) IsRetryable 

func (e *ContentFilterError) IsRetryable() bool

func (*ContentFilterError) Unwrap 

func (e *ContentFilterError) Unwrap() error

type ContentKind 

type ContentKind string

ContentKind discriminates the type of content in a ContentPart. 

const (
	ContentText             ContentKind = "text"
	ContentImage            ContentKind = "image"
	ContentAudio            ContentKind = "audio"
	ContentDocument         ContentKind = "document"
	ContentToolCall         ContentKind = "tool_call"
	ContentToolResult       ContentKind = "tool_result"
	ContentThinking         ContentKind = "thinking"
	ContentRedactedThinking ContentKind = "redacted_thinking"
)

type ContentPart 

type ContentPart struct {
	Kind       ContentKind     `json:"kind"`
	Text       string          `json:"text,omitempty"`
	Image      *ImageData      `json:"image,omitempty"`
	Audio      *AudioData      `json:"audio,omitempty"`
	Document   *DocumentData   `json:"document,omitempty"`
	ToolCall   *ToolCallData   `json:"tool_call,omitempty"`
	ToolResult *ToolResultData `json:"tool_result,omitempty"`
	Thinking   *ThinkingData   `json:"thinking,omitempty"`
}

ContentPart is a single piece of content within a message. It uses a tagged-union pattern: the Kind field determines which data field is populated.

func ImageDataPart 

func ImageDataPart(data []byte, mediaType string) ContentPart

ImageDataPart creates an image ContentPart from raw bytes.

func ImageURLPart 

func ImageURLPart(url string) ContentPart

ImageURLPart creates an image ContentPart from a URL.

func RedactedThinkingPart 

func RedactedThinkingPart(text, signature string) ContentPart

RedactedThinkingPart creates a redacted thinking ContentPart.

func TextPart 

func TextPart(text string) ContentPart

TextPart creates a text ContentPart.

func ThinkingPart 

func ThinkingPart(text, signature string) ContentPart

ThinkingPart creates a thinking ContentPart.

func ToolCallPart 

func ToolCallPart(id, name string, args json.RawMessage) ContentPart

ToolCallPart creates a tool call ContentPart.

func ToolCallPartWithSignature added in v0.9.0 

func ToolCallPartWithSignature(id, name string, args json.RawMessage, signature string) ContentPart

ToolCallPartWithSignature creates a tool call ContentPart with an optional provider signature.

func ToolResultPart 

func ToolResultPart(toolCallID, content string, isError bool) ContentPart

ToolResultPart creates a tool result ContentPart.

type ContextLengthError 

type ContextLengthError struct {
	ProviderError
}

ContextLengthError represents a 413 payload/context too large response. Not retryable.

func (*ContextLengthError) As 

func (e *ContextLengthError) As(target any) bool

func (*ContextLengthError) Error 

func (e *ContextLengthError) Error() string

func (*ContextLengthError) IsRetryable 

func (e *ContextLengthError) IsRetryable() bool

func (*ContextLengthError) Unwrap 

func (e *ContextLengthError) Unwrap() error

type DocumentData 

type DocumentData struct {
	URL       string `json:"url,omitempty"`
	Data      []byte `json:"data,omitempty"`
	MediaType string `json:"media_type,omitempty"`
	FileName  string `json:"file_name,omitempty"`
}

DocumentData holds document content (PDF, etc.).

type FinishReason 

type FinishReason struct {
	Reason string `json:"reason"` // unified: stop, length, tool_calls, content_filter, error, other
	Raw    string `json:"raw,omitempty"`
}

FinishReason indicates why generation stopped, with both unified and raw values.

type GeminiAdapter 

type GeminiAdapter struct {
	// contains filtered or unexported fields
}

GeminiAdapter implements ProviderAdapter for Google's Gemini API. When hitting the default Google endpoint, it uses query-parameter authentication. When a custom base URL is set (e.g. Cloudflare AI Gateway), it uses header-based authentication (x-goog-api-key) for proxy compatibility.

func NewGeminiAdapter deprecated 

func NewGeminiAdapter(apiKey string, opts ...GeminiOption) *GeminiAdapter

NewGeminiAdapter creates a GeminiAdapter with the given API key and options. The BaseAdapter APIKey is set to empty so DoRequest will not add a Bearer token; authentication is handled via query parameter instead.

Deprecated: Use NewMuxAdapter with the appropriate mux/llm client instead. Retained as fallback for custom base URL configurations.

func (*GeminiAdapter) Close 

func (a *GeminiAdapter) Close() error

Close releases any resources held by the adapter.

func (*GeminiAdapter) Complete 

func (a *GeminiAdapter) Complete(ctx context.Context, req Request) (*Response, error)

Complete sends a non-streaming completion request to the Gemini API and returns a unified Response.

func (*GeminiAdapter) Name 

func (a *GeminiAdapter) Name() string

Name returns the provider name "gemini".

func (*GeminiAdapter) Stream 

func (a *GeminiAdapter) Stream(ctx context.Context, req Request) (<-chan StreamEvent, error)

Stream sends a streaming request to the Gemini API and returns a channel of StreamEvents.

type GeminiOption 

type GeminiOption func(*GeminiAdapter)

GeminiOption is a functional option for configuring a GeminiAdapter.

func WithGeminiBaseURL deprecated 

func WithGeminiBaseURL(url string) GeminiOption

WithGeminiBaseURL sets the base URL for the Gemini API. Default is "https://generativelanguage.googleapis.com".

Deprecated: Use NewMuxAdapter with the appropriate mux/llm client instead. Retained as fallback for custom base URL configurations.

func WithGeminiTimeout 

func WithGeminiTimeout(timeout AdapterTimeout) GeminiOption

WithGeminiTimeout sets the timeout configuration for the adapter.

type GenerateOptions 

type GenerateOptions struct {
	Model           string
	Prompt          string    // simple text prompt (mutually exclusive with Messages)
	Messages        []Message // full message history
	System          string    // system message
	Tools           []Tool    // tools with optional execute handlers
	ToolChoice      *ToolChoice
	MaxToolRounds   int // default 1
	StopWhen        StopCondition
	ResponseFormat  *ResponseFormat
	Temperature     *float64
	TopP            *float64
	MaxTokens       *int
	StopSequences   []string
	ReasoningEffort string
	Provider        string
	ProviderOptions map[string]any
	MaxRetries      int // default 2
	Timeout         *TimeoutConfig
	Client          *Client // override default client
}

GenerateOptions configures a Generate, StreamGenerate, or GenerateObject call.

type GenerateResult 

type GenerateResult struct {
	Text         string
	Reasoning    string
	ToolCalls    []ToolCallData
	ToolResults  []ToolResult
	FinishReason FinishReason
	Usage        Usage
	TotalUsage   Usage
	Steps        []StepResult
	Response     *Response
	Output       any // for GenerateObject parsed output
}

GenerateResult is the final output of a Generate call, aggregating all steps.

func Generate 

func Generate(ctx context.Context, opts GenerateOptions) (*GenerateResult, error)

Generate performs a completion request with optional tool execution loop. It builds a Request from the provided GenerateOptions, calls the client, and if the model requests tool calls with active tools, executes them concurrently and continues the loop.

func GenerateObject 

func GenerateObject(ctx context.Context, opts GenerateOptions, schema json.RawMessage) (*GenerateResult, error)

GenerateObject calls Generate with the ResponseFormat set to json_schema, then parses the response text as JSON. It validates the output by unmarshaling into a map and sets result.Output. Returns NoObjectGeneratedError on parse failure.

type ImageData 

type ImageData struct {
	URL       string `json:"url,omitempty"`
	Data      []byte `json:"data,omitempty"`
	MediaType string `json:"media_type,omitempty"`
	Detail    string `json:"detail,omitempty"` // "auto", "low", "high"
}

ImageData holds image content as URL, raw bytes, or both.

type Initializer 

type Initializer interface {
	Initialize() error
}

Initializer is an optional interface that adapters may implement to perform one-time setup (validating credentials, warming caches, etc.).

type InvalidRequestError 

type InvalidRequestError struct {
	ProviderError
}

InvalidRequestError represents a 400 or 422 response. Not retryable.

func (*InvalidRequestError) As 

func (e *InvalidRequestError) As(target any) bool

func (*InvalidRequestError) Error 

func (e *InvalidRequestError) Error() string

func (*InvalidRequestError) IsRetryable 

func (e *InvalidRequestError) IsRetryable() bool

func (*InvalidRequestError) Unwrap 

func (e *InvalidRequestError) Unwrap() error

type InvalidToolCallError 

type InvalidToolCallError struct {
	SDKError
}

InvalidToolCallError represents a malformed or invalid tool call from the model. Not retryable.

func (*InvalidToolCallError) As 

func (e *InvalidToolCallError) As(target any) bool

func (*InvalidToolCallError) Error 

func (e *InvalidToolCallError) Error() string

func (*InvalidToolCallError) IsRetryable 

func (e *InvalidToolCallError) IsRetryable() bool

func (*InvalidToolCallError) Unwrap 

func (e *InvalidToolCallError) Unwrap() error

type Message 

type Message struct {
	Role       Role          `json:"role"`
	Content    []ContentPart `json:"content"`
	Name       string        `json:"name,omitempty"`
	ToolCallID string        `json:"tool_call_id,omitempty"`
}

Message is the fundamental unit of conversation.

func AssistantMessage 

func AssistantMessage(text string) Message

AssistantMessage creates an assistant role message with text.

func DeveloperMessage 

func DeveloperMessage(text string) Message

DeveloperMessage creates a developer role message.

func ExtractSystemMessages 

func ExtractSystemMessages(messages []Message) (systemText string, remaining []Message)

ExtractSystemMessages separates system and developer role messages from the rest. It concatenates the text content of all system/developer messages (joined by newlines) and returns them along with the remaining non-system messages.

func MergeConsecutiveMessages 

func MergeConsecutiveMessages(messages []Message) []Message

MergeConsecutiveMessages combines consecutive messages with the same role by appending their content arrays. This is required for providers like Anthropic that enforce strict message role alternation.

func SystemMessage 

func SystemMessage(text string) Message

SystemMessage creates a system role message.

func ToolResultMessage 

func ToolResultMessage(toolCallID, content string, isError bool) Message

ToolResultMessage creates a tool role message with a result.

func UserMessage 

func UserMessage(text string) Message

UserMessage creates a user role message with text.

func UserMessageWithParts 

func UserMessageWithParts(parts ...ContentPart) Message

UserMessageWithParts creates a user role message with multiple content parts.

func (*Message) ReasoningContent 

func (m *Message) ReasoningContent() string

ReasoningContent returns concatenated text from all thinking content parts.

func (*Message) TextContent 

func (m *Message) TextContent() string

TextContent returns concatenated text from all text content parts.

func (*Message) ToolCalls 

func (m *Message) ToolCalls() []ToolCallData

ToolCalls extracts all tool call data from the message.

type Middleware 

type Middleware func(ctx context.Context, req Request, next NextFunc) (*Response, error)

Middleware is a function that wraps an LLM call, enabling request/response transformation, logging, caching, and other cross-cutting concerns. Middleware executes in registration order for requests and reverse order for responses (onion/chain-of-responsibility pattern).

type ModelInfo 

type ModelInfo struct {
	ID                   string // e.g., "claude-opus-4-6"
	Provider             string // e.g., "anthropic"
	DisplayName          string // e.g., "Claude Opus 4.6"
	ContextWindow        int    // max total tokens
	MaxOutput            int    // max output tokens, 0 if unknown
	SupportsTools        bool
	SupportsVision       bool
	SupportsReasoning    bool
	InputCostPerMillion  float64  // USD per 1M input tokens, 0 if unknown
	OutputCostPerMillion float64  // USD per 1M output tokens, 0 if unknown
	Aliases              []string // shorthand names
}

ModelInfo describes a single LLM model's capabilities and metadata.

type MuxAdapter 

type MuxAdapter struct {
	// contains filtered or unexported fields
}

MuxAdapter wraps a mux/llm.Client as a mammoth ProviderAdapter. This allows mammoth's agent loop to use mux as an LLM provider without any changes to the agent code.

func NewMuxAdapter 

func NewMuxAdapter(name string, client muxllm.Client) *MuxAdapter

NewMuxAdapter creates a MuxAdapter with the given provider name and mux client.

func (*MuxAdapter) Close 

func (a *MuxAdapter) Close() error

Close releases any resources held by the adapter. The underlying mux client does not expose a Close method, so this is a no-op.

func (*MuxAdapter) Complete 

func (a *MuxAdapter) Complete(ctx context.Context, req Request) (*Response, error)

Complete sends a completion request through the mux client. Rate limit errors (429) are automatically retried with exponential backoff.

func (*MuxAdapter) Name 

func (a *MuxAdapter) Name() string

Name returns the provider name for this adapter.

func (*MuxAdapter) Stream 

func (a *MuxAdapter) Stream(ctx context.Context, req Request) (<-chan StreamEvent, error)

Stream sends a streaming request through the mux client, converting each mux stream event into a mammoth StreamEvent. Rate limit errors (429) on the initial connection are automatically retried with exponential backoff.

type NetworkError 

type NetworkError struct {
	SDKError
}

NetworkError represents a network-level failure (DNS, connection refused, etc.). Retryable.

func (*NetworkError) As 

func (e *NetworkError) As(target any) bool

func (*NetworkError) Error 

func (e *NetworkError) Error() string

func (*NetworkError) IsRetryable 

func (e *NetworkError) IsRetryable() bool

func (*NetworkError) Unwrap 

func (e *NetworkError) Unwrap() error

type NextFunc 

type NextFunc func(ctx context.Context, req Request) (*Response, error)

NextFunc is the function signature passed to middleware to continue the chain.

type NoObjectGeneratedError 

type NoObjectGeneratedError struct {
	SDKError
}

NoObjectGeneratedError represents a failure to generate the expected structured output. Not retryable.

func (*NoObjectGeneratedError) As 

func (e *NoObjectGeneratedError) As(target any) bool

func (*NoObjectGeneratedError) Error 

func (e *NoObjectGeneratedError) Error() string

func (*NoObjectGeneratedError) IsRetryable 

func (e *NoObjectGeneratedError) IsRetryable() bool

func (*NoObjectGeneratedError) Unwrap 

func (e *NoObjectGeneratedError) Unwrap() error

type NotFoundError 

type NotFoundError struct {
	ProviderError
}

NotFoundError represents a 404 Not Found response. Not retryable.

func (*NotFoundError) As 

func (e *NotFoundError) As(target any) bool

func (*NotFoundError) Error 

func (e *NotFoundError) Error() string

func (*NotFoundError) IsRetryable 

func (e *NotFoundError) IsRetryable() bool

func (*NotFoundError) Unwrap 

func (e *NotFoundError) Unwrap() error

type OpenAIAdapter 

type OpenAIAdapter struct {
	*BaseAdapter
	// contains filtered or unexported fields
}

OpenAIAdapter implements ProviderAdapter for the OpenAI Responses API.

func NewOpenAIAdapter deprecated 

func NewOpenAIAdapter(apiKey string, opts ...OpenAIOption) *OpenAIAdapter

NewOpenAIAdapter creates a new OpenAIAdapter with the given API key and options.

Deprecated: Use NewMuxAdapter with the appropriate mux/llm client instead. Retained as fallback for custom base URL configurations.

func (*OpenAIAdapter) Close 

func (a *OpenAIAdapter) Close() error

Close releases resources held by the adapter.

func (*OpenAIAdapter) Complete 

func (a *OpenAIAdapter) Complete(ctx context.Context, req Request) (*Response, error)

Complete sends a synchronous completion request to the OpenAI Responses API.

func (*OpenAIAdapter) Name 

func (a *OpenAIAdapter) Name() string

Name returns the provider name for this adapter.

func (*OpenAIAdapter) Stream 

func (a *OpenAIAdapter) Stream(ctx context.Context, req Request) (<-chan StreamEvent, error)

Stream sends a streaming request to the OpenAI Responses API and returns a channel of events.

type OpenAICompatClient added in v0.8.0 

type OpenAICompatClient struct {
	// contains filtered or unexported fields
}

OpenAICompatClient implements muxllm.Client using the OpenAI Chat Completions API. Unlike mux's built-in OpenAIClient, this supports custom base URLs for OpenAI-compatible providers (Cerebras, OpenRouter, Cloudflare AI Gateway, etc.).

func NewOpenAICompatClient added in v0.8.0 

func NewOpenAICompatClient(apiKey, model, baseURL string) *OpenAICompatClient

NewOpenAICompatClient creates a Chat Completions client with a custom base URL. This uses /v1/chat/completions (not /v1/responses), which is the standard endpoint supported by all OpenAI-compatible providers.

func (*OpenAICompatClient) CreateMessage added in v0.8.0 

func (c *OpenAICompatClient) CreateMessage(ctx context.Context, req *muxllm.Request) (*muxllm.Response, error)

CreateMessage sends a message and returns the complete response.

func (*OpenAICompatClient) CreateMessageStream added in v0.8.0 

func (c *OpenAICompatClient) CreateMessageStream(ctx context.Context, req *muxllm.Request) (<-chan muxllm.StreamEvent, error)

CreateMessageStream sends a message and returns a channel of streaming events.

type OpenAIOption 

type OpenAIOption func(*OpenAIAdapter)

OpenAIOption is a functional option for configuring an OpenAIAdapter.

func WithOpenAIBaseURL deprecated 

func WithOpenAIBaseURL(url string) OpenAIOption

WithOpenAIBaseURL sets the base URL for OpenAI API requests.

Deprecated: Use NewMuxAdapter with the appropriate mux/llm client instead. Retained as fallback for custom base URL configurations.

func WithOpenAIOrganization 

func WithOpenAIOrganization(org string) OpenAIOption

WithOpenAIOrganization sets the OpenAI-Organization header for API requests.

func WithOpenAIProject 

func WithOpenAIProject(project string) OpenAIOption

WithOpenAIProject sets the OpenAI-Project header for API requests.

func WithOpenAITimeout 

func WithOpenAITimeout(timeout AdapterTimeout) OpenAIOption

WithOpenAITimeout sets the timeout configuration for OpenAI API requests.

type ProviderAdapter 

type ProviderAdapter interface {
	Name() string
	Complete(ctx context.Context, req Request) (*Response, error)
	Stream(ctx context.Context, req Request) (<-chan StreamEvent, error)
	Close() error
}

ProviderAdapter is the interface that all LLM provider adapters must implement. It provides a uniform way to send completion and streaming requests to different LLM providers (OpenAI, Anthropic, Gemini, etc.).

type ProviderError 

type ProviderError struct {
	SDKError
	Provider   string
	StatusCode int
	ErrorCode  string
	Retryable  bool
	RetryAfter *float64
	Raw        json.RawMessage
}

ProviderError represents an error returned by an LLM provider's API. It carries provider-specific metadata including status code, error code, and raw response.

func (*ProviderError) As 

func (e *ProviderError) As(target any) bool

As enables errors.As to match ProviderError and SDKError from a ProviderError.

func (*ProviderError) Error 

func (e *ProviderError) Error() string

func (*ProviderError) IsRetryable 

func (e *ProviderError) IsRetryable() bool

IsRetryable returns the Retryable flag set on the provider error.

func (*ProviderError) Unwrap 

func (e *ProviderError) Unwrap() error

type QuotaExceededError 

type QuotaExceededError struct {
	ProviderError
}

QuotaExceededError represents a quota exhaustion error. Not retryable.

func (*QuotaExceededError) As 

func (e *QuotaExceededError) As(target any) bool

func (*QuotaExceededError) Error 

func (e *QuotaExceededError) Error() string

func (*QuotaExceededError) IsRetryable 

func (e *QuotaExceededError) IsRetryable() bool

func (*QuotaExceededError) Unwrap 

func (e *QuotaExceededError) Unwrap() error

type RateLimitError 

type RateLimitError struct {
	ProviderError
}

RateLimitError represents a 429 Too Many Requests response. Retryable.

func (*RateLimitError) As 

func (e *RateLimitError) As(target any) bool

func (*RateLimitError) Error 

func (e *RateLimitError) Error() string

func (*RateLimitError) IsRetryable 

func (e *RateLimitError) IsRetryable() bool

func (*RateLimitError) Unwrap 

func (e *RateLimitError) Unwrap() error

type RateLimitInfo 

type RateLimitInfo struct {
	RequestsRemaining *int       `json:"requests_remaining,omitempty"`
	RequestsLimit     *int       `json:"requests_limit,omitempty"`
	TokensRemaining   *int       `json:"tokens_remaining,omitempty"`
	TokensLimit       *int       `json:"tokens_limit,omitempty"`
	ResetAt           *time.Time `json:"reset_at,omitempty"`
}

RateLimitInfo contains rate limit metadata from provider response headers.

type Request 

type Request struct {
	Model           string            `json:"model"`
	Messages        []Message         `json:"messages"`
	Provider        string            `json:"provider,omitempty"`
	Tools           []ToolDefinition  `json:"tools,omitempty"`
	ToolChoice      *ToolChoice       `json:"tool_choice,omitempty"`
	ResponseFormat  *ResponseFormat   `json:"response_format,omitempty"`
	Temperature     *float64          `json:"temperature,omitempty"`
	TopP            *float64          `json:"top_p,omitempty"`
	MaxTokens       *int              `json:"max_tokens,omitempty"`
	StopSequences   []string          `json:"stop_sequences,omitempty"`
	ReasoningEffort string            `json:"reasoning_effort,omitempty"` // "none", "low", "medium", "high"
	Metadata        map[string]string `json:"metadata,omitempty"`
	ProviderOptions map[string]any    `json:"provider_options,omitempty"`
}

Request is the unified input type for both complete() and stream().

type RequestTimeoutError 

type RequestTimeoutError struct {
	SDKError
}

RequestTimeoutError represents a request timeout (408 or client-side). Retryable.

func (*RequestTimeoutError) As 

func (e *RequestTimeoutError) As(target any) bool

func (*RequestTimeoutError) Error 

func (e *RequestTimeoutError) Error() string

func (*RequestTimeoutError) IsRetryable 

func (e *RequestTimeoutError) IsRetryable() bool

func (*RequestTimeoutError) Unwrap 

func (e *RequestTimeoutError) Unwrap() error

type Response 

type Response struct {
	ID           string          `json:"id"`
	Model        string          `json:"model"`
	Provider     string          `json:"provider"`
	Message      Message         `json:"message"`
	FinishReason FinishReason    `json:"finish_reason"`
	Usage        Usage           `json:"usage"`
	Raw          json.RawMessage `json:"raw,omitempty"`
	Warnings     []Warning       `json:"warnings,omitempty"`
	RateLimit    *RateLimitInfo  `json:"rate_limit,omitempty"`
}

Response is the unified output from a complete() call.

func (*Response) Reasoning 

func (r *Response) Reasoning() string

Reasoning returns concatenated reasoning text from the response.

func (*Response) TextContent 

func (r *Response) TextContent() string

TextContent returns the concatenated text from the response message.

func (*Response) ToolCalls 

func (r *Response) ToolCalls() []ToolCallData

ToolCalls returns tool calls from the response message.

type ResponseFormat 

type ResponseFormat struct {
	Type       string          `json:"type"` // "text", "json", or "json_schema"
	JSONSchema json.RawMessage `json:"json_schema,omitempty"`
	Strict     bool            `json:"strict,omitempty"`
}

ResponseFormat specifies the desired output format.

type RetryPolicy 

type RetryPolicy struct {
	// MaxRetries is the maximum number of retry attempts (not counting the initial call).
	MaxRetries int

	// BaseDelay is the initial delay before the first retry.
	BaseDelay time.Duration

	// MaxDelay is the upper bound on the delay between retries.
	MaxDelay time.Duration

	// BackoffMultiplier controls exponential growth of the delay between retries.
	BackoffMultiplier float64

	// Jitter adds randomness to the delay to avoid thundering herd problems.
	Jitter bool

	// OnRetry is an optional callback invoked before each retry attempt.
	// It receives the error that triggered the retry, the attempt number (0-indexed),
	// and the delay that will be applied before the next attempt.
	OnRetry func(err error, attempt int, delay time.Duration)
}

RetryPolicy configures how retry behavior works for LLM API calls.

func DefaultRetryPolicy 

func DefaultRetryPolicy() RetryPolicy

DefaultRetryPolicy returns a RetryPolicy with sensible defaults: 2 retries, 1s base delay, 60s max delay, 2x backoff, jitter enabled.

func (RetryPolicy) CalculateDelay 

func (p RetryPolicy) CalculateDelay(attempt int) time.Duration

CalculateDelay computes the delay for a given retry attempt using exponential backoff. When Jitter is enabled, the delay is randomized between 0 and the calculated backoff value. The result is always capped at MaxDelay.

func (RetryPolicy) ShouldRetry 

func (p RetryPolicy) ShouldRetry(err error, attempt int) bool

ShouldRetry determines whether the operation should be retried based on the error and the current attempt number. It returns false for nil errors, non-retryable errors, and when the attempt count has reached MaxRetries.

type Role 

type Role string

Role represents who produced a message in a conversation. 

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
	RoleDeveloper Role = "developer"
)

type SDKError 

type SDKError struct {
	Message string
	Cause   error
}

SDKError is the base error type for all errors in the LLM SDK. All other error types embed SDKError either directly or transitively.

func (*SDKError) Error 

func (e *SDKError) Error() string

func (*SDKError) IsRetryable 

func (e *SDKError) IsRetryable() bool

IsRetryable returns false for the base SDKError. Subtypes override this.

func (*SDKError) Unwrap 

func (e *SDKError) Unwrap() error

type ServerError 

type ServerError struct {
	ProviderError
}

ServerError represents a 5xx server error response. Retryable.

func (*ServerError) As 

func (e *ServerError) As(target any) bool

func (*ServerError) Error 

func (e *ServerError) Error() string

func (*ServerError) IsRetryable 

func (e *ServerError) IsRetryable() bool

func (*ServerError) Unwrap 

func (e *ServerError) Unwrap() error

type StepResult 

type StepResult struct {
	Text         string
	Reasoning    string
	ToolCalls    []ToolCallData
	ToolResults  []ToolResult
	FinishReason FinishReason
	Usage        Usage
	Response     *Response
	Warnings     []Warning
}

StepResult captures the output of a single iteration in the generate loop.

type StopCondition 

type StopCondition func(steps []StepResult) bool

StopCondition is a predicate that decides whether the tool loop should stop based on the accumulated step results so far.

type StreamAccumulator 

type StreamAccumulator struct {
	// contains filtered or unexported fields
}

StreamAccumulator collects streaming events and builds a complete Response.

func NewStreamAccumulator 

func NewStreamAccumulator() *StreamAccumulator

NewStreamAccumulator creates a new StreamAccumulator ready to process events.

func (*StreamAccumulator) Process 

func (a *StreamAccumulator) Process(event StreamEvent)

Process ingests a single StreamEvent, updating the accumulator's internal state.

func (*StreamAccumulator) Response 

func (a *StreamAccumulator) Response() *Response

Response constructs a complete Response from the accumulated stream events.

type StreamError 

type StreamError struct {
	SDKError
}

StreamError represents an error during response streaming. Retryable.

func (*StreamError) As 

func (e *StreamError) As(target any) bool

func (*StreamError) Error 

func (e *StreamError) Error() string

func (*StreamError) IsRetryable 

func (e *StreamError) IsRetryable() bool

func (*StreamError) Unwrap 

func (e *StreamError) Unwrap() error

type StreamEvent 

type StreamEvent struct {
	Type           StreamEventType  `json:"type"`
	Delta          string           `json:"delta,omitempty"`
	TextID         string           `json:"text_id,omitempty"`
	ReasoningDelta string           `json:"reasoning_delta,omitempty"`
	ToolCall       *ToolCall        `json:"tool_call,omitempty"`
	FinishReason   *FinishReason    `json:"finish_reason,omitempty"`
	Usage          *Usage           `json:"usage,omitempty"`
	Response       *Response        `json:"response,omitempty"`
	Error          error            `json:"-"`
	Raw            *json.RawMessage `json:"raw,omitempty"`
}

StreamEvent represents a single event in a streaming response.

type StreamEventType 

type StreamEventType string

StreamEventType discriminates the type of streaming event. 

const (
	StreamStart       StreamEventType = "stream_start"
	StreamTextStart   StreamEventType = "text_start"
	StreamTextDelta   StreamEventType = "text_delta"
	StreamTextEnd     StreamEventType = "text_end"
	StreamReasonStart StreamEventType = "reasoning_start"
	StreamReasonDelta StreamEventType = "reasoning_delta"
	StreamReasonEnd   StreamEventType = "reasoning_end"
	StreamToolStart   StreamEventType = "tool_call_start"
	StreamToolDelta   StreamEventType = "tool_call_delta"
	StreamToolEnd     StreamEventType = "tool_call_end"
	StreamFinish      StreamEventType = "finish"
	StreamErrorEvt    StreamEventType = "error"
	StreamProviderEvt StreamEventType = "provider_event"
)

type StreamResult 

type StreamResult struct {
	Events <-chan StreamEvent
	// contains filtered or unexported fields
}

StreamResult wraps a channel of streaming events and provides access to the final response after the stream completes.

func StreamGenerate 

func StreamGenerate(ctx context.Context, opts GenerateOptions) (*StreamResult, error)

StreamGenerate performs a streaming completion request. It builds the request from GenerateOptions and wraps the client's Stream call. The tool loop is not performed for streaming (can be added later).

func (*StreamResult) Response 

func (sr *StreamResult) Response() *Response

Response returns the final accumulated response after the stream completes.

type ThinkingData 

type ThinkingData struct {
	Text      string `json:"text"`
	Signature string `json:"signature,omitempty"`
	Redacted  bool   `json:"redacted"`
}

ThinkingData holds model reasoning/thinking content.

type TimeoutConfig 

type TimeoutConfig struct {
	Total   time.Duration `json:"total,omitempty"`
	PerStep time.Duration `json:"per_step,omitempty"`
}

TimeoutConfig specifies timeout durations for generation.

type Tool 

type Tool struct {
	ToolDefinition
	Execute func(args json.RawMessage) (string, error) `json:"-"`
}

Tool is a ToolDefinition with an optional execute handler.

func (*Tool) IsActive 

func (t *Tool) IsActive() bool

IsActive returns true if the tool has an execute handler.

type ToolCall 

type ToolCall struct {
	ID           string          `json:"id"`
	Name         string          `json:"name"`
	Arguments    json.RawMessage `json:"arguments"`
	Signature    string          `json:"signature,omitempty"`
	RawArguments string          `json:"raw_arguments,omitempty"`
}

ToolCall represents a tool invocation extracted from a response.

type ToolCallData 

type ToolCallData struct {
	ID        string          `json:"id"`
	Name      string          `json:"name"`
	Arguments json.RawMessage `json:"arguments"`
	Signature string          `json:"signature,omitempty"`
	Type      string          `json:"type,omitempty"` // "function" (default) or "custom"
}

ToolCallData represents a model-initiated tool invocation.

func (*ToolCallData) ArgumentsMap 

func (tc *ToolCallData) ArgumentsMap() (map[string]any, error)

ArgumentsMap parses the raw JSON arguments into a map.

type ToolChoice 

type ToolChoice struct {
	Mode     string `json:"mode"`                // "auto", "none", "required", "named"
	ToolName string `json:"tool_name,omitempty"` // required when mode is "named"
}

ToolChoice controls whether and how the model uses tools.

type ToolChoiceChecker 

type ToolChoiceChecker interface {
	SupportsToolChoice(mode string) bool
}

ToolChoiceChecker is an optional interface that adapters may implement to indicate which tool choice modes they support.

type ToolDefinition 

type ToolDefinition struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Parameters  json.RawMessage `json:"parameters"` // JSON Schema with root "type": "object"
}

ToolDefinition describes a tool available to the model.

type ToolResult 

type ToolResult struct {
	ToolCallID string `json:"tool_call_id"`
	Content    string `json:"content"`
	IsError    bool   `json:"is_error"`
}

ToolResult represents the output of a tool execution.

type ToolResultData 

type ToolResultData struct {
	ToolCallID     string `json:"tool_call_id"`
	Content        string `json:"content"`
	IsError        bool   `json:"is_error"`
	ImageData      []byte `json:"image_data,omitempty"`
	ImageMediaType string `json:"image_media_type,omitempty"`
}

ToolResultData represents the result of executing a tool call.

type Usage 

type Usage struct {
	InputTokens      int              `json:"input_tokens"`
	OutputTokens     int              `json:"output_tokens"`
	TotalTokens      int              `json:"total_tokens"`
	ReasoningTokens  *int             `json:"reasoning_tokens,omitempty"`
	CacheReadTokens  *int             `json:"cache_read_tokens,omitempty"`
	CacheWriteTokens *int             `json:"cache_write_tokens,omitempty"`
	Raw              *json.RawMessage `json:"raw,omitempty"`
}

Usage tracks token consumption for a single LLM call.

func (Usage) Add 

func (u Usage) Add(other Usage) Usage

Add combines two Usage values, summing all fields.

type Warning 

type Warning struct {
	Message string `json:"message"`
	Code    string `json:"code,omitempty"`
}

Warning represents a non-fatal issue in a response.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.