Documentation
¶
Index ¶
- Variables
- func IsEncryptedReplayRejection(err error) bool
- func UnsupportedStream(provider string) iter.Seq2[Chunk, error]
- type Capabilities
- type Chunk
- type ContentPart
- type ErrorKind
- type FinishReason
- type ImagePart
- type Message
- type ModelInfo
- type Provider
- type ProviderError
- type ReasoningPart
- type Registry
- type Request
- type Response
- type Role
- type TextPart
- type Tool
- type ToolCallPart
- type ToolResultPart
- type Usage
Constants ¶
This section is empty.
Variables ¶
var ( ErrRateLimit = &ProviderError{Kind: KindRateLimit} ErrAuth = &ProviderError{Kind: KindAuth} ErrContextLength = &ProviderError{Kind: KindContextLength} ErrContentFilter = &ProviderError{Kind: KindContentFilter} ErrCanceled = &ProviderError{Kind: KindCanceled} ErrUnsupported = &ProviderError{Kind: KindUnsupported} )
Sentinel values for errors.Is checks. They carry only a Kind.
Functions ¶
func IsEncryptedReplayRejection ¶ added in v0.2.0
IsEncryptedReplayRejection reports the deterministic 400 returned when a provider cannot re-verify an encrypted reasoning item from prior history. The provider's error envelopes vary, so matching intentionally uses the complete error text rather than a provider-specific error code.
func UnsupportedStream ¶
UnsupportedStream is the Stream implementation for providers that do not stream: it yields a single KindUnsupported error and nothing else. Streaming is part of the core Provider interface, but a backend (or a test fake) that only implements Generate can satisfy the method with this instead of hand-writing the iterator.
Types ¶
type Capabilities ¶
Capabilities advertises optional features a provider supports, so callers can branch without triggering a failed request.
type Chunk ¶
type Chunk struct {
// Text is an incremental text delta (empty on non-text chunks).
Text string
// ToolCall is a fully-assembled tool call (nil unless this chunk is a call).
ToolCall *ToolCallPart
// Reasoning is a fully-assembled reasoning trace for the turn (nil unless
// this chunk carries one). The caller must keep it on the assistant turn it
// re-sends, exactly like a ReasoningPart from Generate.
Reasoning *ReasoningPart
// FinishReason is why generation stopped. Set only on the Done chunk.
FinishReason FinishReason
// Usage is token accounting for the whole generation. Set only on the Done chunk.
Usage Usage
// Done is true on the single terminal chunk that carries FinishReason+Usage.
Done bool
}
Chunk is one increment of a streaming generation (decision 5: the core streaming primitive is iter.Seq2[Chunk, error]). A stream yields a sequence of content chunks followed by exactly one terminal chunk:
- Text holds an incremental text delta. It is the common chunk; concatenating every Text across a stream reconstructs the full message text.
- ToolCall is a tool call surfaced ONCE FULLY ASSEMBLED (id + name + complete JSON args), not as raw argument fragments — a streamed tool call arrives as partial JSON across many wire chunks, and almost every caller wants the whole call, so the adapter accumulates it and emits it complete. nil on a text chunk.
- Reasoning is a provider reasoning trace surfaced ONCE FULLY ASSEMBLED, like ToolCall — a stateful trace (Anthropic's signed thinking blocks) is only replayable complete, and replay is the sole reason it exists (see ReasoningPart). A provider that returns none never yields this shape.
- Done marks the single TERMINAL chunk. It carries no content; instead it reports FinishReason and the final Usage for the whole generation. It is the streaming analogue of a Response's tail fields, so a caller that only needs totals can ignore every prior chunk and read this one.
A chunk is one of those four shapes — a text delta, a completed tool call, an assembled reasoning trace, or the terminal Done — never a mix.
type ContentPart ¶
type ContentPart interface {
// contains filtered or unexported methods
}
ContentPart is a single piece of a message's content. It is a closed sum type: the only implementations are those defined in this package (TextPart, ImagePart, ToolCallPart, ToolResultPart).
The content model is part-based from day one (decision 3) because both underlying wire formats are natively part-based and because vision and tool use require it. Text-first helpers keep the common case a one-liner.
type ErrorKind ¶
type ErrorKind string
ErrorKind is a normalized error classification (decision 9). Adapters map provider/SDK errors into one of these so callers can react uniformly across providers (e.g. back off a rate-limited model in a comparison run).
type FinishReason ¶
type FinishReason string
FinishReason is the normalized reason generation stopped.
const ( FinishStop FinishReason = "stop" // natural end FinishLength FinishReason = "length" // hit a token cap FinishToolUse FinishReason = "tool_use" // model wants to call a tool FinishContentFilter FinishReason = "content_filter" // blocked by a safety filter FinishOther FinishReason = "other" // unrecognized / provider-specific )
type ImagePart ¶
type ImagePart struct {
// URL is a remote image reference. Mutually exclusive with Data.
URL string
// Data is raw image bytes. Mutually exclusive with URL.
Data []byte
// MIME is the media type for Data, e.g. "image/png".
MIME string
}
ImagePart is image content for vision-capable models.
type Message ¶
type Message struct {
Role Role
Parts []ContentPart
}
Message is one turn in a conversation: a role plus structured content parts.
func ToolResultMsg ¶
ToolResultMsg builds a tool-result message answering the tool call with id. isErr marks a tool failure — still an observation the model acts on, not a transport error.
func UserParts ¶
func UserParts(parts ...ContentPart) Message
UserParts builds a user message from explicit content parts (e.g. text + image), for when the string helper is not enough.
func (Message) MarshalJSON ¶
MarshalJSON gives Message a durable JSON shape despite Parts being an interface slice. Transcripts depend on being able to read conversations back into Config.History for session resume.
func (*Message) UnmarshalJSON ¶
UnmarshalJSON restores the concrete ContentPart implementations emitted by MarshalJSON.
type ModelInfo ¶ added in v0.2.0
type ModelInfo struct {
ContextWindow int `json:"context_window"`
MaxOutputTokens int `json:"max_output_tokens"`
}
ModelInfo describes model limits that are useful to callers before a request is sent. A zero field means that limit is unknown; callers must treat an unknown limit as having no proactive limit.
func Lookup ¶ added in v0.2.0
Lookup returns cataloged metadata for model. Matching is case-sensitive and accepts either a bare model slug or the same slug prefixed by its provider (for example, "gpt-5.6-luna" and "openai/gpt-5.6-luna"). Unknown models return the zero ModelInfo; the catalog deliberately does not guess.
type Provider ¶
type Provider interface {
// Name is the registered identity of this provider (e.g. "xai").
Name() string
// Capabilities advertises what this provider/model supports.
Capabilities() Capabilities
// Generate runs a single, non-streaming completion.
Generate(ctx context.Context, req Request) (*Response, error)
// Stream runs a completion incrementally, yielding Chunks as they arrive
// (decision 5). The iterator yields content chunks (text deltas, completed
// tool calls) followed by one terminal Done chunk carrying FinishReason and
// Usage; a non-nil error ends the stream. A backend that cannot stream returns
// UnsupportedStream and reports Streaming=false in Capabilities.
Stream(ctx context.Context, req Request) iter.Seq2[Chunk, error]
}
Provider is the uniform interface every LLM backend implements. It is kept small (decision 2) so swapping providers is seamless for the common path.
Capabilities that only some providers have (embeddings, ...) are exposed as separate optional interfaces promoted via type-assertion, not added here.
type ProviderError ¶
type ProviderError struct {
Provider string // which provider produced it (e.g. "xai")
Kind ErrorKind // normalized classification
StatusCode int // HTTP status, if known (0 otherwise)
Retryable bool // whether a retry might succeed
Err error // the wrapped original error
}
ProviderError wraps an underlying error with a normalized classification. The original error is preserved (Unwrap), so errors.Is/As reach the SDK's own error type and the Raw details remain accessible.
func (*ProviderError) Error ¶
func (e *ProviderError) Error() string
func (*ProviderError) Is ¶
func (e *ProviderError) Is(target error) bool
Is matches on Kind, so callers can write errors.Is(err, llm.ErrRateLimit) regardless of which provider or status code produced it.
func (*ProviderError) Unwrap ¶
func (e *ProviderError) Unwrap() error
Unwrap exposes the wrapped error to errors.Is / errors.As.
type ReasoningPart ¶
type ReasoningPart struct{ Raw json.RawMessage }
ReasoningPart carries a provider's OPAQUE reasoning trace for an assistant turn (OpenRouter's `reasoning_details`), so the loop can replay it verbatim when it re-sends that turn. It exists for ENCRYPTED reasoning that is STATEFUL: Gemini returns its chain of thought as an encrypted "thought signature" (reasoning.encrypted / google-gemini-v1) and its API requires that signature be sent back on the following turn — without it the model is amnesiac across tool calls and spirals (re-issuing the same action until a no-progress detector kills it). Plaintext reasoning (e.g. deepseek's reasoning.text) is carried too, but is re-derivable so dropping it is harmless; the encrypted kind is not. The harness never INTERPRETS Raw — it only round-trips it, so the field stays a provider black box.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry holds named providers so they can be configured once and used interchangeably or all at once (decision 7). It is safe for concurrent use.
func (*Registry) Add ¶
Add registers a provider under a name, overwriting any existing entry. It returns the registry so calls can be chained.
type Request ¶
type Request struct {
// Model overrides the provider's default model for this request.
// Empty means "use the provider's configured default".
Model string
// System is a convenience top-level system prompt. Adapters place it
// where the provider expects it (a system message for OpenAI-compatible
// providers, a top-level field for Claude).
System string
// Messages is the conversation so far.
Messages []Message
// MaxTokens caps generated tokens. 0 means unset (provider default).
MaxTokens int
// Temperature, TopP, Seed are optional sampling controls.
Temperature *float64
TopP *float64
Seed *int64
// Stop is an optional set of stop sequences.
Stop []string
// Tools are the functions the model may call this turn. When non-empty, a
// tool-capable adapter advertises them and may return ToolCallParts with
// FinishToolUse; the caller executes them and replies with ToolResultParts.
// Adapters that don't support tools ignore this (preserving swap-ability).
Tools []Tool
// ReasoningEffort asks a reasoning model to think harder or less
// ("minimal" | "low" | "medium" | "high" | "xhigh" — passed through
// untyped, so a new tier needs no code change). Empty means unset: the
// provider's default, which is what every measurement before 2026-07-03
// ran at. Adapters map it to their wire param (openaicompat →
// `reasoning_effort`, the OpenAI chat-completions param OpenRouter
// normalizes into its unified reasoning system); adapters without a
// reasoning surface ignore it.
ReasoningEffort string
// ProviderOptions is a passthrough for provider-specific knobs (e.g.
// reasoning effort, thinking budget). Adapters merge recognized keys into
// the underlying request and ignore the rest. Prefer the typed helpers
// each provider package exposes over setting raw keys here.
ProviderOptions map[string]any
}
Request is a provider-agnostic generation request.
Broadly-shared sampling params are typed fields (decision 6). Optional numeric knobs are pointers so "unset" is distinguishable from a deliberate zero. Provider-specific knobs ride in ProviderOptions and are interpreted by whichever adapter understands them; adapters ignore options they don't.
type Response ¶
type Response struct {
// Content is the generated content parts (text for now).
Content []ContentPart
// FinishReason is why generation stopped.
FinishReason FinishReason
// Usage is token accounting.
Usage Usage
// Model is the model that actually served the request, as reported by the
// provider (may differ from the requested model, e.g. via OpenRouter).
Model string
// Raw is the underlying SDK response. Type-assert to the provider's
// concrete type to reach un-normalized fields (logprobs, reasoning, etc.).
Raw any
}
Response is a provider-agnostic generation result. Raw holds the underlying SDK response for deep inspection of anything not normalized here (decision 8).
type Tool ¶
type Tool struct {
// Name is the function name the model calls and the loop dispatches on.
Name string
// Description tells the model when and how to use the tool.
Description string
// Schema is a JSON Schema object describing the arguments. An empty schema
// declares a no-argument tool.
Schema json.RawMessage
}
Tool is a provider-agnostic description of a function the model may call. It carries NO handler — execution lives in the caller's loop (decision 4), which matches a returned ToolCallPart to its Tool by Name and runs it. Adapters translate Schema into their provider's tool format (OpenAI's `function.parameters`, Claude's `input_schema`), so the same Tool works across providers.
type ToolCallPart ¶
type ToolCallPart struct {
ID string
Name string
Args json.RawMessage
}
ToolCallPart is the model asking to invoke a tool: a provider-assigned call ID, the tool Name, and the raw JSON Args object. It appears in an assistant message when FinishReason is FinishToolUse. The caller runs the named tool and answers with a ToolResultPart carrying the same ID.
type ToolResultPart ¶
ToolResultPart is the caller's answer to a ToolCallPart: the observation text, tagged with the ToolCallID it answers and whether the tool failed. IsError is a real outcome the model should see and react to (a failed tool is information, not a transport error) — not a Go error. Carried in a RoleTool message.
type Usage ¶
type Usage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
CachedTokens int `json:"cached_tokens"`
ReasoningTokens int `json:"reasoning_tokens"`
Cost float64 `json:"cost"` // native provider-reported cost in USD for this call; 0 when the provider does not report one.
}
Usage reports token accounting for a request. CachedTokens is the portion of PromptTokens served from a prompt cache, where the provider reports it. ReasoningTokens is the portion of CompletionTokens the model spent on hidden reasoning (thinking models: Gemini, o-series, DeepSeek). It is a SUBSET of CompletionTokens — already billed inside it — so never add it to a total; it is broken out only to see where completion spend goes.
JSON tags are snake_case so every record that embeds Usage (agent transcripts, eval Trial files, council records, cmd/agent result JSON) uses one consistent shape on disk. Before 2026-07 the struct had no tags and the on-disk corpus (~600+ runs) used PascalCase keys ("PromptTokens"). The custom UnmarshalJSON accepts BOTH shapes so old unversioned records (eval Trial, council) don't silently zero out on read.
Source Files