llm

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 20, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Index

Constants

View Source 
const DiagnosticToolArgumentsRecovered = "tool_arguments_recovered"

DiagnosticToolArgumentsRecovered is the Type of a diagnostic recorded when a tool call's arguments could not be parsed strictly and were repaired, partially recovered, or discarded.

Variables

This section is empty.

Functions

func APIKeyEnvVars 

func APIKeyEnvVars(provider string) []string

APIKeyEnvVars returns the environment variables checked for provider in precedence order. The returned slice is safe for the caller to modify.

func FindEnvAPIKeys 

func FindEnvAPIKeys(provider string) []string

FindEnvAPIKeys returns the names of configured API key environment variables for provider, in lookup order.

func FindEnvAPIKeysWithEnv 

func FindEnvAPIKeysWithEnv(provider string, env ProviderEnv) []string

FindEnvAPIKeysWithEnv is FindEnvAPIKeys with request-scoped overrides.

func GetEnvAPIKey 

func GetEnvAPIKey(provider string) string

GetEnvAPIKey returns the first configured API key for provider.

func GetEnvAPIKeyWithEnv 

func GetEnvAPIKeyWithEnv(provider string, env ProviderEnv) string

GetEnvAPIKeyWithEnv returns the first configured API key for provider, preferring non-empty request-scoped values over process environment values.

func GetProviders 

func GetProviders() []string

GetProviders returns all providers in the built-in model registry.

func IsContextOverflow 

func IsContextOverflow(message AssistantMessage, contextWindow int64) bool

IsContextOverflow reports whether an assistant message indicates that the input exceeded the model's context window. It covers three cases:

  1. Error-based overflow: most providers return StopReasonError with a recognizable error message (matched against overflowPatterns, excluding nonOverflowPatterns such as rate limits).
  2. Silent overflow (e.g. z.ai): the request succeeds but usage.Input exceeds the context window. Pass a non-zero contextWindow to detect this.
  3. Length-stop overflow (e.g. Xiaomi MiMo): the server truncates oversized input to fill the window, leaving no room to generate, so it returns StopReasonLength with zero output and input filling the window.

Pass contextWindow as the model's window size to enable cases 2 and 3; pass 0 to check error messages only.

func OverflowPatterns 

func OverflowPatterns() []*regexp.Regexp

OverflowPatterns returns a copy of the overflow detection patterns, primarily for tests.

func ParseToolArguments 

func ParseToolArguments(raw string) map[string]any

ParseToolArguments decodes a tool call's accumulated JSON arguments into an object on a best-effort basis. Models occasionally emit JSON with unescaped control characters or bad escapes, and a stream may be cut off mid-token, so a strict decode of that input would lose every argument. This therefore tries, in order: a strict decode, a decode of a repaired copy, a partial decode that closes any open containers and strings, and a partial decode of the repaired copy. It always returns a non-nil map, falling back to an empty object when nothing can be salvaged.

Parsing never fails the surrounding stream: a recoverable but invalid tool call surfaces with whatever arguments could be salvaged so an agent can still validate it (see ValidateToolArguments) and let the model self-correct, instead of aborting the whole response.

func RepairJSON 

func RepairJSON(source string) string

RepairJSON fixes malformed JSON string literals by escaping raw control characters inside strings and doubling backslashes before invalid escape characters, while preserving valid escapes and \uXXXX sequences. Only ASCII characters carry special meaning here, so iterating over bytes leaves multi-byte UTF-8 sequences untouched.

func ValidateToolArguments 

func ValidateToolArguments(tool ToolDefinition, toolCall ToolCall) (map[string]any, error)

ValidateToolArguments coerces a tool call's arguments toward the tool's JSON Schema (forgiving common model mistakes such as "3" for a number), then validates them. It returns the coerced arguments, or a detailed error naming the failing fields. The original toolCall.Arguments are left unchanged.

The coercion mirrors pi's coerceWithJsonSchema. Validation covers the JSON Schema features normally emitted for tool definitions, including composition keywords and object, array, string, and numeric constraints.

func ValidateToolCall 

func ValidateToolCall(tools []ToolDefinition, toolCall ToolCall) (map[string]any, error)

ValidateToolCall finds the tool named by the call and validates its arguments. It mirrors pi's helper: a utility callers may invoke before dispatching a tool, not something the library calls itself. It returns the coerced arguments.

Types

type AnthropicMessagesCompatibility 

type AnthropicMessagesCompatibility struct {
	SupportsTemperature       *bool `json:"supportsTemperature,omitempty"`
	SupportsCacheControl      *bool `json:"supportsCacheControl,omitempty"`
	SupportsCacheControlTools *bool `json:"supportsCacheControlOnTools,omitempty"`
	ForceAdaptiveThinking     *bool `json:"forceAdaptiveThinking,omitempty"`
	AllowEmptySignature       *bool `json:"allowEmptySignature,omitempty"`
}

AnthropicMessagesCompatibility describes differences between providers that implement an Anthropic Messages-compatible endpoint. Pointer booleans distinguish an explicit false value from an unspecified provider default. Anthropic-compatible vendors (e.g. MiniMax) are served by pointing the base URL at their endpoint; most need no overrides at all.

func (*AnthropicMessagesCompatibility) Protocol 

func (*AnthropicMessagesCompatibility) Protocol() Protocol

Protocol identifies the API protocol whose request and message dialect this compatibility configuration describes.

type AnthropicStreamOptions added in v0.2.0 

type AnthropicStreamOptions struct {
	// ThinkingDisplay controls how a reasoning model returns its thinking. Empty
	// defaults to summarized.
	ThinkingDisplay ThinkingDisplay
	// ToolChoice controls Anthropic's native tool selection behavior.
	ToolChoice AnthropicToolChoice
}

AnthropicStreamOptions contains settings understood only by the Anthropic Messages protocol. Keeping them nested prevents provider-specific knobs from flattening the shared StreamOptions namespace.

func (*AnthropicStreamOptions) Protocol added in v0.2.0 

func (*AnthropicStreamOptions) Protocol() Protocol

Protocol identifies the protocol that accepts these options.

func (*AnthropicStreamOptions) Validate added in v0.2.0 

func (options *AnthropicStreamOptions) Validate(tools []ToolDefinition) error

Validate checks the Anthropic-specific settings against the available tools.

type AnthropicToolChoice added in v0.2.0 

type AnthropicToolChoice interface {
	// contains filtered or unexported methods
}

AnthropicToolChoice is the native Anthropic tool_choice union. Use one of the AnthropicToolChoice* mode constants or AnthropicToolChoiceTool.

type AnthropicToolChoiceMode added in v0.2.0 

type AnthropicToolChoiceMode string

AnthropicToolChoiceMode is one of Anthropic's string tool-choice modes. 

const (
	AnthropicToolChoiceAuto AnthropicToolChoiceMode = "auto"
	AnthropicToolChoiceAny  AnthropicToolChoiceMode = "any"
	AnthropicToolChoiceNone AnthropicToolChoiceMode = "none"
)

type AnthropicToolChoiceTool added in v0.2.0 

type AnthropicToolChoiceTool struct {
	Name string
}

AnthropicToolChoiceTool forces the model to call Name.

type ArgumentsMode added in v0.2.0 

type ArgumentsMode string

ArgumentsMode reports which layer of ParseToolArgumentsMode produced a result, so callers can tell strictly parsed arguments apart from recovered ones. 

const (
	// ArgumentsStrict means empty input or a clean strict decode: fully trusted.
	ArgumentsStrict ArgumentsMode = "strict"
	// ArgumentsRepaired means the input decoded only after escape repair.
	ArgumentsRepaired ArgumentsMode = "repaired"
	// ArgumentsPartial means truncated input was closed and decoded, so some
	// fields may be missing or cut short.
	ArgumentsPartial ArgumentsMode = "partial"
	// ArgumentsInvalid means nothing could be salvaged and the result is empty.
	ArgumentsInvalid ArgumentsMode = "invalid"
)

func ParseToolArgumentsMode added in v0.2.0 

func ParseToolArgumentsMode(raw string) (map[string]any, ArgumentsMode)

ParseToolArgumentsMode is ParseToolArguments with the recovery mode it used. The arguments are identical to ParseToolArguments; the mode lets a caller decline to execute a tool whose arguments were not strictly parsed.

type AssistantContent 

type AssistantContent interface {
	// contains filtered or unexported methods
}

AssistantContent is content that can appear in an assistant message.

type AssistantMessage 

type AssistantMessage struct {
	Content       []AssistantContent `json:"content"`
	Protocol      Protocol           `json:"protocol"`
	Provider      string             `json:"provider"`
	Model         string             `json:"model"`
	ResponseModel string             `json:"responseModel,omitempty"`
	ResponseID    string             `json:"responseId,omitempty"`
	Usage         Usage              `json:"usage"`
	StopReason    StopReason         `json:"stopReason"`
	ErrorMessage  string             `json:"errorMessage,omitempty"`
	// Diagnostics records non-fatal events (failures recovered from, degraded
	// results) that occurred while producing this response. It is nil for a
	// clean response.
	Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
	Timestamp   int64        `json:"timestamp"`
}

AssistantMessage is the final or partial response returned by a provider.

func NewAssistantMessage 

func NewAssistantMessage(model Model) AssistantMessage

NewAssistantMessage initializes provider-independent response metadata.

func (AssistantMessage) MarshalJSON 

func (message AssistantMessage) MarshalJSON() ([]byte, error)

func (*AssistantMessage) UnmarshalJSON 

func (message *AssistantMessage) UnmarshalJSON(data []byte) error

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client routes LLM requests to the adapter registered for a model protocol.

func NewClient 

func NewClient(registry *Registry) *Client

NewClient creates a client backed by the given provider registry.

func (*Client) Complete 

func (c *Client) Complete(
	ctx context.Context,
	model Model,
	input Context,
	options StreamOptions,
) (AssistantMessage, error)

Complete consumes a provider stream and returns the final assistant message.

func (*Client) Stream 

func (c *Client) Stream(ctx context.Context, model Model, input Context, options StreamOptions) (<-chan Event, error)

Stream starts a streaming completion request for the given model and input.

type Context 

type Context struct {
	SystemPrompt string           `json:"systemPrompt,omitempty"`
	Messages     []Message        `json:"messages"`
	Tools        []ToolDefinition `json:"tools,omitempty"`
}

Context contains the prompt, conversation history, and available tools.

func (Context) MarshalJSON 

func (input Context) MarshalJSON() ([]byte, error)

func (*Context) UnmarshalJSON 

func (input *Context) UnmarshalJSON(data []byte) error

type Diagnostic added in v0.2.0 

type Diagnostic struct {
	// Type categorizes the event, e.g. DiagnosticToolArgumentsRecovered.
	Type string `json:"type"`
	// Timestamp is the Unix millisecond time the diagnostic was recorded.
	Timestamp int64 `json:"timestamp"`
	// Message is an optional human-readable summary.
	Message string `json:"message,omitempty"`
	// Details carries structured, redacted context for the event.
	Details map[string]any `json:"details,omitempty"`
}

Diagnostic records a non-fatal event that occurred while producing an assistant response — a failure that was recovered from, or a degraded result — without affecting the message content or stop reason. Callers may inspect AssistantMessage.Diagnostics to react to recoveries, for example to avoid auto-executing a tool call whose arguments were only partially recovered.

func ToolArgumentsDiagnostic added in v0.2.0 

func ToolArgumentsDiagnostic(toolCallID, toolName string, mode ArgumentsMode) (Diagnostic, bool)

ToolArgumentsDiagnostic builds a diagnostic describing how a tool call's arguments were recovered. ok is false for a clean (strict) parse, which needs no diagnostic.

type Event 

type Event struct {
	Type EventType

	ContentIndex int

	Delta string

	Content string

	ToolCall *ToolCall

	Partial *AssistantMessage

	Message *AssistantMessage

	Err error
}

Event is a single update emitted while streaming a provider response.

type EventType 

type EventType string

EventType identifies the kind of update emitted by a provider stream. 

const (
	// EventStart marks the beginning of a provider stream.
	EventStart EventType = "start"
	// EventTextStart marks the creation of a text content block.
	EventTextStart EventType = "text_start"
	// EventTextDelta carries newly generated text.
	EventTextDelta EventType = "text_delta"
	// EventTextEnd carries the completed text content block.
	EventTextEnd EventType = "text_end"
	// EventThinkingStart marks the creation of a reasoning content block.
	EventThinkingStart EventType = "thinking_start"
	// EventThinkingDelta carries newly generated reasoning content.
	EventThinkingDelta EventType = "thinking_delta"
	// EventThinkingEnd carries the completed reasoning content block.
	EventThinkingEnd EventType = "thinking_end"
	// EventToolCallStart marks the creation of a tool call content block.
	EventToolCallStart EventType = "toolcall_start"
	// EventToolCallDelta carries a fragment of a tool call's arguments as it streams.
	EventToolCallDelta EventType = "toolcall_delta"
	// EventToolCallEnd carries a tool call whose arguments finished streaming.
	// Malformed or truncated argument JSON is parsed best-effort, so callers
	// should validate the arguments and wait for EventDone before executing the
	// call, because a later content block may still fail the overall response.
	EventToolCallEnd EventType = "toolcall_end"
	// EventDone carries the final assistant message.
	EventDone EventType = "done"
	// EventError carries a stream failure.
	EventError EventType = "error"
)

type ImageContent 

type ImageContent struct {
	Data     string `json:"data"`
	MIMEType string `json:"mimeType"`
}

ImageContent represents a base64-encoded image.

func (ImageContent) MarshalJSON 

func (content ImageContent) MarshalJSON() ([]byte, error)

type Message 

type Message interface {
	// contains filtered or unexported methods
}

Message is one item in the conversation context.

func TransformMessages 

func TransformMessages(messages []Message, model Model, normalizeToolCallID func(string) string) []Message

TransformMessages prepares the library's provider-independent conversation history for replay against model. Provider adapters should call it before translating Message values into their own wire-format message types.

Transformation happens per request instead of modifying the Agent's canonical history. The same history may later be sent to a model with different image, reasoning, or tool capabilities. Mutating stored history would make that model switch lose information permanently.

Shared transformations are applied in this order:

  1. Replace images with descriptive text when the target model is text-only.
  2. Reconcile assistant turns produced by a different model: keep reasoning for the same model, downgrade it to text otherwise, and normalize tool-call identifiers via normalizeToolCallID when crossing providers.
  3. Drop assistant turns terminated by an error or cancellation because they may contain partial reasoning or half-streamed tool calls.
  4. Insert synthetic error results for tool calls with no matching result before the conversation continues or ends.

normalizeToolCallID rewrites a tool-call ID for the target provider; pass nil to leave identifiers unchanged.

The returned slice is new and this function does not mutate messages. Message objects requiring no changes may still be shared with the input, so callers should treat the input and result as immutable.

type Model 

type Model struct {
	ID               string                         `json:"id"`
	Name             string                         `json:"name"`
	Protocol         Protocol                       `json:"protocol"`
	Provider         string                         `json:"provider"`
	BaseURL          string                         `json:"baseUrl"`
	Reasoning        bool                           `json:"reasoning"`
	ThinkingLevelMap map[ModelThinkingLevel]*string `json:"thinkingLevelMap,omitempty"`
	Input            []ModelInput                   `json:"input"`
	Cost             ModelCost                      `json:"cost"`
	ContextWindow    int64                          `json:"contextWindow"`
	MaxTokens        int64                          `json:"maxTokens"`
	Headers          map[string]string              `json:"headers,omitempty"`
	Compatibility    ModelCompatibility             `json:"compat,omitempty"`
}

Model identifies a model, its provider endpoint, capabilities, limits, and pricing. ThinkingLevelMap values are provider-specific; nil marks a level as unsupported while a missing key uses the provider default.

func GetModel 

func GetModel(provider, modelID string) Model

GetModel returns a model from the package's built-in model registry. It panics when the provider/model pair is unknown. Use LookupModel when the identifiers come from dynamic or untrusted input.

func GetModels 

func GetModels(provider string) []Model

GetModels returns all built-in models for a provider.

func LookupModel 

func LookupModel(provider, modelID string) (Model, bool)

LookupModel returns a model from the package's built-in model registry.

func (*Model) UnmarshalJSON 

func (model *Model) UnmarshalJSON(data []byte) error

UnmarshalJSON restores the concrete compatibility type selected by Protocol. The protocol acts as the discriminator, mirroring pi's Model<TApi> conditional compatibility type at runtime.

type ModelCompatibility 

type ModelCompatibility interface {
	Protocol() Protocol
}

ModelCompatibility is implemented by protocol-specific compatibility configurations. It keeps Model independent from any one provider protocol while allowing registration and adapters to verify type/protocol agreement.

type ModelCost 

type ModelCost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cacheRead"`
	CacheWrite float64 `json:"cacheWrite"`
}

ModelCost stores prices in US dollars per million tokens.

type ModelInput 

type ModelInput string

ModelInput identifies an input modality accepted by a model. 

const (
	Text  ModelInput = "text"
	Image ModelInput = "image"
)

type ModelRegistry 

type ModelRegistry struct {
	// contains filtered or unexported fields
}

ModelRegistry stores models by provider and model ID. It is safe for concurrent access and returns defensive copies of registered models.

func NewModelRegistry 

func NewModelRegistry() *ModelRegistry

NewModelRegistry creates an empty model registry.

func (*ModelRegistry) Get 

func (registry *ModelRegistry) Get(provider, modelID string) (Model, bool)

Get returns a model registered for provider and modelID.

func (*ModelRegistry) Models 

func (registry *ModelRegistry) Models(provider string) []Model

Models returns a provider's models ordered by model ID.

func (*ModelRegistry) Providers 

func (registry *ModelRegistry) Providers() []string

Providers returns registered provider IDs in lexical order.

func (*ModelRegistry) Register 

func (registry *ModelRegistry) Register(model Model) error

Register adds or replaces a model with the same provider and ID.

type ModelThinkingLevel 

type ModelThinkingLevel string

ModelThinkingLevel is a provider-independent reasoning effort level. 

const (
	ModelThinkingOff     ModelThinkingLevel = "off"
	ModelThinkingMinimal ModelThinkingLevel = "minimal"
	ModelThinkingLow     ModelThinkingLevel = "low"
	ModelThinkingMedium  ModelThinkingLevel = "medium"
	ModelThinkingHigh    ModelThinkingLevel = "high"
	ModelThinkingXHigh   ModelThinkingLevel = "xhigh"
)

func ClampThinkingLevel 

func ClampThinkingLevel(model Model, level ModelThinkingLevel) ModelThinkingLevel

ClampThinkingLevel adjusts a requested level to the nearest one the model supports: it prefers the requested level, then steps up, then down, and falls back to the lowest supported level (or "off").

func SupportedThinkingLevels 

func SupportedThinkingLevels(model Model) []ModelThinkingLevel

SupportedThinkingLevels returns the thinking levels a model accepts. A non-reasoning model supports only "off". For reasoning models, a level mapped to nil is unsupported, and "xhigh" is supported only when explicitly mapped.

type OpenAICompletionsCompatibility 

type OpenAICompletionsCompatibility struct {
	SupportsStore                               *bool  `json:"supportsStore,omitempty"`
	SupportsDeveloperRole                       *bool  `json:"supportsDeveloperRole,omitempty"`
	SupportsReasoningEffort                     *bool  `json:"supportsReasoningEffort,omitempty"`
	MaxTokensField                              string `json:"maxTokensField,omitempty"`
	SupportsStrictMode                          *bool  `json:"supportsStrictMode,omitempty"`
	RequiresReasoningContentOnAssistantMessages *bool  `json:"requiresReasoningContentOnAssistantMessages,omitempty"`
	// RequiresThinkingAsText makes replayed assistant turns carry thinking as a
	// leading text content block instead of a provider reasoning field, for
	// endpoints that reject reasoning fields on input.
	RequiresThinkingAsText *bool  `json:"requiresThinkingAsText,omitempty"`
	ThinkingFormat         string `json:"thinkingFormat,omitempty"`
	ZAIToolStream          *bool  `json:"zaiToolStream,omitempty"`
}

OpenAICompletionsCompatibility describes differences between providers that implement an OpenAI-compatible Chat Completions endpoint. Pointer booleans distinguish an explicit false value from an unspecified provider default.

func (*OpenAICompletionsCompatibility) Protocol 

func (*OpenAICompletionsCompatibility) Protocol() Protocol

Protocol identifies the API protocol whose request and message dialect this compatibility configuration describes.

type OpenAICompletionsStreamOptions added in v0.2.0 

type OpenAICompletionsStreamOptions struct {
	ToolChoice OpenAIToolChoice
}

OpenAICompletionsStreamOptions contains settings understood only by the OpenAI-compatible Chat Completions protocol.

func (*OpenAICompletionsStreamOptions) Protocol added in v0.2.0 

func (*OpenAICompletionsStreamOptions) Protocol() Protocol

Protocol identifies the protocol that accepts these options.

func (*OpenAICompletionsStreamOptions) Validate added in v0.2.0 

func (options *OpenAICompletionsStreamOptions) Validate(tools []ToolDefinition) error

Validate checks the OpenAI-specific settings against the available tools.

type OpenAIToolChoice added in v0.2.0 

type OpenAIToolChoice interface {
	// contains filtered or unexported methods
}

OpenAIToolChoice is the native OpenAI Chat Completions tool_choice union. Use one of the OpenAIToolChoice* mode constants or OpenAIToolChoiceFunction.

type OpenAIToolChoiceFunction added in v0.2.0 

type OpenAIToolChoiceFunction struct {
	Name string
}

OpenAIToolChoiceFunction forces the model to call the named function tool.

type OpenAIToolChoiceMode added in v0.2.0 

type OpenAIToolChoiceMode string

OpenAIToolChoiceMode is one of OpenAI's string tool-choice modes. 

const (
	OpenAIToolChoiceAuto     OpenAIToolChoiceMode = "auto"
	OpenAIToolChoiceNone     OpenAIToolChoiceMode = "none"
	OpenAIToolChoiceRequired OpenAIToolChoiceMode = "required"
)

type Protocol 

type Protocol string

Protocol identifies the API protocol used to communicate with a model. 

const (
	ProtocolOpenAICompletions Protocol = "openai-completions"
	ProtocolAnthropicMessages Protocol = "anthropic-messages"
)

type ProtocolAdapter 

type ProtocolAdapter interface {
	// Protocol returns the registry key used to select this adapter.
	Protocol() Protocol

	// Stream emits response events for the given model and conversation context.
	Stream(ctx context.Context, model Model, input Context, options StreamOptions) (<-chan Event, error)
}

ProtocolAdapter translates between a concrete LLM protocol and the package streaming interface.

type ProtocolStreamOptions added in v0.2.0 

type ProtocolStreamOptions interface {
	Protocol() Protocol
	Validate(tools []ToolDefinition) error
}

ProtocolStreamOptions is the extension point for settings whose semantics cannot be shared across protocols. Custom protocol adapters may provide their own implementation and validate it before the stream starts.

type ProviderEnv 

type ProviderEnv map[string]string

ProviderEnv contains request-scoped environment overrides. Non-empty values take precedence over process environment variables during credential lookup.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry stores protocol adapters and is safe for concurrent access.

func NewRegistry 

func NewRegistry() *Registry

NewRegistry creates an empty provider registry.

func (*Registry) Get 

func (registry *Registry) Get(protocol Protocol) (ProtocolAdapter, bool)

Get returns the adapter registered for the protocol.

func (*Registry) Register 

func (registry *Registry) Register(adapter ProtocolAdapter) error

Register adds or replaces an adapter for its protocol.

type StopReason 

type StopReason string

StopReason explains why the model stopped generating a response. 

const (
	// StopReasonStop marks a normal completion.
	StopReasonStop StopReason = "stop"
	// StopReasonLength marks truncation by the max output token limit.
	StopReasonLength StopReason = "length"
	// StopReasonToolUse marks a stop to let the caller execute tool calls.
	StopReasonToolUse StopReason = "toolUse"
	// StopReasonError marks a provider or runtime failure.
	StopReasonError StopReason = "error"
	// StopReasonAborted marks a cancelled request.
	StopReasonAborted StopReason = "aborted"
)

type StreamOptions 

type StreamOptions struct {
	APIKey string
	Env    ProviderEnv
	// Temperature overrides the model's default sampling temperature when set.
	Temperature *float64
	// MaxTokens caps the output tokens for this request. Zero leaves it unset.
	MaxTokens int64
	// Headers are merged into the request, overriding model default headers.
	Headers map[string]string
	// Reasoning requests a thinking level. The provider clamps it to what the
	// model supports. Empty leaves the model's default; "off" disables thinking.
	Reasoning ModelThinkingLevel
	// ProtocolOptions carries settings specific to exactly one protocol.
	ProtocolOptions ProtocolStreamOptions
	// MaxRetries overrides the SDK client-side retry count for transient failures
	// (HTTP 429 and 5xx, connection errors). Nil leaves the SDK default; a value
	// of 0 disables retries so the caller can manage them.
	MaxRetries *int
	// Timeout caps the total duration of one HTTP attempt. Zero leaves the SDK
	// default. It is independent of the request context, which still cancels the
	// whole call.
	Timeout time.Duration
	// OnResponse, when set, is called with the status and headers of every HTTP
	// response before its body is consumed. It fires once per attempt, so a
	// retried request invokes it for each try, making retries observable.
	OnResponse func(status int, headers http.Header)
	// OnRequest, when set, is called with the method, URL, and full body of every
	// HTTP request before it is sent. The body is the exact JSON serialized for
	// the provider. It fires once per attempt, so a retried request invokes it
	// for each try, making retries observable.
	OnRequest func(method, url string, body []byte)
}

StreamOptions contains shared request settings plus optional protocol-specific extensions. A non-nil extension must match the target model protocol.

func (StreamOptions) Validate added in v0.2.0 

func (options StreamOptions) Validate(protocol Protocol, tools []ToolDefinition) error

Validate checks that explicitly supplied protocol extensions match the target protocol and contain supported values.

type StreamWriter added in v0.2.0 

type StreamWriter struct {
	// contains filtered or unexported fields
}

StreamWriter manages the event channel for one streamed response. It emits a single EventStart, attaches a Partial snapshot to every non-terminal event, and guarantees exactly one terminal event (EventDone or EventError) before the channel closes. A cancelled context is reported as StopReasonAborted.

A protocol adapter builds its AssistantMessage incrementally — the writer is constructed over a pointer to it — emits deltas through Emit, and ends with Done on success or Fail on error. This is the shared machinery behind the built-in adapters; it keeps the single-terminal guarantee and Partial cloning in one place rather than duplicated per provider.

func NewStreamWriter added in v0.2.0 

func NewStreamWriter(ctx context.Context, events chan<- Event, output *AssistantMessage) *StreamWriter

NewStreamWriter returns a writer that sends events to the channel and snapshots output for each event's Partial and for the terminal Message.

func (*StreamWriter) Done added in v0.2.0 

func (w *StreamWriter) Done()

Done emits the single terminal EventDone carrying the final message.

func (*StreamWriter) Emit added in v0.2.0 

func (w *StreamWriter) Emit(event Event)

Emit sends a non-terminal event, first emitting EventStart if needed and attaching a fresh Partial snapshot of the message built so far.

func (*StreamWriter) Fail added in v0.2.0 

func (w *StreamWriter) Fail(err error)

Fail emits the single terminal EventError. A cancelled context is reported as StopReasonAborted and replaces the error with the context error; any other failure is reported as StopReasonError.

func (*StreamWriter) Start added in v0.2.0 

func (w *StreamWriter) Start()

Start emits EventStart. It is idempotent: only the first call emits.

type TextContent 

type TextContent struct {
	Text          string `json:"text"`
	TextSignature string `json:"textSignature,omitempty"`
}

TextContent represents plain text.

func (TextContent) MarshalJSON 

func (content TextContent) MarshalJSON() ([]byte, error)

type ThinkingContent 

type ThinkingContent struct {
	Thinking          string `json:"thinking"`
	ThinkingSignature string `json:"thinkingSignature,omitempty"`
	Redacted          bool   `json:"redacted,omitempty"`
}

ThinkingContent represents model reasoning content.

func (ThinkingContent) MarshalJSON 

func (content ThinkingContent) MarshalJSON() ([]byte, error)

type ThinkingDisplay added in v0.2.0 

type ThinkingDisplay string

ThinkingDisplay controls how a reasoning model returns its thinking. It does not change whether the model reasons or what it is billed; it only governs what travels back. Only Anthropic-protocol models honor it today. 

const (
	// ThinkingDisplaySummarized returns summarized thinking text in the response.
	ThinkingDisplaySummarized ThinkingDisplay = "summarized"
	// ThinkingDisplayOmitted redacts the thinking text but still returns the
	// signature needed for multi-turn tool-use continuity. Use it for backends
	// that never surface reasoning, trading the thinking text for lower
	// time-to-first-token and a lighter response.
	ThinkingDisplayOmitted ThinkingDisplay = "omitted"
)

type ToolCall 

type ToolCall struct {
	ID               string         `json:"id"`
	Name             string         `json:"name"`
	Arguments        map[string]any `json:"arguments"`
	ThoughtSignature string         `json:"thoughtSignature,omitempty"`
}

ToolCall describes a request to invoke a named tool with JSON arguments.

func CloneToolCall added in v0.2.0 

func CloneToolCall(toolCall *ToolCall) *ToolCall

CloneToolCall returns a deep copy of a tool call, including its arguments, for use in an event's ToolCall field.

func (ToolCall) MarshalJSON 

func (content ToolCall) MarshalJSON() ([]byte, error)

type ToolDefinition 

type ToolDefinition struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Parameters  json.RawMessage `json:"parameters"`
}

ToolDefinition describes a tool that the model may call.

type ToolResultContent 

type ToolResultContent interface {
	// contains filtered or unexported methods
}

ToolResultContent is content that can appear in a tool result message.

type ToolResultMessage 

type ToolResultMessage struct {
	ToolCallID string              `json:"toolCallId"`
	ToolName   string              `json:"toolName"`
	Content    []ToolResultContent `json:"content"`
	IsError    bool                `json:"isError"`
}

ToolResultMessage contains the result of an assistant tool call.

func (ToolResultMessage) MarshalJSON 

func (message ToolResultMessage) MarshalJSON() ([]byte, error)

func (*ToolResultMessage) UnmarshalJSON 

func (message *ToolResultMessage) UnmarshalJSON(data []byte) error

type Usage 

type Usage struct {
	Input       int64     `json:"input"`
	Output      int64     `json:"output"`
	CacheRead   int64     `json:"cacheRead"`
	CacheWrite  int64     `json:"cacheWrite"`
	TotalTokens int64     `json:"totalTokens"`
	Cost        UsageCost `json:"cost"`
}

Usage records token consumption for one assistant response.

type UsageCost 

type UsageCost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cacheRead"`
	CacheWrite float64 `json:"cacheWrite"`
	Total      float64 `json:"total"`
}

UsageCost breaks down the US dollar cost of one response by token category.

func CalculateCost 

func CalculateCost(model Model, usage Usage) UsageCost

CalculateCost returns the US dollar cost of usage at the model's prices. Model costs are quoted per million tokens.

type UserContent 

type UserContent interface {
	// contains filtered or unexported methods
}

UserContent is content that can appear in a user message.

type UserMessage 

type UserMessage struct {
	Content []UserContent `json:"content"`
}

UserMessage contains content supplied by the user.

func (UserMessage) MarshalJSON 

func (message UserMessage) MarshalJSON() ([]byte, error)

func (*UserMessage) UnmarshalJSON 

func (message *UserMessage) UnmarshalJSON(data []byte) error

Source Files

View all Source files

Directories

Path Synopsis
cmd
genmodels command
Command genmodels builds llm's checked-in model catalog from the same public catalogs used by pi-ai.
 Command genmodels builds llm's checked-in model catalog from the same public catalogs used by pi-ai.
providers
anthropic
Package anthropic implements the Anthropic Messages protocol on top of the official anthropic-sdk-go.
 Package anthropic implements the Anthropic Messages protocol on top of the official anthropic-sdk-go.
openai

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.