llm

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 25, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package llm is a unified, provider-neutral API for large language models.

It speaks two wire protocols, OpenAI Chat Completions and Anthropic Messages, behind one set of types. The same conversation can be sent to any model on either protocol, and the target model can change between turns; the library re-adapts the history for each request. It is a stateless translation layer: it decides what to send for a request and how to interpret the streamed response, and leaves history storage, context compaction, and tool-loop orchestration to the caller.

Entry points

Stream returns a channel of Event values; Complete consumes that stream and returns the final AssistantMessage. Both dispatch to the adapter registered for the model's protocol. The package keeps a default client; NewClient returns an isolated one with the built-in adapters registered. 

model := llm.GetModel("anthropic", "claude-opus-4-8")
msg, err := llm.Complete(ctx, model, input, llm.StreamOptions{})

Building a request

A Context holds the system prompt, the message history, and the available tools. Messages are provider-neutral and serialize to self-describing JSON, so a conversation can be persisted and replayed later against any model:

Text-only models that receive image content have it downgraded to a placeholder automatically.

Options

StreamOptions carries settings shared by every protocol — API key, temperature, max tokens, headers, retries, timeout — plus observation hooks, OnRequest (the exact serialized request body) and OnResponse (status and headers), and RewriteRequest, which replaces the serialized body before it is sent. Each is invoked once per attempt including retries.

Reasoning is a provider-neutral effort level (ModelThinkingLevel: off, minimal, low, medium, high, xhigh). Each adapter maps it to that provider's native form (Anthropic adaptive or budget thinking; the OpenAI-compatible reasoning fields) and clamps it to what the model supports. It is ignored by non-reasoning models.

Settings with no neutral equivalent live on a protocol-specific extension supplied through StreamOptions.ProtocolOptions and validated against the target protocol before the request is sent:

Streaming

Event values report incremental progress: text, reasoning, and tool-call blocks each emit start, delta, and end events, followed by a single terminal EventDone (carrying the final message) or EventError. Every event carries a Partial snapshot of the message assembled so far. Tool-call arguments are parsed best-effort at end of stream; validate them and wait for EventDone before executing any call.

Typed tools

NewTool and MustTool derive a provider-compatible JSON Schema from a Go struct, and DecodeToolCall decodes a returned call back into that struct. Malformed or truncated argument JSON degrades to a best-effort value and is reported in AssistantMessage.Diagnostics rather than failing the response.

Results

AssistantMessage holds the response content, a StopReason, token Usage with per-category cost, and any non-fatal Diagnostics. CalculateCost prices a usage record against a model.

Switching models

TransformMessages adapts a stored history for a target model before replay: it downgrades unsupported images, preserves reasoning signatures for the same model while downgrading or dropping them across models, normalizes tool-call identifiers, and repairs unanswered tool calls. Stream and Complete apply it automatically. IsContextOverflow reports whether a response exceeded the model's context window.

Models

LookupModel and GetModel resolve models from the built-in catalog; GetProviders and GetModels enumerate it. SupportedThinkingLevels and ClampThinkingLevel report and adjust a model's reasoning levels. A caller may also construct a Model directly, pointing BaseURL at any OpenAI-compatible or Anthropic-compatible endpoint.

Custom protocols

A genuinely different wire protocol is added by implementing ProtocolAdapter and registering it: build a Registry with NewRegistry, add the built-ins with RegisterBuiltins and the custom adapter with Registry.Register, then construct a client with NewClientWithRegistry. NewStreamWriter gives the adapter the same event-stream machinery the built-ins use.

Index

Constants

View Source 
const (
	ProtocolOpenAICompletions = core.ProtocolOpenAICompletions
	ProtocolAnthropicMessages = core.ProtocolAnthropicMessages

	Text  = core.Text
	Image = core.Image

	ModelThinkingOff     = core.ModelThinkingOff
	ModelThinkingMinimal = core.ModelThinkingMinimal
	ModelThinkingLow     = core.ModelThinkingLow
	ModelThinkingMedium  = core.ModelThinkingMedium
	ModelThinkingHigh    = core.ModelThinkingHigh
	ModelThinkingXHigh   = core.ModelThinkingXHigh

	ThinkingDisplaySummarized = core.ThinkingDisplaySummarized
	ThinkingDisplayOmitted    = core.ThinkingDisplayOmitted

	AnthropicToolChoiceAuto = core.AnthropicToolChoiceAuto
	AnthropicToolChoiceAny  = core.AnthropicToolChoiceAny
	AnthropicToolChoiceNone = core.AnthropicToolChoiceNone

	OpenAIToolChoiceAuto     = core.OpenAIToolChoiceAuto
	OpenAIToolChoiceNone     = core.OpenAIToolChoiceNone
	OpenAIToolChoiceRequired = core.OpenAIToolChoiceRequired

	StopReasonStop    = core.StopReasonStop
	StopReasonLength  = core.StopReasonLength
	StopReasonToolUse = core.StopReasonToolUse
	StopReasonError   = core.StopReasonError
	StopReasonAborted = core.StopReasonAborted

	EventStart         = core.EventStart
	EventTextStart     = core.EventTextStart
	EventTextDelta     = core.EventTextDelta
	EventTextEnd       = core.EventTextEnd
	EventThinkingStart = core.EventThinkingStart
	EventThinkingDelta = core.EventThinkingDelta
	EventThinkingEnd   = core.EventThinkingEnd
	EventToolCallStart = core.EventToolCallStart
	EventToolCallDelta = core.EventToolCallDelta
	EventToolCallEnd   = core.EventToolCallEnd
	EventDone          = core.EventDone
	EventError         = core.EventError
)
View Source 
const (
	ArgumentsStrict   = core.ArgumentsStrict
	ArgumentsRepaired = core.ArgumentsRepaired
	ArgumentsPartial  = core.ArgumentsPartial
	ArgumentsInvalid  = core.ArgumentsInvalid

	DiagnosticToolArgumentsRecovered = core.DiagnosticToolArgumentsRecovered
)

Variables

This section is empty.

Functions

func DecodeToolCall 

func DecodeToolCall[T any](tool ToolDefinition, call ToolCall) (T, error)

DecodeToolCall validates and coerces call arguments with tool's schema, then decodes them into T. The original ToolCall is not modified.

func GetProviders 

func GetProviders() []string

GetProviders returns the provider IDs in the built-in model catalog.

func IsContextOverflow 

func IsContextOverflow(message AssistantMessage, contextWindow int64) bool

IsContextOverflow reports whether a response indicates a context overflow.

func ParseToolArguments 

func ParseToolArguments(raw string) map[string]any

ParseToolArguments parses streamed tool argument JSON on a best-effort basis, repairing malformed string escapes and closing truncated input. It always returns a non-nil map, falling back to an empty object when nothing can be salvaged, so a recoverable but invalid tool call never aborts the stream. Enforce correctness separately with ValidateToolArguments before dispatching.

func RegisterBuiltins added in v0.2.0 

func RegisterBuiltins(registry *Registry)

RegisterBuiltins registers the OpenAI-compatible and Anthropic adapters into registry, so a custom client keeps the built-in protocols alongside any added ones.

func Stream 

func Stream(ctx context.Context, model Model, input Context, options StreamOptions) (<-chan Event, error)

Stream uses the default client to start a streaming model request.

func ValidateToolArguments 

func ValidateToolArguments(tool ToolDefinition, toolCall ToolCall) (map[string]any, error)

ValidateToolArguments validates and coerces one tool call against tool.

func ValidateToolCall 

func ValidateToolCall(tools []ToolDefinition, toolCall ToolCall) (map[string]any, error)

ValidateToolCall validates and coerces a tool call against its definition.

Types

type AnthropicMessagesCompatibility 

type AnthropicMessagesCompatibility = core.AnthropicMessagesCompatibility

Core conversation, model, and streaming types are aliases so callers only need this package.

type AnthropicStreamOptions added in v0.2.0 

type AnthropicStreamOptions = core.AnthropicStreamOptions

Core conversation, model, and streaming types are aliases so callers only need this package.

type AnthropicToolChoice added in v0.2.0 

type AnthropicToolChoice = core.AnthropicToolChoice

Core conversation, model, and streaming types are aliases so callers only need this package.

type AnthropicToolChoiceMode added in v0.2.0 

type AnthropicToolChoiceMode = core.AnthropicToolChoiceMode

Core conversation, model, and streaming types are aliases so callers only need this package.

type AnthropicToolChoiceTool added in v0.2.0 

type AnthropicToolChoiceTool = core.AnthropicToolChoiceTool

Core conversation, model, and streaming types are aliases so callers only need this package.

type ArgumentsMode added in v0.2.0 

type ArgumentsMode = core.ArgumentsMode

Tool-related conversation types are aliased here so callers only need this package.

func ParseToolArgumentsMode added in v0.2.0 

func ParseToolArgumentsMode(raw string) (map[string]any, ArgumentsMode)

ParseToolArgumentsMode is ParseToolArguments with the recovery mode it used, so a caller can decline to execute a tool whose arguments were not strictly parsed (ArgumentsPartial or ArgumentsInvalid).

type AssistantContent 

type AssistantContent = core.AssistantContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type AssistantMessage 

type AssistantMessage = core.AssistantMessage

Core conversation, model, and streaming types are aliases so callers only need this package.

func AssistantText added in v0.5.1 

func AssistantText(text string) *AssistantMessage

AssistantText builds an assistant message containing a single text block. It is handy for seeding conversation history with a prior model reply.

func Complete 

func Complete(ctx context.Context, model Model, input Context, options StreamOptions) (AssistantMessage, error)

Complete uses the default client and returns the final assistant message.

type Client 

type Client = core.Client

Core conversation, model, and streaming types are aliases so callers only need this package.

func NewClient 

func NewClient() *Client

NewClient returns an isolated client with all built-in protocol adapters registered. Most callers can use the package-level Stream and Complete functions instead.

func NewClientWithRegistry added in v0.2.0 

func NewClientWithRegistry(registry *Registry) *Client

NewClientWithRegistry returns a client backed by the given registry. Combine it with NewRegistry, RegisterBuiltins, and Registry.Register to serve a custom ProtocolAdapter.

type Context 

type Context = core.Context

Core conversation, model, and streaming types are aliases so callers only need this package.

func NewContext added in v0.5.1 

func NewContext(messages ...Message) Context

NewContext assembles a Context from the given messages.

func Prompt added in v0.5.1 

func Prompt(text string) Context

Prompt builds a Context holding a single user text message. It is the shortest way to start a one-shot completion.

func PromptWithSystem added in v0.5.1 

func PromptWithSystem(system, user string) Context

PromptWithSystem builds a Context with a system prompt and a single user text message.

type Diagnostic added in v0.2.0 

type Diagnostic = core.Diagnostic

Core conversation, model, and streaming types are aliases so callers only need this package.

type Event 

type Event = core.Event

Core conversation, model, and streaming types are aliases so callers only need this package.

type EventType 

type EventType = core.EventType

Core conversation, model, and streaming types are aliases so callers only need this package.

type ImageContent 

type ImageContent = core.ImageContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type Message 

type Message = core.Message

Core conversation, model, and streaming types are aliases so callers only need this package.

func TransformMessages 

func TransformMessages(messages []Message, model Model, normalizeToolCallID func(string) string) []Message

TransformMessages prepares history for replay against model.

type Model 

type Model = core.Model

Core conversation, model, and streaming types are aliases so callers only need this package.

func GetModel 

func GetModel(provider, modelID string) Model

GetModel returns a model from the built-in catalog and panics when unknown.

func GetModels 

func GetModels(provider string) []Model

GetModels returns the built-in models for provider.

func LookupModel 

func LookupModel(provider, modelID string) (Model, bool)

LookupModel returns a model from the built-in catalog.

type ModelCompatibility 

type ModelCompatibility = core.ModelCompatibility

Core conversation, model, and streaming types are aliases so callers only need this package.

type ModelCost 

type ModelCost = core.ModelCost

Core conversation, model, and streaming types are aliases so callers only need this package.

type ModelInput 

type ModelInput = core.ModelInput

Core conversation, model, and streaming types are aliases so callers only need this package.

type ModelThinkingLevel 

type ModelThinkingLevel = core.ModelThinkingLevel

Core conversation, model, and streaming types are aliases so callers only need this package.

func ClampThinkingLevel 

func ClampThinkingLevel(model Model, level ModelThinkingLevel) ModelThinkingLevel

ClampThinkingLevel returns the nearest reasoning level accepted by model.

func SupportedThinkingLevels 

func SupportedThinkingLevels(model Model) []ModelThinkingLevel

SupportedThinkingLevels returns the reasoning levels accepted by model.

type OpenAICompletionsCompatibility 

type OpenAICompletionsCompatibility = core.OpenAICompletionsCompatibility

Core conversation, model, and streaming types are aliases so callers only need this package.

type OpenAICompletionsStreamOptions added in v0.2.0 

type OpenAICompletionsStreamOptions = core.OpenAICompletionsStreamOptions

Core conversation, model, and streaming types are aliases so callers only need this package.

type OpenAIToolChoice added in v0.2.0 

type OpenAIToolChoice = core.OpenAIToolChoice

Core conversation, model, and streaming types are aliases so callers only need this package.

type OpenAIToolChoiceFunction added in v0.2.0 

type OpenAIToolChoiceFunction = core.OpenAIToolChoiceFunction

Core conversation, model, and streaming types are aliases so callers only need this package.

type OpenAIToolChoiceMode added in v0.2.0 

type OpenAIToolChoiceMode = core.OpenAIToolChoiceMode

Core conversation, model, and streaming types are aliases so callers only need this package.

type Protocol 

type Protocol = core.Protocol

Core conversation, model, and streaming types are aliases so callers only need this package.

type ProtocolAdapter added in v0.2.0 

type ProtocolAdapter = core.ProtocolAdapter

Core conversation, model, and streaming types are aliases so callers only need this package.

type ProtocolStreamOptions added in v0.2.0 

type ProtocolStreamOptions = core.ProtocolStreamOptions

Core conversation, model, and streaming types are aliases so callers only need this package.

type ProviderEnv 

type ProviderEnv = core.ProviderEnv

Core conversation, model, and streaming types are aliases so callers only need this package.

type Registry added in v0.2.0 

type Registry = core.Registry

Extension points for registering a custom protocol adapter.

func NewRegistry added in v0.2.0 

func NewRegistry() *Registry

NewRegistry returns an empty protocol-adapter registry. Use it with RegisterBuiltins and Register to assemble a client that also serves a custom protocol. Most callers want NewClient instead.

type StopReason 

type StopReason = core.StopReason

Core conversation, model, and streaming types are aliases so callers only need this package.

type StreamOptions 

type StreamOptions = core.StreamOptions

Core conversation, model, and streaming types are aliases so callers only need this package.

type StreamWriter added in v0.2.0 

type StreamWriter = core.StreamWriter

Core conversation, model, and streaming types are aliases so callers only need this package.

func NewStreamWriter added in v0.2.0 

func NewStreamWriter(ctx context.Context, events chan<- Event, output *AssistantMessage) *StreamWriter

NewStreamWriter helps a custom ProtocolAdapter emit a well-formed event stream: it sends a single EventStart, attaches a Partial snapshot to each event, guarantees one terminal event, and turns context cancellation into an aborted terminal error. output is the assistant message the adapter builds.

type TextContent 

type TextContent = core.TextContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type ThinkingContent 

type ThinkingContent = core.ThinkingContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type ThinkingDisplay added in v0.2.0 

type ThinkingDisplay = core.ThinkingDisplay

Core conversation, model, and streaming types are aliases so callers only need this package.

type ToolCall 

type ToolCall = core.ToolCall

Tool-related conversation types are aliased here so callers only need this package.

func CloneToolCall added in v0.2.0 

func CloneToolCall(toolCall *ToolCall) *ToolCall

CloneToolCall returns a deep copy of a tool call for use in an event's ToolCall field while streaming.

type ToolDefinition 

type ToolDefinition = core.ToolDefinition

Tool-related conversation types are aliased here so callers only need this package.

func MustTool 

func MustTool[T any](name, description string) ToolDefinition

MustTool is NewTool for statically declared tools. It panics when the tool name or argument type cannot produce a valid definition.

func NewTool 

func NewTool[T any](name, description string) (ToolDefinition, error)

NewTool creates a tool definition whose parameters are generated from T. T must be a struct or pointer to a struct. Fields without json omitempty are required, and jsonschema tags can add descriptions, enums, and constraints.

type ToolResultContent 

type ToolResultContent = core.ToolResultContent

Tool-related conversation types are aliased here so callers only need this package.

type ToolResultMessage 

type ToolResultMessage = core.ToolResultMessage

Tool-related conversation types are aliased here so callers only need this package.

func ToolResult added in v0.5.1 

func ToolResult(callID, toolName, text string) *ToolResultMessage

ToolResult builds a tool result message answering the assistant tool call with the given ID. The text becomes a single text block in the result.

type Usage 

type Usage = core.Usage

Core conversation, model, and streaming types are aliases so callers only need this package.

type UsageCost 

type UsageCost = core.UsageCost

Core conversation, model, and streaming types are aliases so callers only need this package.

func CalculateCost 

func CalculateCost(model Model, usage Usage) UsageCost

CalculateCost calculates the model cost for usage.

type UserContent 

type UserContent = core.UserContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type UserMessage 

type UserMessage = core.UserMessage

Core conversation, model, and streaming types are aliases so callers only need this package.

func UserImage added in v0.5.1 

func UserImage(data, mimeType string) *UserMessage

UserImage builds a user message containing a single base64-encoded image.

func UserText added in v0.5.1 

func UserText(text string) *UserMessage

UserText builds a user message containing a single text block.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.