llm

Documentation

Overview

Package llm provides the batteries-included entry point for model inference. It keeps a ready-to-use default client for common calls while the lower-level implementation remains internal to the module.

Index

• Constants
• func DecodeToolCall[T any](tool ToolDefinition, call ToolCall) (T, error)
• func GetProviders() []string
• func IsContextOverflow(message AssistantMessage, contextWindow int64) bool
• func ParseToolArguments(raw string) map[string]any
• func Stream(ctx context.Context, model Model, input Context, options StreamOptions) (<-chan Event, error)
• func ValidateToolArguments(tool ToolDefinition, toolCall ToolCall) (map[string]any, error)
• func ValidateToolCall(tools []ToolDefinition, toolCall ToolCall) (map[string]any, error)
• type AnthropicMessagesCompatibility
• type AssistantContent
• type AssistantMessage
  • func Complete(ctx context.Context, model Model, input Context, options StreamOptions) (AssistantMessage, error)
• type Client
  • func NewClient() *Client
• type Context
• type Event
• type EventType
• type ImageContent
• type Message
  • func TransformMessages(messages []Message, model Model, normalizeToolCallID func(string) string) []Message
• type Model
  • func GetModel(provider, modelID string) Model
  • func GetModels(provider string) []Model
  • func LookupModel(provider, modelID string) (Model, bool)
• type ModelCompatibility
• type ModelCost
• type ModelInput
• type ModelThinkingLevel
  • func ClampThinkingLevel(model Model, level ModelThinkingLevel) ModelThinkingLevel
  • func SupportedThinkingLevels(model Model) []ModelThinkingLevel
• type OpenAICompletionsCompatibility
• type Protocol
• type ProviderEnv
• type StopReason
• type StreamOptions
• type TextContent
• type ThinkingContent
• type ToolCall
• type ToolDefinition
  • func MustTool[T any](name, description string) ToolDefinition
  • func NewTool[T any](name, description string) (ToolDefinition, error)
• type ToolResultContent
• type ToolResultMessage
• type Usage
• type UsageCost
  • func CalculateCost(model Model, usage Usage) UsageCost
• type UserContent
• type UserMessage

Constants

const (
	ProtocolOpenAICompletions = core.ProtocolOpenAICompletions
	ProtocolAnthropicMessages = core.ProtocolAnthropicMessages

	Text  = core.Text
	Image = core.Image

	ModelThinkingOff     = core.ModelThinkingOff
	ModelThinkingMinimal = core.ModelThinkingMinimal
	ModelThinkingLow     = core.ModelThinkingLow
	ModelThinkingMedium  = core.ModelThinkingMedium
	ModelThinkingHigh    = core.ModelThinkingHigh
	ModelThinkingXHigh   = core.ModelThinkingXHigh

	StopReasonStop    = core.StopReasonStop
	StopReasonLength  = core.StopReasonLength
	StopReasonToolUse = core.StopReasonToolUse
	StopReasonError   = core.StopReasonError
	StopReasonAborted = core.StopReasonAborted

	EventStart         = core.EventStart
	EventTextStart     = core.EventTextStart
	EventTextDelta     = core.EventTextDelta
	EventTextEnd       = core.EventTextEnd
	EventThinkingStart = core.EventThinkingStart
	EventThinkingDelta = core.EventThinkingDelta
	EventThinkingEnd   = core.EventThinkingEnd
	EventToolCallStart = core.EventToolCallStart
	EventToolCallDelta = core.EventToolCallDelta
	EventToolCallEnd   = core.EventToolCallEnd
	EventDone          = core.EventDone
	EventError         = core.EventError
)

Functions

func DecodeToolCall

func DecodeToolCall[T any](tool ToolDefinition, call ToolCall) (T, error)

DecodeToolCall validates and coerces call arguments with tool's schema, then decodes them into T. The original ToolCall is not modified.

func GetProviders

func GetProviders() []string

GetProviders returns the provider IDs in the built-in model catalog.

func IsContextOverflow

func IsContextOverflow(message AssistantMessage, contextWindow int64) bool

IsContextOverflow reports whether a response indicates a context overflow.

func ParseToolArguments

func ParseToolArguments(raw string) map[string]any

ParseToolArguments parses and repairs streamed tool argument JSON.

func Stream

func Stream(ctx context.Context, model Model, input Context, options StreamOptions) (<-chan Event, error)

Stream uses the default client to start a streaming model request.

func ValidateToolArguments

func ValidateToolArguments(tool ToolDefinition, toolCall ToolCall) (map[string]any, error)

ValidateToolArguments validates and coerces one tool call against tool.

func ValidateToolCall

func ValidateToolCall(tools []ToolDefinition, toolCall ToolCall) (map[string]any, error)

ValidateToolCall validates and coerces a tool call against its definition.

Types

type AnthropicMessagesCompatibility

type AnthropicMessagesCompatibility = core.AnthropicMessagesCompatibility

Core conversation, model, and streaming types are aliases so callers only need this package.

type AssistantContent

type AssistantContent = core.AssistantContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type AssistantMessage

type AssistantMessage = core.AssistantMessage

Core conversation, model, and streaming types are aliases so callers only need this package.

func Complete

func Complete(ctx context.Context, model Model, input Context, options StreamOptions) (AssistantMessage, error)

Complete uses the default client and returns the final assistant message.

type Client

type Client = core.Client

Core conversation, model, and streaming types are aliases so callers only need this package.

func NewClient

func NewClient() *Client

NewClient returns an isolated client with all built-in protocol adapters registered. Most callers can use the package-level Stream and Complete functions instead.

type Context

type Context = core.Context

Core conversation, model, and streaming types are aliases so callers only need this package.

type Event

type Event = core.Event

Core conversation, model, and streaming types are aliases so callers only need this package.

type EventType

type EventType = core.EventType

Core conversation, model, and streaming types are aliases so callers only need this package.

type ImageContent

type ImageContent = core.ImageContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type Message

type Message = core.Message

Core conversation, model, and streaming types are aliases so callers only need this package.

func TransformMessages

func TransformMessages(messages []Message, model Model, normalizeToolCallID func(string) string) []Message

TransformMessages prepares history for replay against model.

type Model

type Model = core.Model

Core conversation, model, and streaming types are aliases so callers only need this package.

func GetModel

func GetModel(provider, modelID string) Model

GetModel returns a model from the built-in catalog and panics when unknown.

func GetModels

func GetModels(provider string) []Model

GetModels returns the built-in models for provider.

func LookupModel

func LookupModel(provider, modelID string) (Model, bool)

LookupModel returns a model from the built-in catalog.

type ModelCompatibility

type ModelCompatibility = core.ModelCompatibility

Core conversation, model, and streaming types are aliases so callers only need this package.

type ModelCost

type ModelCost = core.ModelCost

Core conversation, model, and streaming types are aliases so callers only need this package.

type ModelInput

type ModelInput = core.ModelInput

Core conversation, model, and streaming types are aliases so callers only need this package.

type ModelThinkingLevel

type ModelThinkingLevel = core.ModelThinkingLevel

Core conversation, model, and streaming types are aliases so callers only need this package.

func ClampThinkingLevel

func ClampThinkingLevel(model Model, level ModelThinkingLevel) ModelThinkingLevel

ClampThinkingLevel returns the nearest reasoning level accepted by model.

func SupportedThinkingLevels

func SupportedThinkingLevels(model Model) []ModelThinkingLevel

SupportedThinkingLevels returns the reasoning levels accepted by model.

type OpenAICompletionsCompatibility

type OpenAICompletionsCompatibility = core.OpenAICompletionsCompatibility

Core conversation, model, and streaming types are aliases so callers only need this package.

type Protocol

type Protocol = core.Protocol

Core conversation, model, and streaming types are aliases so callers only need this package.

type ProviderEnv

type ProviderEnv = core.ProviderEnv

Core conversation, model, and streaming types are aliases so callers only need this package.

type StopReason

type StopReason = core.StopReason

Core conversation, model, and streaming types are aliases so callers only need this package.

type StreamOptions

type StreamOptions = core.StreamOptions

Core conversation, model, and streaming types are aliases so callers only need this package.

type TextContent

type TextContent = core.TextContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type ThinkingContent

type ThinkingContent = core.ThinkingContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type ToolCall

type ToolCall = core.ToolCall

Core conversation, model, and streaming types are aliases so callers only need this package.

type ToolDefinition

type ToolDefinition = core.ToolDefinition

Core conversation, model, and streaming types are aliases so callers only need this package.

func MustTool

func MustTool[T any](name, description string) ToolDefinition

MustTool is NewTool for statically declared tools. It panics when the tool name or argument type cannot produce a valid definition.

func NewTool

func NewTool[T any](name, description string) (ToolDefinition, error)

NewTool creates a tool definition whose parameters are generated from T. T must be a struct or pointer to a struct. Fields without json omitempty are required, and jsonschema tags can add descriptions, enums, and constraints.

type ToolResultContent

type ToolResultContent = core.ToolResultContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type ToolResultMessage

type ToolResultMessage = core.ToolResultMessage

Core conversation, model, and streaming types are aliases so callers only need this package.

type Usage

type Usage = core.Usage

Core conversation, model, and streaming types are aliases so callers only need this package.

type UsageCost

type UsageCost = core.UsageCost

Core conversation, model, and streaming types are aliases so callers only need this package.

func CalculateCost

func CalculateCost(model Model, usage Usage) UsageCost

CalculateCost calculates the model cost for usage.

type UserContent

type UserContent = core.UserContent

Core conversation, model, and streaming types are aliases so callers only need this package.

type UserMessage

type UserMessage = core.UserMessage

Core conversation, model, and streaming types are aliases so callers only need this package.