llm

Documentation

Overview

Package llm defines the provider-agnostic LLM interface used by the agent engine. Concrete providers (Anthropic, OpenAI, Ollama) live in subpackages and self-register via init(). The engine never imports a provider directly.

Index

• Variables
• func Register(name string, f Factory)
• type BlockType
• type ContentBlock
• type Event
• type EventKind
• type Factory
• type Message
  • func TextMessage(role Role, text string) Message
• type Provider
  • func Resolve(model string) (Provider, string, error)
• type Request
• type Role
• type ToolSchema
• type Usage

Variables

var ErrUnknownProvider = errors.New("unknown llm provider")

ErrUnknownProvider is returned when a model string references a provider that has not been registered.

Functions

func Register

func Register(name string, f Factory)

Register makes a provider available under name. Typically called from a provider package's init().

Types

type BlockType

type BlockType string

BlockType discriminates ContentBlock payloads. Mirrors Anthropic's content-block model since it is the most expressive of the providers; other adapters translate as needed.

const (
	BlockText       BlockType = "text"
	BlockToolUse    BlockType = "tool_use"
	BlockToolResult BlockType = "tool_result"
)

type ContentBlock

type ContentBlock struct {
	Type BlockType

	// BlockText
	Text string

	// BlockToolUse (assistant → tool call request)
	ToolID    string
	ToolName  string
	ToolInput json.RawMessage

	// BlockToolResult (user → tool execution response)
	ToolUseID string
	Output    string
	IsError   bool
}

ContentBlock is a single typed piece of a Message. A text-only message has one block of type BlockText. An assistant turn that calls a tool has one or more BlockToolUse blocks (possibly interleaved with BlockText). The result returned to the model is a user message containing BlockToolResult blocks.

type Event

type Event struct {
	Kind EventKind

	// EventText
	Text string

	// EventToolCall
	ToolID    string
	ToolName  string
	ToolInput json.RawMessage

	// EventDone
	StopReason string

	// EventUsage
	Usage Usage

	// EventError
	Err error
}

Event is a single item in a streamed response.

type EventKind

type EventKind int

EventKind discriminates streamed Event payloads.

const (
	// EventText is a text token chunk to append to the assistant reply.
	EventText EventKind = iota
	// EventToolCall is emitted once per tool invocation, after the model has
	// produced the full tool_use block (id, name, input JSON).
	EventToolCall
	// EventDone is emitted once when the model has finished its turn.
	// StopReason carries the reason ("end_turn", "tool_use", "max_tokens"...).
	EventDone
	// EventUsage is emitted once per response with token counts. Providers
	// that don't report usage simply omit this event.
	EventUsage
	// EventError is a terminal failure event; the channel closes after.
	EventError
)

type Factory

type Factory func() (Provider, error)

Factory constructs a Provider on demand. It runs once per Resolve call so providers can read env vars / secrets at construction time and surface configuration errors clearly.

type Message

type Message struct {
	Role    Role
	Content []ContentBlock
}

Message is one turn in a conversation.

func TextMessage

func TextMessage(role Role, text string) Message

TextMessage is a convenience constructor for plain-text turns.

type Provider

type Provider interface {
	// Stream returns a channel that yields Events until the response is
	// complete or ctx is cancelled. The channel is always closed by the
	// implementation. An error returned synchronously means no events were
	// emitted (e.g. transport failure before the stream opened).
	Stream(ctx context.Context, req Request) (<-chan Event, error)
}

Provider is the contract every backend implements.

func Resolve

func Resolve(model string) (Provider, string, error)

Resolve splits a "provider/model" string, looks up the provider factory, and returns the constructed provider plus the bare model id.

type Request

type Request struct {
	Model       string
	System      string
	Messages    []Message
	Tools       []ToolSchema
	Temperature *float64
	MaxTokens   int
	// ResponseSchema, when non-nil, constrains the model's output to valid
	// JSON matching this schema. Providers implement this via their native
	// structured-output mechanism (OpenAI json_schema, Anthropic tool-use
	// trick, Ollama format:json). Nil means unconstrained text output.
	ResponseSchema json.RawMessage
}

Request is a single completion call. Model is the bare model id (provider prefix already stripped by Resolve).

type Role

type Role string

Role is the speaker of a Message.

const (
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
)

type ToolSchema

type ToolSchema struct {
	Name        string
	Description string
	InputSchema json.RawMessage
}

ToolSchema describes a tool the model may call.

type Usage

type Usage struct {
	InputTokens  int
	OutputTokens int
}

Usage carries token counts for a single provider round-trip.