llm

Documentation

Overview

Package llm defines the provider-agnostic types and the Client interface every LLM provider satisfies.

The shapes here are deliberately the lowest common denominator across providers. Provider-specific knobs live on each provider's Options struct.

See docs/v1/API.md for the rationale behind every type.

Index

• Variables
• func Collect(ctx context.Context, c Client, req Request) (text string, calls []ToolCall, images []ImagePart, usage *Usage, err error)
• func Float32(v float32) *float32
• type CacheBreakpoint
• type CacheTTL
• type Client
• type Event
• type EventKind
• type ImagePart
• type Message
• type Request
• type Role
• type ThinkingConfig
  • func ThinkingByBudget(n int) *ThinkingConfig
  • func ThinkingByEffort(e ThinkingEffort) *ThinkingConfig
  • func (t *ThinkingConfig) Budget() int
  • func (t *ThinkingConfig) Effort() ThinkingEffort
• type ThinkingEffort
• type ToolCall
• type ToolDef
• type Usage

Variables

var (
	ErrAuth           = errors.New("aikido/llm: authentication failed")
	ErrRateLimited    = errors.New("aikido/llm: rate limited")
	ErrServerError    = errors.New("aikido/llm: provider server error")
	ErrInvalidRequest = errors.New("aikido/llm: invalid request")
)

Errors providers wrap with %w when mapping HTTP status to a typed cause.

Functions

func Collect

func Collect(ctx context.Context, c Client, req Request) (text string, calls []ToolCall, images []ImagePart, usage *Usage, err error)

Collect drains a stream into a final result. Useful for non-streaming callers.

Returns text accumulated from EventTextDelta, all complete tool calls, all images surfaced by the provider, final Usage if the provider emitted one, and the first error encountered. Thinking text is not included in the returned text.

Collect respects ctx cancellation: if ctx is cancelled before the stream closes, Collect returns ctx.Err() without waiting for the producer.

func Float32

func Float32(v float32) *float32

Float32 returns a pointer to v.

Convenience for SessionOptions.Temperature so callers can write inline values. Float32(0) returns a non-nil pointer to zero — the deterministic-zero case.

Types

type CacheBreakpoint

type CacheBreakpoint struct {
	TTL CacheTTL
}

CacheBreakpoint marks a message as a cache breakpoint on providers that support it. nil = no breakpoint. &CacheBreakpoint{} = default 5m TTL.

type CacheTTL

type CacheTTL string

const (
	CacheTTL5Min  CacheTTL = "5m"
	CacheTTL1Hour CacheTTL = "1h"
)

type Client

type Client interface {
	Stream(ctx context.Context, req Request) (<-chan Event, error)
}

type Event

type Event struct {
	Kind  EventKind
	Text  string
	Tool  *ToolCall
	Image *ImagePart
	Usage *Usage
	Err   error
}

Event is one streaming event emitted by a Client. Channel closes after EventEnd.

type EventKind

type EventKind string

EventKind identifies the kind of a streaming event.

const (
	EventTextDelta EventKind = "text_delta"
	EventToolCall  EventKind = "tool_call"
	EventThinking  EventKind = "thinking"
	EventImage     EventKind = "image"
	EventUsage     EventKind = "usage"
	EventError     EventKind = "error"
	EventEnd       EventKind = "end"
)

type ImagePart

type ImagePart struct {
	URL         string
	ContentType string
	Data        []byte
}

ImagePart is one image attached to a message or returned by an image-capable model. URL is set when the provider returned a remote URL or when the caller supplied a data URI / remote URL on input. Data is set when the provider returned inline bytes (decoded from a data: URI) or when the caller wants to inline an image on output. ContentType is the MIME type when known ("image/png", "image/jpeg", ...) — empty when not provided by the wire.

type Message

type Message struct {
	Role       Role
	Content    string
	Images     []ImagePart
	ToolCalls  []ToolCall
	ToolCallID string
	Cache      *CacheBreakpoint
}

type Request

type Request struct {
	Model         string
	Messages      []Message
	Tools         []ToolDef
	MaxTokens     int
	Temperature   *float32
	Thinking      *ThinkingConfig
	StopSequences []string
}

type Role

type Role string

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type ThinkingConfig

type ThinkingConfig struct {
	// contains filtered or unexported fields
}

ThinkingConfig configures provider-side thinking. Use ThinkingByEffort or ThinkingByBudget — the unexported fields prevent setting both.

func ThinkingByBudget

func ThinkingByBudget(n int) *ThinkingConfig

func ThinkingByEffort

func ThinkingByEffort(e ThinkingEffort) *ThinkingConfig

func (*ThinkingConfig) Budget

func (t *ThinkingConfig) Budget() int

Budget returns the configured token budget, 0 if effort-based.

func (*ThinkingConfig) Effort

func (t *ThinkingConfig) Effort() ThinkingEffort

Effort returns the configured coarse effort, empty string if budget-based.

type ThinkingEffort

type ThinkingEffort string

const (
	ThinkingEffortLow    ThinkingEffort = "low"
	ThinkingEffortMedium ThinkingEffort = "medium"
	ThinkingEffortHigh   ThinkingEffort = "high"
)

type ToolCall

type ToolCall struct {
	ID        string
	Name      string
	Arguments string
}

type ToolDef

type ToolDef struct {
	Name        string
	Description string
	Parameters  json.RawMessage
}

ToolDef is one tool the model may call. Parameters is a JSON Schema.

type Usage

type Usage struct {
	PromptTokens     int
	CompletionTokens int
	CacheReadTokens  int
	CacheWriteTokens int
	CostUSD          float64
}