llm

README

pi-llm-go

A minimal, Go-native LLM adapter with streaming, tool calling, and extended thinking. Provider-agnostic interface with built-in support for Anthropic Messages, OpenAI Chat Completions (covering OpenAI, Azure OpenAI, Groq, Together, vLLM, OpenRouter, Ollama), and the OpenAI Responses API (GPT-5-family server-side state + reasoning summaries).

Status: v0.x — pre-1.0. API may change between minor versions; see CHANGELOG.md.

Why

The Go LLM library landscape forces you to pick between heavy vendor SDKs that don't compose, code-generated client surfaces that don't track new providers, or "framework" libraries that ship more concepts than you want. pi-llm-go is the opposite: ~1.5kLoc of plain Go that gives you an interface, two providers, and a streaming model that uses iter.Seq2 like a normal iterator. No HTTP wrappers, no model registries, no provider-specific magic.

Installation

go get github.com/amit-timalsina/pi-llm-go

Requires Go 1.23 or later (for iter.Seq2).

Quickstart

package main

import (
    "context"
    "fmt"
    "os"

    llm "github.com/amit-timalsina/pi-llm-go"
    "github.com/amit-timalsina/pi-llm-go/providers/anthropic"
)

func main() {
    p, _ := anthropic.New(anthropic.Options{APIKey: os.Getenv("ANTHROPIC_API_KEY")})

    msg, err := llm.Complete(context.Background(), p, llm.Request{
        Model:     anthropic.ClaudeSonnet4_6,
        MaxTokens: 1024,
        Messages: []llm.Message{
            {Role: llm.RoleUser, Content: []llm.Block{llm.TextBlock{Text: "hello"}}},
        },
    })
    if err != nil { panic(err) }

    for _, block := range msg.Content {
        if tb, ok := block.(llm.TextBlock); ok {
            fmt.Println(tb.Text)
        }
    }
}

Streaming is iter.Seq2[llm.StreamEvent, error] — range over it:

for event, err := range p.Stream(ctx, req) {
    if err != nil { /* handle */ }
    if d, ok := event.(llm.EventTextDelta); ok {
        fmt.Print(d.Delta)
    }
}

Features

• Streaming-first. Stream() returns iter.Seq2[StreamEvent, error] — Go 1.23 iterators, no callbacks, no goroutine leaks. Complete() is the synchronous helper for one-shot use.
• Sealed sum types. Block and StreamEvent are interfaces with package-private marker methods. Type-switch exhaustively; the compiler tells you if you miss a case.
• Tool calling. Declare tools on Request.Tools; receive ToolCallBlocks on the response; send ToolResultBlocks back. Pi-llm-go does not execute tools — that's pi-agent-go's job.
• Extended thinking. ThinkingConfig{BudgetTokens: N} on requests, surfaced as ThinkingBlock content. Anthropic-only at v1.
• Open-closed providers. Implement LLM.Stream to add custom providers; no plugin registry needed.
• Errors that branch cleanly. errors.Is(err, llm.ErrRateLimit) works through *APIError wraps; errors.As(err, &apiErr) gives you status + body.
• Cancellation = context.Context. No bespoke abort signal types.

Providers

Anthropic

import "github.com/amit-timalsina/pi-llm-go/providers/anthropic"

p, _ := anthropic.New(anthropic.Options{
    APIKey: os.Getenv("ANTHROPIC_API_KEY"),
    // BaseURL: defaults to https://api.anthropic.com
    // Version: defaults to "2023-06-01"
    // Beta:    optional anthropic-beta header values
})

Honors Request.Thinking. Surfaces all content-block types (text, thinking, tool_use).

OpenAI-compatible

import "github.com/amit-timalsina/pi-llm-go/providers/openai"

p, _ := openai.New(openai.Options{
    APIKey:  os.Getenv("OPENAI_API_KEY"),
    BaseURL: "https://api.openai.com/v1",  // or "https://api.groq.com/openai/v1", etc.
})

Talks the Chat Completions wire format, so the same provider works against any compatible host. Request.Thinking is ignored at v1 — reasoning-effort dialects vary too much across compatible hosts to map portably.

Gemini

import "github.com/amit-timalsina/pi-llm-go/providers/gemini"

p, _ := gemini.New(gemini.Options{APIKey: os.Getenv("GEMINI_API_KEY")})

Native support for the Gemini 2.5 / 3 / Robotics ER 1.6 families. Same LLM interface as the other providers, plus llm.VideoBlock for native video input (Gemini is the only provider that accepts video natively; Anthropic and OpenAI reject VideoBlock at the wire boundary with a clear pointer to the frame-extraction workaround). YouTube URLs work directly:

llm.VideoBlock{URI: "https://www.youtube.com/watch?v=..."}

For files larger than ~20 MB, the sibling providers/gemini/files sub-package handles the multipart upload + ACTIVE-state polling:

import "github.com/amit-timalsina/pi-llm-go/providers/gemini/files"

fc, _ := files.New(files.Options{APIKey: os.Getenv("GEMINI_API_KEY")})
ref, _ := fc.Upload(ctx, mp4Reader, "video/mp4", files.UploadOptions{DisplayName: "demo.mp4"})
ref, _ = fc.Wait(ctx, ref, files.WaitOptions{}) // polls until ACTIVE
defer fc.Delete(context.Background(), ref.Name) // ~48h server-side TTL if you forget

// Now use the URI in a generateContent call:
content := []llm.Block{llm.TextBlock{Text: "describe"}, llm.VideoBlock{URI: ref.URI}}

Vertex AI (gs:// URIs + OAuth) is a planned future addition; v0.5 only supports the Google AI direct endpoint.

Error handling

Non-2xx HTTP responses surface as *llm.APIError wrapping one of the typed sentinels — errors.Is works through the wrapping so consumers branch on category, not status code:

ErrAuth           // 401, 403
ErrRateLimit      // 429
ErrInvalidRequest // other 4xx
ErrProvider       // generic provider problem (parent of the next two)
├─ ErrServerError // 5xx (excluding 529)
└─ ErrOverloaded  // 529 (Anthropic infra overload)

ErrServerError and ErrOverloaded both wrap ErrProvider via %w, so legacy errors.Is(err, llm.ErrProvider) keeps matching 5xx + 529 unchanged.

Sugar helpers and the parsed Retry-After:

for ev, err := range provider.Stream(ctx, req) {
    if err == nil { /* consume ev */ continue }

    var apiErr *llm.APIError
    if errors.As(err, &apiErr) {
        switch {
        case llm.IsRateLimited(err):     // 429 → respect apiErr.RetryAfter
        case llm.IsOverloaded(err):      // 529 → short backoff, consider failover
        case llm.IsServerError(err):     // 5xx → retry, escalate if sustained
        case errors.Is(err, llm.ErrAuth):
        }
    }
    return err
}

APIError.RetryAfter is populated by all four built-in providers when the response carries a Retry-After (RFC 7231 delta-seconds or HTTP-date) or retry-after-ms (OpenAI's millisecond form, which wins when both are present).

Examples

Runnable examples in examples/:

• examples/streaming — basic streaming of a text response.
• examples/tool_calling — hand-rolled tool-call loop against get_current_time.
• examples/multimodal — image input on Anthropic + OpenAI.
• examples/multimodal_gemini — text / image / video against Gemini; --video-upload PATH exercises the Files API end-to-end.
• examples/prompt_caching — CacheRetention knob driving Anthropic prompt-cache hits.

Run them with go run ./examples/streaming (set ANTHROPIC_API_KEY first).

Versioning

This package is pre-1.0. Anything can change between minor versions; refer to CHANGELOG.md for each release.

v1.0 lands when the API surface has been used in production for ≥4 weeks without churn. Post-1.0 follows semver strictly.

License

MIT. See LICENSE.

Acknowledgements

Designed and named after pi-ai by Mario Zechner (TypeScript, MIT). The wire-level vocabulary and event types follow the upstream's lead; the Go-native API surface (interface-based providers, iterator streaming, sealed sum types) is a from-scratch redesign for Go idioms.

Built and maintained by Amit Timalsina with Claude Code assistance — all design decisions and release calls are human-owned.

Documentation

Overview

Package llm is a minimal, provider-agnostic Go interface for streaming LLM completions with tool calling. Built-in providers cover the Anthropic Messages API and OpenAI-compatible Chat Completions APIs (OpenAI, Groq, Together, vLLM, OpenRouter, Ollama, and similar).

Two patterns of use:

• Streaming events: range over LLM.Stream to react to deltas in real time.
• One-shot completion: call Complete to drain the stream and receive the final assistant message.

Cancellation propagates through context.Context. Provider errors surface through the iterator's error return as *APIError wrapping a sentinel (ErrAuth, ErrRateLimit, ErrInvalidRequest, ErrProvider).

pi-llm-go does not execute tools. It declares tool schemas on requests and surfaces ToolCallBlocks on responses. The companion pi-agent-go module adds the execution loop.

Index

• Variables
• func Accumulate(events iter.Seq2[StreamEvent, error]) iter.Seq2[*Message, error]
• func IsOverloaded(err error) bool
• func IsRateLimited(err error) bool
• func IsServerError(err error) bool
• func ParseRetryAfter(headers http.Header) time.Duration
• func SentinelForStatus(status int) error
• type APIError
  • func (e *APIError) Error() string
  • func (e *APIError) Unwrap() error
• type Block
• type CacheRetention
• type EventMessageEnd
• type EventMessageStart
• type EventTextDelta
• type EventTextEnd
• type EventTextStart
• type EventThinkingDelta
• type EventThinkingEnd
• type EventThinkingStart
• type EventToolCallDelta
• type EventToolCallEnd
• type EventToolCallStart
• type ImageBlock
  • func (i ImageBlock) Validate() error
• type LLM
• type Message
  • func Complete(ctx context.Context, l LLM, req Request) (*Message, error)
• type Request
• type Role
• type StopReason
• type StreamEvent
• type TextBlock
• type ThinkingBlock
• type ThinkingConfig
• type Tool
• type ToolCallBlock
• type ToolResultBlock
• type Usage
• type VideoBlock
  • func (v VideoBlock) Validate() error

Variables

var (
	ErrAuth           = errors.New("llm: authentication failed")
	ErrRateLimit      = errors.New("llm: rate limited")
	ErrInvalidRequest = errors.New("llm: invalid request")
	ErrProvider       = errors.New("llm: provider error")
	// ErrServerError signals a generic 5xx response (excluding 529).
	// Recommended consumer policy: retry with backoff; if sustained
	// past a threshold, surface for engineer escalation.
	ErrServerError = fmt.Errorf("%w: server error (5xx)", ErrProvider)
	// ErrOverloaded signals an Anthropic-style 529 "overloaded"
	// response. Recommended consumer policy: short backoff (~60s)
	// then retry; consider provider fallback if sustained.
	ErrOverloaded = fmt.Errorf("%w: overloaded (529)", ErrProvider)
)

Sentinel errors. Wrap via APIError or return directly. Use errors.Is to branch on them in caller retry / fallback logic.

Hierarchy:

ErrProvider           // generic "something provider-side broke"
├─ ErrServerError     // HTTP 5xx (excluding 529)
└─ ErrOverloaded      // HTTP 529 (Anthropic infra overload)

ErrServerError and ErrOverloaded each wrap ErrProvider via fmt.Errorf("%w"), so existing callers using errors.Is(err, ErrProvider) continue to match 5xx + 529 responses (backward compatible). The two child sentinels add specificity for consumers that need distinct retry / escalation policies per the category guidance in issue #11.

Functions

func Accumulate

func Accumulate(events iter.Seq2[StreamEvent, error]) iter.Seq2[*Message, error]

Accumulate folds a stream of events into a stream of progressively-built messages. Each yield emits a snapshot of the assistant message as it stands after the most recent event. The final yielded value (when err is nil) is the fully-assembled assistant message.

Callers that only want the final message should prefer Complete. Use Accumulate when intermediate snapshots are useful (e.g. driving a UI that renders each delta against a complete message tree rather than against individual deltas).

Each snapshot is an independent value — internal slices and strings are copied on emit so callers can retain previous snapshots without aliasing.

func IsOverloaded

func IsOverloaded(err error) bool

IsOverloaded reports whether err is an Anthropic-style 529 overloaded response.

func IsRateLimited

func IsRateLimited(err error) bool

IsRateLimited reports whether err (or anything in its Unwrap chain) is a 429 rate-limit error. Equivalent to errors.Is(err, ErrRateLimit) — sugar for the common caller-side branch.

func IsServerError

func IsServerError(err error) bool

IsServerError reports whether err is a generic 5xx response (excluding 529; check IsOverloaded for that case).

func ParseRetryAfter

func ParseRetryAfter(headers http.Header) time.Duration

ParseRetryAfter extracts a wait hint from a provider response's Retry-After / retry-after-ms headers. Returns 0 if no header is present or none of them parse.

Header precedence (`retry-after-ms` wins when both are present because it carries sub-second precision):

• retry-after-ms: integer milliseconds (OpenAI convention).
• Retry-After: integer seconds (RFC 7231 delta-seconds).
• Retry-After: HTTP-date (RFC 7231; computed as now() delta).

Negative deltas (HTTP-date already in the past) are clamped to 0. Callers should still apply their own minimum bound — a 0-second server hint usually means "as soon as you can" but pummeling the API immediately is rarely productive.

func SentinelForStatus

func SentinelForStatus(status int) error

SentinelForStatus maps an HTTP status code to the matching sentinel error. Used by provider implementations when constructing APIError.

Status to sentinel:

401, 403       → ErrAuth
429            → ErrRateLimit
other 4xx      → ErrInvalidRequest
529            → ErrOverloaded (Anthropic-style)
other 5xx      → ErrServerError
otherwise      → ErrProvider

ErrServerError and ErrOverloaded both wrap ErrProvider, so legacy errors.Is(err, ErrProvider) keeps working for 5xx / 529 responses.

Types

type APIError

type APIError struct {
	Provider string
	Status   int
	Body     []byte
	Inner    error
	// RetryAfter, when > 0, is the server's hint for how long the
	// caller should wait before retrying. Parsed from the response's
	// Retry-After header (RFC 7231 seconds-or-HTTP-date) or, for
	// OpenAI-family providers, the `retry-after-ms` header. Zero
	// means "no header present" — callers fall back to their own
	// backoff schedule.
	RetryAfter time.Duration
}

APIError wraps a non-2xx HTTP response from a provider. The Inner field is one of the sentinel errors above so that errors.Is works through the wrapping. Status and Body let callers inspect the raw failure (e.g. to parse a structured provider error payload). RetryAfter, when non-zero, is the parsed value of the response's Retry-After or retry-after-ms header — populated by providers for 429 / 529 responses to support caller-side rate-limit / overload backoff.

func (*APIError) Error

func (e *APIError) Error() string

func (*APIError) Unwrap

func (e *APIError) Unwrap() error

type Block

type Block interface {
	// contains filtered or unexported methods
}

Block is the sealed sum type for message content. Concrete implementations are TextBlock, ThinkingBlock, ToolCallBlock, and ToolResultBlock — all defined in this package. The unexported marker method keeps the set closed: provider converters need exhaustive type-switches to serialize content correctly, so new block types must be added inside the package.

type CacheRetention

type CacheRetention string

CacheRetention controls Anthropic prompt-cache breakpoint placement. It is a single-knob abstraction: callers pick a retention tier and the provider decides where to place the cache_control markers on the underlying wire format.

The marker tells Anthropic "everything from the start of the request up to and including this block is cacheable; on a subsequent request with byte-identical content up to this marker, return a cache hit and bill cache-read rates instead of full input rates."

Behavior by value:

• CacheRetentionNone (zero value, "") — no markers emitted.
• CacheRetentionShort — ephemeral markers with the default ~5 minute lifetime, placed at: (a) the System prompt's trailing block, (b) the final Tool in Request.Tools, (c) the last block (any type) of the most recent user-role message. The last-block placement is type-agnostic so that subsequent calls in a tool loop reuse the cached tool_result round-trip instead of re-billing it.
• CacheRetentionLong — same placement as Short with TTL "1h" and the "extended-cache-ttl-2025-04-11" beta header auto-attached to the outgoing HTTP request.

OpenAI's Chat Completions and Responses providers silently ignore CacheRetention — OpenAI caches automatically with no caller-side breakpoint API.

All currently-shipped Claude models support both 1h cache TTL and tool-level cache_control; pi-llm-go therefore does not gate on per- model compat flags. Third-party Anthropic-compatible hosts that lack either capability are outside the scope of v1.

The caller owns prompt determinism: cached sections must be byte-stable across iterations for the cache to hit. Any change (timestamps, map iteration order, reordered items) invalidates the cache from that point forward in the request.

See https://docs.claude.com/en/docs/build-with-claude/prompt-caching for the full discipline.

const (
	// CacheRetentionNone disables prompt caching for this request. This is
	// the zero value of CacheRetention; an unset field and an explicit
	// CacheRetentionNone are byte-identical and produce no cache_control
	// markers.
	CacheRetentionNone CacheRetention = ""

	// CacheRetentionShort places ephemeral cache breakpoints with the
	// default ~5 minute lifetime. The right default for iterative agent
	// loops where the prefix is reused within a single session.
	CacheRetentionShort CacheRetention = "short"

	// CacheRetentionLong places ephemeral cache breakpoints with the 1-hour
	// TTL and auto-attaches the "extended-cache-ttl-2025-04-11" beta header.
	// For long-lived static prefixes (large system prompts, big tool sets)
	// that survive across many sessions.
	CacheRetentionLong CacheRetention = "long"
)

type EventMessageEnd

type EventMessageEnd struct {
	StopReason StopReason
	Usage      Usage
}

EventMessageEnd is the terminal event. It carries the normalized stop reason and the final usage tally.

type EventMessageStart

type EventMessageStart struct {
	Model string
}

EventMessageStart is emitted once at the start of an assistant turn, before any block events.

type EventTextDelta

type EventTextDelta struct {
	BlockIndex int
	Delta      string
}

EventTextDelta appends Delta to the text in the block at BlockIndex.

type EventTextEnd

type EventTextEnd struct {
	BlockIndex int
}

EventTextEnd marks the end of the TextBlock at BlockIndex.

type EventTextStart

type EventTextStart struct {
	BlockIndex int
}

EventTextStart marks the beginning of a TextBlock.

type EventThinkingDelta

type EventThinkingDelta struct {
	BlockIndex int
	Delta      string
}

EventThinkingDelta appends Delta to the thinking block at BlockIndex.

type EventThinkingEnd

type EventThinkingEnd struct {
	BlockIndex int
	Signature  string
}

EventThinkingEnd marks the end of the ThinkingBlock. Signature is the opaque provider-supplied token to round-trip on follow-up messages.

type EventThinkingStart

type EventThinkingStart struct {
	BlockIndex int
}

EventThinkingStart marks the beginning of a ThinkingBlock.

type EventToolCallDelta

type EventToolCallDelta struct {
	BlockIndex int
	Delta      string
}

EventToolCallDelta delivers a fragment of the streaming JSON arguments. Callers that need the assembled arguments should wait for EventToolCallEnd rather than accumulate Delta bytes themselves (provider JSON delta framing is not guaranteed to be at value boundaries).

type EventToolCallEnd

type EventToolCallEnd struct {
	BlockIndex int
	Arguments  json.RawMessage
}

EventToolCallEnd marks the end of a ToolCallBlock and carries the assembled arguments.

type EventToolCallStart

type EventToolCallStart struct {
	BlockIndex int
	ID         string
	Name       string
}

EventToolCallStart marks the beginning of a ToolCallBlock. ID and Name are available immediately; arguments stream as deltas and are emitted in fully assembled form on EventToolCallEnd.

type ImageBlock

type ImageBlock struct {
	// Data is the raw base64-encoded image bytes. Do NOT include the
	// "data:<mime>;base64," prefix — providers add it where required.
	Data string

	// MimeType is the image's MIME type (e.g. "image/png"). Required.
	MimeType string
}

ImageBlock holds image data for multimodal input. Data is the raw base64-encoded image bytes (no "data:" URI prefix); MimeType is the standard MIME identifier (e.g. "image/png"). Providers convert to their on-wire format at the boundary.

pi-llm-go does NOT fetch image URLs. Callers that want to attach a remote image must download it themselves first, then construct an ImageBlock with the resulting bytes encoded. This keeps the library network-free except for the LLM provider call itself — no surprise timeouts, no surprise 404s mid-stream.

Portable MIME types accepted by every built-in provider:

• "image/jpeg"
• "image/png"
• "image/gif"
• "image/webp"

Other types may work on specific providers (e.g. OpenAI accepts more) but pi-llm-go does not pre-validate — the provider returns an ErrInvalidRequest if the type is unsupported.

v0.3.0 supports ImageBlock as USER-message input only. Assistant image output is provider-specific and a separate, future feature.

func (ImageBlock) Validate

func (i ImageBlock) Validate() error

Validate enforces the ImageBlock contract: Data must be raw base64-encoded bytes (without the "data:<mime>;base64," URI prefix) and MimeType must be set. Providers call this at the wire boundary and surface a wrapped error if the contract is violated.

type LLM

type LLM interface {
	Stream(ctx context.Context, req Request) iter.Seq2[StreamEvent, error]
}

LLM is the provider-agnostic streaming interface. Anthropic and OpenAI providers implement this; third-party providers may do the same to plug into Complete and Accumulate.

Implementations must:

• Honor cancellation of ctx by terminating the underlying HTTP request and yielding (nil, ctx.Err()) from the iterator.
• Surface HTTP errors as *APIError values via the iterator's error half.
• Emit events in the order documented on StreamEvent.

type Message

type Message struct {
	Role    Role
	Content []Block

	Usage      Usage
	StopReason StopReason
	Model      string
}

Message is one turn in the transcript. Content holds a sequence of blocks — the model emits assistant messages with mixed text / thinking / tool-call content; the caller sends user messages with text and tool messages with tool-result content.

Usage, StopReason, and Model are populated on assistant messages produced by Complete or Accumulate; they are zero on user / tool messages and on messages sent into Stream.

func Complete

func Complete(ctx context.Context, l LLM, req Request) (*Message, error)

Complete drains a streaming completion and returns the final assistant message. It is equivalent to iterating Stream and folding each event into a Message via Accumulate.

Returns the partial message and a wrapped error if the stream terminates early; the partial may be useful for debugging or replay.

type Request

type Request struct {
	Model       string
	System      string
	Messages    []Message
	Tools       []Tool
	Temperature *float64
	MaxTokens   int
	Thinking    *ThinkingConfig
	StopReasons []string

	// CacheRetention controls Anthropic prompt-cache breakpoint placement.
	// When unset or "none", no cache markers are emitted. When "short" or
	// "long", the Anthropic provider auto-places ephemeral cache_control
	// markers at the static prefix boundary: the last block of the System
	// prompt, the final Tool in Tools, and the last text block of the most
	// recent user message. "long" additionally selects the 1h TTL and
	// auto-attaches the extended-cache-ttl-2025-04-11 beta header.
	//
	// Ignored by OpenAI providers (their cache is automatic and opaque).
	CacheRetention CacheRetention
}

Request is the common payload for a completion. Provider-specific tunables that have no portable meaning live on the provider's own Options struct (passed to its constructor), not here.

Temperature is a pointer so the zero value can be distinguished from "unset"; callers that want temperature=0 must set *Temperature to 0.

type Role

type Role string

Role enumerates message roles in a transcript. RoleTool messages carry tool results back to the model and may hold only ToolResultBlock content.

const (
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type StopReason

type StopReason string

StopReason is the normalized reason a model stopped generating. Provider-specific stop reasons map to one of these; unmappable values surface as errors rather than leaking provider strings.

const (
	StopReasonEnd       StopReason = "end"        // natural end of turn
	StopReasonMaxTokens StopReason = "max_tokens" // hit MaxTokens cap
	StopReasonToolUse   StopReason = "tool_use"   // model requested tool calls
	StopReasonStop      StopReason = "stop"       // matched a stop sequence
)

type StreamEvent

type StreamEvent interface {
	// contains filtered or unexported methods
}

StreamEvent is the sealed sum type emitted during a streaming completion. Errors flow through the iterator's error half rather than as event values, so consumers do not need an EventError variant.

Event order for one assistant turn:

EventMessageStart
  ( EventTextStart, EventTextDelta*, EventTextEnd
  | EventThinkingStart, EventThinkingDelta*, EventThinkingEnd
  | EventToolCallStart, EventToolCallDelta*, EventToolCallEnd
  )*
EventMessageEnd

BlockIndex on per-block events is the position of the block inside the emitted message's Content slice — it lets consumers route deltas when rendering or reconstructing incrementally.

type TextBlock

type TextBlock struct {
	Text string
}

TextBlock holds plain text content.

type ThinkingBlock

type ThinkingBlock struct {
	Thinking  string
	Signature string
}

ThinkingBlock holds an extended-thinking segment emitted by reasoning models. Signature is an opaque provider-supplied token that must be preserved and replayed for multi-turn thinking continuity (Anthropic).

type ThinkingConfig

type ThinkingConfig struct {
	// BudgetTokens is the maximum number of thinking tokens the model may
	// emit before producing the final response. Required when ThinkingConfig
	// is non-nil. Provider minimums apply (Anthropic: 1024).
	//
	// IMPORTANT: Anthropic requires Request.MaxTokens > BudgetTokens because
	// thinking tokens are counted against max_tokens. A common safe choice
	// is MaxTokens == BudgetTokens * 2, giving roughly equal budget to the
	// reasoning trace and the visible answer.
	BudgetTokens int
}

ThinkingConfig enables extended thinking on supported models. Honored by the Anthropic provider. Ignored by the OpenAI-compatible provider in v1.

type Tool

type Tool struct {
	Name        string
	Description string
	InputSchema json.RawMessage
}

Tool is the wire-level declaration of a callable function exposed to the model. pi-llm-go does not execute tools — it surfaces ToolCallBlocks on the response and accepts ToolResultBlocks in follow-up messages. Execution lives in pi-agent-go.

InputSchema is a JSON Schema document describing the tool's expected input. Both Anthropic and OpenAI accept JSON Schema draft-07; the schema is forwarded to the provider as-is, so the caller is responsible for dialect choice.

type ToolCallBlock

type ToolCallBlock struct {
	ID        string
	Name      string
	Arguments json.RawMessage
}

ToolCallBlock represents a tool invocation requested by the model. Arguments is the raw JSON object the model emitted, matching the tool's declared InputSchema. The agent layer validates and dispatches.

type ToolResultBlock

type ToolResultBlock struct {
	ToolCallID string
	Content    string
	IsError    bool
}

ToolResultBlock carries the result of a tool invocation back to the model. ToolCallID matches the ID on the originating ToolCallBlock.

type Usage

type Usage struct {
	InputTokens      int
	OutputTokens     int
	CacheReadTokens  int
	CacheWriteTokens int
	TotalTokens      int
}

Usage records token accounting returned by a provider for a single completion request. Cache fields are zero when the provider doesn't bill or report cache reads/writes separately.

type VideoBlock

type VideoBlock struct {
	// Data is the raw base64-encoded video bytes for inline emission.
	// Do NOT include the "data:" URI prefix. Mutually exclusive with URI.
	Data string

	// URI is a pre-uploaded reference (Files API handle, YouTube URL,
	// or provider-specific URI). Mutually exclusive with Data.
	URI string

	// MimeType is the video's MIME type (e.g. "video/mp4"). Required
	// when Data is set; optional when only URI is set.
	MimeType string

	// StartOffset, when non-nil, clips the start of the segment. The
	// provider must support clipping; Gemini does via videoMetadata.
	StartOffset *time.Duration

	// EndOffset, when non-nil, clips the end of the segment.
	EndOffset *time.Duration

	// FPS, when non-nil, overrides the provider's default sampling rate.
	// Gemini defaults to 1 FPS (1 video frame per second analyzed);
	// pass a higher value for action-dense content or lower for long
	// static footage. Float for fractional rates (0.5 = 1 frame per 2s).
	FPS *float64
}

VideoBlock holds video data for multimodal input. Today only the built-in Gemini provider accepts video natively; Anthropic and OpenAI providers reject VideoBlock at the wire boundary (callers wanting video understanding on those providers must extract frames client-side and submit them as ImageBlocks).

VideoBlock has three mutually-exclusive emission shapes:

• **Data + MimeType set**: inline base64. Total request body must stay under the provider's inline cap (Gemini: ~20 MB).
• **URI set**: a pre-uploaded reference. For Gemini, this is either an `https://generativelanguage.googleapis.com/v1beta/files/...` handle from the Files API (see providers/gemini/files) or a YouTube URL (public videos only; free-tier 8h/day cap).
• Exactly one of (Data, URI) must be non-empty; both empty or both set is a contract violation rejected by Validate().

Optional StartOffset, EndOffset, and FPS let callers clip the segment and override Gemini's default 1 FPS sampling. nil = use the provider's default. FPS is float for fractional rates (e.g. 0.5).

MimeType uses standard video MIME identifiers: video/mp4, video/quicktime, video/webm, video/mpeg, etc. Required when Data is set; ignored when only URI is set (server infers from the file).

func (VideoBlock) Validate

func (v VideoBlock) Validate() error

Validate enforces the VideoBlock contract:

• exactly one of (Data, URI) must be non-empty
• Data must not carry a "data:" URI prefix (raw base64 only)
• MimeType is required when Data is set