llm

package
v0.0.0-...-20cd8ba Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 30, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrChannelClosed = errors.New("delta channel closed unexpectedly")

ErrChannelClosed is returned when an internal helper sees a channel close unexpectedly. Not used by CollectStream directly (which returns ErrFinalMissing), but exposed for custom consumers.

View Source 
var ErrFinalMissing = errors.New("stream closed without Final delta")

ErrFinalMissing is returned by stream consumers when a channel was closed without a Final marker. It indicates a provider bug.

View Source 
var ErrInvalidToolSchema = errors.New("invalid tool schema")

ErrInvalidToolSchema is returned by Validate when a ToolSchema is malformed (empty name, parameters not a JSON object, parameters with type!=object).

View Source 
var ErrSSEClosed = errors.New("sse: stream closed without final event")

ErrSSEClosed is returned by ReadSSE when the underlying reader returns io.EOF without a terminal blank line.

View Source 
var ErrUnknownContentBlock = errors.New("unknown content block type")

ErrUnknownContentBlock is returned by UnmarshalContentBlock when the JSON "type" tag is missing or unknown.

Functions

func IsAbort 

func IsAbort(err error) bool

IsAbort reports whether err is context.Canceled or context.DeadlineExceeded. Providers wrap aborts in Final.Err; the agent loop uses this helper to distinguish aborts from provider errors.

func IsRetryableStatus 

func IsRetryableStatus(status int) bool

IsRetryableStatus reports whether status is in the retryable set.

func IsTransientNetworkError 

func IsTransientNetworkError(err error) bool

IsTransientNetworkError reports whether err is a network-layer error that providers should mark retryable via NewRetryableError.

func NewRetryableError 

func NewRetryableError(err error) error

NewRetryableError wraps err as retryable. Returns nil if err is nil.

func ReadSSE 

func ReadSSE(r io.Reader, fn func(SSEEvent) error) error

ReadSSE scans r as an SSE stream and emits events to fn. Returns nil on a clean EOF (the stream ended with a complete event followed by EOF). The caller MUST close r after ReadSSE returns.

fn is invoked for each complete event. If fn returns an error, ReadSSE stops and returns that error.

Types

type ContentBlock 

type ContentBlock interface {
	// contains filtered or unexported methods
}

ContentBlock is a sealed interface: only the five concrete types in this file implement it (the unexported isContentBlock method seals it). Variants:

  • TextContent plain text
  • ThinkingContent extended-thinking output (reasoning models)
  • ImageContent base64 image
  • ToolUse model requests a tool call
  • ToolResult result of a previous ToolUse

The interface value is the persistence shape — it round-trips through JSON without loss via the type discriminator.

func UnmarshalContentBlock 

func UnmarshalContentBlock(data []byte) (ContentBlock, error)

UnmarshalContentBlock decodes one JSON object into the appropriate ContentBlock variant based on the "type" discriminator. It is the single entry point for parsing the tagged union from any context (top-level Message.Content, nested ToolResult.Content, fixtures, etc.).

type Cost 

type Cost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cacheRead"`
	CacheWrite float64 `json:"cacheWrite"`
	Total      float64 `json:"total"`
}

Cost is the dollar cost of a single turn. All fields are in USD.

type Delta 

type Delta interface {
	// contains filtered or unexported methods
}

Delta is a sealed interface: only the five concrete types in this file implement it (the unexported isDelta method seals it). Variants:

  • TextDelta incremental assistant text
  • ThinkingDelta incremental thinking text
  • ToolCallDelta incremental tool-call input JSON for a given tool-use id
  • UsageDelta per-turn token accounting (emitted once)
  • Final terminal marker carrying StopReason and optional error

Concatenation: TextDelta.Text fragments concatenate in order to form the assistant text. ToolCallDelta.PartialInput fragments with the same ID concatenate to form the full tool-input JSON (which is then parsed).

type Final 

type Final struct {
	StopReason    StopReason
	ResponseID    string
	ResponseModel string
	Err           error
}

Final is the terminal marker. The provider MUST emit exactly one Final after all other deltas, then close the channel.

  • StopReason is set to StopReasonError if Err is non-nil; the converse is not guaranteed (StopReasonError may have a nil Err when the provider could not classify the failure).
  • ResponseID carries the provider's response/message id when available (e.g., Anthropic `msg_…`, OpenAI `chatcmpl_…`).
  • ResponseModel is the concrete model that served the request when the provider returns a different id (e.g., OpenRouter auto-routing).

type HTTPStatusError 

type HTTPStatusError struct {
	Status int
	Body   string // body preview captured on demand (empty if not requested)
}

HTTPStatusError is returned when a request fails with a non-retryable HTTP status code. The body has already been consumed and discarded.

func (*HTTPStatusError) Error 

func (e *HTTPStatusError) Error() string

Error implements error.

type ImageContent 

type ImageContent struct {
	Data     string `json:"data"`
	MimeType string `json:"mimeType"`
}

ImageContent is a base64-encoded image. Data MUST NOT include a "data:" URL prefix — the provider marshaler adds the prefix on the wire if needed.

func (ImageContent) MarshalJSON 

func (i ImageContent) MarshalJSON() ([]byte, error)

MarshalJSON emits ImageContent as {"type":"image","data":"...","mimeType":"..."}.

func (*ImageContent) UnmarshalJSON 

func (i *ImageContent) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts the same shape produced by MarshalJSON.

type LLMClient 

type LLMClient interface {
	// Stream issues a streaming completion request. See the package comment
	// for the channel protocol.
	Stream(ctx context.Context, req Request) (<-chan Delta, error)
}

LLMClient is the provider abstraction. Implementations MUST be safe for concurrent use; the agent loop may issue parallel Stream calls in tests and via plugins.

type Message 

type Message struct {
	Role       Role           `json:"role"`
	Content    []ContentBlock `json:"content"`
	ToolCallID string         `json:"toolCallId,omitempty"`
	ToolName   string         `json:"toolName,omitempty"`
	Model      string         `json:"model,omitempty"`
	ProviderID string         `json:"providerId,omitempty"`
	ResponseID string         `json:"responseId,omitempty"`
	Usage      *Usage         `json:"usage,omitempty"`
	StopReason StopReason     `json:"stopReason,omitempty"`
	Timestamp  time.Time      `json:"timestamp"`
}

Message is a single conversational turn.

Field usage by Role:

  • RoleSystem: Content = [TextContent{...}] (one block, no more)
  • RoleUser: Content = [TextContent, ImageContent, ToolResult, ...]
  • RoleAssistant: Content = [TextContent, ThinkingContent, ToolUse, ...] Model/ProviderID/ResponseID/Usage/StopReason set
  • RoleTool: Content = [ToolResult{...}] (or [TextContent{...}]) ToolCallID and ToolName set

The Marshalers in internal/llm/provider/{anthropic,openai}/ read these fields and translate to the provider's wire format. OpenAI uses RoleTool + ToolCallID directly; Anthropic inlines ToolResult as a user-role content block.

func CollectStream 

func CollectStream(ctx context.Context, ch <-chan Delta, model, providerID string) (Message, error)

CollectStream drains a Delta channel and returns the assembled Message. Returns an error if:

  • the channel closed without a Final (ErrFinalMissing),
  • Accumulate returned an error for any delta,
  • Final.Err was set.

The caller's ctx bounds the wait. If ctx is cancelled before the channel closes, the partial Message assembled so far is discarded and ctx.Err() is returned.

func (Message) MarshalJSON 

func (m Message) MarshalJSON() ([]byte, error)

MarshalJSON encodes Message, dispatching each ContentBlock to its concrete MarshalJSON so the type discriminator is emitted.

func (*Message) UnmarshalJSON 

func (m *Message) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes Message, dispatching each Content block via UnmarshalContentBlock.

type Request 

type Request struct {
	Model          string            // provider-specific model id
	System         []ContentBlock    // system prompt blocks (typically [TextContent])
	Messages       []Message         // conversation turns
	Tools          []ToolSchema      // tool schemas offered this turn
	ThinkingBudget *int              // 0 disables thinking; nil = provider default
	MaxTokens      *int              // hard output cap; nil = provider default
	Temperature    *float64          // 0..2; nil = provider default
	Stop           []string          // stop sequences
	Transport      Transport         // SSE / WebSocket / auto
	Headers        map[string]string // extra headers merged after auth
}

Request is the provider-agnostic request shape.

Fields are pointers where zero needs to be distinguishable from unset (MaxTokens, Temperature, ThinkingBudget). The System slice is rendered as the provider expects (Anthropic: top-level system blocks; OpenAI: a role="system" message prepended to the conversation).

type RetryPolicy 

type RetryPolicy struct {
	// MaxRetries is the number of retry attempts after the initial request.
	// Zero disables retries. Default 4.
	MaxRetries int
	// BaseDelay is the initial backoff interval. Default 500ms.
	BaseDelay time.Duration
	// MaxDelay caps the backoff interval. Default 30s.
	MaxDelay time.Duration
}

RetryPolicy controls the behavior of Do.

func DefaultRetryPolicy 

func DefaultRetryPolicy() RetryPolicy

DefaultRetryPolicy returns the policy matching the llm-client spec (MaxRetries=4, MaxDelay=30s, BaseDelay=500ms).

type RetryResult 

type RetryResult struct {
	Attempts  int           // total requests issued (1 + retries)
	WaitTotal time.Duration // sum of pre-retry sleeps
	LastErr   error         // nil if last attempt succeeded
	Status    int           // HTTP status of last attempt (0 if transport-level)
}

Err is the terminal error returned by Do after retries are exhausted or a non-retryable error occurs. Use errors.As to extract.

func Do 

func Do(ctx context.Context, policy RetryPolicy, send SendFunc) (*http.Response, *RetryResult, error)

Do issues the request via send, retrying per policy on retryable failures. The caller's ctx bounds the total wait.

On success returns the final *http.Response and a nil error. The caller must close resp.Body.

On failure returns a nil response and a non-nil error (which may be a *RetryableError wrapping the last transport error, an *HTTPStatusError for a non-retryable status, or ctx.Err() if the context was cancelled).

type RetryableError 

type RetryableError struct {
	Err error
}

RetryableError marks a network-layer error as eligible for retry. HTTP-status retries are decided by status code; this is for the transport layer (DNS, connection, EOF, etc.).

func (*RetryableError) Error 

func (e *RetryableError) Error() string

Error implements error.

func (*RetryableError) Unwrap 

func (e *RetryableError) Unwrap() error

Unwrap exposes the wrapped error for errors.Is / errors.As.

type Role 

type Role string

Role identifies the speaker of a Message. 

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type SSEEvent 

type SSEEvent struct {
	// Type is the value of the "event:" line, or "message" if omitted
	// (the SSE default per the spec).
	Type string
	// Data is the concatenated payload (one line per "data:" line).
	Data string
}

SSEEvent is one decoded Server-Sent Event.

type SendFunc 

type SendFunc func(ctx context.Context) (*http.Response, error)

SendFunc issues one HTTP request and returns the response or an error. The response body MUST be closed by the caller (or by tryDecode) — the retry loop reads the body only to inspect error payloads.

type StopReason 

type StopReason string

StopReason is why the model stopped generating this turn. 

const (
	StopReasonEndTurn StopReason = "stop"
	StopReasonLength  StopReason = "length"
	StopReasonToolUse StopReason = "toolUse"
	StopReasonError   StopReason = "error"
	StopReasonAborted StopReason = "aborted"
)

type StreamAccumulator 

type StreamAccumulator struct {
	// contains filtered or unexported fields
}

StreamAccumulator consumes a sequence of Delta values and produces an assistant Message. The zero value is NOT ready to use — call NewAccumulator.

Accumulator is NOT safe for concurrent use. The agent loop owns a single goroutine that reads from the delta channel and calls Accumulate.

func NewAccumulator 

func NewAccumulator(model, providerID string) *StreamAccumulator

NewAccumulator returns a ready-to-use StreamAccumulator. model and providerID are stamped on the resulting Message (use the model id the request was issued with; ResponseModel from Final overrides this if set).

func (*StreamAccumulator) Accumulate 

func (a *StreamAccumulator) Accumulate(d Delta) error

Accumulate processes one delta. Returns an error if the delta violates the protocol (e.g., a delta arriving after Final, or a malformed tool fragment). Once Final is seen, subsequent deltas are rejected.

func (*StreamAccumulator) Err 

func (a *StreamAccumulator) Err() error

Err returns the error from Final.Err, or nil. Convenience for callers that only care whether the stream ended cleanly.

func (*StreamAccumulator) Message 

func (a *StreamAccumulator) Message() Message

Message returns the assistant Message assembled so far. May be called at any point; the returned Message is a snapshot and safe to retain.

Calling Message before Final is seen produces a partial message — useful for live TUI updates. After Final it produces the persisted shape.

type TextContent 

type TextContent struct {
	Text string `json:"text"`
}

TextContent is plain text.

func (TextContent) MarshalJSON 

func (t TextContent) MarshalJSON() ([]byte, error)

MarshalJSON emits TextContent as {"type":"text","text":"..."}.

func (*TextContent) UnmarshalJSON 

func (t *TextContent) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts the same shape produced by MarshalJSON.

type TextDelta 

type TextDelta struct {
	// ContentIndex is the position of the text block within the assistant
	// message. The agent loop uses this to write into the correct slot when
	// the model interleaves text and tool_use blocks.
	ContentIndex int
	Text         string
}

TextDelta carries an incremental fragment of assistant text output.

type ThinkingContent 

type ThinkingContent struct {
	Thinking  string `json:"thinking"`
	Signature string `json:"signature,omitempty"`
	Redacted  bool   `json:"redacted,omitempty"`
}

ThinkingContent is the output of an extended-thinking ("reasoning") block. Signature is an opaque provider-specific token used for multi-turn continuity (Anthropic signature, OpenAI reasoning item id). When Redacted is true the model declined to share the reasoning text; the opaque payload is in Signature so subsequent turns still see it.

func (ThinkingContent) MarshalJSON 

func (t ThinkingContent) MarshalJSON() ([]byte, error)

MarshalJSON emits ThinkingContent as {"type":"thinking",...}.

func (*ThinkingContent) UnmarshalJSON 

func (t *ThinkingContent) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts the same shape produced by MarshalJSON.

type ThinkingDelta 

type ThinkingDelta struct {
	ContentIndex int
	Text         string
	// Signature is non-empty only when the provider includes one with this
	// fragment (Anthropic signature_delta events).
	Signature string
}

ThinkingDelta carries an incremental fragment of thinking text.

type ToolCallDelta 

type ToolCallDelta struct {
	// ContentIndex is the position of the tool_use block within the
	// assistant message.
	ContentIndex int
	// ID is the tool-use id assigned by the provider. The first delta for a
	// given content index carries the id; subsequent deltas for the same
	// content index repeat it.
	ID string
	// Name is the tool name. Only set on the first delta for a content
	// index; consumers should latch it on first sight.
	Name string
	// PartialInput is a fragment of the input JSON. Concatenate in order.
	PartialInput string
}

ToolCallDelta carries an incremental fragment of a tool-call's input JSON. The provider streams the JSON arguments in chunks; the consumer concatenates PartialInput for matching ID to recover the full payload.

type ToolResult 

type ToolResult struct {
	ToolUseID string         `json:"toolUseId"`
	Content   []ContentBlock `json:"content"`
	IsError   bool           `json:"isError,omitempty"`
}

ToolResult is the result of executing a ToolUse. Content typically holds a single TextContent; the slice shape matches Anthropic's tool_result.content which can carry multiple blocks (text + image). When IsError is true the provider renders the result as a tool-call failure to the model.

func (ToolResult) MarshalJSON 

func (r ToolResult) MarshalJSON() ([]byte, error)

MarshalJSON emits ToolResult as {"type":"toolResult",...} with nested content blocks recursively tagged.

func (*ToolResult) UnmarshalJSON 

func (r *ToolResult) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts the same shape produced by MarshalJSON.

type ToolSchema 

type ToolSchema struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Parameters  json.RawMessage `json:"parameters"`
}

ToolSchema describes one tool to the model. It mirrors the fields a tool implementation publishes: Name, Description, and a JSON Schema for parameters. The schema is encoded as raw JSON so any schema source (Go struct tags via invopop/jsonschema, hand-written maps, dynamic plugin schemas) plugs in without conversion.

func NewToolSchemaFromJSON 

func NewToolSchemaFromJSON(name, description, paramsJSON string) (ToolSchema, error)

NewToolSchemaFromJSON builds a ToolSchema from a hand-written JSON Schema string. Useful in tests and for plugins that ship schemas verbatim.

func NewToolSchemaFromStruct 

func NewToolSchemaFromStruct[T any](name, description string, sample T) (ToolSchema, error)

NewToolSchemaFromStruct builds a ToolSchema by reflecting over an example value of T. The struct MUST have json tags or jsonschema tags; fields without tags get their Go field name. This is the primary constructor for built-in tools whose parameters are typed Go structs.

func (ToolSchema) ParameterMap 

func (t ToolSchema) ParameterMap() (map[string]any, error)

ParameterMap returns Parameters as a parsed map. Provider marshalers use this when they need to mutate the schema before serialization (e.g., add "additionalProperties": false for OpenAI strict mode).

func (ToolSchema) Validate 

func (t ToolSchema) Validate() error

Validate enforces the minimum invariants all providers expect:

  • Name is non-empty
  • Parameters parses as a JSON object
  • Parameters.type is "object" (or omitted, which we normalize to "object")

type ToolUse 

type ToolUse struct {
	ID    string          `json:"id"`
	Name  string          `json:"name"`
	Input json.RawMessage `json:"input"`
}

ToolUse is the model's request to invoke a tool. Input is the raw JSON arguments payload from the model; callers validate it against the tool's parameter schema before execution.

func (ToolUse) MarshalJSON 

func (u ToolUse) MarshalJSON() ([]byte, error)

MarshalJSON emits ToolUse as {"type":"toolUse",...}.

func (*ToolUse) UnmarshalJSON 

func (u *ToolUse) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts the same shape produced by MarshalJSON.

type Transport 

type Transport string

Transport selects the streaming transport for a single request. 

const (
	TransportAuto      Transport = "auto"
	TransportSSE       Transport = "sse"
	TransportWebSocket Transport = "websocket"
)

type Usage 

type Usage struct {
	Input       int  `json:"input"`
	Output      int  `json:"output"`
	CacheRead   int  `json:"cacheRead"`
	CacheWrite  int  `json:"cacheWrite"`
	TotalTokens int  `json:"totalTokens"`
	Cost        Cost `json:"cost"`
}

Usage reports per-turn token accounting. The agent loop reads Usage to drive compaction decisions. Cost is filled in by the provider when the model definition in models.json carries nonzero Cost rates.

type UsageDelta 

type UsageDelta struct {
	// InputTokens counts the prompt tokens.
	InputTokens int
	// OutputTokens counts the generated tokens.
	OutputTokens int
	// CacheReadTokens is the count of input tokens served from a prompt
	// cache (Anthropic-only; zero for providers without prompt caching).
	CacheReadTokens int
	// CacheWriteTokens is the count of input tokens written to the prompt
	// cache (Anthropic-only).
	CacheWriteTokens int
}

UsageDelta carries per-turn token accounting. Emitted exactly once per stream, after the last content delta and before Final. Some providers (OpenAI) only emit usage when stream_options.include_usage is set; tau's OpenAI provider always requests it.

Source Files

View all Source files

Directories

Path Synopsis
provider
anthropic
openai
Package tokencounter provides model-aware token counting for the llm layer.
 Package tokencounter provides model-aware token counting for the llm layer.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.