llm

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 28, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package llm defines zkit's provider-neutral language-model contract.

The core abstraction is Provider: a deliberately narrow interface with one streaming completion method and one name method. Provider implementations live in subpackages such as openai, anthropic, google, ollama, llamacpp, deepseek, claudecode, and openaicodex. Consumers should depend on this package's request, response, tool-call, response-format, and streaming types rather than on provider SDK DTOs.

Streaming providers must own and close the returned channel and emit exactly one terminal CompletionChunk with Done set. On success the terminal chunk carries final usage/finish metadata when available. On failure the terminal chunk carries Error and Done. This lets runners and shells treat stream completion consistently across providers.

Richer capabilities, such as model discovery or OAuth-backed construction, should remain separate opt-in interfaces or backend helpers rather than widening Provider.

Index

Constants

View Source 
const (
	RoleSystem    = "system"
	RoleUser      = "user"
	RoleAssistant = "assistant"
	RoleTool      = "tool"
)

Message roles, as carried on Message.Role across every provider. The strings match the OpenAI-compatible wire values so histories serialize without translation.

Variables

View Source 
var (
	ErrProviderUnavailable   = errors.New("llm provider unavailable")
	ErrInvalidAPIKey         = errors.New("invalid api key")
	ErrModelNotSupported     = errors.New("model not supported")
	ErrRateLimitExceeded     = errors.New("rate limit exceeded")
	ErrContextLengthExceeded = errors.New("context length exceeded")
)

Common errors.

View Source 
var ErrParseLLMProvider = errors.New("invalid input provided to parse to LLMProvider")
View Source 
var ErrParseReasoningHistory = errors.New("invalid input provided to parse to ReasoningHistory")
View Source 
var LLMProviders = llmProvidersContainer{
	OPENAI: LLMProvider{
		// contains filtered or unexported fields
	},
	DEEPSEEK: LLMProvider{
		// contains filtered or unexported fields
	},
	OPENAICODEX: LLMProvider{
		// contains filtered or unexported fields
	},
	GOOGLE: LLMProvider{
		// contains filtered or unexported fields
	},
	ANTHROPIC: LLMProvider{
		// contains filtered or unexported fields
	},
	CLAUDECODE: LLMProvider{
		// contains filtered or unexported fields
	},
	LLAMACPP: LLMProvider{
		// contains filtered or unexported fields
	},
	OLLAMA: LLMProvider{
		// contains filtered or unexported fields
	},
}

LLMProviders is a main entry point using the LLMProvider type. It it a container for all enum values and provides a convenient way to access all enum values and perform operations, with convenience methods for common use cases.

View Source 
var ReasoningHistories = reasoningHistoriesContainer{
	INLINE: ReasoningHistory{
		// contains filtered or unexported fields
	},
	FIELD: ReasoningHistory{
		// contains filtered or unexported fields
	},
	STRIP: ReasoningHistory{
		// contains filtered or unexported fields
	},
}

ReasoningHistories is a main entry point using the ReasoningHistory type. It it a container for all enum values and provides a convenient way to access all enum values and perform operations, with convenience methods for common use cases.

Functions

func ExhaustiveLLMProviders 

func ExhaustiveLLMProviders(f func(LLMProvider))

ExhaustiveLLMProviders iterates over all enum values and calls the provided function for each value. This function is useful for performing operations on all valid enum values in a loop.

func ExhaustiveReasoningHistories 

func ExhaustiveReasoningHistories(f func(ReasoningHistory))

ExhaustiveReasoningHistories iterates over all enum values and calls the provided function for each value. This function is useful for performing operations on all valid enum values in a loop.

func IsRateLimitError added in v0.2.0 

func IsRateLimitError(err error) bool

Types

type AudioData 

type AudioData struct {
	DataURI string `json:"data_uri,omitempty"`
	Format  string `json:"format,omitempty"`
}

AudioData is the audio payload for an audio content part. Format is the codec hint, e.g. "wav", "mp3". DataURI must be a base64 data URI.

type ChatTemplateKwargs added in v0.2.1 

type ChatTemplateKwargs struct {
	EnableThinking   bool `json:"enable_thinking"`
	PreserveThinking bool `json:"preserve_thinking,omitempty"`
}

ChatTemplateKwargs is the typed payload serialized into the non-standard chat_template_kwargs request extension used by llama.cpp/vLLM chat templates. Providers render it to raw JSON only at the transport edge.

func (ChatTemplateKwargs) AsMap added in v0.2.1 

func (k ChatTemplateKwargs) AsMap() map[string]any

AsMap renders k as a generic map for transports that pass arbitrary JSON extension fields. It returns nil when no fields are set.

func (ChatTemplateKwargs) IsZero added in v0.2.1 

func (k ChatTemplateKwargs) IsZero() bool

IsZero reports whether k carries no template overrides.

type CompletionChunk 

type CompletionChunk struct {
	Content      string
	Thinking     string
	FinishReason string // "stop", "length", "error"
	Done         bool
	// Error is NOT the streaming error channel. Provider errors ride the
	// iter.Seq2's second return value, which the consumer (runner drain)
	// reads; this field is populated only by the conformance test harness
	// (folding the yield error in for legacy assertions). A provider that
	// sets it directly will have it silently ignored — don't.
	Error     error
	ToolCalls []ToolCall // Function calls made by the LLM (transport format)

	// Usage information (only in final chunk)
	Usage *Usage
}

CompletionChunk represents a piece of streaming response.

Content and Thinking are the two output channels, and they are disjoint by contract: for any given byte of model output, exactly one of them carries it. Content holds the visible answer; Thinking holds reasoning / extended-thinking / chain-of-thought, routed out-of-band so consumers can render it in a dedicated surface. Every provider (Anthropic extended thinking, DeepSeek/OpenAI reasoning_content, Gemini thought parts, openaicodex reasoning events) lands on this contract — there is no inline-tag escape hatch at the provider boundary.

type CompletionRequest 

type CompletionRequest struct {
	Messages    []Message
	Temperature float32
	MaxTokens   int
	Stream      bool
	Tools       []Tool // Function calling tools available to the LLM

	// ChatTemplateKwargs is the llama.cpp / vLLM extension field that
	// gets serialised as `chat_template_kwargs` on the wire. Providers
	// that don't recognise it ignore it. The runner builds this from the
	// active ChatTemplate's ThinkingKwargs each request.
	ChatTemplateKwargs ChatTemplateKwargs

	// ResponseFormat constrains the model's output shape. When set to
	// a JSONSchema variant, llama.cpp converts the schema to a GBNF
	// grammar and constrains sampling so the model literally cannot
	// emit a token that violates the schema — including invented enum
	// values. The OpenAI hosted API enforces structured output the
	// same way when strict=true; Anthropic maps it to its native
	// structured-outputs config and Gemini to responseJsonSchema. The
	// claude-code CLI can't grammar-constrain, so it falls back to a
	// prompt directive.
	//
	// Leave zero-valued for free-form text output.
	ResponseFormat ResponseFormat

	// Thinking requests the model's extended reasoning for this request.
	// Each provider maps it to its native mechanism — Anthropic's thinking
	// budget, Gemini's thinking config, OpenAI/codex reasoning effort,
	// llama.cpp's chat_template_kwargs. Providers that surface reasoning
	// unconditionally (Gemini, the OpenAI-compatible reasoning_content
	// path) ignore the toggle. Zero value leaves the provider default.
	Thinking ThinkingConfig

	// Provider-specific options
	Options ModelOptions
}

CompletionRequest represents a request to generate text.

type ContentPart 

type ContentPart struct {
	Type ContentPartType `json:"type"`

	// Text is set when Type == ContentTypeText.
	Text string `json:"text,omitempty"`

	// Image is set when Type == ContentTypeImage.
	Image *ImageData `json:"image,omitempty"`

	// Audio is set when Type == ContentTypeAudio.
	Audio *AudioData `json:"audio,omitempty"`

	// Video is set when Type == ContentTypeVideo.
	Video *VideoData `json:"video,omitempty"`
}

ContentPart is one element of a multimodal Message.Parts slice. Exactly one of the typed fields (Text/Image/Audio/Video) is meaningful per instance; Type tells you which.

func ImagePartFromDataURI 

func ImagePartFromDataURI(dataURI, mime string) ContentPart

ImagePartFromDataURI is a convenience constructor for an image ContentPart backed by a base64 data URI.

func ImagePartFromURL 

func ImagePartFromURL(url string) ContentPart

ImagePartFromURL is a convenience constructor for an image ContentPart referenced by remote URL.

func TextPart 

func TextPart(text string) ContentPart

TextPart is a convenience constructor for a text ContentPart.

func VideoPartFromDataURI added in v0.3.0 

func VideoPartFromDataURI(dataURI, mime string) ContentPart

VideoPartFromDataURI is a convenience constructor for a video ContentPart backed by a base64 data URI.

func VideoPartFromURL added in v0.3.0 

func VideoPartFromURL(url string) ContentPart

VideoPartFromURL is a convenience constructor for a video ContentPart referenced by remote URL.

type ContentPartType 

type ContentPartType string

ContentPartType discriminates which kind of payload a ContentPart carries. Adding a new modality is a matter of adding a constant and the matching field. 

const (
	ContentTypeText  ContentPartType = "text"
	ContentTypeImage ContentPartType = "image"
	ContentTypeAudio ContentPartType = "audio"
	ContentTypeVideo ContentPartType = "video"
)

The supported content-part modalities; each selects the matching typed field on ContentPart.

type ImageData 

type ImageData struct {
	URL      string `json:"url,omitempty"`
	DataURI  string `json:"data_uri,omitempty"`
	MIMEType string `json:"mime_type,omitempty"`

	// Detail is OpenAI-specific: "low" | "high" | "auto" (default).
	// Other providers ignore it.
	Detail string `json:"detail,omitempty"`
}

ImageData is the image payload for an image content part. Either URL (remote http(s) URL) or DataURI ("data:image/png;base64,...") must be set; providers prefer DataURI when both are present.

type LLMProvider 

type LLMProvider struct {
	// contains filtered or unexported fields
}

LLMProvider is a type that represents a single enum value. It combines the core information about the enum constant and it's defined fields.

func ParseLLMProvider 

func ParseLLMProvider(input any) (LLMProvider, error)

ParseLLMProvider parses the input value into an enum value. It returns the parsed enum value or an error if the input is invalid. It is a convenience function that can be used to parse enum values from various input types, such as strings, byte slices, or other enum types.

func (LLMProvider) IsValid 

func (l LLMProvider) IsValid() bool

IsValid checks whether the LLMProviders value is valid. A valid value is one that is defined in the original enum and not marked as invalid.

func (LLMProvider) MarshalBinary 

func (l LLMProvider) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface for LLMProvider. It returns the binary representation of the enum value as a byte slice.

func (LLMProvider) MarshalJSON 

func (l LLMProvider) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for LLMProvider. It returns the JSON representation of the enum value as a byte slice.

func (LLMProvider) MarshalText 

func (l LLMProvider) MarshalText() ([]byte, error)

MarshalText implements the encoding.TextMarshaler interface for LLMProvider. It returns the string representation of the enum value as a byte slice

func (LLMProvider) MarshalYAML 

func (l LLMProvider) MarshalYAML() ([]byte, error)

MarshalYAML implements the yaml.Marshaler interface for LLMProvider. It returns the string representation of the enum value.

func (*LLMProvider) Scan 

func (l *LLMProvider) Scan(value any) error

Scan implements the database/sql.Scanner interface for LLMProvider. It parses the string representation of the enum value from the database row. It returns an error if the row does not contain a valid enum value.

func (LLMProvider) String 

func (l LLMProvider) String() string

String implements the Stringer interface. It returns the canonical absolute name of the enum value.

func (*LLMProvider) UnmarshalBinary 

func (l *LLMProvider) UnmarshalBinary(by []byte) error

UnmarshalBinary implements the encoding.BinaryUnmarshaler interface for LLMProvider. It parses the binary representation of the enum value from the byte slice. It returns an error if the byte slice does not contain a valid enum value.

func (*LLMProvider) UnmarshalJSON 

func (l *LLMProvider) UnmarshalJSON(by []byte) error

UnmarshalJSON implements the json.Unmarshaler interface for LLMProvider. It parses the JSON representation of the enum value from the byte slice. It returns an error if the input is not a valid JSON representation.

func (*LLMProvider) UnmarshalText 

func (l *LLMProvider) UnmarshalText(by []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface for LLMProvider. It parses the string representation of the enum value from the byte slice. It returns an error if the byte slice does not contain a valid enum value.

func (*LLMProvider) UnmarshalYAML 

func (l *LLMProvider) UnmarshalYAML(by []byte) error

UnmarshalYAML implements the yaml.Unmarshaler interface for Planet. It parses the byte slice representation of the enum value and returns an error if the YAML byte slice does not contain a valid enum value.

func (LLMProvider) Value 

func (l LLMProvider) Value() (driver.Value, error)

Value implements the database/sql/driver.Valuer interface for LLMProvider. It returns the string representation of the enum value.

type Message 

type Message struct {
	Role    string `json:"role"` // Use messages.RoleSystem, messages.RoleUser, etc.
	Content string `json:"content"`

	// ReasoningContent stores an assistant turn's reasoning out-of-band
	// from Content. The runner populates it from CompletionChunk.Thinking
	// at end-of-turn; per-provider history serializers reshape it back
	// onto the wire (openai's ReasoningInline re-wraps it as
	// `<think>…</think>`, ReasoningField forwards it via the
	// reasoning_content extra field, ReasoningStrip drops it). Disjoint
	// from Content — see CompletionChunk's docstring for the channel
	// contract.
	ReasoningContent string `json:"reasoning_content,omitempty"`

	// Parts carries multimodal content (text + image + audio + video) for
	// vision/audio/video-capable models. When non-nil, providers SHOULD send
	// Parts in preference to Content; if a provider doesn't support
	// multimodal, it falls back to flattening the Text parts to
	// Content. Parts is typically only set on role="user" messages —
	// assistant turns return text via Content.
	Parts []ContentPart `json:"parts,omitempty"`

	// ToolCalls is set on assistant messages that requested tool
	// invocations. Carrying these in the conversation history is what
	// lets the model see "I already called this tool" rather than
	// re-emitting the same call on every iteration.
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`

	// ToolCallID is set on role="tool" messages and matches the ID of
	// the assistant's tool call this message is responding to. OpenAI
	// (and llama.cpp's OAI shim) reject tool messages without it.
	ToolCallID string `json:"tool_call_id,omitempty"`
}

Message represents a single message in a conversation. Includes vision/audio/video-capable parts for user input.

type Model 

type Model struct {
	ID          string
	Name        string
	Description string
	MaxTokens   int
	InputCost   float64 // per 1k tokens
	OutputCost  float64 // per 1k tokens

	// Model capabilities metadata
	Capabilities ModelCapabilities `json:"capabilities"`
}

Model represents an available LLM model.

type ModelCapabilities 

type ModelCapabilities struct {
	SupportsStreaming bool `json:"supports_streaming"`
	SupportsVision    bool `json:"supports_vision"`   // Image input processing
	SupportsVideo     bool `json:"supports_video"`    // Video input processing
	SupportsTools     bool `json:"supports_tools"`    // Function/tool calling
	SupportsSystem    bool `json:"supports_system"`   // System messages
	SupportsThinking  bool `json:"supports_thinking"` // DeepSeek R1 style reasoning
}

ModelCapabilities describes what features a specific model supports.

type ModelConfig 

type ModelConfig struct {
	Provider string `json:"provider"` // "openai", "anthropic", "google", etc.
	Model    string `json:"model"`    // "gpt-4", "claude-3-5-sonnet", "gemini-pro", etc.
}

ModelConfig specifies a model with its provider.

type ModelOptions 

type ModelOptions map[string]any

ModelOptions holds semantic model configuration options as a map[string]any.

type ModelPreferences 

type ModelPreferences struct {
	Primary   ModelConfig   `json:"primary"`
	Fallbacks []ModelConfig `json:"fallbacks,omitempty"`
}

ModelPreferences represents model preferences for user configuration.

type PersonalityModifiers 

type PersonalityModifiers map[string]any

PersonalityModifiers holds trait-driven adjustments to LLM behavior.

type Provider 

type Provider interface {
	// Complete streams a completion as an iter.Seq2: each chunk is yielded
	// with a nil error, and a mid-stream failure is the yield's second value
	// (not a field on the chunk). The returned error is a pre-stream setup
	// failure only. Consumers range it: `for chunk, err := range seq`.
	Complete(ctx context.Context, req CompletionRequest) (iter.Seq2[CompletionChunk, error], error)
	Name() string
}

Provider is the minimum contract a backend has to satisfy. It is deliberately narrow: a single streaming completion entry-point plus a name for identification. Anything richer (model discovery, image generation, MCP, thinking-mode toggles) belongs on a separate opt-in interface that consumers type-assert for when they need it.

func Named 

func Named(inner Provider, name string) Provider

Named returns a Provider that delegates to inner but reports name from Name(). Useful when one adapter type (e.g. openai.Provider) is reused under different provider identities — llamacpp and ollama both wrap openai.Provider, and the registry needs their Name() to reflect the wrapper identity rather than "openai".

type RateLimitError added in v0.2.0 

type RateLimitError struct {
	Message    string
	RetryAfter time.Duration
	ResetAt    time.Time
	Permanent  bool
	// Retryable indicates the runner may retry the request if its retry
	// budget permits. Permanent == true takes precedence (never retry).
	// When both Permanent and Retryable are false the error is treated
	// as terminal for backward compatibility with providers that have
	// not yet been updated.
	Retryable bool
}

RateLimitError carries retry-after/reset information from provider 429 (or similar rate-limit) responses. Consumers (runner, TUI) can type-assert the error from RunResult.Err with errors.As to render countdowns, back-pressure warnings, or billing notices.

func (*RateLimitError) Error added in v0.2.0 

func (e *RateLimitError) Error() string

type ReasoningHistory 

type ReasoningHistory struct {
	// contains filtered or unexported fields
}

ReasoningHistory is a type that represents a single enum value. It combines the core information about the enum constant and it's defined fields.

func ParseReasoningHistory 

func ParseReasoningHistory(input any) (ReasoningHistory, error)

ParseReasoningHistory parses the input value into an enum value. It returns the parsed enum value or an error if the input is invalid. It is a convenience function that can be used to parse enum values from various input types, such as strings, byte slices, or other enum types.

func (ReasoningHistory) IsValid 

func (r ReasoningHistory) IsValid() bool

IsValid checks whether the ReasoningHistories value is valid. A valid value is one that is defined in the original enum and not marked as invalid.

func (ReasoningHistory) MarshalBinary 

func (r ReasoningHistory) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface for ReasoningHistory. It returns the binary representation of the enum value as a byte slice.

func (ReasoningHistory) MarshalJSON 

func (r ReasoningHistory) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for ReasoningHistory. It returns the JSON representation of the enum value as a byte slice.

func (ReasoningHistory) MarshalText 

func (r ReasoningHistory) MarshalText() ([]byte, error)

MarshalText implements the encoding.TextMarshaler interface for ReasoningHistory. It returns the string representation of the enum value as a byte slice

func (ReasoningHistory) MarshalYAML 

func (r ReasoningHistory) MarshalYAML() ([]byte, error)

MarshalYAML implements the yaml.Marshaler interface for ReasoningHistory. It returns the string representation of the enum value.

func (*ReasoningHistory) Scan 

func (r *ReasoningHistory) Scan(value any) error

Scan implements the database/sql.Scanner interface for ReasoningHistory. It parses the string representation of the enum value from the database row. It returns an error if the row does not contain a valid enum value.

func (ReasoningHistory) String 

func (r ReasoningHistory) String() string

String implements the Stringer interface. It returns the canonical absolute name of the enum value.

func (*ReasoningHistory) UnmarshalBinary 

func (r *ReasoningHistory) UnmarshalBinary(by []byte) error

UnmarshalBinary implements the encoding.BinaryUnmarshaler interface for ReasoningHistory. It parses the binary representation of the enum value from the byte slice. It returns an error if the byte slice does not contain a valid enum value.

func (*ReasoningHistory) UnmarshalJSON 

func (r *ReasoningHistory) UnmarshalJSON(by []byte) error

UnmarshalJSON implements the json.Unmarshaler interface for ReasoningHistory. It parses the JSON representation of the enum value from the byte slice. It returns an error if the input is not a valid JSON representation.

func (*ReasoningHistory) UnmarshalText 

func (r *ReasoningHistory) UnmarshalText(by []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface for ReasoningHistory. It parses the string representation of the enum value from the byte slice. It returns an error if the byte slice does not contain a valid enum value.

func (*ReasoningHistory) UnmarshalYAML 

func (r *ReasoningHistory) UnmarshalYAML(by []byte) error

UnmarshalYAML implements the yaml.Unmarshaler interface for Planet. It parses the byte slice representation of the enum value and returns an error if the YAML byte slice does not contain a valid enum value.

func (ReasoningHistory) Value 

func (r ReasoningHistory) Value() (driver.Value, error)

Value implements the database/sql/driver.Valuer interface for ReasoningHistory. It returns the string representation of the enum value.

type ResponseFormat 

type ResponseFormat struct {
	// Type selects the output mode. Zero value is unconstrained text.
	Type ResponseFormatType

	// Name labels the schema for OpenAI's structured-output API
	// (required there) and shows up in llama.cpp's logs. Free-form;
	// stick to identifier-ish strings ("verdict", "skill_pick").
	Name string

	// Schema is the typed JSON Schema document the model's output must
	// satisfy (the same Schema type used for tool parameters, so there is
	// one schema representation across the package). The zero value means
	// no schema. Hand-author inline as a Schema literal, or build from a
	// map with SchemaFromMap; the provider serialises it into the request.
	//
	// For enum-constrained decisions, the canonical shape is:
	//
	//	{
	//	    "type": "object",
	//	    "properties": {
	//	        "verdict": {"type": "string", "enum": ["a", "b", "c"]},
	//	    },
	//	    "required": ["verdict"],
	//	    "additionalProperties": false,
	//	}
	Schema Schema

	// Strict asks OpenAI's API to refuse responses that don't satisfy
	// the schema rather than best-effort matching. llama.cpp ignores
	// it (grammar-constrained sampling is always strict by
	// construction). Default false to match OpenAI's default.
	Strict bool
}

ResponseFormat carries the per-request structured-output directive in a provider-neutral shape that mirrors OpenAI's response_format payload (which llama.cpp and vLLM also accept). Type discriminates; the other fields are only consulted when Type == ResponseFormatJSONSchema.

func JSONObjectResponseFormat added in v0.3.1 

func JSONObjectResponseFormat() ResponseFormat

JSONObjectResponseFormat returns provider JSON-object mode without a pinned schema.

func JSONSchemaResponseFormat added in v0.3.1 

func JSONSchemaResponseFormat(name string, schema Schema, strict bool) ResponseFormat

JSONSchemaResponseFormat returns a schema-constrained response format.

func TextResponseFormat added in v0.3.1 

func TextResponseFormat() ResponseFormat

TextResponseFormat returns an unconstrained text response format.

func (ResponseFormat) Validate added in v0.3.1 

func (f ResponseFormat) Validate() error

Validate reports whether f has the fields required by its Type.

type ResponseFormatType 

type ResponseFormatType string

ResponseFormatType discriminates how a provider should constrain the model's output. Zero value (ResponseFormatText) means no constraint — the historical default. 

const (
	// ResponseFormatText is unconstrained free-form output. The zero
	// value, so a CompletionRequest with no ResponseFormat set picks
	// this implicitly.
	ResponseFormatText ResponseFormatType = ""

	// ResponseFormatJSONObject asks the model for valid JSON without
	// pinning the shape — OpenAI's classic "JSON mode". Useful when
	// the prompt already describes the keys but the schema would be
	// noisy to author. llama.cpp accepts the same flag.
	ResponseFormatJSONObject ResponseFormatType = "json_object"

	// ResponseFormatJSONSchema constrains output to exactly the
	// supplied JSON Schema. This is the form to use for enum-driven
	// classifications, verdicts, and routing decisions — the model
	// cannot mis-spell or invent an enum value because the sampler
	// rejects any token sequence that would.
	ResponseFormatJSONSchema ResponseFormatType = "json_schema"
)

type Schema 

type Schema struct {
	Type        string
	Description string
	// Properties are stored by value — map entries aren't addressable, so a
	// pointer would only add nil checks and GC pressure with no aliasing win.
	Properties map[string]Schema
	Required   []string
	Enum       []any
	// Items is a pointer because a struct can't embed itself by value
	// (infinite size); nil means "no items constraint".
	Items *Schema
	// AdditionalProperties is bool | Schema (JSON Schema permits either).
	AdditionalProperties any
	// Extra carries every JSON-Schema key not modelled above, preserved
	// verbatim so externally-sourced schemas reach the model losslessly.
	Extra map[string]any

	// PropertyOrder fixes the order Properties serialize in. Property order
	// is load-bearing for grammar-constrained sampling: llama.cpp's
	// schema-to-GBNF converter emits properties in document order, so a
	// schema that wants a free-text rationale generated BEFORE an enum
	// commitment must serialize rationale first. Without this, MarshalJSON
	// falls back to Go's map marshalling — alphabetical — which silently
	// reorders ("action" before "rationale"). Names not present in
	// Properties are skipped; properties not named here follow in sorted
	// order so none are dropped. Ignored by Map(), which cannot carry order.
	PropertyOrder []string
}

Schema is the typed JSON-Schema fragment describing a tool's parameters. It replaces the old map[string]any so required/enum have exactly one Go type each (no []string-vs-[]any reconciliation at every consumer), while Extra preserves the open-world JSON-Schema keys (oneOf, format, minimum, $ref, patternProperties, …) that MCP servers and hand-written schemas send straight to the model — they must round-trip untouched.

All the JSON<->Go juggling lives in (Un)MarshalJSON / SchemaFromMap, so every other producer and consumer works in typed fields.

func SchemaFromMap 

func SchemaFromMap(m map[string]any) Schema

SchemaFromMap builds a Schema from a generic JSON-Schema map — the ingest path for MCP servers, dynamic-tool --describe output, and persisted catalog entries, where the schema arrives via json.Unmarshal (so required/enum are []any). A nil map yields the zero Schema.

func (Schema) IsZero 

func (s Schema) IsZero() bool

IsZero reports whether the schema carries no constraints — the no-arguments case. Used where the old map representation was checked for nil/len==0.

func (Schema) Map 

func (s Schema) Map() map[string]any

Map returns the schema as a generic JSON-Schema map for the few consumers (e.g. SDKs whose field type is map[string]any) that still need one.

func (Schema) MarshalJSON 

func (s Schema) MarshalJSON() ([]byte, error)

MarshalJSON uses a value receiver so json also invokes it for the non-addressable Schema values inside Properties. When PropertyOrder is set, properties are emitted in that order (see the field doc — grammar converters read document order); otherwise the map path applies and keys serialize alphabetically.

func (*Schema) UnmarshalJSON 

func (s *Schema) UnmarshalJSON(b []byte) error

UnmarshalJSON funnels through the generic map so a single code path handles both the []string (in-process literals) and []any (post-json) shapes.

type ThinkingConfig 

type ThinkingConfig struct {
	// Enabled turns extended thinking on. Off (the default) leaves the
	// provider/model default in place.
	Enabled bool

	// BudgetTokens optionally caps the thinking-token budget for providers
	// that accept one (Anthropic requires >= 1024 and clamps). Zero lets
	// the provider choose a sane default.
	BudgetTokens int
}

ThinkingConfig is the provider-neutral request for extended reasoning.

type Tool 

type Tool struct {
	Type     string       `json:"type"` // "function"
	Function ToolFunction `json:"function"`
}

Tool represents a function that the LLM can call.

type ToolCall 

type ToolCall struct {
	ID       string           `json:"id"`
	Type     string           `json:"type"` // "function"
	Function ToolCallFunction `json:"function"`
}

ToolCall represents a function call made by the LLM (raw transport format).

type ToolCallFunction 

type ToolCallFunction struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"` // JSON string of arguments
}

ToolCallFunction represents the function details in a tool call.

type ToolFunction 

type ToolFunction struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	Parameters  Schema `json:"parameters"`
}

ToolFunction defines a callable function for the LLM.

func (ToolFunction) ParametersMap 

func (f ToolFunction) ParametersMap() map[string]any

ParametersMap renders the function's parameter schema as a generic JSON Schema map for SDKs whose parameter field is map[string]any. Returns nil when the function takes no arguments.

type Usage 

type Usage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
	CachedTokens     int
}

Usage tracks token usage.

CachedTokens is the subset of PromptTokens served from the provider's prompt cache — Anthropic reports this as `cache_read_input_tokens`, OpenAI as `prompt_tokens_details.cached_tokens`, and llama.cpp's openai-compat endpoint approximates it from the KV-cache reuse. Adapters that can't distinguish cached vs uncached leave this at 0.

type VideoData added in v0.3.0 

type VideoData struct {
	URL      string `json:"url,omitempty"`
	DataURI  string `json:"data_uri,omitempty"`
	MIMEType string `json:"mime_type,omitempty"`
}

VideoData is the video payload for a video content part. Either URL (remote http(s) URL) or DataURI ("data:video/...;base64,...") must be set; providers prefer DataURI when both are present.

Source Files

View all Source files

Directories

Path Synopsis
Package anthropic adapts Anthropic Claude models to the shared LLM provider interface.
 Package anthropic adapts Anthropic Claude models to the shared LLM provider interface.
Package backends maps user-facing backend names to concrete LLM provider implementations.
 Package backends maps user-facing backend names to concrete LLM provider implementations.
Package claudecode adapts the OAuth-backed Claude Code CLI surface to the shared llm.Provider interface.
 Package claudecode adapts the OAuth-backed Claude Code CLI surface to the shared llm.Provider interface.
Package deepseek provides an LLM provider for DeepSeek's OpenAI-compatible chat completions API.
 Package deepseek provides an LLM provider for DeepSeek's OpenAI-compatible chat completions API.
Package google adapts Google Gemini models to the shared LLM provider interface.
 Package google adapts Google Gemini models to the shared LLM provider interface.
Package llamacpp provides an LLM provider for llama.cpp's HTTP server, which exposes an OpenAI-compatible /v1/* API surface.
 Package llamacpp provides an LLM provider for llama.cpp's HTTP server, which exposes an OpenAI-compatible /v1/* API surface.
Package ollama provides an LLM provider for Ollama, which exposes an OpenAI-compatible /v1/* API surface alongside its native /api/* surface.
 Package ollama provides an LLM provider for Ollama, which exposes an OpenAI-compatible /v1/* API surface alongside its native /api/* surface.
Package openai adapts OpenAI-compatible chat completion APIs to the shared LLM provider interface.
 Package openai adapts OpenAI-compatible chat completion APIs to the shared LLM provider interface.
Package openaicodex implements an LLM provider that talks to OpenAI's ChatGPT Codex backend using a ChatGPT Plus/Pro OAuth credential instead of a paid OpenAI Platform API key.
 Package openaicodex implements an LLM provider that talks to OpenAI's ChatGPT Codex backend using a ChatGPT Plus/Pro OAuth credential instead of a paid OpenAI Platform API key.
Package providertest exercises every llm.Provider implementation against the common behaviours the runner depends on.
 Package providertest exercises every llm.Provider implementation against the common behaviours the runner depends on.
Package repair tries to recover usable JSON from malformed model output.
 Package repair tries to recover usable JSON from malformed model output.
Package templates provides per-model chat-template helpers for local-LLM servers (llama.cpp, Ollama via /v1/) where the wire-format reasoning negotiation differs by model family.
 Package templates provides per-model chat-template helpers for local-LLM servers (llama.cpp, Ollama via /v1/) where the wire-format reasoning negotiation differs by model family.
Package toolparse recovers structured tool calls from assistant text when the provider's own structured-output path produced none — the model emitted a known protocol artifact (tagged block, JSON object, bare array) as plain text instead of the wire-format tool_calls field.
 Package toolparse recovers structured tool calls from assistant text when the provider's own structured-output path produced none — the model emitted a known protocol artifact (tagged block, JSON object, bare array) as plain text instead of the wire-format tool_calls field.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.