chat

package
v0.1.20 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 30, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package chat defines provider-agnostic chat types and the Provider interface that all model backends implement.

The types in this package form the canonical request/response shape used across the SDK. Concrete providers (OpenAI, Anthropic, local models, ...) translate to and from these types so that higher-level code can remain backend-independent.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNoProvider indicates the Client has no underlying Provider configured.
	ErrNoProvider = errors.New("chat: no provider configured")

	// ErrInvalidRequest indicates the Request is malformed or missing
	// required fields (for example, no Model or no Messages).
	ErrInvalidRequest = errors.New("chat: invalid request")

	// ErrProviderUnavailable indicates the upstream provider is temporarily
	// unreachable or returned a transient failure.
	ErrProviderUnavailable = errors.New("chat: provider unavailable")

	// ErrRateLimited indicates the upstream provider rejected the request
	// due to rate limiting or quota exhaustion.
	ErrRateLimited = errors.New("chat: rate limited")

	// ErrAuthFailed indicates the provider rejected the supplied credentials.
	ErrAuthFailed = errors.New("chat: authentication failed")

	// ErrContextLength indicates the request exceeds the model's maximum
	// supported context length.
	ErrContextLength = errors.New("chat: context length exceeded")

	// ErrUnsupported indicates the provider does not support a requested
	// capability (for example, streaming or tool calls).
	ErrUnsupported = errors.New("chat: unsupported operation")
)
View Source 
var ErrInvalidPart = errors.New("chat: invalid part")

ErrInvalidPart indicates a Part is malformed (e.g. neither URL nor Data set on an ImagePart, or both set, or missing MediaType for inline data). Provider implementations may wrap this when rejecting content.

View Source 
var ErrUnsupportedContent = errors.New("chat: unsupported content")

ErrUnsupportedContent indicates a Part kind is not supported by the provider/model for the current request. Providers wrap this with context (provider, model, part type) when rejecting content.

Functions

func ContextHeaders 

func ContextHeaders(ctx context.Context) (map[string]string, bool)

ContextHeaders returns the extra headers stored in ctx, or nil if none were set.

func ProviderOptionsFor 

func ProviderOptionsFor[T any](po map[string]any, providerName string) (T, error)

ProviderOptionsFor extracts the provider-specific options bucket from a ProviderOptions map (typically Request.ProviderOptions or Message.ProviderOptions) into a typed value.

providerName is the key used to namespace the bucket — by convention the provider's Provider.Name return value, e.g. "openai", "ollama".

Two input shapes are supported transparently:

  • The bucket is already the typed Options struct (or a pointer to one) — it is returned as-is.
  • The bucket is a map[string]any (e.g. constructed from JSON) — it is re-marshalled and decoded into T using encoding/json so that JSON tags on T's fields are honoured.

If po is nil or the providerName key is absent, the zero value of T is returned with a nil error. Decoding errors are wrapped.

Example provider-side use: 

type Options struct {
    ReasoningEffort string `json:"reasoning_effort,omitempty"`
}
opts, err := chat.ProviderOptionsFor[Options](req.ProviderOptions, "openai")

func SanitizeErrorBody added in v0.1.12 

func SanitizeErrorBody(body []byte) string

SanitizeErrorBody trims an HTTP error response body down to a short, display-safe snippet for embedding in an error message. HTML bodies (edge/gateway error pages such as Cloudflare's 502/504 pages, which can run to several KB of markup) are collapsed to a short marker instead of being embedded verbatim — the status code already conveys the failure, and embedding raw HTML has repeatedly overwhelmed caller UIs that print error text directly to a terminal or notification banner. Non-HTML bodies are truncated to maxErrorBodySnippet bytes with a marker appended when trimmed.

func ValidateToolCallArguments 

func ValidateToolCallArguments(args string) error

ValidateToolCallArguments reports whether the given Arguments string is parseable as a JSON object. It is a best-effort sanity check; providers may emit pre-validated JSON, or fragments that need concatenation upstream of this call.

func WithContextHeaders 

func WithContextHeaders(ctx context.Context, headers map[string]string) context.Context

WithContextHeaders attaches extra HTTP headers to a context so that provider implementations can inject them into outbound requests. Multiple callers can attach headers; later calls overwrite earlier values for the same key.

Types

type Chunk 

type Chunk struct {
	Delta            string          `json:"delta,omitempty"`
	ReasoningDelta   string          `json:"reasoning_delta,omitempty"`
	Role             Role            `json:"role,omitempty"`
	ToolCallDeltas   []ToolCallDelta `json:"tool_call_deltas,omitempty"`
	FinishReason     string          `json:"finish_reason,omitempty"`
	Usage            *Usage          `json:"usage,omitempty"`
	Warnings         []Warning       `json:"warnings,omitempty"`
	ProviderMetadata map[string]any  `json:"provider_metadata,omitempty"`
	Done             bool            `json:"done,omitempty"`
}

Chunk is a single increment in a streaming chat completion.

Usage is a pointer because token totals are typically only known at the end of a stream; intermediate chunks carry Usage == nil and the final chunk (Done == true) carries the aggregate usage.

ReasoningDelta carries incremental reasoning/thinking text emitted by providers that support it (Anthropic thinking, Gemini thinking, OpenAI o1). Providers that do not produce reasoning leave it empty.

ProviderMetadata carries opaque provider response data that must be preserved when a streamed assistant turn is sent back to that provider.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is a thin, provider-agnostic facade over a Provider. It centralises concerns that are independent of the underlying backend (such as toggling the streaming flag on the request) and provides a single entry point that higher-level code can depend on.

func NewClient 

func NewClient(p Provider) *Client

NewClient returns a Client backed by the given Provider. The Provider may be nil; in that case the Client's methods will return ErrNoProvider.

func (*Client) Chat 

func (c *Client) Chat(ctx context.Context, req Request) (Response, error)

Chat performs a non-streaming chat completion. It forces req.Stream to false before delegating to the underlying Provider. If the Client or its Provider is nil, Chat returns ErrNoProvider.

func (*Client) ChatStream 

func (c *Client) ChatStream(ctx context.Context, req Request) (Stream, error)

ChatStream performs a streaming chat completion. It forces req.Stream to true before delegating to the underlying Provider. If the Client or its Provider is nil, ChatStream returns ErrNoProvider.

func (*Client) Provider 

func (c *Client) Provider() Provider

Provider returns the underlying Provider, which may be nil.

type FilePart 

type FilePart struct {
	URL              string         `json:"url,omitempty"`
	Data             []byte         `json:"data,omitempty"`
	MediaType        string         `json:"media_type,omitempty"`
	Name             string         `json:"name,omitempty"`
	ProviderMetadata map[string]any `json:"provider_metadata,omitempty"`
}

FilePart is a generic file content fragment (PDF, audio, etc.). Exactly one of URL or Data must be set; MediaType is required when Data is set.

func NewFileData 

func NewFileData(name, mediaType string, data []byte) FilePart

NewFileData constructs a FilePart from raw bytes.

func NewFileURL 

func NewFileURL(url, mediaType string) FilePart

NewFileURL constructs a FilePart from a URL.

func (FilePart) Type 

func (FilePart) Type() PartType

Type returns PartTypeFile.

type ImagePart 

type ImagePart struct {
	// URL points at a remote image or a data: URI. Mutually exclusive
	// with Data.
	URL string `json:"url,omitempty"`
	// Data carries the raw image bytes. Mutually exclusive with URL.
	Data []byte `json:"data,omitempty"`
	// MediaType is the IANA media type (e.g. "image/png"). Required
	// when Data is set.
	MediaType string `json:"media_type,omitempty"`
	// ProviderMetadata carries provider-specific per-part options
	// (e.g. Anthropic cache control, OpenAI image detail level).
	ProviderMetadata map[string]any `json:"provider_metadata,omitempty"`
}

ImagePart is an image content fragment. Exactly one of URL or Data must be set. MediaType (e.g. "image/png") is required when Data is set and recommended for URLs that lack a discriminating extension.

func NewImageData 

func NewImageData(mediaType string, data []byte) ImagePart

NewImageData constructs an ImagePart from raw bytes plus MediaType. MediaType is required; if empty the part is still constructed but downstream providers may reject or warn on it.

func NewImageURL 

func NewImageURL(url string) ImagePart

NewImageURL constructs an ImagePart from a URL (remote or data: URI).

func (ImagePart) Type 

func (ImagePart) Type() PartType

Type returns PartTypeImage.

type Message 

type Message struct {
	Role            Role           `json:"role"`
	Content         string         `json:"content,omitempty"`
	Parts           Parts          `json:"parts,omitempty"`
	Name            string         `json:"name,omitempty"`
	ToolCalls       []ToolCall     `json:"tool_calls,omitempty"`
	ToolCallID      string         `json:"tool_call_id,omitempty"`
	ProviderOptions map[string]any `json:"provider_options,omitempty"`
}

Message is a single entry in a chat conversation.

Multimodal content is carried on Message.Parts. The legacy Message.Content string remains for ergonomic text-only construction — when Parts is nil, providers treat Content as a single TextPart; when Parts is non-nil, Parts is canonical and Content is ignored on the request path. On the response path, providers populate both fields: Parts as the source of truth, Content as the concatenation of all TextPart text for back-compat.

Tool integration uses two additional fields:

  • Assistant messages that called tools carry Message.ToolCalls; Content/Parts may also be present alongside tool calls for some providers.
  • Messages with RoleTool carry the tool's output in Content and reference the originating call via Message.ToolCallID.

ProviderOptions allows per-message provider-specific options keyed by provider name, mirroring Request.ProviderOptions. This is rarely needed at the call site but is useful for things like Anthropic's per-block cache control or OpenAI's per-message attachments.

func (Message) GetParts 

func (m Message) GetParts() Parts

GetParts returns the canonical Parts slice for the message:

  • If Parts is non-nil, it is returned as-is.
  • Otherwise, if Content is non-empty, a single-element slice containing a TextPart is returned.
  • Otherwise, nil is returned.

Provider implementations should call GetParts (not access Message.Content directly) so that callers using either field are handled uniformly.

func (Message) Text 

func (m Message) Text() string

Text returns the textual content of the message: Parts.Text() if Parts is non-nil, otherwise Content.

type Part 

type Part interface {
	// Type returns the Part's discriminator.
	Type() PartType
	// contains filtered or unexported methods
}

Part is a single content fragment of a Message or Response.

Parts replace the historical "Content string" field as the canonical representation of multimodal content. The Part interface is sealed (only types defined in this package implement it) so that providers can safely type-switch on the concrete kind.

Note on tool calls/results: tool invocations and their outputs are currently carried on Message.ToolCalls / Message.ToolCallID rather than as parts. This means Parts does not represent a fully-ordered transcript when tool calls are interleaved with text in an assistant turn. Providers that require strict ordering of content blocks (e.g. Anthropic Messages API with thinking signatures) will gain dedicated ToolCallPart / ToolResultPart types in a follow-up iteration.

type PartType 

type PartType string

PartType identifies a Part's concrete kind. It is used both for type discrimination on the wire ("type" JSON tag) and for callers inspecting a Parts slice without a type switch. 

const (
	PartTypeText      PartType = "text"
	PartTypeImage     PartType = "image"
	PartTypeFile      PartType = "file"
	PartTypeReasoning PartType = "reasoning"
)

Standard Part kinds.

type Parts 

type Parts []Part

Parts is an ordered slice of Part. It implements custom JSON marshal/unmarshal so that the "type" discriminator round-trips correctly through encoding/json.

func (Parts) HasNonText 

func (ps Parts) HasNonText() bool

HasNonText reports whether any part is something other than a TextPart. This is the canonical capability check for providers that can only handle text and need to emit warnings or errors.

func (Parts) MarshalJSON 

func (ps Parts) MarshalJSON() ([]byte, error)

MarshalJSON encodes the slice as a JSON array of objects, each carrying a "type" discriminator alongside the part's fields.

func (Parts) Text 

func (ps Parts) Text() string

Text concatenates the text content of every TextPart, preserving order. Non-text parts are skipped entirely. No separator is inserted between adjacent text parts; callers that need a lossy textual representation including non-text placeholders should iterate Parts themselves.

func (*Parts) UnmarshalJSON 

func (ps *Parts) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes a JSON array of part objects back into concrete Part values, dispatching on the "type" discriminator.

func (Parts) Validate 

func (ps Parts) Validate() error

Validate checks the Parts slice for structural validity and returns the first error encountered, or nil. It does not enforce provider-specific capability limits — those are reported by the provider as warnings or errors at request time.

type Provider 

type Provider interface {
	// Name returns a short, stable identifier for the provider
	// (for example, "openai", "anthropic", "ollama").
	Name() string

	// Chat performs a non-streaming chat completion.
	Chat(ctx context.Context, req Request) (Response, error)

	// ChatStream performs a streaming chat completion. Callers must Close
	// the returned Stream when finished.
	ChatStream(ctx context.Context, req Request) (Stream, error)
}

Provider is implemented by chat model backends. Implementations translate between the provider-agnostic Request/Response/Chunk types defined in this package and their underlying API.

type ReasoningPart 

type ReasoningPart struct {
	Text             string         `json:"text"`
	ProviderMetadata map[string]any `json:"provider_metadata,omitempty"`
}

ReasoningPart carries chain-of-thought / thinking content emitted by the model. ProviderMetadata holds opaque provider replay tokens that must be preserved verbatim for multi-turn conversations (notably Anthropic thinking-block signatures); the SDK does not interpret it.

func (ReasoningPart) Type 

func (ReasoningPart) Type() PartType

Type returns PartTypeReasoning.

type Request 

type Request struct {
	Model           string            `json:"model"`
	Messages        []Message         `json:"messages"`
	MaxTokens       int               `json:"max_tokens,omitempty"`
	Temperature     float32           `json:"temperature,omitempty"`
	TopP            float32           `json:"top_p,omitempty"`
	Stop            []string          `json:"stop,omitempty"`
	Stream          bool              `json:"stream,omitempty"`
	Tools           []Tool            `json:"tools,omitempty"`
	ToolChoice      *ToolChoice       `json:"tool_choice,omitempty"`
	Metadata        map[string]string `json:"metadata,omitempty"`
	ProviderOptions map[string]any    `json:"provider_options,omitempty"`
}

Request is a provider-agnostic chat completion request.

Only Model and Messages are required; all other fields are optional and providers should treat zero values as "unspecified" and apply their own defaults.

ProviderOptions carries provider-specific options keyed by provider name (for example "openai", "anthropic", "ollama"). Each provider reads only its own bucket and ignores keys for other providers, so the same Request can be shared across providers without modification. Use the ProviderOptionsFor helper to extract a typed options struct from inside a provider implementation.

type Response 

type Response struct {
	ID               string         `json:"id,omitempty"`
	Model            string         `json:"model,omitempty"`
	Role             Role           `json:"role,omitempty"`
	Content          string         `json:"content"`
	Parts            Parts          `json:"parts,omitempty"`
	ToolCalls        []ToolCall     `json:"tool_calls,omitempty"`
	FinishReason     string         `json:"finish_reason,omitempty"`
	Usage            Usage          `json:"usage"`
	Warnings         []Warning      `json:"warnings,omitempty"`
	ProviderMetadata map[string]any `json:"provider_metadata,omitempty"`
}

Response is a non-streaming chat completion result.

When the model invoked tools, ToolCalls is non-empty and FinishReason is "tool_calls". Content may still be present alongside tool calls for some providers.

Parts is the canonical multimodal representation of the assistant's reply (text + reasoning + future image/file outputs). Content is populated as the concatenation of all TextPart text for back-compat and ergonomic text-only consumption.

type Role 

type Role string

Role identifies the author of a Message in a chat conversation. 

const (
	// RoleSystem is used for high-level instructions or persona setup.
	RoleSystem Role = "system"
	// RoleUser is used for end-user messages.
	RoleUser Role = "user"
	// RoleAssistant is used for model-generated replies.
	RoleAssistant Role = "assistant"
	// RoleTool is used for tool/function call results fed back to the model.
	RoleTool Role = "tool"
)

Standard chat roles. Providers map these to their own role vocabularies.

type Stream 

type Stream interface {
	// Next returns the next chunk. It returns io.EOF when the stream is
	// exhausted.
	Next(ctx context.Context) (Chunk, error)

	// Close releases resources associated with the stream.
	Close() error
}

Stream is an iterator over Chunks produced by a streaming chat completion.

Next returns io.EOF (and a zero Chunk) when the stream is exhausted; any other non-nil error indicates a stream failure. Callers must Close the stream exactly once, even after receiving io.EOF, to release resources.

type TextPart 

type TextPart struct {
	Text string `json:"text"`
}

TextPart is a plain-text content fragment.

func NewTextPart 

func NewTextPart(text string) TextPart

NewTextPart returns a TextPart wrapping text.

func (TextPart) Type 

func (TextPart) Type() PartType

Type returns PartTypeText.

type Tool 

type Tool struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters,omitempty"`
}

Tool describes a callable tool/function that the model may invoke.

Parameters is a JSON Schema (RFC 8927-compatible / OpenAPI-compatible) describing the tool's expected input. Providers translate Tool into their wire format (OpenAI/DeepSeek "function", Gemini "functionDeclaration", Ollama "tools").

type ToolCall 

type ToolCall struct {
	ID        string `json:"id"`
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

ToolCall is a single tool invocation requested by the model.

ID is the provider-assigned identifier used to match a subsequent RoleTool message to this call. Some providers (notably Ollama) do not emit IDs; in that case the SDK synthesises one (call_<index>) so that downstream code can correlate calls and results uniformly.

Arguments is the JSON-encoded argument object as a string, matching the wire shape used by every provider we target.

func AssembleToolCalls 

func AssembleToolCalls(deltas []ToolCallDelta) []ToolCall

AssembleToolCalls reconstructs complete [ToolCall]s from an ordered sequence of [ToolCallDelta]s as emitted by a streaming provider.

Deltas with the same Index are folded into a single ToolCall: the first delta carrying ID/Name sets those fields, and ArgsDelta is concatenated across all deltas for that Index. Indices that never receive an ID are assigned a synthetic id of "call_<index>" so that downstream tool execution can still correlate results back to calls when the upstream provider does not supply IDs (e.g. Ollama).

The returned slice is sorted by Index ascending.

type ToolCallDelta 

type ToolCallDelta struct {
	Index     int    `json:"index"`
	ID        string `json:"id,omitempty"`
	Name      string `json:"name,omitempty"`
	ArgsDelta string `json:"args_delta,omitempty"`
}

ToolCallDelta is an incremental update to a single tool call within a streaming response. Multiple parallel tool calls are distinguished by Index.

On the first delta for a given Index, ID and Name are populated; on subsequent deltas only ArgsDelta is appended (the JSON arguments arrive token-by-token). Consumers concatenate ArgsDelta across all deltas with the same Index to reconstruct the full Arguments string.

Some providers (Ollama) emit a complete tool call in a single chunk rather than streaming arguments — in that case ID/Name are populated and ArgsDelta carries the entire JSON arguments at once.

type ToolChoice 

type ToolChoice struct {
	Type ToolChoiceType `json:"type"`
	Name string         `json:"name,omitempty"`
}

ToolChoice constrains the model's tool selection behaviour. When Type is ToolChoiceTool, Name identifies the required tool.

type ToolChoiceType 

type ToolChoiceType string

ToolChoiceType controls how the model decides which (if any) tool to call. 

const (
	// ToolChoiceAuto lets the model choose freely (default when tools are present).
	ToolChoiceAuto ToolChoiceType = "auto"
	// ToolChoiceNone forbids tool calls; the model must answer with text.
	ToolChoiceNone ToolChoiceType = "none"
	// ToolChoiceRequired forces the model to call any tool.
	ToolChoiceRequired ToolChoiceType = "required"
	// ToolChoiceTool forces the model to call a specific named tool.
	ToolChoiceTool ToolChoiceType = "tool"
)

type UnsupportedContentError 

type UnsupportedContentError struct {
	Provider string
	Model    string
	PartType PartType
}

UnsupportedContentError is the typed form of ErrUnsupportedContent carrying enough context to compose a useful error message and to be matched via errors.As. It always errors.Is matches ErrUnsupportedContent.

func (*UnsupportedContentError) Error 

func (e *UnsupportedContentError) Error() string

Error implements error.

func (*UnsupportedContentError) Unwrap 

func (e *UnsupportedContentError) Unwrap() error

Unwrap allows errors.Is(err, ErrUnsupportedContent) matching.

type Usage 

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`

	// CachedTokens is the portion of PromptTokens served from a cache
	// (OpenAI's automatic prefix cache, or an Anthropic cache-control
	// read), billed at a reduced rate. Zero when the provider doesn't
	// report it or nothing was served from cache.
	CachedTokens int `json:"cached_tokens,omitempty"`
	// CacheCreationTokens is Anthropic-specific: the token count written
	// to the cache on this call (billed at a premium over normal input
	// tokens, the cost of the cache warming up). Zero for providers with
	// no separate cache-write charge, e.g. OpenAI's automatic caching.
	CacheCreationTokens int `json:"cache_creation_tokens,omitempty"`
}

Usage reports token accounting for a chat completion.

type Warning 

type Warning struct {
	// Message is the human-readable warning text.
	Message string `json:"message"`
	// Type is an optional machine-readable category, e.g.
	// "unsupported-content", "deprecated-option".
	Type string `json:"type,omitempty"`
}

Warning is a non-fatal provider message (e.g. "image part dropped: model is text-only"). Providers attach warnings to Response.Warnings or Chunk.Warnings; core aggregates them onto StepResult/GenerateResult.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.