ai

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 5, 2026 License: MIT Imports: 13 Imported by: 0

README

deps.dev License License Stay with Ukraine

ai

ai is a small, provider-agnostic interface for talking to large language model APIs, plus the shared request and response types every provider driver speaks. It is the core that goloop's provider packages (anthropic, openai, gemini, and so on) build on.

Like the standard library's database/sql with its drivers, or log/slog with its handlers, this package holds the common contract while a separate package per provider implements it. A driver depends only on ai, so the whole set stays free of third-party dependencies.

Installation

go get github.com/goloop/ai

The interface

type Client interface {
	Generate(ctx context.Context, req *Request) (*Response, error)
	Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}

Types

  • Role and Message (a role plus a list of content Part values).
  • Part: Text, Image (multimodal), ToolUse, ToolResult (tool calling).
  • Tool and ToolChoice for function calling.
  • Request (model, system, messages, tools, sampling knobs).
  • Format for structured output: ask for JSON, or JSON matching a schema.
  • Response with Text(), JSON(&v) and ToolCalls() helpers; Chunk for streaming.
  • Usage for token counts; APIError for normalized provider errors.

Using a provider

import (
	"github.com/goloop/ai"
	"github.com/goloop/anthropic"
)

c := anthropic.New(os.Getenv("ANTHROPIC_API_KEY"))
resp, err := c.Generate(ctx, &ai.Request{
	Model:    anthropic.ModelClaude37SonnetLatest,
	Messages: []ai.Message{ai.UserText("Hello!")},
})

Any provider client is an ai.Client, so code written against the interface works with all of them, which makes multi-provider setups straightforward.

Plumbing for drivers

Drivers reuse the shared configuration and transport:

  • Options and functional options (WithBaseURL, WithHTTPClient, WithTimeout, WithMaxRetries, WithHeader), built with NewOptions.
  • Options.Do - an HTTP request with retries on 429 and 5xx.
  • SSEEvents - an iterator over Server-Sent Events data payloads.

Endpoints providers do not share (embeddings, images, audio, files, batches) are not part of the interface; each driver exposes them as native methods.

Documentation

Full reference: DOC.md (Ukrainian: DOC.UK.md).

License

MIT - see LICENSE.

Documentation

Overview

Package ai defines a single, provider-agnostic interface for talking to large language model APIs, together with the shared request and response types every provider driver speaks.

The design mirrors the standard library's split between an interface and its drivers: like database/sql with its drivers, or log/slog with its handlers, package ai holds the common contract while a separate package per provider (anthropic, openai, gemini, and so on) implements it. A driver depends only on this package, so the whole set stays free of third-party dependencies.

The contract is the Client interface: 

type Client interface {
    Generate(ctx context.Context, req *Request) (*Response, error)
    Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}

A Request carries a model, an optional system prompt, a list of Messages, optional Tools and the usual sampling knobs. A Message is a role plus a list of content Parts (Text, Image, ToolUse, ToolResult), which is enough to express multimodal input and tool calling across providers. Generate returns a whole Response; Stream yields Chunks as they arrive.

Structured output

Request.Format asks for JSON, optionally matching a JSON Schema, and Response.JSON decodes the reply into a Go value. Providers differ in what they can enforce: a driver uses native support where it exists, asks for the format in the prompt where it does not, and reports which it did in Response.Format. No driver drops a Format silently.

Endpoints that providers do not share (embeddings, image generation, audio, files, batches, and so on) are not part of this interface. Each driver exposes those as its own native methods, so the common surface stays small and honest while provider-specific power is still available.

This package also carries the plumbing drivers reuse: Options and its functional configuration, Options.Do for HTTP requests with retries, and SSEEvents for reading Server-Sent Events streams.

Index

Examples

Constants

View Source 
const DefaultSchemaName = "response"

DefaultSchemaName is the schema name drivers use when a Format that needs one leaves Name empty.

Variables

View Source 
var (
	ErrNoRequest  = errors.New("ai: request is nil")
	ErrNoModel    = errors.New("ai: model is required")
	ErrNoMessages = errors.New("ai: at least one message is required")

	// Format errors, reported by Request.Validate before any driver work.
	ErrBadFormat = errors.New("ai: unknown format type")
	ErrNoSchema  = errors.New("ai: FormatJSONSchema requires a schema")
	ErrBadSchema = errors.New("ai: format schema must be a JSON Schema object")

	// ErrNoFormat is returned by a driver whose provider can neither enforce
	// the requested format nor be asked for it in a way that is worth
	// trusting. A driver never drops a Format quietly.
	ErrNoFormat = errors.New("ai: provider cannot produce the requested format")

	// ErrNoText is returned by Response.JSON when the reply carried no text
	// to decode, for example a response made up entirely of tool calls.
	ErrNoText = errors.New("ai: response has no text")
)

Sentinel errors returned before a request reaches the network.

Functions

func SSEEvents 

func SSEEvents(r io.Reader) iter.Seq2[string, error]

SSEEvents returns an iterator over the data payloads of a Server-Sent Events stream read from r. Each yielded string is the concatenated "data:" content of one event, with the field prefix and the single optional leading space removed and multi-line data joined by newlines. Comment lines (starting with ":") and other fields (event:, id:, retry:) are ignored, which is enough for the streaming chat APIs the drivers target. A read error is yielded once with an empty payload, after which iteration stops.

The caller is responsible for closing r.

Types

type APIError 

type APIError struct {
	Status  int             // HTTP status code
	Type    string          // provider error type, when given
	Code    string          // provider error code, when given
	Message string          // human-readable message, when given
	Raw     json.RawMessage // original error body
}

APIError is a normalized error for a non-success HTTP response from a provider. Drivers fill the fields they can parse from the provider's error body and keep the original JSON in Raw.

func (*APIError) Error 

func (e *APIError) Error() string

Error implements the error interface.

type Chunk 

type Chunk struct {
	Text     string
	ToolCall *ToolUse
	Usage    *Usage
	Done     bool
	Raw      json.RawMessage
}

Chunk is one increment of a streaming response from Client.Stream. Text is the incremental text delta; ToolCall is set when the chunk carries a completed tool call; Done marks the final chunk. Drivers set Usage on the Done chunk; its counts are zero when the provider did not report usage. Raw keeps the provider's original event JSON.

type Client 

type Client interface {
	Generate(ctx context.Context, req *Request) (*Response, error)
	Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}

Client is the contract every provider driver implements. Generate performs a single request and returns the whole response. Stream performs a request and returns an iterator over response chunks; the iterator yields a zero Chunk with a non-nil error and stops if the stream fails.

Example (Generate) 
package main

import (
	"context"
	"fmt"
	"iter"

	"github.com/goloop/ai"
)

// mockClient is a trivial in-memory [ai.Client]. A real driver sends the
// request to a provider; this shows the contract Generate and Stream must
// satisfy - the same shape every goloop AI provider implements.
type mockClient struct{}

func (mockClient) Generate(ctx context.Context, req *ai.Request) (*ai.Response, error) {
	if err := req.Validate(); err != nil {
		return nil, err
	}
	return &ai.Response{
		Model:      req.Model,
		Parts:      []ai.Part{ai.Text{Text: "Hello!"}},
		StopReason: "stop",
		Usage:      ai.Usage{InputTokens: 3, OutputTokens: 2},
	}, nil
}

func (mockClient) Stream(ctx context.Context, req *ai.Request) iter.Seq2[ai.Chunk, error] {
	return func(yield func(ai.Chunk, error) bool) {
		if err := req.Validate(); err != nil {
			yield(ai.Chunk{}, err)
			return
		}
		for _, part := range []string{"Hel", "lo!"} {
			if !yield(ai.Chunk{Text: part}, nil) {
				return
			}
		}
		yield(ai.Chunk{Done: true, Usage: &ai.Usage{InputTokens: 3, OutputTokens: 2}}, nil)
	}
}

func main() {
	var c ai.Client = mockClient{}

	resp, err := c.Generate(context.Background(), &ai.Request{
		Model: "demo",
		Messages: []ai.Message{
			ai.SystemText("You are concise."),
			ai.UserText("Say hi."),
		},
	})
	if err != nil {
		panic(err)
	}
	fmt.Println(resp.Text())
	fmt.Printf("%d in / %d out\n", resp.Usage.InputTokens, resp.Usage.OutputTokens)
}

Output:
Hello!
3 in / 2 out
Example (Stream) 
package main

import (
	"context"
	"fmt"
	"iter"

	"github.com/goloop/ai"
)

// mockClient is a trivial in-memory [ai.Client]. A real driver sends the
// request to a provider; this shows the contract Generate and Stream must
// satisfy - the same shape every goloop AI provider implements.
type mockClient struct{}

func (mockClient) Generate(ctx context.Context, req *ai.Request) (*ai.Response, error) {
	if err := req.Validate(); err != nil {
		return nil, err
	}
	return &ai.Response{
		Model:      req.Model,
		Parts:      []ai.Part{ai.Text{Text: "Hello!"}},
		StopReason: "stop",
		Usage:      ai.Usage{InputTokens: 3, OutputTokens: 2},
	}, nil
}

func (mockClient) Stream(ctx context.Context, req *ai.Request) iter.Seq2[ai.Chunk, error] {
	return func(yield func(ai.Chunk, error) bool) {
		if err := req.Validate(); err != nil {
			yield(ai.Chunk{}, err)
			return
		}
		for _, part := range []string{"Hel", "lo!"} {
			if !yield(ai.Chunk{Text: part}, nil) {
				return
			}
		}
		yield(ai.Chunk{Done: true, Usage: &ai.Usage{InputTokens: 3, OutputTokens: 2}}, nil)
	}
}

func main() {
	var c ai.Client = mockClient{}

	for chunk, err := range c.Stream(context.Background(), &ai.Request{
		Model:    "demo",
		Messages: []ai.Message{ai.UserText("Say hi.")},
	}) {
		if err != nil {
			panic(err)
		}
		fmt.Print(chunk.Text)
	}
	fmt.Println()
}

Output:
Hello!

type Format added in v0.4.0 

type Format struct {
	// Type is the shape wanted. The zero value, FormatText, asks for nothing.
	Type FormatType

	// Name labels the schema. Some providers require a name; drivers that do
	// substitute DefaultSchemaName when this is empty. It has no effect for
	// FormatJSON.
	Name string

	// Schema is a JSON Schema object describing the wanted value. It is
	// required for FormatJSONSchema and ignored otherwise.
	Schema json.RawMessage

	// Strict asks the provider to enforce the schema exactly rather than
	// treat it as guidance. Providers that cannot do this ignore the flag;
	// it never changes whether the request is accepted.
	Strict bool
}

Format asks for a structured response. It is provider-agnostic: a driver maps it onto whatever its provider offers natively, and asks for it in the prompt when the provider offers nothing. Either way a driver never ignores it silently - see FormatMode. 

req.Format = &ai.Format{
	Type:   ai.FormatJSONSchema,
	Name:   "seo",
	Schema: schema, // a JSON Schema object
}

Nil means no structured output was requested, which is the default.

func (*Format) AppendInstruction added in v0.4.0 

func (f *Format) AppendInstruction(system string) string

AppendInstruction returns system with Instruction appended after a blank line, which is how a driver folds the request into an existing system prompt. An empty system prompt yields the instruction alone, and a Format that asks for nothing leaves system untouched.

func (*Format) Instruction added in v0.4.0 

func (f *Format) Instruction() string

Instruction returns the wording a driver puts in the prompt when its provider cannot enforce the format itself. It lives here so that all nine drivers ask for the same thing in the same words; a driver with native support never calls it.

It returns an empty string when nothing was asked for.

func (*Format) SchemaName added in v0.4.0 

func (f *Format) SchemaName() string

SchemaName returns Name, or DefaultSchemaName when Name is empty.

func (*Format) Validate added in v0.4.0 

func (f *Format) Validate() error

Validate reports whether the format is one a driver can act on. A nil Format is valid: it asks for nothing.

An unknown Type is rejected rather than treated as one of the known ones. Nine drivers switch over this value, and a type none of them agrees on would mean nine different guesses about what the caller wanted.

type FormatMode added in v0.4.0 

type FormatMode int

FormatMode says how a driver satisfied the Format of a request. It is reported on Response so that a caller can tell an enforced answer from a requested one - a provider that was merely asked nicely can still reply with prose, and that is worth knowing when output turns out malformed. 

const (
	FormatNone     FormatMode = iota // no format was requested
	FormatNative                     // the provider enforced it
	FormatEmulated                   // the driver asked for it in the prompt
)

How a format was satisfied.

func (FormatMode) String added in v0.4.0 

func (m FormatMode) String() string

String renders the mode for diagnostics.

type FormatType added in v0.4.0 

type FormatType int

FormatType selects the shape the model must produce. 

const (
	FormatText       FormatType = iota // free-form text, the provider default
	FormatJSON                         // a single valid JSON value, any shape
	FormatJSONSchema                   // JSON matching Format.Schema
)

The response shapes. FormatText is the zero value and asks for nothing, so a Request without a Format behaves exactly as before.

func (FormatType) String added in v0.4.0 

func (t FormatType) String() string

String renders the type for diagnostics.

type Image 

type Image struct {
	MIME string // for example "image/png" or "image/jpeg"
	Data []byte // inline image bytes, or nil when URL is set
	URL  string // remote image URL, or "" when Data is set
}

Image is an image content part. Provide either inline Data with its MIME type, or a URL when the provider supports fetching remote images. Drivers encode Data as base64 as required by their wire format.

type Message 

type Message struct {
	Role  Role
	Parts []Part
}

Message is one turn in a conversation: a role and its content parts.

func AssistantText 

func AssistantText(s string) Message

AssistantText returns an assistant Message containing a single text part.

func SystemText added in v0.1.1 

func SystemText(s string) Message

SystemText returns a system Message containing a single text part. Drivers fold system messages into the provider's system prompt; the Request.System field is an equivalent shorthand for a single instruction.

func UserText 

func UserText(s string) Message

UserText returns a user Message containing a single text part.

func (Message) MarshalJSON added in v0.3.0 

func (m Message) MarshalJSON() ([]byte, error)

MarshalJSON encodes the message with each part tagged by its "type".

func (*Message) UnmarshalJSON added in v0.3.0 

func (m *Message) UnmarshalJSON(data []byte) error

UnmarshalJSON reconstructs the message and its concrete part types.

type Option 

type Option func(*Options)

Option configures Options. The same options work across every provider, so client construction looks the same everywhere.

func WithBaseURL 

func WithBaseURL(u string) Option

WithBaseURL overrides the provider's default API base URL. It is useful for proxies, gateways, mock servers and self-hosted deployments.

func WithHTTPClient 

func WithHTTPClient(c *http.Client) Option

WithHTTPClient sets the HTTP client used for requests. When set, its own timeout takes precedence over WithTimeout.

func WithHeader 

func WithHeader(key, value string) Option

WithHeader adds a header sent with every request, for example a custom API version or a beta feature flag.

func WithMaxRetries 

func WithMaxRetries(n int) Option

WithMaxRetries sets how many times Options.Do retries a request on HTTP 429 or 5xx responses. Zero disables retrying.

func WithTimeout 

func WithTimeout(d time.Duration) Option

WithTimeout sets the per-request timeout used when no custom HTTP client is provided.

type Options 

type Options struct {
	APIKey     string
	BaseURL    string
	HTTPClient *http.Client
	Timeout    time.Duration
	MaxRetries int
	Headers    http.Header
}

Options is the shared client configuration every driver understands. Drivers build it from an API key and functional options with NewOptions and reuse it for transport (see Options.Do).

func NewOptions 

func NewOptions(apiKey string, opts ...Option) Options

NewOptions builds Options from an API key and functional options, filling in defaults: a 60s timeout, two retries and an HTTP client when none is given.

func (Options) Do 

func (o Options) Do(
	ctx context.Context,
	method, url string,
	body []byte,
	headers http.Header,
) (*http.Response, error)

Do sends an HTTP request using the configured client and headers, retrying transient responses (HTTP 429, 502, 503, 504 and 529) up to MaxRetries with jittered exponential backoff. A Retry-After header on the response is honored, capped at 30s. HTTP 500 is not retried, because driver requests are non-idempotent POSTs. The Options headers are applied first, then the per-call headers override them.

On the final attempt the response is returned as-is even when its status is an error, so the caller can read the provider's error body (drivers check the status and call their own error parser). Do only returns a non-nil error for a transport-level failure (no response) or a canceled context. The caller owns the returned response body and must close it.

Retries repeat the request, including non-idempotent POSTs. A transport error may occur after the server already accepted the request, so a retried POST can execute twice; use WithMaxRetries(0) to disable retrying when that is a concern.

func (Options) GoString added in v0.3.0 

func (o Options) GoString() string

GoString redacts the API key for the %#v verb, matching String.

func (Options) String added in v0.3.0 

func (o Options) String() string

String renders the Options with the API key redacted, so printing a client or its options with %v or %+v never leaks the credential into logs.

type Part 

type Part interface {
	// contains filtered or unexported methods
}

Part is a single piece of a Message's content. The concrete part types are Text, Image, ToolUse and ToolResult. The set is closed: Part cannot be implemented outside this package, so drivers can switch over it exhaustively.

type Request 

type Request struct {
	Model       string
	System      string // optional system prompt
	Messages    []Message
	Tools       []Tool
	ToolChoice  ToolChoice
	MaxTokens   int
	Temperature *float64
	TopP        *float64
	Stop        []string

	// Format asks for a structured response, JSON or JSON matching a schema.
	// Nil, the default, asks for nothing. Drivers map it onto native provider
	// support where it exists and request it in the prompt where it does not;
	// [Response.Format] reports which happened. See [Format].
	Format *Format
}

Request is a provider-agnostic generation request. Only Model and Messages are required; the remaining fields are applied when set. Temperature and TopP are pointers so that "unset" is distinct from an explicit zero.

func (*Request) Validate 

func (r *Request) Validate() error

Validate reports whether the request has the minimum a provider needs. A nil request is reported as ErrNoRequest rather than panicking, so a driver can forward a bad call as a normal error.

type Response 

type Response struct {
	Model      string
	Parts      []Part
	StopReason string
	Usage      Usage
	Raw        json.RawMessage

	// Format reports how the driver satisfied [Request.Format]: enforced by
	// the provider, asked for in the prompt, or not requested at all. An
	// emulated format is a request, not a guarantee, so a malformed reply is
	// worth reading differently depending on this. See [FormatMode].
	Format FormatMode
}

Response is the result of a non-streaming Client.Generate call. Parts holds the assistant's output blocks (text and any tool calls); Raw keeps the provider's original JSON for access to fields this package does not model.

func (*Response) JSON added in v0.4.0 

func (r *Response) JSON(v any) error

JSON decodes the response text into v, which is what a request carrying a Format is asking for: 

var seo SEO
if err := resp.JSON(&seo); err != nil { ... }

The text must be one JSON value. A reply wrapped whole in a Markdown code fence is unwrapped first, because a provider that was only asked for JSON in the prompt tends to add one; anything else - prose around the value, a second value after it - is an error rather than something to go hunting through. Salvaging JSON out of arbitrary text is how a wrong answer gets read as a right one.

JSON does not consult Response.Format: text that is JSON decodes whatever produced it.

func (Response) MarshalJSON added in v0.3.0 

func (r Response) MarshalJSON() ([]byte, error)

MarshalJSON encodes the response with each output part tagged by its "type".

func (*Response) Text 

func (r *Response) Text() string

Text returns the concatenation of all text parts in the response.

func (*Response) ToolCall added in v0.1.1 

func (r *Response) ToolCall(name string) (ToolUse, bool)

ToolCall returns the first tool call with the given name and whether one was found. It is a convenience for dispatching a single expected tool.

Example 
package main

import (
	"encoding/json"
	"fmt"

	"github.com/goloop/ai"
)

func main() {
	resp := &ai.Response{Parts: []ai.Part{
		ai.Text{Text: "let me check"},
		ai.ToolUse{ID: "call_1", Name: "get_weather", Input: json.RawMessage(`{"city":"Kyiv"}`)},
	}}

	if call, ok := resp.ToolCall("get_weather"); ok {
		fmt.Printf("%s(%s)\n", call.Name, call.Input)
	}
}

Output:
get_weather({"city":"Kyiv"})

func (*Response) ToolCalls 

func (r *Response) ToolCalls() []ToolUse

ToolCalls returns the tool-call parts the model produced, in order.

func (*Response) UnmarshalJSON added in v0.3.0 

func (r *Response) UnmarshalJSON(data []byte) error

UnmarshalJSON reconstructs the response and its concrete part types.

type Role 

type Role string

Role identifies who a Message comes from in a conversation. 

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

The roles a Message can take. Providers map these onto their own wire values; RoleTool marks a message that carries tool results back to the model.

type Text 

type Text struct {
	Text string
}

Text is a plain-text content part.

type Tool 

type Tool struct {
	Name        string
	Description string
	Schema      json.RawMessage
}

Tool describes a function the model may call. Schema is a JSON Schema object describing the tool's input; drivers pass it through to the provider in the shape that provider expects.

Example 
package main

import (
	"encoding/json"
	"fmt"

	"github.com/goloop/ai"
)

func main() {
	tool := ai.Tool{
		Name:        "get_weather",
		Description: "Get the current weather for a city.",
		Schema:      json.RawMessage(`{"type":"object","properties":{"city":{"type":"string"}}}`),
	}
	fmt.Println(tool.Name)
}

Output:
get_weather

type ToolChoice 

type ToolChoice int

ToolChoice controls whether and how the model may call tools in a Request. 

const (
	ToolAuto ToolChoice = iota
	ToolNone
	ToolRequired
)

The tool-calling strategies. ToolAuto lets the model decide, ToolNone forbids tool calls, and ToolRequired forces the model to call at least one tool.

type ToolResult 

type ToolResult struct {
	ID      string
	Content string
	IsError bool
}

ToolResult carries the result of a tool call back to the model. ID must match the ToolUse it answers. Set IsError to report that the tool failed.

type ToolUse 

type ToolUse struct {
	ID    string          // provider-assigned call identifier
	Name  string          // name of the tool to invoke
	Input json.RawMessage // arguments as a JSON object
}

ToolUse is a request from the assistant to call a tool. Input is the raw JSON arguments object produced by the model, validated against the matching Tool schema by the caller.

type Usage 

type Usage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

Usage reports how many tokens a request consumed.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.