README
¶
ai
ai is a small, provider-agnostic interface for talking to large language model
APIs, plus the shared request and response types every provider driver speaks.
It is the core that goloop's provider packages (anthropic, openai, gemini,
and so on) build on.
Like the standard library's database/sql with its drivers, or log/slog with
its handlers, this package holds the common contract while a separate package
per provider implements it. A driver depends only on ai, so the whole set
stays free of third-party dependencies.
Installation
go get github.com/goloop/ai
The interface
type Client interface {
Generate(ctx context.Context, req *Request) (*Response, error)
Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}
Types
RoleandMessage(a role plus a list of contentPartvalues).Part:Text,Image(multimodal),ToolUse,ToolResult(tool calling).ToolandToolChoicefor function calling.Request(model, system, messages, tools, sampling knobs).ResponsewithText()andToolCalls()helpers;Chunkfor streaming.Usagefor token counts;APIErrorfor normalized provider errors.
Using a provider
import (
"github.com/goloop/ai"
"github.com/goloop/anthropic"
)
c := anthropic.New(os.Getenv("ANTHROPIC_API_KEY"))
resp, err := c.Generate(ctx, &ai.Request{
Model: anthropic.ModelClaude37SonnetLatest,
Messages: []ai.Message{ai.UserText("Hello!")},
})
Any provider client is an ai.Client, so code written against the interface
works with all of them, which makes multi-provider setups straightforward.
Plumbing for drivers
Drivers reuse the shared configuration and transport:
Optionsand functional options (WithBaseURL,WithHTTPClient,WithTimeout,WithMaxRetries,WithHeader), built withNewOptions.Options.Do- an HTTP request with retries on 429 and 5xx.SSEEvents- an iterator over Server-Sent Events data payloads.
Endpoints providers do not share (embeddings, images, audio, files, batches) are not part of the interface; each driver exposes them as native methods.
Documentation
Full reference: DOC.md (Ukrainian: DOC.UK.md).
License
MIT - see LICENSE.
Documentation
¶
Overview ¶
Package ai defines a single, provider-agnostic interface for talking to large language model APIs, together with the shared request and response types every provider driver speaks.
The design mirrors the standard library's split between an interface and its drivers: like database/sql with its drivers, or log/slog with its handlers, package ai holds the common contract while a separate package per provider (anthropic, openai, gemini, and so on) implements it. A driver depends only on this package, so the whole set stays free of third-party dependencies.
The contract is the Client interface:
type Client interface {
Generate(ctx context.Context, req *Request) (*Response, error)
Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}
A Request carries a model, an optional system prompt, a list of Messages, optional Tools and the usual sampling knobs. A Message is a role plus a list of content Parts (Text, Image, ToolUse, ToolResult), which is enough to express multimodal input and tool calling across providers. Generate returns a whole Response; Stream yields Chunks as they arrive.
Endpoints that providers do not share (embeddings, image generation, audio, files, batches, and so on) are not part of this interface. Each driver exposes those as its own native methods, so the common surface stays small and honest while provider-specific power is still available.
This package also carries the plumbing drivers reuse: Options and its functional configuration, Options.Do for HTTP requests with retries, and SSEEvents for reading Server-Sent Events streams.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrNoRequest = errors.New("ai: request is nil") ErrNoModel = errors.New("ai: model is required") ErrNoMessages = errors.New("ai: at least one message is required") )
Sentinel errors returned before a request reaches the network.
Functions ¶
func SSEEvents ¶
SSEEvents returns an iterator over the data payloads of a Server-Sent Events stream read from r. Each yielded string is the concatenated "data:" content of one event, with the field prefix and the single optional leading space removed and multi-line data joined by newlines. Comment lines (starting with ":") and other fields (event:, id:, retry:) are ignored, which is enough for the streaming chat APIs the drivers target. A read error is yielded once with an empty payload, after which iteration stops.
The caller is responsible for closing r.
Types ¶
type APIError ¶
type APIError struct {
Status int // HTTP status code
Type string // provider error type, when given
Code string // provider error code, when given
Message string // human-readable message, when given
Raw json.RawMessage // original error body
}
APIError is a normalized error for a non-success HTTP response from a provider. Drivers fill the fields they can parse from the provider's error body and keep the original JSON in Raw.
type Chunk ¶
Chunk is one increment of a streaming response from Client.Stream. Text is the incremental text delta; ToolCall is set when the chunk carries a completed tool call; Done marks the final chunk. Drivers set Usage on the Done chunk; its counts are zero when the provider did not report usage. Raw keeps the provider's original event JSON.
type Client ¶
type Client interface {
Generate(ctx context.Context, req *Request) (*Response, error)
Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}
Client is the contract every provider driver implements. Generate performs a single request and returns the whole response. Stream performs a request and returns an iterator over response chunks; the iterator yields a zero Chunk with a non-nil error and stops if the stream fails.
Example (Generate) ¶
package main
import (
"context"
"fmt"
"iter"
"github.com/goloop/ai"
)
// mockClient is a trivial in-memory [ai.Client]. A real driver sends the
// request to a provider; this shows the contract Generate and Stream must
// satisfy - the same shape every goloop AI provider implements.
type mockClient struct{}
func (mockClient) Generate(ctx context.Context, req *ai.Request) (*ai.Response, error) {
if err := req.Validate(); err != nil {
return nil, err
}
return &ai.Response{
Model: req.Model,
Parts: []ai.Part{ai.Text{Text: "Hello!"}},
StopReason: "stop",
Usage: ai.Usage{InputTokens: 3, OutputTokens: 2},
}, nil
}
func (mockClient) Stream(ctx context.Context, req *ai.Request) iter.Seq2[ai.Chunk, error] {
return func(yield func(ai.Chunk, error) bool) {
if err := req.Validate(); err != nil {
yield(ai.Chunk{}, err)
return
}
for _, part := range []string{"Hel", "lo!"} {
if !yield(ai.Chunk{Text: part}, nil) {
return
}
}
yield(ai.Chunk{Done: true, Usage: &ai.Usage{InputTokens: 3, OutputTokens: 2}}, nil)
}
}
func main() {
var c ai.Client = mockClient{}
resp, err := c.Generate(context.Background(), &ai.Request{
Model: "demo",
Messages: []ai.Message{
ai.SystemText("You are concise."),
ai.UserText("Say hi."),
},
})
if err != nil {
panic(err)
}
fmt.Println(resp.Text())
fmt.Printf("%d in / %d out\n", resp.Usage.InputTokens, resp.Usage.OutputTokens)
}
Output: Hello! 3 in / 2 out
Example (Stream) ¶
package main
import (
"context"
"fmt"
"iter"
"github.com/goloop/ai"
)
// mockClient is a trivial in-memory [ai.Client]. A real driver sends the
// request to a provider; this shows the contract Generate and Stream must
// satisfy - the same shape every goloop AI provider implements.
type mockClient struct{}
func (mockClient) Generate(ctx context.Context, req *ai.Request) (*ai.Response, error) {
if err := req.Validate(); err != nil {
return nil, err
}
return &ai.Response{
Model: req.Model,
Parts: []ai.Part{ai.Text{Text: "Hello!"}},
StopReason: "stop",
Usage: ai.Usage{InputTokens: 3, OutputTokens: 2},
}, nil
}
func (mockClient) Stream(ctx context.Context, req *ai.Request) iter.Seq2[ai.Chunk, error] {
return func(yield func(ai.Chunk, error) bool) {
if err := req.Validate(); err != nil {
yield(ai.Chunk{}, err)
return
}
for _, part := range []string{"Hel", "lo!"} {
if !yield(ai.Chunk{Text: part}, nil) {
return
}
}
yield(ai.Chunk{Done: true, Usage: &ai.Usage{InputTokens: 3, OutputTokens: 2}}, nil)
}
}
func main() {
var c ai.Client = mockClient{}
for chunk, err := range c.Stream(context.Background(), &ai.Request{
Model: "demo",
Messages: []ai.Message{ai.UserText("Say hi.")},
}) {
if err != nil {
panic(err)
}
fmt.Print(chunk.Text)
}
fmt.Println()
}
Output: Hello!
type Image ¶
type Image struct {
MIME string // for example "image/png" or "image/jpeg"
Data []byte // inline image bytes, or nil when URL is set
URL string // remote image URL, or "" when Data is set
}
Image is an image content part. Provide either inline Data with its MIME type, or a URL when the provider supports fetching remote images. Drivers encode Data as base64 as required by their wire format.
type Message ¶
Message is one turn in a conversation: a role and its content parts.
func AssistantText ¶
AssistantText returns an assistant Message containing a single text part.
func SystemText ¶ added in v0.1.1
SystemText returns a system Message containing a single text part. Drivers fold system messages into the provider's system prompt; the Request.System field is an equivalent shorthand for a single instruction.
func (Message) MarshalJSON ¶ added in v0.3.0
MarshalJSON encodes the message with each part tagged by its "type".
func (*Message) UnmarshalJSON ¶ added in v0.3.0
UnmarshalJSON reconstructs the message and its concrete part types.
type Option ¶
type Option func(*Options)
Option configures Options. The same options work across every provider, so client construction looks the same everywhere.
func WithBaseURL ¶
WithBaseURL overrides the provider's default API base URL. It is useful for proxies, gateways, mock servers and self-hosted deployments.
func WithHTTPClient ¶
WithHTTPClient sets the HTTP client used for requests. When set, its own timeout takes precedence over WithTimeout.
func WithHeader ¶
WithHeader adds a header sent with every request, for example a custom API version or a beta feature flag.
func WithMaxRetries ¶
WithMaxRetries sets how many times Options.Do retries a request on HTTP 429 or 5xx responses. Zero disables retrying.
func WithTimeout ¶
WithTimeout sets the per-request timeout used when no custom HTTP client is provided.
type Options ¶
type Options struct {
APIKey string
BaseURL string
HTTPClient *http.Client
Timeout time.Duration
MaxRetries int
Headers http.Header
}
Options is the shared client configuration every driver understands. Drivers build it from an API key and functional options with NewOptions and reuse it for transport (see Options.Do).
func NewOptions ¶
NewOptions builds Options from an API key and functional options, filling in defaults: a 60s timeout, two retries and an HTTP client when none is given.
func (Options) Do ¶
func (o Options) Do( ctx context.Context, method, url string, body []byte, headers http.Header, ) (*http.Response, error)
Do sends an HTTP request using the configured client and headers, retrying transient responses (HTTP 429, 502, 503, 504 and 529) up to MaxRetries with jittered exponential backoff. A Retry-After header on the response is honored, capped at 30s. HTTP 500 is not retried, because driver requests are non-idempotent POSTs. The Options headers are applied first, then the per-call headers override them.
On the final attempt the response is returned as-is even when its status is an error, so the caller can read the provider's error body (drivers check the status and call their own error parser). Do only returns a non-nil error for a transport-level failure (no response) or a canceled context. The caller owns the returned response body and must close it.
Retries repeat the request, including non-idempotent POSTs. A transport error may occur after the server already accepted the request, so a retried POST can execute twice; use WithMaxRetries(0) to disable retrying when that is a concern.
type Part ¶
type Part interface {
// contains filtered or unexported methods
}
Part is a single piece of a Message's content. The concrete part types are Text, Image, ToolUse and ToolResult. The set is closed: Part cannot be implemented outside this package, so drivers can switch over it exhaustively.
type Request ¶
type Request struct {
Model string
System string // optional system prompt
Messages []Message
Tools []Tool
ToolChoice ToolChoice
MaxTokens int
Temperature *float64
TopP *float64
Stop []string
}
Request is a provider-agnostic generation request. Only Model and Messages are required; the remaining fields are applied when set. Temperature and TopP are pointers so that "unset" is distinct from an explicit zero.
type Response ¶
type Response struct {
Model string
Parts []Part
StopReason string
Usage Usage
Raw json.RawMessage
}
Response is the result of a non-streaming Client.Generate call. Parts holds the assistant's output blocks (text and any tool calls); Raw keeps the provider's original JSON for access to fields this package does not model.
func (Response) MarshalJSON ¶ added in v0.3.0
MarshalJSON encodes the response with each output part tagged by its "type".
func (*Response) ToolCall ¶ added in v0.1.1
ToolCall returns the first tool call with the given name and whether one was found. It is a convenience for dispatching a single expected tool.
Example ¶
package main
import (
"encoding/json"
"fmt"
"github.com/goloop/ai"
)
func main() {
resp := &ai.Response{Parts: []ai.Part{
ai.Text{Text: "let me check"},
ai.ToolUse{ID: "call_1", Name: "get_weather", Input: json.RawMessage(`{"city":"Kyiv"}`)},
}}
if call, ok := resp.ToolCall("get_weather"); ok {
fmt.Printf("%s(%s)\n", call.Name, call.Input)
}
}
Output: get_weather({"city":"Kyiv"})
func (*Response) UnmarshalJSON ¶ added in v0.3.0
UnmarshalJSON reconstructs the response and its concrete part types.
type Tool ¶
type Tool struct {
Name string
Description string
Schema json.RawMessage
}
Tool describes a function the model may call. Schema is a JSON Schema object describing the tool's input; drivers pass it through to the provider in the shape that provider expects.
Example ¶
package main
import (
"encoding/json"
"fmt"
"github.com/goloop/ai"
)
func main() {
tool := ai.Tool{
Name: "get_weather",
Description: "Get the current weather for a city.",
Schema: json.RawMessage(`{"type":"object","properties":{"city":{"type":"string"}}}`),
}
fmt.Println(tool.Name)
}
Output: get_weather
type ToolChoice ¶
type ToolChoice int
ToolChoice controls whether and how the model may call tools in a Request.
const ( ToolAuto ToolChoice = iota ToolNone ToolRequired )
The tool-calling strategies. ToolAuto lets the model decide, ToolNone forbids tool calls, and ToolRequired forces the model to call at least one tool.
type ToolResult ¶
ToolResult carries the result of a tool call back to the model. ID must match the ToolUse it answers. Set IsError to report that the tool failed.
type ToolUse ¶
type ToolUse struct {
ID string // provider-assigned call identifier
Name string // name of the tool to invoke
Input json.RawMessage // arguments as a JSON object
}
ToolUse is a request from the assistant to call a tool. Input is the raw JSON arguments object produced by the model, validated against the matching Tool schema by the caller.
Source Files