ai

package module
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 15, 2026 License: MIT Imports: 14 Imported by: 0

README

deps.dev License License Stay with Ukraine

ai

ai is a small, provider-agnostic interface for talking to large language model APIs, plus the shared request and response types every provider driver speaks. It is the core that goloop's provider packages (anthropic, openai, gemini, and so on) build on.

Like the standard library's database/sql with its drivers, or log/slog with its handlers, this package holds the common contract while a separate package per provider implements it. A driver depends only on ai, so the whole set stays free of third-party dependencies.

Installation

go get github.com/goloop/ai

The interface

type Client interface {
	Generate(ctx context.Context, req *Request) (*Response, error)
	Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}

Types

  • Role and Message (a role plus a list of content Part values).
  • Part: Text, Image (multimodal), ToolUse, ToolResult (tool calling).
  • Tool and ToolChoice for function calling.
  • Request (model, system, messages, tools, sampling knobs).
  • Format for structured output: ask for JSON, or JSON matching a schema.
  • Hosted for work the provider runs itself, such as web search, with Citation for the sources it came back with and HostedReport for whether it ran at all.
  • Response with Text(), JSON(&v), ToolCalls() and Citations() helpers; Chunk for streaming.
  • Usage for token counts; APIError for normalized provider errors.

Using a provider

import (
	"github.com/goloop/ai"
	"github.com/goloop/anthropic"
)

c := anthropic.New(os.Getenv("ANTHROPIC_API_KEY"))
resp, err := c.Generate(ctx, &ai.Request{
	Model:    anthropic.ModelClaude37SonnetLatest,
	Messages: []ai.Message{ai.UserText("Hello!")},
})

Any provider client is an ai.Client, so code written against the interface works with all of them, which makes multi-provider setups straightforward.

Plumbing for drivers

Drivers reuse the shared configuration and transport:

  • Options and functional options (WithBaseURL, WithHTTPClient, WithTimeout, WithMaxRetries, WithHeader), built with NewOptions.
  • Options.Do - an HTTP request with retries on 429 and 5xx.
  • SSEEvents - an iterator over Server-Sent Events data payloads.

Endpoints providers do not share (embeddings, images, audio, files, batches) are not part of the interface; each driver exposes them as native methods.

Documentation

Full reference: DOC.md (Ukrainian: DOC.UK.md).

License

MIT - see LICENSE.

Documentation

Overview

Package ai defines a single, provider-agnostic interface for talking to large language model APIs, together with the shared request and response types every provider driver speaks.

The design mirrors the standard library's split between an interface and its drivers: like database/sql with its drivers, or log/slog with its handlers, package ai holds the common contract while a separate package per provider (anthropic, openai, gemini, and so on) implements it. A driver depends only on this package, so the whole set stays free of third-party dependencies.

The contract is the Client interface: 

type Client interface {
    Generate(ctx context.Context, req *Request) (*Response, error)
    Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}

A Request carries a model, an optional system prompt, a list of Messages, optional Tools and the usual sampling knobs. A Message is a role plus a list of content Parts (Text, Image, ToolUse, ToolResult), which is enough to express multimodal input and tool calling across providers. Generate returns a whole Response; Stream yields Chunks as they arrive.

Structured output

Request.Format asks for JSON, optionally matching a JSON Schema, and Response.JSON decodes the reply into a Go value. Providers differ in what they can enforce: a driver uses native support where it exists, asks for the format in the prompt where it does not, and reports which it did in Response.Format. No driver drops a Format silently.

FormatEmulated is not a weaker FormatNative, it is a different thing: the model was asked, not constrained. Code that cannot proceed without an enforced shape should check for FormatNative, or validate the decoded value itself - Response.JSON proves the reply parses, not that it matches the schema.

Hosted capabilities

Request.Hosted asks the provider to do work on its own side, web search first among them: 

req.Hosted = []ai.Hosted{{Kind: ai.HostedWebSearch}}

A hosted capability is not a Tool and does not live in Request.Tools. A Tool is a promise that the caller will answer a ToolUse with a ToolResult; a hosted capability never comes back to the caller at all. Keeping them apart is what lets a tool loop written before this existed keep working: it never sees a call it does not know how to answer.

The sources behind an answer arrive as Citation values on the Text they support, and Response.Citations flattens them when only the list matters. An answer from a search that cannot be checked against its sources is worth less than no answer, so a driver whose provider reports sources always carries them through.

Response.Hosted reports what each requested capability actually did. HostedSkipped is the state worth watching for: a provider can accept a search tool and then answer from the model's own memory, and from the outside the two answers are identical. A caller that cannot accept that asks with Policy: ai.HostedRequired, and gets ErrHostedRequired instead of an answer that was never researched.

Not every provider can run every capability. A driver that cannot returns ErrNoHosted rather than answering without the search: unlike a Format, which a driver can ask for in the prompt, a search cannot be emulated, because a driver has no search engine of its own. A caller who would rather have the answer anyway asks again without Hosted, which is one visible if rather than a silent difference in what an answer is based on.

Knowing before asking

A driver may describe itself through the optional Capable interface, read with CapabilitiesOf, HostedCapabilityOf and SupportsHosted: 

if ai.SupportsHosted(client, ai.Hosted{Kind: ai.HostedWebSearch}) {
    // show the "search the web" control
}

This is a hint, not a permission. Whether a request succeeds depends on the model, the account and the region as much as on the driver, so ErrNoHosted and ErrNoFormat stay the source of truth and a caller still handles them. What Capabilities is for is the decision taken before the call: whether to offer a feature at all, and whether it needs one request or two. A driver that does not describe itself reports nothing rather than no, so code written against this degrades to asking and handling the answer.

SupportsImages answers the same kind of question for image generation: 

if ai.SupportsImages(client) {
    // offer the "generate an image" control
}

Image generation is not part of the Client interface - each driver exposes its own GenerateImage with its own types, because providers do not share the shape - so this hint is only what a UI needs to decide whether to offer it.

Structured output and hosted capabilities together

Several providers refuse a strict schema and a server-side tool in the same call, and the ones that accept it do not all honor both. A driver that knows its provider cannot combine them returns ErrFormatWithHosted before the request leaves, rather than letting the provider reject it in its own words or, worse, return JSON that does not match the schema.

The way around it is two calls: one that searches and answers in prose, and one without Hosted that reshapes that prose into the schema. It costs twice and it is predictable, which is the better trade when the alternative is a well-formed value that quietly does not match what was asked for. HostedCapability.WithFormat says in advance which of the two a provider needs.

Format.Strict is worth its own warning. Without it a schema is a request: the model is shown the shape and asked to follow it, and the classic failure is a reply that is valid JSON and is the schema itself rather than data matching it - nothing errors, and the result reads as an empty answer. With it, providers that enforce schemas also demand that every object set additionalProperties to false and list all of its properties in required; a schema that forgets one is rejected by the provider, over the network, at run time. ValidateStrictSchema checks those rules against a schema literal before it is ever sent, and names the path to what is wrong.

Endpoints that providers do not share (embeddings, image generation, audio, files, batches, and so on) are not part of this interface. Each driver exposes those as its own native methods, so the common surface stays small and honest while provider-specific power is still available.

This package also carries the plumbing drivers reuse: Options and its functional configuration, Options.Do for HTTP requests with retries, and SSEEvents for reading Server-Sent Events streams.

Index

Examples

Constants

View Source 
const DefaultSchemaName = "response"

DefaultSchemaName is the schema name drivers use when a Format that needs one leaves Name empty.

Variables

View Source 
var (
	ErrNoRequest  = errors.New("ai: request is nil")
	ErrNoModel    = errors.New("ai: model is required")
	ErrNoMessages = errors.New("ai: at least one message is required")

	// Format errors, reported by Request.Validate before any driver work.
	ErrBadFormat = errors.New("ai: unknown format type")
	ErrNoSchema  = errors.New("ai: FormatJSONSchema requires a schema")
	ErrBadSchema = errors.New("ai: format schema must be a JSON Schema object")

	// ErrNoFormat is returned by a driver whose provider cannot produce the
	// requested format: it can neither enforce it nor be asked for it in a
	// way that is worth trusting, or the chosen model rejected it. A driver
	// never drops a Format quietly.
	//
	// It says what could not be done, not when that became known. A driver
	// that knows in advance returns it before the request leaves; one that
	// learns from the provider's own refusal wraps that refusal in it, so a
	// caller degrades with one errors.Is either way. The provider's original
	// [APIError], where there was one, stays reachable with errors.As.
	ErrNoFormat = errors.New("ai: provider cannot produce the requested format")

	// ErrBadStrictSchema is returned by [ValidateStrictSchema], and by drivers
	// that run it, when a schema cannot be enforced strictly as written.
	ErrBadStrictSchema = errors.New("ai: strict schema is not provider-compatible")

	// ErrNoText is returned by Response.JSON when the reply carried no text
	// to decode, for example a response made up entirely of tool calls.
	ErrNoText = errors.New("ai: response has no text")

	// Hosted-capability errors, reported by Request.Validate before any
	// driver work.
	ErrBadHosted       = errors.New("ai: unknown hosted capability")
	ErrBadHostedPolicy = errors.New("ai: unknown hosted policy")
	ErrDupHosted       = errors.New("ai: hosted capability requested twice")

	// ErrNoHosted is returned by a driver whose provider cannot run the
	// requested capability, or cannot run it under the constraints given. A
	// format can be emulated by asking the model nicely; a search cannot,
	// because a driver has no search engine of its own. A caller who would
	// rather have the answer without the search asks again without Hosted,
	// which is one visible if.
	//
	// Drivers wrap it to say what exactly they could not do:
	//
	//	fmt.Errorf("%w: blocked domains", ai.ErrNoHosted)
	//
	// Like [ErrNoFormat], it says what could not be done rather than when that
	// became known. Some providers accept the capability for one model and
	// reject it for another, so a driver that only learns from the provider's
	// refusal wraps that refusal in this error rather than handing back a bare
	// [APIError] for the caller to read prose out of. The original APIError
	// stays reachable with errors.As.
	ErrNoHosted = errors.New("ai: provider cannot run the requested hosted capability")

	// ErrHostedRequired is returned when a capability requested with
	// HostedRequired was offered to the provider and the model did not use
	// it. Unlike ErrNoHosted this is not a provider limitation: the request
	// was sent and the answer simply is not the one that was asked for.
	ErrHostedRequired = errors.New("ai: required hosted capability was not used")

	// ErrFormatWithHosted is returned by a driver whose provider cannot
	// combine a structured Format with a hosted capability in one call. It
	// is a driver decision rather than a Request.Validate one because the
	// combination is legal at some providers and not at others. The way
	// around it is two calls: one that searches and answers in prose, one
	// that reshapes that prose into the schema. See the package
	// documentation.
	ErrFormatWithHosted = errors.New("ai: provider cannot combine the requested format with a hosted capability")
)

Sentinel errors returned before a request reaches the network.

Functions

func SSEEvents 

func SSEEvents(r io.Reader) iter.Seq2[string, error]

SSEEvents returns an iterator over the data payloads of a Server-Sent Events stream read from r. Each yielded string is the concatenated "data:" content of one event, with the field prefix and the single optional leading space removed and multi-line data joined by newlines. Comment lines (starting with ":") and other fields (event:, id:, retry:) are ignored, which is enough for the streaming chat APIs the drivers target. A read error is yielded once with an empty payload, after which iteration stops.

The caller is responsible for closing r.

func SupportsHosted added in v1.1.0 

func SupportsHosted(c Client, h Hosted) bool

SupportsHosted reports whether a client claims it can run a capability as asked for, constraints included. A client that does not describe itself reports false, so this answers "is this known to work", never "is this known to fail" - the difference matters, because the honest answer to the second only comes from making the request.

func SupportsImages added in v1.2.0 

func SupportsImages(c Client) bool

SupportsImages reports whether a client claims it can generate images. It mirrors SupportsHosted: a client that does not describe itself reports false, so this answers "is this known to draw", not "is this known not to". Image generation itself is a native method on each driver, not part of the shared interface; this is only the hint a UI uses to decide whether to offer it.

func ValidateStrictSchema added in v1.1.0 

func ValidateStrictSchema(schema json.RawMessage) error

ValidateStrictSchema reports whether a schema can be enforced strictly as written, by the rules providers that offer strict structured output share: every object that declares properties must also set additionalProperties to false and list every one of those properties in required.

It exists because a schema is a literal, fully known before the first call, and the alternative to checking it here is learning about a typo from a 400 on a live key after a deploy. It walks properties, items, prefixItems, $defs/definitions and anyOf/oneOf/allOf, so a mistake nested three levels down is reported with the path that leads to it: 

properties.stories.items: "required" is missing "placeLocal"

It is deliberately not part of Request.Validate: strictness is a provider dialect, and a provider-agnostic validator has no business enforcing one. Drivers whose provider needs these rules call it themselves when Format.Strict is set; anyone assembling schemas can call it in a test.

This is not a JSON Schema validator. It checks the rules that decide whether strict mode is accepted, and says nothing about whether the schema describes what you meant.

Types

type APIError 

type APIError struct {
	Status  int             // HTTP status code
	Type    string          // provider error type, when given
	Code    string          // provider error code, when given
	Message string          // human-readable message, when given
	Raw     json.RawMessage // original error body
}

APIError is a normalized error for a non-success HTTP response from a provider. Drivers fill the fields they can parse from the provider's error body and keep the original JSON in Raw.

func (*APIError) Error 

func (e *APIError) Error() string

Error implements the error interface.

type Capabilities added in v1.1.0 

type Capabilities struct {
	// Hosted maps each capability the driver can run to what it accepts.
	// A kind that is absent is one the driver cannot run at all.
	Hosted map[HostedKind]HostedCapability

	// Format says which structured-output shapes the driver offers on an
	// ordinary request, and whether the provider enforces them or the driver
	// asks for them in the prompt. See [FormatCapability].
	Format FormatCapability

	// Images reports whether the driver can generate images. Image generation
	// is not part of the shared Client interface - each driver exposes its own
	// GenerateImage method with its own request and response types, because
	// providers do not share the shape - so this is only a hint: it lets a UI
	// build a "providers that can draw" list the same way it builds one for
	// web search, without a hand-kept table that goes stale. The zero value,
	// false, means the driver does not draw.
	Images bool
}

Capabilities describes what a driver can be asked for.

IT IS A HINT, NOT A PERMISSION. Whether a request actually succeeds depends on the model, the account, the region and the endpoint, none of which this value knows about. ErrNoHosted, ErrNoFormat and ErrFormatWithHosted remain the source of truth; a caller still handles them. What this is good for is the decision made before the call: whether to show a "search the web" button at all, and whether a feature needs one request or two.

func CapabilitiesOf added in v1.1.0 

func CapabilitiesOf(c Client) Capabilities

CapabilitiesOf returns what a client says it can do, or the zero Capabilities for a client that does not describe itself. The zero value claims nothing, so code written against it degrades to asking and handling the error, which is what it should have done anyway.

type Capable added in v1.1.0 

type Capable interface {
	Capabilities() Capabilities
}

Capable is implemented by drivers that can describe themselves. It is optional: a driver that does not implement it simply offers no hint, and everything else keeps working.

It exists because the knowledge of what a provider can do lives in the driver and, without this, gets copied into every application as a table of provider names - one that goes quietly out of date the day a driver learns something new.

type Chunk 

type Chunk struct {
	Text     string
	ToolCall *ToolUse
	Usage    *Usage
	Done     bool
	Raw      json.RawMessage

	// Citations carries the sources a hosted capability reported, as the
	// events announcing them arrive. It is a list because one event can name
	// several sources. A citation is delivered in the chunk the provider
	// reports it in, which is not necessarily the chunk carrying the text it
	// supports, so a caller assembling a document keeps its own list rather
	// than pairing them position by position. See [Citation].
	Citations []Citation

	// Hosted reports what each requested capability did. Drivers set it on
	// the Done chunk, where the counts are finally known, so it mirrors
	// [Response.Hosted] for a stream. See [HostedReport].
	Hosted []HostedReport
}

Chunk is one increment of a streaming response from Client.Stream. Text is the incremental text delta; ToolCall is set when the chunk carries a completed tool call; Done marks the final chunk. Drivers set Usage on the Done chunk; its counts are zero when the provider did not report usage. Raw keeps the provider's original event JSON.

type Citation added in v1.0.0 

type Citation struct {
	// URL is where the source lives, and Title is what the provider called
	// it. Title is empty when the provider does not report one.
	URL   string `json:"url,omitempty"`
	Title string `json:"title,omitempty"`

	// CitedText is the fragment of the source that supports the text, as the
	// provider reported it. It is the source's words, not ours: it is what
	// makes a citation checkable without fetching the URL.
	CitedText string `json:"cited_text,omitempty"`

	// StartByte and EndByte bound the span of the enclosing Text.Text that
	// this source supports. Both zero means the source supports the whole
	// part, which is also what a provider that reports no offsets gets.
	//
	// They are byte offsets into a Go string, so a driver whose provider
	// counts in something else - runes, UTF-16 code units, tokens - converts
	// them or leaves them zero. Passing another provider's units through
	// would put the boundary mid-character on any text that is not ASCII,
	// and Ukrainian or Greek prose is not ASCII.
	StartByte int `json:"start_byte,omitempty"`
	EndByte   int `json:"end_byte,omitempty"`
}

Citation is one source behind a piece of generated text.

Citations are attached to the Text they support rather than collected in one list on the response, because which sentence a source backs is the whole point of having sources at all. Response.Citations flattens them when the order is all that matters.

An answer produced by a hosted search without citations cannot be checked, so a driver that receives sources from its provider always carries them through.

type Client 

type Client interface {
	Generate(ctx context.Context, req *Request) (*Response, error)
	Stream(ctx context.Context, req *Request) iter.Seq2[Chunk, error]
}

Client is the contract every provider driver implements. Generate performs a single request and returns the whole response. Stream performs a request and returns an iterator over response chunks; the iterator yields a zero Chunk with a non-nil error and stops if the stream fails.

Example (Generate) 
package main

import (
	"context"
	"fmt"
	"iter"

	"github.com/goloop/ai"
)

// mockClient is a trivial in-memory [ai.Client]. A real driver sends the
// request to a provider; this shows the contract Generate and Stream must
// satisfy - the same shape every goloop AI provider implements.
type mockClient struct{}

func (mockClient) Generate(ctx context.Context, req *ai.Request) (*ai.Response, error) {
	if err := req.Validate(); err != nil {
		return nil, err
	}
	return &ai.Response{
		Model:      req.Model,
		Parts:      []ai.Part{ai.Text{Text: "Hello!"}},
		StopReason: "stop",
		Usage:      ai.Usage{InputTokens: 3, OutputTokens: 2},
	}, nil
}

func (mockClient) Stream(ctx context.Context, req *ai.Request) iter.Seq2[ai.Chunk, error] {
	return func(yield func(ai.Chunk, error) bool) {
		if err := req.Validate(); err != nil {
			yield(ai.Chunk{}, err)
			return
		}
		for _, part := range []string{"Hel", "lo!"} {
			if !yield(ai.Chunk{Text: part}, nil) {
				return
			}
		}
		yield(ai.Chunk{Done: true, Usage: &ai.Usage{InputTokens: 3, OutputTokens: 2}}, nil)
	}
}

func main() {
	var c ai.Client = mockClient{}

	resp, err := c.Generate(context.Background(), &ai.Request{
		Model: "demo",
		Messages: []ai.Message{
			ai.SystemText("You are concise."),
			ai.UserText("Say hi."),
		},
	})
	if err != nil {
		panic(err)
	}
	fmt.Println(resp.Text())
	fmt.Printf("%d in / %d out\n", resp.Usage.InputTokens, resp.Usage.OutputTokens)
}

Output:
Hello!
3 in / 2 out
Example (Stream) 
package main

import (
	"context"
	"fmt"
	"iter"

	"github.com/goloop/ai"
)

// mockClient is a trivial in-memory [ai.Client]. A real driver sends the
// request to a provider; this shows the contract Generate and Stream must
// satisfy - the same shape every goloop AI provider implements.
type mockClient struct{}

func (mockClient) Generate(ctx context.Context, req *ai.Request) (*ai.Response, error) {
	if err := req.Validate(); err != nil {
		return nil, err
	}
	return &ai.Response{
		Model:      req.Model,
		Parts:      []ai.Part{ai.Text{Text: "Hello!"}},
		StopReason: "stop",
		Usage:      ai.Usage{InputTokens: 3, OutputTokens: 2},
	}, nil
}

func (mockClient) Stream(ctx context.Context, req *ai.Request) iter.Seq2[ai.Chunk, error] {
	return func(yield func(ai.Chunk, error) bool) {
		if err := req.Validate(); err != nil {
			yield(ai.Chunk{}, err)
			return
		}
		for _, part := range []string{"Hel", "lo!"} {
			if !yield(ai.Chunk{Text: part}, nil) {
				return
			}
		}
		yield(ai.Chunk{Done: true, Usage: &ai.Usage{InputTokens: 3, OutputTokens: 2}}, nil)
	}
}

func main() {
	var c ai.Client = mockClient{}

	for chunk, err := range c.Stream(context.Background(), &ai.Request{
		Model:    "demo",
		Messages: []ai.Message{ai.UserText("Say hi.")},
	}) {
		if err != nil {
			panic(err)
		}
		fmt.Print(chunk.Text)
	}
	fmt.Println()
}

Output:
Hello!

type Format added in v0.4.0 

type Format struct {
	// Type is the shape wanted. The zero value, FormatText, asks for nothing.
	Type FormatType

	// Name labels the schema. Some providers require a name; drivers that do
	// substitute DefaultSchemaName when this is empty. It has no effect for
	// FormatJSON.
	Name string

	// Schema is a JSON Schema object describing the wanted value. It is
	// required for FormatJSONSchema and ignored otherwise.
	Schema json.RawMessage

	// Strict asks the provider to enforce the schema exactly rather than
	// treat it as guidance. Providers that cannot do this ignore the flag;
	// it never changes whether the request is accepted.
	Strict bool
}

Format asks for a structured response. It is provider-agnostic: a driver maps it onto whatever its provider offers natively, and asks for it in the prompt when the provider offers nothing. Either way a driver never ignores it silently - see FormatMode. 

req.Format = &ai.Format{
	Type:   ai.FormatJSONSchema,
	Name:   "seo",
	Schema: schema, // a JSON Schema object
}

Nil means no structured output was requested, which is the default.

func (*Format) AppendInstruction added in v0.4.0 

func (f *Format) AppendInstruction(system string) string

AppendInstruction returns system with Instruction appended after a blank line, which is how a driver folds the request into an existing system prompt. An empty system prompt yields the instruction alone, and a Format that asks for nothing leaves system untouched.

func (*Format) Instruction added in v0.4.0 

func (f *Format) Instruction() string

Instruction returns the wording a driver puts in the prompt when its provider cannot enforce the format itself. It lives here so that all nine drivers ask for the same thing in the same words; a driver with native support never calls it.

It returns an empty string when nothing was asked for.

func (*Format) SchemaName added in v0.4.0 

func (f *Format) SchemaName() string

SchemaName returns Name, or DefaultSchemaName when Name is empty.

func (*Format) Validate added in v0.4.0 

func (f *Format) Validate() error

Validate reports whether the format is one a driver can act on. A nil Format is valid: it asks for nothing.

An unknown Type is rejected rather than treated as one of the known ones. Nine drivers switch over this value, and a type none of them agrees on would mean nine different guesses about what the caller wanted.

type FormatCapability added in v1.1.0 

type FormatCapability struct {
	JSON       FormatMode // FormatJSON
	JSONSchema FormatMode // FormatJSONSchema
	Strict     FormatMode // FormatJSONSchema with Format.Strict set
}

FormatCapability says how each structured-output shape is satisfied. The zero value, FormatNone in every field, means the shape is not available at all - which for HostedCapability.WithFormat is the common case and the reason a feature may need two calls.

type FormatMode added in v0.4.0 

type FormatMode int

FormatMode says how a driver satisfied the Format of a request. It is reported on Response so that a caller can tell an enforced answer from a requested one - a provider that was merely asked nicely can still reply with prose, and that is worth knowing when output turns out malformed. 

const (
	FormatNone     FormatMode = iota // no format was requested
	FormatNative                     // the provider enforced it
	FormatEmulated                   // the driver asked for it in the prompt
)

How a format was satisfied.

func (FormatMode) String added in v0.4.0 

func (m FormatMode) String() string

String renders the mode for diagnostics.

type FormatType added in v0.4.0 

type FormatType int

FormatType selects the shape the model must produce. 

const (
	FormatText       FormatType = iota // free-form text, the provider default
	FormatJSON                         // a single valid JSON value, any shape
	FormatJSONSchema                   // JSON matching Format.Schema
)

The response shapes. FormatText is the zero value and asks for nothing, so a Request without a Format behaves exactly as before.

func (FormatType) String added in v0.4.0 

func (t FormatType) String() string

String renders the type for diagnostics.

type Hosted added in v1.0.0 

type Hosted struct {
	// Kind is the capability wanted. It has no default: the zero value is
	// not a capability and is rejected by Request.Validate.
	Kind HostedKind

	// Policy says whether the model may skip the capability. See
	// [HostedPolicy].
	Policy HostedPolicy

	// Web carries the knobs that only mean something for HostedWebSearch.
	// They live in their own type so that the shape of web search does not
	// become the shape every later capability has to wear. Nil asks for the
	// provider's own defaults.
	Web *HostedWeb
}

Hosted asks the provider to run a capability itself. It is deliberately not a Tool: a Tool is a promise that the caller will answer a ToolUse with a ToolResult, while a hosted capability never comes back to the caller at all. Keeping the two in separate request fields means a tool loop written before this existed keeps working unchanged, because it never sees a call it does not know how to answer. 

req.Hosted = []ai.Hosted{{Kind: ai.HostedWebSearch}}

A driver whose provider cannot run the capability returns ErrNoHosted rather than answering without it. See HostedMode for why.

func (Hosted) Validate added in v1.0.0 

func (h Hosted) Validate() error

Validate reports whether the capability is one a driver can act on. An unknown kind or policy is rejected rather than treated as the nearest known one, for the same reason Format.Validate rejects an unknown type: nine drivers read this value, and a value none of them agrees on would mean nine different guesses about what the caller wanted.

type HostedCapability added in v1.1.0 

type HostedCapability struct {
	// Web is set for [HostedWebSearch] and says which of the settings in
	// [HostedWeb] this provider can express. A setting reported false is one
	// that makes the request [ErrNoHosted] rather than one that is ignored.
	Web *HostedWebCapability

	// Policy says whether [HostedRequired] can be honored.
	Policy HostedPolicyCapability

	// WithFormat says whether this capability survives in the same call as a
	// structured format, and how. It decides the shape of a feature rather
	// than the details of a call: a provider that reports nothing here needs
	// two requests - one that searches and answers in prose, one that
	// reshapes that prose - while a provider that reports FormatEmulated or
	// FormatNative does it in one.
	//
	// FormatEmulated here means what it means on [Response.Format]: the model
	// was asked, not constrained. Reporting it as FormatNative would hand back
	// a guarantee where there is only a request.
	WithFormat FormatCapability
}

HostedCapability describes one capability a driver can run.

func HostedCapabilityOf added in v1.1.0 

func HostedCapabilityOf(c Client, k HostedKind) (HostedCapability, bool)

HostedCapabilityOf returns what a client says about one capability, and whether it claims to run it at all.

type HostedKind added in v1.0.0 

type HostedKind int

HostedKind identifies a capability the provider runs on its own side.

The zero value is deliberately not a capability: a Hosted built by mistake, or one decoded from an older payload, should fail validation instead of quietly asking for a web search nobody wanted. 

const (
	HostedWebSearch HostedKind = iota + 1
)

The hosted capabilities. Web search is the first of a family; code execution and file search fit the same shape when a provider offers them.

func (HostedKind) String added in v1.0.0 

func (k HostedKind) String() string

String renders the kind for diagnostics.

type HostedMode added in v1.0.0 

type HostedMode int

HostedMode says what became of one capability the request asked for. 

const (
	HostedNone    HostedMode = iota // it was not requested
	HostedNative                    // the provider ran it
	HostedSkipped                   // it was offered and the model did not use it
)

What happened to a hosted capability.

func (HostedMode) String added in v1.0.0 

func (m HostedMode) String() string

String renders the mode for diagnostics.

type HostedPolicy added in v1.0.0 

type HostedPolicy int

HostedPolicy says whether a hosted capability is an offer or part of the contract. Providers treat a hosted capability as optional by default: the model decides whether to use it, and often decides not to when it believes it already knows the answer. 

const (
	HostedAuto     HostedPolicy = iota // the model may use it
	HostedRequired                     // the answer must come from using it
)

The policies. HostedAuto is the zero value and leaves the choice to the model, which is what every provider does on its own.

func (HostedPolicy) String added in v1.0.0 

func (p HostedPolicy) String() string

String renders the policy for diagnostics.

type HostedPolicyCapability added in v1.1.0 

type HostedPolicyCapability struct {
	Required bool
}

HostedPolicyCapability says which policies a driver can honor. Auto is always true for a capability the driver runs at all; Required is separate because a provider may offer a capability and no way to insist on it.

type HostedReport added in v1.0.0 

type HostedReport struct {
	// Kind is the capability this report is about.
	Kind HostedKind

	// Mode is what happened. See [HostedMode].
	Mode HostedMode

	// Calls is how many times the provider ran the capability, when it says
	// so. Hosted work is usually billed separately from tokens, so this is
	// the only place the cost of a request shows up. A driver that can prove
	// the capability ran but cannot get a count reports 1.
	Calls int
}

HostedReport says what one requested capability actually did.

HostedSkipped is the state worth having: a provider can accept a search tool and then answer from the model's own memory, and the two answers look identical from the outside. Without this report a caller cannot tell "looked and found nothing" from "never looked", which is exactly the distinction that decides whether an answer is worth trusting.

type HostedWeb added in v1.0.0 

type HostedWeb struct {
	// MaxUses bounds how many searches the provider may run for one request.
	// Zero leaves it to the provider. It counts searches rather than results
	// because that is the unit providers actually meter and bill.
	MaxUses int

	// AllowDomains restricts results to these domains; BlockDomains excludes
	// them. Empty means no restriction. Providers generally accept one list
	// or the other but not both at once, so a driver given both returns
	// ErrNoHosted rather than picking one and dropping the other.
	AllowDomains []string
	BlockDomains []string

	// Region is an ISO 3166-1 alpha-2 country code that biases results
	// ("UA", "DE"). Empty leaves it to the provider.
	Region string
}

HostedWeb configures hosted web search. Every field is a constraint, not a hint: a driver that cannot express a non-zero field returns ErrNoHosted instead of running a wider search than was asked for. A caller who wants the wider search can ask for it by leaving the field alone.

type HostedWebCapability added in v1.1.0 

type HostedWebCapability struct {
	MaxUses      bool
	AllowDomains bool
	BlockDomains bool

	// BothDomainLists says whether an allow list and a block list can be sent
	// together. Most providers take one or the other.
	BothDomainLists bool

	Region bool
}

HostedWebCapability says which web-search settings a provider can express. Every field mirrors one field of HostedWeb; false means a non-zero value there is refused rather than silently widened.

type Image 

type Image struct {
	MIME string // for example "image/png" or "image/jpeg"
	Data []byte // inline image bytes, or nil when URL is set
	URL  string // remote image URL, or "" when Data is set
}

Image is an image content part. Provide either inline Data with its MIME type, or a URL when the provider supports fetching remote images. Drivers encode Data as base64 as required by their wire format.

type Message 

type Message struct {
	Role  Role
	Parts []Part
}

Message is one turn in a conversation: a role and its content parts.

func AssistantText 

func AssistantText(s string) Message

AssistantText returns an assistant Message containing a single text part.

func SystemText added in v0.1.1 

func SystemText(s string) Message

SystemText returns a system Message containing a single text part. Drivers fold system messages into the provider's system prompt; the Request.System field is an equivalent shorthand for a single instruction.

func UserText 

func UserText(s string) Message

UserText returns a user Message containing a single text part.

func (Message) MarshalJSON added in v0.3.0 

func (m Message) MarshalJSON() ([]byte, error)

MarshalJSON encodes the message with each part tagged by its "type".

func (*Message) UnmarshalJSON added in v0.3.0 

func (m *Message) UnmarshalJSON(data []byte) error

UnmarshalJSON reconstructs the message and its concrete part types.

type Option 

type Option func(*Options)

Option configures Options. The same options work across every provider, so client construction looks the same everywhere.

func WithBaseURL 

func WithBaseURL(u string) Option

WithBaseURL overrides the provider's default API base URL. It is useful for proxies, gateways, mock servers and self-hosted deployments.

func WithHTTPClient 

func WithHTTPClient(c *http.Client) Option

WithHTTPClient sets the HTTP client used for requests. When set, its own timeout takes precedence over WithTimeout.

func WithHeader 

func WithHeader(key, value string) Option

WithHeader adds a header sent with every request, for example a custom API version or a beta feature flag.

func WithMaxRetries 

func WithMaxRetries(n int) Option

WithMaxRetries sets how many times Options.Do retries a request on HTTP 429 or 5xx responses. Zero disables retrying.

func WithTimeout 

func WithTimeout(d time.Duration) Option

WithTimeout sets the per-request timeout used when no custom HTTP client is provided.

type Options 

type Options struct {
	APIKey     string
	BaseURL    string
	HTTPClient *http.Client
	Timeout    time.Duration
	MaxRetries int
	Headers    http.Header
}

Options is the shared client configuration every driver understands. Drivers build it from an API key and functional options with NewOptions and reuse it for transport (see Options.Do).

func NewOptions 

func NewOptions(apiKey string, opts ...Option) Options

NewOptions builds Options from an API key and functional options, filling in defaults: a 60s timeout, two retries and an HTTP client when none is given.

func (Options) Do 

func (o Options) Do(
	ctx context.Context,
	method, url string,
	body []byte,
	headers http.Header,
) (*http.Response, error)

Do sends an HTTP request using the configured client and headers, retrying transient responses (HTTP 429, 502, 503, 504 and 529) up to MaxRetries with jittered exponential backoff. A Retry-After header on the response is honored, capped at 30s. HTTP 500 is not retried, because driver requests are non-idempotent POSTs. The Options headers are applied first, then the per-call headers override them.

On the final attempt the response is returned as-is even when its status is an error, so the caller can read the provider's error body (drivers check the status and call their own error parser). Do only returns a non-nil error for a transport-level failure (no response) or a canceled context. The caller owns the returned response body and must close it.

Retries repeat the request, including non-idempotent POSTs. A transport error may occur after the server already accepted the request, so a retried POST can execute twice; use WithMaxRetries(0) to disable retrying when that is a concern.

func (Options) GoString added in v0.3.0 

func (o Options) GoString() string

GoString redacts the API key for the %#v verb, matching String.

func (Options) String added in v0.3.0 

func (o Options) String() string

String renders the Options with the API key redacted, so printing a client or its options with %v or %+v never leaks the credential into logs.

type Part 

type Part interface {
	// contains filtered or unexported methods
}

Part is a single piece of a Message's content. The concrete part types are Text, Image, ToolUse and ToolResult. The set is closed: Part cannot be implemented outside this package, so drivers can switch over it exhaustively.

type Request 

type Request struct {
	Model       string
	System      string // optional system prompt
	Messages    []Message
	Tools       []Tool
	ToolChoice  ToolChoice
	MaxTokens   int
	Temperature *float64
	TopP        *float64
	Stop        []string

	// Format asks for a structured response, JSON or JSON matching a schema.
	// Nil, the default, asks for nothing. Drivers map it onto native provider
	// support where it exists and request it in the prompt where it does not;
	// [Response.Format] reports which happened. See [Format].
	Format *Format

	// Hosted asks the provider to run capabilities of its own, web search
	// first among them. It is separate from Tools on purpose: a hosted
	// capability never produces a [ToolUse] the caller has to answer, so code
	// that loops over tool calls and knows nothing about this field keeps
	// working unchanged. [Response.Hosted] reports what each one did. See
	// [Hosted].
	//
	// Not every provider can run every capability, and a driver that cannot
	// returns [ErrNoHosted] rather than answering without it.
	Hosted []Hosted
}

Request is a provider-agnostic generation request. Only Model and Messages are required; the remaining fields are applied when set. Temperature and TopP are pointers so that "unset" is distinct from an explicit zero.

func (*Request) HostedByKind added in v1.0.0 

func (r *Request) HostedByKind(k HostedKind) (Hosted, bool)

HostedByKind returns the requested capability of the given kind and whether it was requested at all. Request.Validate rejects a kind listed twice, so there is at most one to find.

func (*Request) HostedReports added in v1.0.0 

func (r *Request) HostedReports(calls map[HostedKind]int) ([]HostedReport, error)

HostedReports pairs everything the request asked for with what the provider did, in request order, and enforces HostedRequired while it is at it. It lives here so that nine drivers agree on the answer instead of each deciding separately what "the provider skipped it" means, the same reason Format.Instruction is shared.

A driver calls it once it has read the reply, passing how many times each capability ran: 

reports, err := req.HostedReports(map[ai.HostedKind]int{
	ai.HostedWebSearch: searches,
})

A driver that can prove a capability ran but cannot get a count from its provider passes 1; a kind missing from calls, or present with a zero count, was not used. A request that asked for nothing yields no reports and no error, which is what every call written before this existed does.

A capability asked for with HostedRequired and not used is ErrHostedRequired rather than a response the caller has to inspect: a required search that did not happen makes the answer something other than what was asked for, and an answer from the model's own memory is indistinguishable from a researched one by the time it reaches the caller.

func (*Request) Validate 

func (r *Request) Validate() error

Validate reports whether the request has the minimum a provider needs. A nil request is reported as ErrNoRequest rather than panicking, so a driver can forward a bad call as a normal error.

type Response 

type Response struct {
	Model      string
	Parts      []Part
	StopReason string
	Usage      Usage
	Raw        json.RawMessage

	// Format reports how the driver satisfied [Request.Format]: enforced by
	// the provider, asked for in the prompt, or not requested at all. An
	// emulated format is a request, not a guarantee, so a malformed reply is
	// worth reading differently depending on this. See [FormatMode].
	Format FormatMode

	// Hosted reports what became of each capability [Request.Hosted] asked
	// for, in request order. It is a list rather than one value because a
	// request may ask for several capabilities and they do not share a fate:
	// the model can search and skip running code in the same reply. A
	// request that asked for nothing gets no reports. See [HostedReport].
	Hosted []HostedReport
}

Response is the result of a non-streaming Client.Generate call. Parts holds the assistant's output blocks (text and any tool calls); Raw keeps the provider's original JSON for access to fields this package does not model.

func (*Response) Citations added in v1.0.0 

func (r *Response) Citations() []Citation

Citations returns every citation across the response's text parts, in the order the parts appear. It is a convenience for callers that only want the list of sources; the association between a source and the sentence it backs lives on Text.Citations.

func (*Response) JSON added in v0.4.0 

func (r *Response) JSON(v any) error

JSON decodes the response text into v, which is what a request carrying a Format is asking for: 

var seo SEO
if err := resp.JSON(&seo); err != nil { ... }

The text must be one JSON value. A reply wrapped whole in a Markdown code fence is unwrapped first, because a provider that was only asked for JSON in the prompt tends to add one; anything else - prose around the value, a second value after it - is an error rather than something to go hunting through. Salvaging JSON out of arbitrary text is how a wrong answer gets read as a right one.

JSON does not consult Response.Format: text that is JSON decodes whatever produced it.

func (Response) MarshalJSON added in v0.3.0 

func (r Response) MarshalJSON() ([]byte, error)

MarshalJSON encodes the response with each output part tagged by its "type".

func (*Response) Text 

func (r *Response) Text() string

Text returns the concatenation of all text parts in the response.

func (*Response) ToolCall added in v0.1.1 

func (r *Response) ToolCall(name string) (ToolUse, bool)

ToolCall returns the first tool call with the given name and whether one was found. It is a convenience for dispatching a single expected tool.

Example 
package main

import (
	"encoding/json"
	"fmt"

	"github.com/goloop/ai"
)

func main() {
	resp := &ai.Response{Parts: []ai.Part{
		ai.Text{Text: "let me check"},
		ai.ToolUse{ID: "call_1", Name: "get_weather", Input: json.RawMessage(`{"city":"Kyiv"}`)},
	}}

	if call, ok := resp.ToolCall("get_weather"); ok {
		fmt.Printf("%s(%s)\n", call.Name, call.Input)
	}
}

Output:
get_weather({"city":"Kyiv"})

func (*Response) ToolCalls 

func (r *Response) ToolCalls() []ToolUse

ToolCalls returns the tool-call parts the model produced, in order.

func (*Response) UnmarshalJSON added in v0.3.0 

func (r *Response) UnmarshalJSON(data []byte) error

UnmarshalJSON reconstructs the response and its concrete part types.

type Role 

type Role string

Role identifies who a Message comes from in a conversation. 

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

The roles a Message can take. Providers map these onto their own wire values; RoleTool marks a message that carries tool results back to the model.

type Text 

type Text struct {
	Text string

	// Citations are the sources behind this text, when it came from a hosted
	// capability that reports them. It is empty everywhere else, so code
	// written before hosted capabilities existed reads exactly the same
	// values it always did. See [Citation].
	Citations []Citation
}

Text is a plain-text content part.

type Tool 

type Tool struct {
	Name        string
	Description string
	Schema      json.RawMessage
}

Tool describes a function the model may call. Schema is a JSON Schema object describing the tool's input; drivers pass it through to the provider in the shape that provider expects.

Example 
package main

import (
	"encoding/json"
	"fmt"

	"github.com/goloop/ai"
)

func main() {
	tool := ai.Tool{
		Name:        "get_weather",
		Description: "Get the current weather for a city.",
		Schema:      json.RawMessage(`{"type":"object","properties":{"city":{"type":"string"}}}`),
	}
	fmt.Println(tool.Name)
}

Output:
get_weather

type ToolChoice 

type ToolChoice int

ToolChoice controls whether and how the model may call tools in a Request. 

const (
	ToolAuto ToolChoice = iota
	ToolNone
	ToolRequired
)

The tool-calling strategies. ToolAuto lets the model decide, ToolNone forbids tool calls, and ToolRequired forces the model to call at least one tool.

type ToolResult 

type ToolResult struct {
	ID      string
	Content string
	IsError bool
}

ToolResult carries the result of a tool call back to the model. ID must match the ToolUse it answers. Set IsError to report that the tool failed.

type ToolUse 

type ToolUse struct {
	ID    string          // provider-assigned call identifier
	Name  string          // name of the tool to invoke
	Input json.RawMessage // arguments as a JSON object
}

ToolUse is a request from the assistant to call a tool. Input is the raw JSON arguments object produced by the model, validated against the matching Tool schema by the caller.

type Usage 

type Usage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

Usage reports how many tokens a request consumed.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.