aisdk

README

ai-sdk-go

A Go backend for the Vercel AI SDK 6 UIMessage stream protocol, plus streaming LLM providers and an agentic tool loop.

Four layers, each usable on its own:

Import	What it gives you
github.com/xujiahua/ai-sdk-go	The AI SDK 6 data stream protocol: an SSE writer emitting the events @ai-sdk/react's useChat expects, and conversion between UI messages and model messages.
github.com/xujiahua/ai-sdk-go/llm	One streaming Provider contract over OpenAI-compatible and Anthropic APIs.
github.com/xujiahua/ai-sdk-go/llm/acp	A local agent as the model, driven over the Agent Client Protocol on stdio.
github.com/xujiahua/ai-sdk-go/agent	The chat endpoint: an http.Handler running the tool loop, with skill prompts and capability-gated tools.

The core has no third-party dependencies. Tools, logging and the HTTP router are interfaces you supply.

Status: incubating. The API is not yet stable.

Serving a chat endpoint

a := agent.New(llm.WithRetry(llm.NewOpenAI()),
    agent.WithLLMConfig(llm.Config{APIKey: key, Model: "gpt-4o"}),
    agent.WithTools(myTools, map[string]string{"search": "execution"}),
    agent.WithLogger(slog.Default()), // *slog.Logger satisfies the interface as-is
)

mux := http.NewServeMux()
mux.Handle("/chat", a) // point useChat at this

The agent is a plain http.Handler, so any router mounts it in one line:

Router
net/http	mux.Handle("POST /chat", a)
chi	r.Method("POST", "/chat", a)
gin	g.POST("/chat", gin.WrapH(a))
echo	e.POST("/chat", echo.WrapHandler(a))

Running it without a browser

Agent.Run drives one turn against a Sink, so HTTP is not required: the SSE renderer is simply the Sink that speaks the AI SDK protocol. cmd/aisdk-agent is a terminal agent built on that seam — the same loop, the same tools, the same capability gate, printed to a terminal.

# a local Kiro CLI as the model — no API key needed
go run ./cmd/aisdk-agent

# the tool loop, deterministically and with no model at all
go run ./cmd/aisdk-agent -provider script -once "roll two dice"

# the capability gate: withheld, then authorized
go run ./cmd/aisdk-agent -provider script -skill operator -once "wipe the state"
go run ./cmd/aisdk-agent -provider script -skill operator -privileged -once "wipe the state"

# an OpenAI-compatible endpoint instead
AISDK_API_KEY=sk-… go run ./cmd/aisdk-agent -provider openai -model gpt-4o

The REPL does its own line editing (backspace, Ctrl-U, Ctrl-W) because a terminal in canonical mode erases one byte per backspace: deleting a CJK character would otherwise leave a broken UTF-8 fragment in the line and half a glyph on screen. With no terminal — piped input — it falls back to reading whole lines.

When the provider misbehaves

Failures are classified, not stringly typed. llm.APIError carries the status, the provider's own error type and its message, and answers the only question a caller has: Kind() is one of rate-limit, server-error, auth, invalid-request or context-overflow, and Retryable() follows from it.

llm.WithRetry(p) acts on that — a rate limit or a brief overload is sent again with capped, jittered backoff (honoring Retry-After) instead of becoming an error event in someone's browser. It stops the moment a delta has been forwarded: those bytes cannot be taken back, so replaying would duplicate the text and double-count the usage.

What a turn cost

Providers normalize token counts into aisdk.Usage, the AI SDK's own usage shape. The agent accumulates them and emits a message-metadata event before finish, which is where useChat's onFinish reads them — along with the finish reason, so a client can say "I ran out of steps" instead of presenting a truncated turn as an answer.

Two numbers, because they answer different questions: usage is the whole turn added up (what it cost), while contextTokens is the prompt size of the last model call (how full the window got). A tool loop re-sends a growing transcript every step, so summing the input tokens overstates the context by the step count.

llm.Config.PromptCache asks Anthropic to cache the stable prefix — after the tools, after the system prompt, and at the end of the transcript, so each step of a tool loop reads back what the previous one wrote. Off by default: a cache write costs more than an ordinary input token, so whether the reads pay for the writes is the application's call.

When the transcript stops fitting

A conversation that outgrows the context window gets a hard 400 mid-chat — KindContextOverflow, the one provider failure a retry cannot fix and a caller can, by sending less. agent.WithCompactor is the seam: on an overflow the loop hands the transcript over and calls the model again, once, so a compactor that does not shrink enough costs one extra call rather than a step budget's worth.

The policy is not here, deliberately. What to summarize, what to keep verbatim and which model to summarize with differ completely between a support chat and a coding agent, and deciding it here would also mean counting tokens properly — which means a BPE implementation, which means a dependency.

Attachments

A file part posted by useChat becomes a ContentPart on the model message, in the order the user composed it — "look at this, then answer that" is not the same prompt as the reverse. ModelMessage.Parts is the general form of Content: when it is set a provider renders it and ignores Content, so a text-only chat never builds a slice it does not need.

Images go to both APIs; Anthropic also takes PDFs. A data: URL, which is what the client produces for a locally picked file, is passed through to OpenAI and taken apart for Anthropic, which wants the media type and the payload as separate fields.

A provider that cannot express an attachment returns an error naming the media type. It does not drop it: a model asked "what is in this screenshot?" without the screenshot answers anyway, which is the worst outcome available.

Capabilities

A tool carries a capability tag; a skill declares which tags it may call; and the host separately declares whether this endpoint may expose privileged tags at all. A tool is offered to the model only when both agree, so a client that picks its own skill cannot escalate. An untagged tool is treated as privileged.

That gate is resolved before the turn starts, so it never sees an argument. agent.WithApproval is the per-call counterpart: it keeps the tool offered to the model and withholds the call, which reaches the client unexecuted and ends the turn caller-pending — the same shape as a tool the agent does not own, because the client's next move is the same one. A browser shows a confirmation and answers with addToolResult.

Before running anything the loop refuses what it can already see is wrong: arguments that do not parse (a truncated tool call is a normal streaming outcome), and arguments missing a field the tool's own schema marks required. Both come back to the model as a failed tool call, so it gets a step to fix them. This is not a JSON Schema validator and does not pretend to be — a ToolSource knows the Go types behind its schema and validates properly by unmarshalling into them.

agent.WithParallelTools(n) lets a batch of calls overlap — a model that asks for three independent searches then waits for the slowest instead of the sum. Off by default, because a ToolSource is the application's code and overlapping its calls is not a decision to make on its behalf. Results are always reported in the order the model asked for them, whatever order they finish in, so turning it on changes when the client hears, never what it hears.

The protocol is pinned

testdata/ holds golden files for the response headers, every event the writer emits, and whole turns through the agent — text, a server-executed tool round trip, a provider-executed tool event, a failed tool, an error, and a gated tool. They are the contract with a client this module does not control:

go test ./...                  # verify
go test . ./agent -update      # rewrite, then review the diff

Only those two packages hold golden files, so only their test binaries define -update; passing it to the others fails with "flag provided but not defined".

Golden files alone cannot say whether the bytes are right — they are just as happy to lock in a misunderstanding of the protocol. So conformance/ puts the same output through the client that consumes it:

cd conformance && npm ci && node conform.mjs

Three levels. Every event of every fixture is validated against the SDK's own uiMessageChunkSchema, not a transcription of it. Every fixture is then reduced by readUIMessageStream to the UIMessage a browser would render, which catches the ordering and lifecycle mistakes a per-event check cannot see. Finally a real server — the ordinary agent.Agent, with the model replaced by a deterministic script — is driven over a socket through AbstractChat and DefaultChatTransport, the class useChat is and the transport it uses, so the response head, the SSE framing and the [DONE] marker are exercised too.

It earned its keep immediately: it found that an error event terminates stream processing, so anything after it — including the turn's token usage — never reaches the client. That rule is not in the protocol document.

The client is pinned to an exact version, because the point is to check against a known one. Compatibility beyond that is load-bearing elsewhere: this backend serves @ai-sdk/react in production.

Documentation

Overview

Package aisdk implements the Vercel AI SDK 6 data stream protocol (the SSE-based UIMessage stream) for Go backends: a streaming writer emitting the events @ai-sdk/react's useChat hook expects, and conversion between UI messages and model messages.

Protocol reference: https://ai-sdk.dev/docs/ai-sdk-ui/stream-protocol

Index

• type ChatRequest
• type ContentPart
• type Logger
  • func OrNop(l Logger) Logger
• type LoggerAware
• type MessagePart
• type ModelMessage
  • func ConvertToModelMessages(msgs []UIMessage) []ModelMessage
  • func (m ModelMessage) Text() string
• type NopLogger
  • func (NopLogger) Debug(string, ...any)
  • func (NopLogger) Error(string, ...any)
  • func (NopLogger) Info(string, ...any)
  • func (NopLogger) Warn(string, ...any)
• type Tool
• type ToolCall
• type ToolFunction
• type UIMessage
• type UIMessageStream
  • func NewUIMessageStream(w http.ResponseWriter) *UIMessageStream
  • func (s *UIMessageStream) Done()
  • func (s *UIMessageStream) File(url, mediaType string) error
  • func (s *UIMessageStream) Finish() error
  • func (s *UIMessageStream) FinishStep() error
  • func (s *UIMessageStream) MessageMetadata(metadata any) error
  • func (s *UIMessageStream) MessageStart(messageID string) error
  • func (s *UIMessageStream) ReasoningDelta(id, delta string) error
  • func (s *UIMessageStream) ReasoningEnd(id string) error
  • func (s *UIMessageStream) ReasoningStart(id string) error
  • func (s *UIMessageStream) Send(event any) error
  • func (s *UIMessageStream) SendError(msg string) error
  • func (s *UIMessageStream) StartStep() error
  • func (s *UIMessageStream) TextDelta(id, delta string) error
  • func (s *UIMessageStream) TextEnd(id string) error
  • func (s *UIMessageStream) TextStart(id string) error
  • func (s *UIMessageStream) ToolInputAvailable(toolCallID, toolName string, input any) error
  • func (s *UIMessageStream) ToolInputDelta(toolCallID, inputTextDelta string) error
  • func (s *UIMessageStream) ToolInputStart(toolCallID, toolName string) error
  • func (s *UIMessageStream) ToolOutputAvailable(toolCallID string, output any) error
  • func (s *UIMessageStream) ToolOutputError(toolCallID, errorText string) error
• type Usage
  • func (u *Usage) Add(other Usage)

Types

type ChatRequest

type ChatRequest struct {
	Messages    []UIMessage     `json:"messages"`
	PageContext json.RawMessage `json:"pageContext,omitempty"`
	ClientTools []Tool          `json:"clientTools,omitempty"`
}

ChatRequest is the JSON body POSTed by useChat to the chat endpoint. PageContext and ClientTools are extensions a client may add; both are optional, so a plain chat client keeps working unchanged.

type ContentPart

type ContentPart struct {
	// Type is "text" or "file".
	Type string `json:"type"`
	// Text is set when Type is "text".
	Text string `json:"text,omitempty"`
	// URL, MediaType and Filename describe a file part. URL is remote or a data:
	// URL; providers translate both.
	URL       string `json:"url,omitempty"`
	MediaType string `json:"mediaType,omitempty"`
	Filename  string `json:"filename,omitempty"`
}

ContentPart is one piece of a message's content. It is the general form of ModelMessage.Content: a message that is only text needs none of this, but a message with an attachment has to say what came in which order, because "look at this, then answer that" is not the same prompt as the reverse.

type Logger

type Logger interface {
	Debug(msg string, args ...any)
	Info(msg string, args ...any)
	Warn(msg string, args ...any)
	Error(msg string, args ...any)
}

Logger is the whole logging surface this module needs.

The method set deliberately mirrors *slog.Logger, so a plain slog logger satisfies it with no adapter at all. Nothing in the chat backend reaches for a package-global logger: an embedding application injects its own logging stack (a logrus or zap adapter is a handful of lines) and gets every log line in its own format.

func OrNop

func OrNop(l Logger) Logger

OrNop normalizes a possibly-nil logger, so call sites can log unconditionally instead of guarding each one.

type LoggerAware

type LoggerAware interface {
	SetLogger(Logger)
}

LoggerAware is implemented by components that can adopt a logger after construction (the providers do). It lets one injection reach the whole request path without every constructor taking a logger.

type MessagePart

type MessagePart struct {
	Type       string `json:"type"`
	Text       string `json:"text,omitempty"`
	ToolCallID string `json:"toolCallId,omitempty"`
	ToolName   string `json:"toolName,omitempty"`
	State      string `json:"state,omitempty"`
	Input      any    `json:"input,omitempty"`
	Output     any    `json:"output,omitempty"`
	// URL locates a file part's content. It is either a remote URL or a data: URL,
	// which is what the client produces for a file picked locally.
	URL string `json:"url,omitempty"`
	// MediaType is the file part's IANA media type ("image/png").
	MediaType string `json:"mediaType,omitempty"`
	// Filename is the file part's original name, when the client sent one.
	Filename string `json:"filename,omitempty"`
}

MessagePart is a union part inside a UIMessage. Tool parts carry the tool name in Type ("tool-<name>") for static tools, or in ToolName for the "dynamic-tool" type. A completed tool part has a non-nil Output. A "file" part carries URL and MediaType — that is what useChat posts for an attachment.

type ModelMessage

type ModelMessage struct {
	Role string `json:"role"`
	// Content is the message's text, for a message that is only text.
	Content string `json:"content"`
	// Parts is the general form: text and files, in the order they arrived. When
	// set it is authoritative and Content is ignored.
	Parts      []ContentPart `json:"parts,omitempty"`
	ToolCallID string        `json:"toolCallId,omitempty"`
	ToolCalls  []ToolCall    `json:"toolCalls,omitempty"`
}

ModelMessage represents a simplified message for the LLM API call.

Content and Parts are two ways to carry the same thing, and Parts wins: when it is non-empty a provider renders it and ignores Content. Content is the shorthand for the overwhelmingly common case of a message that is only text, so a plain chat never builds a slice it does not need.

func ConvertToModelMessages

func ConvertToModelMessages(msgs []UIMessage) []ModelMessage

ConvertToModelMessages converts UIMessages to role/content messages for LLM APIs. Text parts are concatenated. Completed tool parts (those with an Output) are reconstructed into an assistant message carrying tool_calls plus a following tool-result message — so tool context (including the results of client-executed tools) survives the re-POST that resumes the agent loop. Pending tool parts (no Output) are skipped so we never emit a dangling tool_call the LLM API would reject.

func (ModelMessage) Text

func (m ModelMessage) Text() string

Text is the message's text however it is carried, so a caller that only wants the words does not have to know which form was used. Non-text parts contribute nothing.

type NopLogger

type NopLogger struct{}

NopLogger drops every line. It is the fallback whenever a component was built without a logger, so a missing injection degrades to silence rather than to a nil-pointer panic.

func (NopLogger) Debug

func (NopLogger) Debug(string, ...any)

func (NopLogger) Error

func (NopLogger) Error(string, ...any)

func (NopLogger) Info

func (NopLogger) Info(string, ...any)

func (NopLogger) Warn

func (NopLogger) Warn(string, ...any)

type Tool

type Tool struct {
	Type     string       `json:"type"`
	Function ToolFunction `json:"function"`
}

Tool is one tool advertised to the model, in the OpenAI function-calling shape. The Anthropic and ACP providers translate from it, so this stays the single internal tool representation.

type ToolCall

type ToolCall struct {
	ID        string `json:"id"`
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

ToolCall is one tool the model asked for. Arguments stays the raw JSON the model emitted: it is forwarded verbatim rather than decoded and re-encoded, so nothing about the model's own formatting changes underneath it, and a fragment that fails to parse stays diagnosable.

type ToolFunction

type ToolFunction struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	Parameters  any    `json:"parameters"`
}

ToolFunction describes a callable function: its name, its purpose, and its JSON Schema parameters.

type UIMessage

type UIMessage struct {
	ID    string        `json:"id"`
	Role  string        `json:"role"` // "user", "assistant", "system"
	Parts []MessagePart `json:"parts"`
}

UIMessage is the shape received from the AI SDK 6 frontend (useChat).

type UIMessageStream

type UIMessageStream struct {
	// contains filtered or unexported fields
}

UIMessageStream writes AI SDK 6 Data Stream Protocol events to an http.ResponseWriter. Each event is a Server-Sent Event (SSE) line:

data: {"type":"...","...":"..."}\n\n

The stream ends with:

data: [DONE]\n\n

func NewUIMessageStream

func NewUIMessageStream(w http.ResponseWriter) *UIMessageStream

NewUIMessageStream prepares the ResponseWriter for SSE streaming and returns a stream handle. This is the sole owner of the SSE response headers — callers set only their own (e.g. CORS) before handing the writer over. A writer that already carries a Content-Type is left untouched, so an application that insists on writing the head itself is not overridden.

func (*UIMessageStream) Done

func (s *UIMessageStream) Done()

Done writes the terminal [DONE] marker and marks the stream closed.

func (*UIMessageStream) File

func (s *UIMessageStream) File(url, mediaType string) error

File attaches a file to the message: a generated image, a rendered chart. The URL is either remote or a data: URL, and the client renders it as a file part.

func (*UIMessageStream) Finish

func (s *UIMessageStream) Finish() error

Finish emits the finish event (message complete).

func (*UIMessageStream) FinishStep

func (s *UIMessageStream) FinishStep() error

FinishStep emits a finish-step event.

func (*UIMessageStream) MessageMetadata

func (s *UIMessageStream) MessageMetadata(metadata any) error

MessageMetadata attaches host-defined metadata to the message. The AI SDK merges it into the UIMessage's metadata field, which is where useChat's onFinish reads a turn's token usage from.

The payload is deliberately opaque to this package: the protocol reserves the envelope, not what an application chooses to put in it.

func (*UIMessageStream) MessageStart

func (s *UIMessageStream) MessageStart(messageID string) error

MessageStart emits the start-of-message event.

func (*UIMessageStream) ReasoningDelta

func (s *UIMessageStream) ReasoningDelta(id, delta string) error

ReasoningDelta emits an incremental reasoning chunk.

func (*UIMessageStream) ReasoningEnd

func (s *UIMessageStream) ReasoningEnd(id string) error

ReasoningEnd closes a reasoning block.

func (*UIMessageStream) ReasoningStart

func (s *UIMessageStream) ReasoningStart(id string) error

ReasoningStart begins a reasoning block.

func (*UIMessageStream) Send

func (s *UIMessageStream) Send(event any) error

Send marshals the event to JSON and writes it as an SSE data line.

func (*UIMessageStream) SendError

func (s *UIMessageStream) SendError(msg string) error

SendError writes an error event to the stream.

func (*UIMessageStream) StartStep

func (s *UIMessageStream) StartStep() error

StartStep emits a start-step event.

func (*UIMessageStream) TextDelta

func (s *UIMessageStream) TextDelta(id, delta string) error

TextDelta emits an incremental text chunk.

func (*UIMessageStream) TextEnd

func (s *UIMessageStream) TextEnd(id string) error

TextEnd closes a text block.

func (*UIMessageStream) TextStart

func (s *UIMessageStream) TextStart(id string) error

TextStart begins a text block with the given ID.

func (*UIMessageStream) ToolInputAvailable

func (s *UIMessageStream) ToolInputAvailable(toolCallID, toolName string, input any) error

ToolInputAvailable signals that tool input is fully available.

func (*UIMessageStream) ToolInputDelta

func (s *UIMessageStream) ToolInputDelta(toolCallID, inputTextDelta string) error

ToolInputDelta sends a partial tool input text chunk.

func (*UIMessageStream) ToolInputStart

func (s *UIMessageStream) ToolInputStart(toolCallID, toolName string) error

ToolInputStart begins streaming tool input.

func (*UIMessageStream) ToolOutputAvailable

func (s *UIMessageStream) ToolOutputAvailable(toolCallID string, output any) error

ToolOutputAvailable sends the tool execution result.

func (*UIMessageStream) ToolOutputError

func (s *UIMessageStream) ToolOutputError(toolCallID, errorText string) error

ToolOutputError reports a tool call that failed. It is a distinct event from ToolOutputAvailable on purpose: a failure sent as ordinary output renders as a successful call that happened to return a sentence about an error.

type Usage

type Usage struct {
	// InputTokens is the whole prompt, including any part of it served from the
	// provider's cache. CachedInputTokens is the subset that was cached, so
	// InputTokens - CachedInputTokens is what was billed at full rate.
	InputTokens       int `json:"inputTokens"`
	OutputTokens      int `json:"outputTokens"`
	TotalTokens       int `json:"totalTokens"`
	ReasoningTokens   int `json:"reasoningTokens"`
	CachedInputTokens int `json:"cachedInputTokens"`
}

Usage is the token accounting of one or more model calls, in the AI SDK's LanguageModelUsage shape so a client reads it without translation.

A zero field means "not reported", not "zero": providers differ in what they count, and the ACP path reports nothing at all. Callers that need to tell the two apart should check whether any usage arrived, not whether a field is zero.

func (*Usage) Add

func (u *Usage) Add(other Usage)

Add accumulates another call's usage into this one. A turn is several model calls, and its cost is their sum.