promptr

README

promptr

Typed prompts for Go. promptr makes a language model's output typed and reliable — the way BAML does, but as idiomatic, dependency-free Go.

Declare your prompts and their input/output types in a small .promptr schema; promptr compiles it to ordinary Go functions you call. Each generated function renders the prompt, calls a model through a tiny Provider interface you wire up (no vendor SDK in the core), and turns the loose reply into your typed value — retrying with the parse error fed back if the first attempt won't fit.

Its core is schema-aligned parsing: instead of forcing a model into a rigid JSON mode (which empirically degrades answer quality), you let it write naturally and a tolerant, schema-driven parser coerces the result into your Go types — recovering from prose preambles, Markdown fences, trailing commas, single quotes, fuzzy enum spellings, and truncated objects that real model output is full of.

.promptr schema ──promptr generate──▶ typed Go function
                                            │ renders prompt (schema baked in)
                                            ▼
                                    your Provider (any model)
                                            │ loose text reply
                                            ▼
                            coerce ──▶ typed value  (retry on misfit)

The pipeline in one example

// ticket.promptr
enum Severity { LOW HIGH CRITICAL }

class Ticket {
  title    string
  severity Severity
  tags     string[]
  due_days int?
}

client GPT4o { provider "openai" model "gpt-4o" }

function ExtractTicket(text: string) -> Ticket {
  client GPT4o
  prompt #"
    Extract a support ticket from the message.
    {{ ctx.output_schema }}      // compiler injects a schema description of Ticket
    Message: {{ text }}
  "#
}

promptr generate ./...     # -> ticket.promptr.go

ticket, err := ExtractTicket(ctx, provider, "my server is down!!")
// ticket.Severity == SeverityCRITICAL, even if the model wrote "critical priority"

The generated function bakes a human-readable schema of Ticket into the prompt (BAML's "show the model the shape you want" trick), then coerces the reply and re-asks on a parse miss — all in a few lines you can read.

coerce — the schema-aligned parser (usable on its own)

Don't want the DSL? Use the kernel directly on the structs you already have:

import "github.com/zkrebbekx/promptr/coerce"

ticket, err := coerce.Into[Ticket](modelText) // modelText: fences, prose and all

coerce.Into[T] digests input that encoding/json would reject outright:

the model emitted	encoding/json	coerce
```json { ... } ``` wrapped in prose	✗	✓ extracts the payload
{ title: 'x', tags: [1,2,], } (unquoted key, single quotes, trailing comma)	✗	✓
{"due_days": "7"} into an int field	✗	✓ coerces "7" → 7
{"amount": "$1,200.50"} into a float64	✗	✓ → 1200.5
{"severity": "high priority"} into an enum	✗	✓ fuzzy-matches → HIGH
{"title": "x", "tags": ["a", (truncated)	✗	✓ recovers what parsed

Nested structs, *T optionals, maps, snake/camel/case-insensitive keys, and a single value where a list was expected are all handled. Bare prose where a struct was expected returns a *coerce.Error — the signal the runtime retries on.

Discriminated unions

u := coerce.NewUnion(Search{}, Escalate{})
act, err := coerce.ResolveInto[Action](modelText, u) // best-fit by shape or "type" discriminator

Streaming

for p := range coerce.Stream[Ticket](tokenChan) {
    render(p.Value)        // a progressively-completed Ticket
    if p.Complete { break }
}

Type system

Beyond classes and enums, the schema language expresses the shapes real LLM tasks need — and each maps to idiomatic Go the coerce kernel already parses.

Unions — classify output into one of several typed shapes. Compiles to a sealed interface (sumx-style) plus a coerce resolver that picks the best-fit variant (by shape, or an explicit type/kind discriminator):

class Search   { query string }
class Escalate { reason string }
union Action = Search | Escalate          // or inline:  -> Search | Escalate

function Route(message: string) -> Action {
  client Default
  prompt #"Search or escalate? {{ ctx.output_schema }} Message: {{ message }}"#
}

act, err := Route(ctx, p, "I want a refund NOW")
switch a := act.(type) {            // exhaustive, type-safe
case Escalate: alertHuman(a.Reason)
case Search:   run(a.Query)
}

Maps — map<string, int> → map[string]int.

Field attributes tune the schema shown to the model (better prompt ⇒ better parse), the BAML "symbol tuning" idea:

class Profile {
  name  string @description("the person's full legal name") @alias("full_name")
  score int
}

@description annotates the field in the baked schema; @alias renames it on the wire — the model is shown full_name, and coerce binds that back to Name.

Validation — @assert & @check

Coercion shapes a reply into your type; validation enforces what the values must be. Two field attributes compile to valx rules the runtime applies after coercion:

class Account {
  email    string @assert("required")
  username string @assert("min=3,max=20")
  age      int    @assert("gt=0,lt=130") @check("min=18")
  seats    int    @check("min=1,max=100")
}

• @assert is hard. A violation is fed back to the model as a repair re-ask — exactly like a parse failure — so it self-corrects within Attempts. If it never satisfies the rules, the call returns the validation error.
• @check is soft. Violations never block the value; they're delivered to an OnCheck sink so you can log or meter them while still using the result.

Generated functions take a trailing ...promptr.Option, so checks (and retry budgets, a system preamble, …) are opt-in without the signature changing:

acc, err := ExtractAccount(ctx, p, msg,
    promptr.OnCheck(func(e error) { log.Println("soft:", e) }),
    promptr.WithAttempts(3),
)

The validator lives in generated code, so the core packages stay zero-dependency — importing promptr pulls in nothing extra. See examples/validate.

Prompt templates

Prompts are more than string interpolation — the template engine supports control flow over your runtime values, so one function adapts its prompt to the input:

prompt #"
  Extract a support ticket.
  {{ if examples }}Here are examples of good tickets:
  {{ for e in examples }}- {{ e }}
  {{ end }}{{ end }}
  {{ ctx.output_schema }}
  Message: {{ text }}
"#

Supported inside {{ }}: {{ var }} (with dotted paths {{ user.name }}), {{ if cond }}…{{ else }}…{{ end }} (truthiness, not, and == "lit" / != "lit"), {{ for x in items }}…{{ end }}, and the compiler-injected {{ ctx.output_schema }}. Unknown names render empty rather than erroring, so a prompt never panics on real model context. The engine (promptr.Render) is usable directly, too.

Streaming & multimodal

Mark a function -> stream T and the generated function returns a channel of progressively-completed T values — the schema-aligned parser coerces each growing prefix, so you can render a partial object while tokens are still arriving:

function SummarizeArticle(article: string) -> stream Summary {
  client Default
  prompt #"Summarize it. {{ ctx.output_schema }} Article: {{ article }}"#
}

ch, err := SummarizeArticle(ctx, provider, text)
for part := range ch {           // promptr.Partial[Summary]
    if part.Err != nil { break }
    render(part.Value)           // headline fills in before the bullets do
    if part.Complete { break }
}

Any provider implementing the optional StreamProvider (openai, anthropic, fake) streams real tokens via SSE; others transparently fall back to a single complete value. promptr.ExtractStream[T] is usable directly, too.

Multimodal inputs. Give a parameter the type image, audio, pdf or file and it becomes a promptr.Part attached to the user message (not templated into the prompt text):

function CaptionImage(photo: image, hint: string) -> Summary { … }

cap, err := CaptionImage(ctx, provider, promptr.ImagePart("image/png", bytes), "be terse")

Providers map parts to their native content arrays (OpenAI image_url, Anthropic image blocks; inline bytes are base64 data-URLs, or pass a URL with promptr.ImageURL). See examples/stream.

Tool-calling & agents

Declare tools and hand them to a function with tools [...]. promptr runs the bounded model → tool → model loop for you and still returns a typed Go value at the end — single-shot extraction becomes a typed agent without giving up type safety:

tool GetWeather(city: string) -> Weather {
  description "Look up the current weather for a city."
}
tool SearchFlights(from: string, to: string) -> Flight[] {
  description "Find available flights between two cities."
}

function PlanTrip(goal: string) -> Itinerary {
  client Smart
  tools [GetWeather, SearchFlights]
  prompt #"Plan a trip for this goal. {{ ctx.output_schema }} Goal: {{ goal }}"#
}

The generated function takes a typed handlers struct — one func per tool, its argument a generated <Tool>Args struct coerced from the model's JSON:

itin, err := PlanTrip(ctx, provider, "see the northern lights", PlanTripTools{
    GetWeather: func(ctx context.Context, a GetWeatherArgs) (Weather, error) {
        return lookupWeather(a.City)
    },
    SearchFlights: func(ctx context.Context, a SearchFlightsArgs) ([]Flight, error) {
        return searchFlights(a.From, a.To)
    },
})

The loop dispatches each requested tool, feeds the result back, and repeats up to Options.MaxSteps (default 8) until the model answers — coerced into Itinerary. Unknown-tool and handler-error turns are fed back as text so the model can recover rather than aborting. promptr.RunTools[T] is usable directly, too. When the model requests several tools in one turn, pass promptr.ParallelTools() to dispatch them concurrently (results still feed back in request order; opt in only when your handlers are goroutine-safe).

promptr.RunToolsStream[T] drives the same agent loop but streams the model's answer: it returns a <-chan Partial[T], emitting a progressively coerced value as the final turn's tokens arrive and a Complete partial once it parses. A provider implementing StreamToolProvider (openai, anthropic, and the fake) streams token-by-token; one that only implements ToolProvider (gemini, ollama) falls back transparently to a single final Partial. Works on providers implementing the optional ToolProvider interface — openai, anthropic, gemini (native functionDeclarations), ollama (native /api/chat tools), and fake; others return a clear "does not support tool calls" error. See examples/agent.

Providers

The core imports no LLM SDK. A Provider is one method:

type Provider interface {
    Complete(ctx context.Context, messages []Message) (string, error)
}

Package	Backend
providers/openai	OpenAI Chat Completions — and anything compatible: Azure OpenAI, Groq, Together, OpenRouter, llama.cpp/vLLM/LM Studio (just set BaseURL)
providers/anthropic	Anthropic Messages API
providers/gemini	Google Gemini (Generative Language API)
providers/ollama	Local models via Ollama
providers/fake	Deterministic scripted replies for tests and the playground
providers/recorded	Replays hand-authored JSON cassettes — a VCR for deterministic, offline tests

Each is net/http only — import just the one you use. Wiring any other model is a dozen lines.

Client reliability policies

Declare retry/fallback/round-robin in the DSL; the compiler generates registry-resolving constructors that wrap your wired providers:

client Fast  { provider "openai"    model "gpt-4o-mini" }
client Smart { provider "anthropic" model "claude-opus-4-8" }
client Reliable {
  fallback [Smart, Fast]   // try Smart, fall over to Fast
  retry 3                  // each up to 3 times on transient error
}

reg := promptr.Registry{"Smart": anthropicClient, "Fast": openaiClient}
provider := ClientReliable(reg) // promptr.Retry(promptr.Fallback(...), 3, 0)
ticket, err := ExtractTicket(ctx, provider, msg)

promptr.Retry, promptr.Fallback, and promptr.RoundRobin are also usable directly — each is just a Provider that wraps other Providers.

Observability

A Middleware is just a Provider-to-Provider function, so you can wrap any provider without touching generated code. The built-in Collector records latency and token usage per call (exact when a provider implements UsageReporter, otherwise a chars/4 estimate):

col := &promptr.Collector{}
p := promptr.Chain(openaiClient, col.Collect) // outermost-first
_, _ = ExtractTicket(ctx, p, msg)

s := col.Stats()
log.Printf("%d calls, %d tokens, %s avg", s.Calls, s.TotalTokens(), s.AvgLatency())

Hooks (streaming + tool calls too). Middleware/Chain wrap only Complete, so wrapping a streaming or tool-calling provider hides those capabilities. For observability across all paths use WithHooks, which is capability-preserving — the wrapped provider still streams and runs tools — and fires a Hook before and after every Complete, Stream and CompleteTools:

type Hook interface {
    BeforeCall(ctx context.Context, info CallInfo) AfterFunc // AfterFunc(Outcome)
}

col := &promptr.Collector{}
p := promptr.WithHooks(openaiClient,
    col.Hook(),                       // latency + tokens on every path
    promptr.LogHook(slog.Default()),  // structured logs, zero deps
)
itin, _ := PlanTrip(ctx, p, goal, handlers) // tool calls still work, now observed

A Hook is the whole extension surface: an OpenTelemetry span exporter is a ~20-line Hook that opens a span in BeforeCall and ends it in the returned AfterFunc, kept out of core so it stays dependency-free. LogHook (built on stdlib log/slog) is the reference implementation.

Tooling & editor support

promptr generate ./...   # compile .promptr -> Go (run under //go:generate)
promptr check ./...       # parse + validate without writing Go (CI-friendly)

promptr check reports unresolved types/clients, malformed unions and test blocks whose args don't match their function — the same checks the language server surfaces in your editor. Install cmd/promptr-lsp for live diagnostics and see editor/ for the tree-sitter grammar and editor wiring.

Deferred for a focused follow-up: promptr fmt (a canonical formatter needs the lexer to retain comments, currently dropped as trivia) and a live-execution test runner with typed assertions (best done by emitting Go tests from test blocks, which deserves its own provider-wiring design). providers/recorded is the deterministic substrate both will build on.

Install

# library
go get github.com/zkrebbekx/promptr

# CLI compiler
go install github.com/zkrebbekx/promptr/cmd/promptr@latest

Also published as a container image (ghcr.io/zkrebbekx/promptr) and a Homebrew cask.

Playground

A WebAssembly playground (DSL → Go, and paste-messy-output → repaired value) runs entirely client-side: playground/, deployed to GitHub Pages.

Develop

make test     # go test ./...
make race     # -race
make fuzz     # fuzz the tolerant parser
make lint
go generate ./...   # regenerate examples

License

MIT

Documentation

Overview

Package promptr is the runtime that generated .promptr code calls into: a minimal, provider-agnostic Provider interface plus the parse-with-repair loop that turns a model's loose reply into a typed Go value via the coerce kernel.

The compiler (cmd/promptr) turns each `function` in a .promptr file into a Go function whose body is a single call to Extract — so the generated code stays thin and readable, and all the retry/coercion logic lives here, tested once.

Index

• func Extract[T any](ctx context.Context, p Provider, prompt string, opts Options) (T, error)
• func ExtractStream[T any](ctx context.Context, p Provider, prompt string, opts Options) (<-chan Partial[T], error)
• func ExtractUnion[I any](ctx context.Context, p Provider, prompt string, opts Options, u *Union) (I, error)
• func Render(tmpl string, ctx map[string]any) (string, error)
• func RunTools[T any](ctx context.Context, p Provider, prompt string, tools []Tool, opts Options) (T, error)
• func RunToolsStream[T any](ctx context.Context, p Provider, prompt string, tools []Tool, opts Options) (<-chan Partial[T], error)
• type AfterFunc
• type Call
• type CallInfo
• type CallKind
• type Collector
  • func (c *Collector) Calls() []Call
  • func (c *Collector) Collect(p Provider) Provider
  • func (c *Collector) CollectTools(p ToolProvider) Provider
  • func (c *Collector) Hook() Hook
  • func (c *Collector) Reset()
  • func (c *Collector) Stats() Stats
• type Hook
  • func LogHook(logger *slog.Logger) Hook
• type Message
• type Middleware
• type Option
  • func OnCheck(fn func(err error)) Option
  • func ParallelTools() Option
  • func WithAttempts(n int) Option
  • func WithMaxSteps(n int) Option
  • func WithSystem(s string) Option
• type Options
• type Outcome
• type Part
  • func AudioPart(mime string, data []byte) Part
  • func FilePart(mime string, data []byte) Part
  • func ImagePart(mime string, data []byte) Part
  • func ImageURL(url string) Part
  • func TextPart(s string) Part
• type PartKind
• type Partial
• type Provider
  • func Chain(p Provider, mws ...Middleware) Provider
  • func Fallback(providers ...Provider) Provider
  • func Retry(p Provider, attempts int, backoff time.Duration) Provider
  • func RoundRobin(providers ...Provider) Provider
  • func WithHooks(p Provider, hooks ...Hook) Provider
• type ProviderFunc
  • func (f ProviderFunc) Complete(ctx context.Context, messages []Message) (string, error)
• type Registry
  • func (r Registry) Get(name string) Provider
• type Reply
• type Stats
  • func (s Stats) AvgLatency() time.Duration
  • func (s Stats) TotalTokens() int
• type StreamProvider
• type StreamToolProvider
• type TemplateError
  • func (e *TemplateError) Error() string
• type Tool
• type ToolCall
• type ToolDef
• type ToolProvider
• type ToolStream
• type Union
  • func NewUnion(samples ...any) *Union
• type UsageReporter

Examples

• WithHooks

Functions

func Extract

func Extract[T any](ctx context.Context, p Provider, prompt string, opts Options) (T, error)

Extract runs prompt against the provider and coerces the reply into T. If the reply will not coerce, it re-asks — appending the unparseable reply and the parse error so the model can correct itself — up to Options.Attempts times.

This is the function every generated .promptr function calls.

func ExtractStream

func ExtractStream[T any](ctx context.Context, p Provider, prompt string, opts Options) (<-chan Partial[T], error)

ExtractStream runs prompt against the provider and emits a progressively completed T after each chunk, coercing the accumulated text with the tolerant kernel so callers can render partial state. If the provider implements StreamProvider the reply streams token-by-token; otherwise a single Complete call yields one final Partial. The returned channel closes when the model is done.

This is what a generated `-> stream T` function calls.

func ExtractUnion

func ExtractUnion[I any](ctx context.Context, p Provider, prompt string, opts Options, u *Union) (I, error)

ExtractUnion runs prompt against the provider and classifies the reply into one of the union's variants, returned as the sealed interface I. Like Extract it re-asks on a parse miss, feeding the error back, up to Options.Attempts.

This is what a generated function with a union return type calls.

func Render

func Render(tmpl string, ctx map[string]any) (string, error)

Render expands a prompt template against a context map and returns the finished prompt. It is the small, dependency-free template engine the compiler targets: generated functions call Render with the call's parameters (plus a "ctx" entry carrying the baked output-schema) so prompts can use control flow over runtime values.

Supported syntax (all inside {{ }}):

{{ name }}              value lookup, with dotted paths: {{ user.name }}
{{ ctx.output_schema }} the compiler-injected schema description
{{ if cond }}…{{ end }} conditional, with optional {{ else }}
{{ for x in items }}…{{ end }}   iterate a slice, binding x in the body

A condition is a path (truthy test), `not <path>`, or `<path> == "lit"` / `<path> != "lit"`. Lookups miss softly: an unknown name renders as empty rather than erroring, so a template can never panic on real model context. Render only returns an error for a structurally broken template (an unterminated {{ if }}/{{ for }}).

func RunTools

func RunTools[T any](ctx context.Context, p Provider, prompt string, tools []Tool, opts Options) (T, error)

RunTools runs prompt against a tool-capable provider, executing the Go tools the model asks for and feeding their results back, until the model returns a final answer that coerces into T. The loop is bounded by Options.MaxSteps (default 8).

An unknown tool name or a handler error is fed back to the model as a tool result so it can recover, rather than aborting the run. If the model's final answer will not coerce into T, the parse error is fed back for one repair re-ask (within the step budget), mirroring Extract.

This is what a generated tool-using function calls. The provider must implement ToolProvider; otherwise RunTools returns an error.

func RunToolsStream

func RunToolsStream[T any](ctx context.Context, p Provider, prompt string, tools []Tool, opts Options) (<-chan Partial[T], error)

RunToolsStream drives the same bounded agent loop as RunTools, but streams the model's text: it emits a progressively coerced Partial[T] after each token of the turn that is currently generating, and a final Complete partial once the model answers and the reply coerces into T. Intermediate tool-calling turns dispatch their tools (honoring Options.ParallelTools) and feed the results back, exactly like RunTools.

The provider must implement StreamToolProvider to stream; otherwise this falls back to a blocking RunTools call delivered as a single final Partial, so a generated `-> stream T` tool-using function works against any tool provider. The returned channel closes when the loop ends (a final answer, an error, or the step budget); a non-nil terminal error rides on the last Partial's Err.

Types

type AfterFunc

type AfterFunc func(Outcome)

AfterFunc observes a call's Outcome. A nil AfterFunc is fine — it is skipped.

type Call

type Call struct {
	Start    time.Time
	Duration time.Duration
	// PromptTokens / ReplyTokens are exact when the provider reports usage (see
	// UsageReporter); otherwise they are a chars/4 estimate.
	PromptTokens int
	ReplyTokens  int
	Estimated    bool // true when token counts are estimated, not reported
	Err          error
}

Call is one observed Provider.Complete invocation.

type CallInfo

type CallInfo struct {
	Kind     CallKind
	Messages []Message
	Tools    []ToolDef // set only when Kind is KindTools
	Start    time.Time
}

CallInfo describes a provider call as a Hook sees it begin.

type CallKind

type CallKind string

CallKind identifies which provider capability a Hook is observing.

const (
	KindComplete CallKind = "complete" // Provider.Complete
	KindStream   CallKind = "stream"   // StreamProvider.Stream
	KindTools    CallKind = "tools"    // ToolProvider.CompleteTools
)

The call kinds a Hook can observe, one per provider capability.

type Collector

type Collector struct {
	// contains filtered or unexported fields
}

Collector aggregates per-call latency and token usage across every Complete it wraps. It is safe for concurrent use. Wire it as middleware with Collect.

func (*Collector) Calls

func (c *Collector) Calls() []Call

Calls returns a snapshot copy of every recorded call.

func (*Collector) Collect

func (c *Collector) Collect(p Provider) Provider

Collect returns a Middleware that records every Complete into c. It captures only the Complete path; to record streaming and tool calls as well, wire the collector with WithHooks(p, c.Hook()) instead.

func (*Collector) CollectTools

func (c *Collector) CollectTools(p ToolProvider) Provider

CollectTools wraps a ToolProvider so each CompleteTools call is recorded into c on the same latency/usage path as Collect. The returned Provider also implements ToolProvider, so it can be passed straight to RunTools while staying observable.

func (*Collector) Hook

func (c *Collector) Hook() Hook

Hook adapts a Collector to the Hook seam so it records every Complete, Stream and CompleteTools call when wired via WithHooks — capturing the streaming and tool paths that the older Collect/CollectTools wrappers miss.

func (*Collector) Reset

func (c *Collector) Reset()

Reset clears all recorded calls.

func (*Collector) Stats

func (c *Collector) Stats() Stats

Stats rolls up the recorded calls into totals.

type Hook

type Hook interface {
	BeforeCall(ctx context.Context, info CallInfo) AfterFunc
}

Hook observes provider calls for cross-cutting concerns — logging, metrics, tracing — without having to implement a Provider. BeforeCall fires just before a call; the AfterFunc it returns (which may be nil) fires when the call returns or, for a stream, when the channel closes. Register hooks with WithHooks, which preserves the wrapped provider's streaming and tool-calling capabilities — the problem a plain Provider-to-Provider Middleware cannot solve.

func LogHook

func LogHook(logger *slog.Logger) Hook

LogHook returns a Hook that logs each call to logger (slog.Default when nil): a debug line before the call and an info/error line after, with kind, latency and token counts. It is the zero-dependency reference Hook; an OpenTelemetry span hook is a handful of lines following the same shape.

type Message

type Message struct {
	Role    string
	Content string
	// Parts, when non-empty, carries a multimodal message (text + images/etc).
	// Providers that support multimodal input map Parts to their content array;
	// otherwise Content is used. Text-only callers leave Parts nil.
	Parts []Part
	// ToolCalls, on an "assistant" turn, are the tool invocations the model
	// requested (see ToolProvider). ToolCallID, on a "tool" turn, correlates a
	// tool result back to the call that produced it. Both are zero for ordinary
	// text turns, so non-tool callers are unaffected.
	ToolCalls  []ToolCall
	ToolCallID string
}

Message is one turn in a chat-style exchange. Roles follow the usual "system" / "user" / "assistant" convention; a Provider maps them to whatever its backend expects.

type Middleware

type Middleware func(Provider) Provider

Middleware wraps a Provider to add cross-cutting behaviour — logging, metrics, caching, rate limiting — without touching the generated code. A Middleware is just a Provider-to-Provider function, so they compose by nesting and a wrapped provider is still a plain Provider the runtime can call.

Middleware only intercepts Complete: the wrapped value does not re-expose StreamProvider or ToolProvider, so wrapping a streaming or tool-calling provider hides those capabilities. To observe streaming and tool calls too, prefer WithHooks (hooks.go), which is capability-preserving.

type Option

type Option func(*Options)

Option tunes the Options a generated function uses, applied after its built-in defaults. Generated functions take a trailing `...Option`, so callers can set retry budgets, a system preamble or a soft-check sink without the signature changing. The zero set leaves the defaults untouched.

func OnCheck

func OnCheck(fn func(err error)) Option

OnCheck installs a sink for soft @check violations. Without it, generated @check constraints are evaluated into a no-op and effectively skipped; with it, each non-fatal violation is delivered here while the value is still returned.

func ParallelTools

func ParallelTools() Option

ParallelTools enables concurrent dispatch of the tool calls within a single model turn (see Options.ParallelTools). Opt in only when the tool handlers are goroutine-safe.

func WithAttempts

func WithAttempts(n int) Option

WithAttempts caps the number of model calls (the first try plus repair re-asks).

func WithMaxSteps

func WithMaxSteps(n int) Option

WithMaxSteps bounds the tool-calling agent loop's model⇄tool round-trips.

func WithSystem

func WithSystem(s string) Option

WithSystem prepends a system message to the exchange.

type Options

type Options struct {
	// Attempts is the maximum number of model calls (default 2): the first
	// try plus repair re-asks when the reply will not coerce into T.
	Attempts int
	// System, when non-empty, is prepended as a system message.
	System string
	// UserParts, when non-empty, makes the user turn multimodal: the rendered
	// prompt becomes the leading text Part, followed by these (images, files…).
	UserParts []Part
	// MaxSteps bounds the tool-calling agent loop (default 8): the most
	// model⇄tool round-trips RunTools will take before giving up. Ignored by the
	// non-tool Extract paths.
	MaxSteps int
	// Validate, when set, runs after a reply coerces into T. A non-nil error is
	// treated like a parse failure: its message is shown to the model and the call
	// retried, so a generated @assert constraint drives the model to self-correct.
	// It receives the value boxed as any (a struct or pointer to one).
	Validate func(v any) error
	// Check, when set, runs after Validate passes. Its violations are advisory:
	// the value is still returned and any error is handed to OnCheck. This backs
	// soft @check constraints. Check only runs when OnCheck is also set.
	Check func(v any) error
	// OnCheck receives the non-fatal result of Check. A nil OnCheck disables the
	// Check pass entirely.
	OnCheck func(err error)
	// ParallelTools, when set, dispatches the tool calls of a single model turn
	// concurrently instead of one after another (results still feed back in
	// request order). The calls within one turn are independent, so this cuts
	// latency when the model asks for several at once. Off by default, since tool
	// handlers that share mutable state must be goroutine-safe to opt in. Ignored
	// by the non-tool Extract paths.
	ParallelTools bool
}

Options tunes an Extract call.

type Outcome

type Outcome struct {
	// Text is the final text reply (KindComplete/KindStream, or KindTools when
	// the model answered without requesting a tool). For a stream it is the full
	// accumulated text once the channel closes.
	Text string
	// Calls is the tool calls the model requested (KindTools only).
	Calls []ToolCall
	// PromptTokens/ReplyTokens are exact when the provider implements
	// UsageReporter; otherwise they are a chars/4 estimate and Estimated is true.
	PromptTokens int
	ReplyTokens  int
	Estimated    bool
	Duration     time.Duration
	Err          error
}

Outcome describes how a provider call ended. A Hook's AfterFunc receives it.

type Part

type Part struct {
	Kind PartKind
	Text string // PartText
	MIME string // media: e.g. "image/png", "application/pdf"
	Data []byte // inline media bytes (base64-encoded by the provider)
	URL  string // or a reference to remote media (used when Data is empty)
}

Part is one piece of a multimodal message: text, an image, audio, or a file. A media Part carries either inline Data (+MIME) or a URL reference. Providers map Parts to their own content-array formats; a provider that does not support a Part kind may ignore or reject it.

func AudioPart

func AudioPart(mime string, data []byte) Part

AudioPart builds an inline audio Part from bytes and a MIME type.

func FilePart

func FilePart(mime string, data []byte) Part

FilePart builds an inline document Part (e.g. a PDF) from bytes and a MIME.

func ImagePart

func ImagePart(mime string, data []byte) Part

ImagePart builds an inline image Part from raw bytes and a MIME type.

func ImageURL

func ImageURL(url string) Part

ImageURL builds an image Part that references a remote URL.

func TextPart

func TextPart(s string) Part

TextPart builds a text Part.

type PartKind

type PartKind uint8

PartKind tags a multimodal message Part.

const (
	// PartText is a plain-text span.
	PartText PartKind = iota
	// PartImage is an image (PNG/JPEG/…), by inline bytes or URL.
	PartImage
	// PartAudio is an audio clip.
	PartAudio
	// PartFile is a document (e.g. a PDF), by inline bytes or URL.
	PartFile
)

type Partial

type Partial[T any] struct {
	Value    T
	Complete bool
	Err      error
}

Partial carries a best-effort value parsed from an as-yet-incomplete stream. Complete flips to true once the payload parses cleanly. It mirrors coerce.Partial so generated code only imports promptr.

type Provider

type Provider interface {
	Complete(ctx context.Context, messages []Message) (string, error)
}

Provider is the single seam between promptr and a language model. Implement it with net/http against any chat API — the core imports no vendor SDK. A deterministic fake lives in providers/fake for tests and the playground.

func Chain

func Chain(p Provider, mws ...Middleware) Provider

Chain applies middlewares to p, outermost first: Chain(p, a, b) yields a(b(p)), so a sees each call before b does.

func Fallback

func Fallback(providers ...Provider) Provider

Fallback wraps providers so Complete tries each in order, returning the first success; if all fail, the last error is returned. Use it to fail over from a primary model to a backup.

func Retry

func Retry(p Provider, attempts int, backoff time.Duration) Provider

Retry wraps p so that Complete is retried up to attempts times on error, sleeping backoff between tries (respecting context cancellation). attempts < 1 is treated as 1. This is reliability against transient failures, distinct from Extract's parse-repair loop (which re-asks on unparseable but successful replies).

func RoundRobin

func RoundRobin(providers ...Provider) Provider

RoundRobin wraps providers so each Complete call goes to the next provider in rotation — load-spreading across keys or endpoints. It is safe for concurrent use.

func WithHooks

func WithHooks(p Provider, hooks ...Hook) Provider

WithHooks wraps p so every Complete, Stream and CompleteTools call fires the given hooks, and returns a provider that still satisfies StreamProvider and ToolProvider exactly when p does. This is the capability-preserving seam for observability: unlike Chain (which only wraps Complete), WithHooks keeps a streaming/tool provider usable for streaming and tool calls. With no hooks it returns p unchanged.

Example

package main

import (
	"context"
	"fmt"

	"github.com/zkrebbekx/promptr"
)

// spanHook is the shape an OpenTelemetry exporter would take: open a span in
// BeforeCall, close it in the returned AfterFunc. Here it just prints, to keep
// the example dependency-free and deterministic.
type spanHook struct{}

func (spanHook) BeforeCall(_ context.Context, info promptr.CallInfo) promptr.AfterFunc {
	fmt.Printf("start %s\n", info.Kind)
	return func(o promptr.Outcome) {
		fmt.Printf("end   %s err=%v\n", info.Kind, o.Err)
	}
}

func main() {
	// Any Provider; here a trivial inline one.
	base := promptr.ProviderFunc(func(context.Context, []promptr.Message) (string, error) {
		return "ok", nil
	})

	p := promptr.WithHooks(base, spanHook{})
	_, _ = p.Complete(context.Background(), nil)

}

Output:
start complete
end   complete err=<nil>

type ProviderFunc

type ProviderFunc func(ctx context.Context, messages []Message) (string, error)

ProviderFunc adapts a plain function to the Provider interface, for inline or test providers and for middleware that build wrappers without a named type.

func (ProviderFunc) Complete

func (f ProviderFunc) Complete(ctx context.Context, messages []Message) (string, error)

Complete calls the underlying function.

type Registry

type Registry map[string]Provider

Registry maps the client names declared in a .promptr file to live Providers the caller has wired up. Generated client constructors resolve their provider references through a Registry, so the DSL stays free of credentials.

func (Registry) Get

func (r Registry) Get(name string) Provider

Get returns the Provider registered under name, or an error Provider that fails every call — so a missing wiring surfaces at call time with a clear message rather than a nil-pointer panic.

type Reply

type Reply struct {
	Text  string
	Calls []ToolCall
}

Reply is one turn from a ToolProvider: either a final answer in Text, or one or more tool Calls the model wants run before it can answer. When Calls is non-empty the loop dispatches them and asks again; otherwise Text is final.

type Stats

type Stats struct {
	Calls        int
	Errors       int
	PromptTokens int
	ReplyTokens  int
	TotalLatency time.Duration
}

Stats is a rollup of a Collector's recorded calls.

func (Stats) AvgLatency

func (s Stats) AvgLatency() time.Duration

AvgLatency is the mean Complete duration, or zero when no calls were recorded.

func (Stats) TotalTokens

func (s Stats) TotalTokens() int

TotalTokens is the sum of prompt and reply tokens across all calls.

type StreamProvider

type StreamProvider interface {
	Stream(ctx context.Context, messages []Message) (<-chan string, error)
}

StreamProvider is an optional capability a Provider may implement to deliver a reply incrementally. Stream returns a channel of text chunks (e.g. server-sent token deltas); the channel closes when the model is done. A Provider that does not implement this is still usable for streaming extraction — ExtractStream falls back to a single Complete call.

type StreamToolProvider

type StreamToolProvider interface {
	StreamTools(ctx context.Context, messages []Message, tools []ToolDef) (ToolStream, error)
}

StreamToolProvider is an optional capability combining StreamProvider and ToolProvider: a tool-enabled turn whose text streams token-by-token while the model is still free to request tool calls instead of answering. A Provider that implements it lets RunToolsStream surface partial answers during the final turn of an agent loop. One that does not is still usable — RunToolsStream falls back to a single blocking RunTools call.

type TemplateError

type TemplateError struct{ Msg string }

TemplateError reports a structurally invalid template.

func (*TemplateError) Error

func (e *TemplateError) Error() string

type Tool

type Tool struct {
	Def    ToolDef
	Invoke func(ctx context.Context, argsJSON string) (resultJSON string, err error)
}

Tool binds a ToolDef to its Go implementation. Invoke receives the model's raw JSON arguments and returns the result as JSON to feed back into the conversation. Generated code builds these by wrapping a typed handler (decode args → call handler → marshal result); you can also construct them by hand.

type ToolCall

type ToolCall struct {
	ID        string
	Name      string
	Arguments string
}

ToolCall is the model's request to invoke a tool: an opaque ID the provider uses to correlate the result, the tool Name, and the raw JSON Arguments. The agent loop decodes Arguments tolerantly via the coerce kernel.

type ToolDef

type ToolDef struct {
	Name        string
	Description string
	Params      string // schema description of the JSON argument object
}

ToolDef describes a tool offered to the model: its name, a one-line description, and a human-readable schema of its argument object (built the same way output schemas are, so the model sees one consistent style). A ToolProvider marshals these into whatever its backend's tool/function-calling API expects.

type ToolProvider

type ToolProvider interface {
	CompleteTools(ctx context.Context, messages []Message, tools []ToolDef) (Reply, error)
}

ToolProvider is an optional capability a Provider may implement to support tool/function-calling. CompleteTools sends the conversation along with the available tool definitions and returns either the model's final text or the tool calls it wants executed. A Provider that does not implement this cannot run tool functions — RunTools returns a clear error.

type ToolStream

type ToolStream struct {
	Deltas <-chan string
	Reply  func() (Reply, error)
}

ToolStream is one streamed tool-enabled turn. Deltas carries the assistant's text as it generates (closed when the turn ends); Reply, valid only once Deltas has fully drained, returns the turn's outcome — the complete text and any tool calls the model requested. The contract is "drain Deltas, then call Reply": a provider may block sending on Deltas until the consumer reads, so a caller that stops reading early must cancel ctx to release it.

type Union

type Union = coerce.Union

Union is a resolver over several candidate variant types — the building block generated code uses for a function that returns a union. It is an alias for coerce.Union so generated code only needs to import promptr.

func NewUnion

func NewUnion(samples ...any) *Union

NewUnion builds a Union resolver from zero values of each variant type.

type UsageReporter

type UsageReporter interface {
	LastUsage() (prompt, reply int)
}

UsageReporter is an optional interface a Provider may implement to expose the exact token counts of its most recent Complete call. The Collector uses it when present and falls back to a chars/4 estimate otherwise. It is consulted immediately after Complete returns, so an implementation should report the usage of that call.