inference

package module
v0.12.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	StructuredOutputToolName = "_looprig_final_output"
	StructuredOutputRevision = "structured-output/v1"
)
View Source 
const (
	// MaxStructuredOutputDiagnosticBytes bounds caller-controlled string
	// metadata retained by structured-output errors.
	MaxStructuredOutputDiagnosticBytes = 128

	// MaxStructuredResultBytes bounds a native structured result before JSON
	// parsing or compaction. It matches the 1 MiB request-schema bound so both
	// structured-output boundaries have a single conservative memory ceiling.
	MaxStructuredResultBytes = 1 << 20

	// StructuredOutputFinishReasonOther classifies a non-empty finish reason
	// outside the provider-neutral set without retaining provider input.
	StructuredOutputFinishReasonOther stream.FinishReason = "other"
)

Variables

View Source 
var (
	ToolChoiceAuto     = ToolAuto()
	ToolChoiceRequired = ToolRequired()
)

ToolChoiceAuto and ToolChoiceRequired preserve the values exposed by the original two-state ToolChoice API. New code may use ToolAuto and ToolRequired; both spellings are equal and may be compared directly.

Functions

func DecodeMessageOutput 

func DecodeMessageOutput(msg *content.AIMessage, out any) error

DecodeMessageOutput extracts and strictly decodes a message into a non-nil concrete pointer. Required and other domain invariants remain caller-owned.

func DecodeOutput 

func DecodeOutput(resp *Response, out any) error

DecodeOutput extracts and strictly decodes a response into a non-nil concrete pointer. Required and other domain invariants remain the caller's concern.

func StructuredMessageResult 

func StructuredMessageResult(msg *content.AIMessage) (json.RawMessage, error)

StructuredMessageResult extracts exactly one JSON-object representation from assistant text fragments or one reserved terminal-tool input. Thinking blocks are ignored. The returned bytes are compacted and independently owned.

func StructuredResult 

func StructuredResult(resp *Response) (json.RawMessage, error)

StructuredResult extracts one structured JSON object from a complete response and verifies that the finish reason agrees with its representation.

func ValidateOutputSchema 

func ValidateOutputSchema(output OutputSchema) error

ValidateOutputSchema validates output against the bounded portable JSON Schema subset shared by provider codecs.

func ValidateRequestFeatures 

func ValidateRequestFeatures(req Request) error

ValidateRequestFeatures validates provider-neutral request feature combinations before a codec attempts to encode them.

Types

type Client 

type Client interface {
	Invoke(ctx context.Context, req Request) (*Response, error)
	Stream(ctx context.Context, req Request) (*stream.StreamReader[content.Chunk], error)
}

Client is the provider-neutral inference interface.

type ImageInputUnsupportedError 

type ImageInputUnsupportedError struct {
	Model string
}

ImageInputUnsupportedError reports that a request thread carries image blocks but the model does not advertise the AcceptsImages capability. Model is bounded diagnostic metadata only.

func (*ImageInputUnsupportedError) Error 

func (e *ImageInputUnsupportedError) Error() string

type InvalidTransientMessagesError 

type InvalidTransientMessagesError struct {
	Transient int
	Messages  int
}

InvalidTransientMessagesError reports a transient-message count that falls outside the request's message slice.

func (*InvalidTransientMessagesError) Error 

func (e *InvalidTransientMessagesError) Error() string

type MalformedStructuredOutputError 

type MalformedStructuredOutputError struct {
	ReasonCode MalformedStructuredOutputReason
	Length     int
	SHA256     [sha256.Size]byte
}

MalformedStructuredOutputError reports bounded metadata about malformed model output. SHA256 and Length support correlation without retaining or exposing the raw output bytes.

func (*MalformedStructuredOutputError) Error 

func (e *MalformedStructuredOutputError) Error() string

type MalformedStructuredOutputReason 

type MalformedStructuredOutputReason string

MalformedStructuredOutputReason is a bounded classification for an invalid structured response representation. It never contains model output. 

const (
	MalformedReasonNilResponse           MalformedStructuredOutputReason = "nil response"
	MalformedReasonNilMessage            MalformedStructuredOutputReason = "nil message"
	MalformedReasonWrongRole             MalformedStructuredOutputReason = "wrong role"
	MalformedReasonEmpty                 MalformedStructuredOutputReason = "empty"
	MalformedReasonMalformedJSON         MalformedStructuredOutputReason = "malformed JSON"
	MalformedReasonRootNotObject         MalformedStructuredOutputReason = "root is not object"
	MalformedReasonInvalidRepresentation MalformedStructuredOutputReason = "invalid representation"
	MalformedReasonAmbiguous             MalformedStructuredOutputReason = "ambiguous"
	MalformedReasonInvalidBlock          MalformedStructuredOutputReason = "invalid block"
	MalformedReasonNilBlock              MalformedStructuredOutputReason = "nil block"
	MalformedReasonTooLarge              MalformedStructuredOutputReason = "too large"
)

type OutputSchema 

type OutputSchema struct {
	Name        string
	Description string
	Schema      json.RawMessage
	Strict      bool
}

OutputSchema is a provider-neutral request for one schema-constrained JSON object. Description must be valid UTF-8 and is limited to 4096 bytes. Schema must satisfy the portable subset checked by ValidateOutputSchema.

func (OutputSchema) Clone 

func (o OutputSchema) Clone() OutputSchema

Clone returns an independent copy of the output schema.

type Request 

type Request struct {
	Model             model.Model
	System            string
	Messages          content.AgenticMessages
	TransientMessages int
	Tools             []Tool
	Output            *OutputSchema
	ToolChoice        ToolChoice
	Override          *model.Sampling
}

Request is the provider-neutral inference request. It carries a secret-free Model descriptor for this turn, the per-agent System prompt, the message thread, the count of trailing transient messages, the exposed tools, an optional structured Output contract, the ToolChoice (zero value: auto), and an optional per-call sampling Override (nil means use Model.Sampling).

type Response 

type Response struct {
	Message      *content.AIMessage
	Usage        *content.Usage
	Model        string
	FinishReason stream.FinishReason
	// Attempts is how many attempts produced this response when served
	// through a retrying decorator; 0 means the serving client does not
	// count attempts, 1 means first-try success.
	Attempts int
}

Response is the complete provider-neutral response.

type SchemaValidationError 

type SchemaValidationError struct {
	Field      SchemaValidationField
	ReasonCode SchemaValidationReason
}

SchemaValidationError reports a stable, bounded validation classification. It never retains schema bytes, property names, descriptions, or JSON decoder errors because those values may contain sensitive caller input.

func (*SchemaValidationError) Error 

func (e *SchemaValidationError) Error() string

type SchemaValidationField 

type SchemaValidationField string

SchemaValidationField identifies the output-schema component that failed validation. Values are stable classifications suitable for errors.As callers. 

const (
	SchemaFieldName                 SchemaValidationField = "Name"
	SchemaFieldDescription          SchemaValidationField = "Description"
	SchemaFieldSchema               SchemaValidationField = "Schema"
	SchemaFieldKeyword              SchemaValidationField = "Keyword"
	SchemaFieldType                 SchemaValidationField = "Type"
	SchemaFieldProperties           SchemaValidationField = "Properties"
	SchemaFieldItems                SchemaValidationField = "Items"
	SchemaFieldEnum                 SchemaValidationField = "Enum"
	SchemaFieldRequired             SchemaValidationField = "Required"
	SchemaFieldAdditionalProperties SchemaValidationField = "AdditionalProperties"
	SchemaFieldOutput               SchemaValidationField = "Output"
)

type SchemaValidationReason 

type SchemaValidationReason string

SchemaValidationReason identifies why an output-schema component failed validation. It intentionally contains no caller-provided content. 

const (
	SchemaReasonEmpty             SchemaValidationReason = "empty"
	SchemaReasonInvalid           SchemaValidationReason = "invalid"
	SchemaReasonReserved          SchemaValidationReason = "reserved"
	SchemaReasonTooLong           SchemaValidationReason = "too long"
	SchemaReasonInvalidUTF8       SchemaValidationReason = "invalid UTF-8"
	SchemaReasonMalformed         SchemaValidationReason = "malformed"
	SchemaReasonTooLarge          SchemaValidationReason = "too large"
	SchemaReasonRootNotObject     SchemaValidationReason = "root is not an object schema"
	SchemaReasonUnknownKeyword    SchemaValidationReason = "unknown keyword"
	SchemaReasonMissing           SchemaValidationReason = "missing"
	SchemaReasonUnsupported       SchemaValidationReason = "unsupported"
	SchemaReasonMustBeFalse       SchemaValidationReason = "must be false"
	SchemaReasonDuplicate         SchemaValidationReason = "duplicate"
	SchemaReasonUnknownProperty   SchemaValidationReason = "unknown property"
	SchemaReasonTypeMismatch      SchemaValidationReason = "type mismatch"
	SchemaReasonTooDeep           SchemaValidationReason = "too deep"
	SchemaReasonTooManyProperties SchemaValidationReason = "too many properties"
	SchemaReasonInvalidTarget     SchemaValidationReason = "invalid target"
	SchemaReasonDecodeFailed      SchemaValidationReason = "decode failed"
)

type StructuredOutputConflictError 

type StructuredOutputConflictError struct {
	Feature string
}

StructuredOutputConflictError reports an invalid request feature combination. Feature is a bounded classification, never a schema or tool payload supplied by the caller.

func (*StructuredOutputConflictError) Error 

func (e *StructuredOutputConflictError) Error() string

type StructuredOutputFinishError 

type StructuredOutputFinishError struct {
	Reason stream.FinishReason
}

StructuredOutputFinishError reports finish metadata that cannot safely produce the requested structured result.

func (*StructuredOutputFinishError) Error 

func (e *StructuredOutputFinishError) Error() string

type StructuredOutputUnsupportedError 

type StructuredOutputUnsupportedError struct {
	Model string
}

StructuredOutputUnsupportedError reports that a model does not advertise native structured output. Model is diagnostic metadata only.

func (*StructuredOutputUnsupportedError) Error 

func (e *StructuredOutputUnsupportedError) Error() string

type StructuredOutputWithToolsUnsupportedError 

type StructuredOutputWithToolsUnsupportedError struct {
	Model string
}

StructuredOutputWithToolsUnsupportedError reports that a model does not advertise the distinct native structured-output-with-tools capability. Model is diagnostic metadata only.

func (*StructuredOutputWithToolsUnsupportedError) Error 

func (e *StructuredOutputWithToolsUnsupportedError) Error() string

type Tool 

type Tool struct {
	Name        string
	Description string
	Schema      json.RawMessage
}

Tool is a callable function definition exposed to the model.

type ToolChoice 

type ToolChoice struct {
	// contains filtered or unexported fields
}

ToolChoice controls whether the model may choose between text and tools, must call some tool, or must call one named tool.

It is an opaque comparable value built only by ToolAuto, ToolRequired and ToolNamed, so the forced tool name cannot be separated from the variant that gives it meaning: "named choice, no name" and "name, but not a named choice" are unspellable rather than merely validated. Carrying the name in a sibling Request field would make both reachable and would need a validation error code apiece to catch after the fact; keeping it inside the variant removes the states instead of policing them.

One cross-field invariant a type cannot encode remains, and is checked in ValidateRequestFeatures: a named choice whose name matches no declared tool. No provider schema enforces that either — measured across all five.

The zero value is ToolAuto(), so a Request struct literal that never mentions ToolChoice keeps the automatic behavior. That requirement is why this is a struct and not a sealed interface in the shape of content.Block: an interface has nil for a zero value, so "auto" would have two spellings and every consumer would need a nil arm — reintroducing exactly the kind of invalid state this type exists to remove.

A variant is added here, not by callers: the discriminant is unexported and the constructors are the only way in. A future multi-name allowlist (Gemini's allowedFunctionNames, OpenAI's allowed_tools) is therefore an additive change to this file — one mode, one field, one constructor — with the single cross-cutting cost that a slice-valued field would end ToolChoice's comparability with ==.

func ToolAuto 

func ToolAuto() ToolChoice

ToolAuto lets the model choose between text and tools. It is the zero value.

func ToolNamed 

func ToolNamed(name string) ToolChoice

ToolNamed forces the model to call the single tool called name, which the request must declare. Every dialect this module encodes has a wire form for it.

func ToolRequired 

func ToolRequired() ToolChoice

ToolRequired forces the model to call some tool of its own choosing. The request must declare at least one tool.

func (ToolChoice) Mode 

func (c ToolChoice) Mode() ToolChoiceMode

Mode reports which variant this choice is.

func (ToolChoice) Named 

func (c ToolChoice) Named() (name string, ok bool)

Named reports the single tool the model must call. ok is false for every variant other than ToolChoiceModeNamed, so a caller cannot read a forced name out of a choice that does not force one.

func (ToolChoice) String 

func (c ToolChoice) String() string

String renders the choice for diagnostics.

type ToolChoiceMode 

type ToolChoiceMode uint8

ToolChoiceMode enumerates the tool-choice variants. It is the discriminant a codec switches on; it is not itself a choice, because two of the three variants carry no data and the third is meaningless without its tool name. 

const (
	// ToolChoiceModeAuto lets the model decide between text and tools.
	ToolChoiceModeAuto ToolChoiceMode = iota
	// ToolChoiceModeRequired forces some tool call, leaving the model the choice
	// of which.
	ToolChoiceModeRequired
	// ToolChoiceModeNamed forces the one tool the choice names.
	ToolChoiceModeNamed
)

Source Files

View all Source files

Directories

Path Synopsis
Package auth provides the legacy inference authorization facade.
 Package auth provides the legacy inference authorization facade.
anthropicapi
bedrockconverse
conformance
Package conformance is the schema conformance gate for provider wire fixtures.
 Package conformance is the schema conformance gate for provider wire fixtures.
conformance/cmd/schemagen command
Command schemagen derives the checked-in JSON Schema 2020-12 documents that back the provider conformance gate.
 Command schemagen derives the checked-in JSON Schema 2020-12 documents that back the provider conformance gate.
geminiapi
openaiapi
openairesponses
Package openairesponses is the OpenAI Responses API wire dialect (POST /v1/responses): a genuinely different, items-based shape from OpenAI Chat Completions (codec/openaiapi) — not a flat messages array.
 Package openairesponses is the OpenAI Responses API wire dialect (POST /v1/responses): a genuinely different, items-based shape from OpenAI Chat Completions (codec/openaiapi) — not a flat messages array.
servertest
Package servertest provides a reusable contract suite for codec.ServerCodec implementations.
 Package servertest provides a reusable contract suite for codec.ServerCodec implementations.
Package contextcount provides deterministic complete-request context counting.
 Package contextcount provides deterministic complete-request context counting.
examples
gateway command
invoke command
retry command
stream command
Package failure owns provider-neutral inference failures shared by codecs, transports, and provider integrations.
 Package failure owns provider-neutral inference failures shared by codecs, transports, and provider integrations.
Package gateway provides a local HTTP compatibility layer that lets coding-harness clients speaking different model-API dialects (Anthropic Messages, OpenAI Responses, OpenAI Chat Completions, Gemini) reach any injected inference.Client/model.Model target.
 Package gateway provides a local HTTP compatibility layer that lets coding-harness clients speaking different model-API dialects (Anthropic Messages, OpenAI Responses, OpenAI Chat Completions, Gemini) reach any injected inference.Client/model.Model target.
internal
jsonstrict
Package jsonstrict provides a small, dialect-agnostic JSON scan used by every codec/*api server-decode path to reject a request body that smuggles a duplicate object member name.
 Package jsonstrict provides a small, dialect-agnostic JSON scan used by every codec/*api server-decode path to reject a request body that smuggles a duplicate object member name.
usagenorm
Package usagenorm validates and normalizes provider token-usage wire values.
 Package usagenorm validates and normalizes provider token-usage wire values.
Package retry decorates an inference.Client with bounded, classified retry and exponential backoff.
 Package retry decorates an inference.Client with bounded, classified retry and exponential backoff.
Package route holds concrete route.Router builders for the bundled wire APIs: a static chat route (OpenAI/Anthropic style) and Gemini's mode-aware model-in-path route.
 Package route holds concrete route.Router builders for the bundled wire APIs: a static chat route (OpenAI/Anthropic style) and Gemini's mode-aware model-in-path route.
Package transport is a generic, connection-bound HTTP client for the inference seam.
 Package transport is a generic, connection-bound HTTP client for the inference seam.
wire
eventstream
Package eventstream frames AWS Event Stream messages into raw inference stream frames.
 Package eventstream frames AWS Event Stream messages into raw inference stream frames.
jsonbody
Package jsonbody holds small stdlib helpers for JSON HTTP bodies: marshal a value into a request-body reader (with its content type) and unmarshal response bytes back into a value.
 Package jsonbody holds small stdlib helpers for JSON HTTP bodies: marshal a value into a request-body reader (with its content type) and unmarshal response bytes back into a value.
ndjson
Package ndjson frames a newline-delimited JSON body into one raw stream frame per line: StreamFrame.Data is the line's bytes and Name is empty.
 Package ndjson frames a newline-delimited JSON body into one raw stream frame per line: StreamFrame.Data is the line's bytes and Name is empty.
sse
Package sse is a real Server-Sent Events event framer (WHATWG event stream model), not an OpenAI-only "data: " line stripper.
 Package sse is a real Server-Sent Events event framer (WHATWG event stream model), not an OpenAI-only "data: " line stripper.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.