models

Documentation

Overview

Package models defines the ModelSpec value type — a typed object combining a model identity with its inference configuration — and the supporting ThinkingLevel type used to express reasoning effort.

ModelSpec is the canonical argument to provider invocations. It supersedes the bare-string model identity and the per-call option type that previously lived in the provider package. The split is deliberate: a Spec describes a model (identity + how to run it), while a provider translates a Spec to a vendor's wire format.

The Spec type is intentionally permissive: pointer-typed fields distinguish "set" from "use the framework / model default", and unknown fields are silently ignored by adapters. This lets the framework grow new Spec fields (and lets vendors grow new catalog entries) without breaking every existing call site.

Per-vendor model catalogs live in the unified catalog module (x/catalog/models), exported as named Spec values for the p80 case (models.Catalog().OpenAI().GPT4o, models.Catalog().Anthropic().ClaudeOpus45, …). Per-vendor provider adapters in x/provider/<vendor>/ expose the same names to applications; ad-hoc construction covers the long tail:

models.Spec{
    Name: "ft:gpt-3.5-turbo:my-org:custom",
    Temperature: ptr(0.3),
}

Index

• type Spec
• type ThinkingLevel
  • func ParseThinkingLevel(s string) (ThinkingLevel, error)
  • func (l ThinkingLevel) Valid() bool

Types

type Spec

type Spec struct {
	// Name is the model identifier as understood by the provider.
	Name string

	// Window is the model's context window size in tokens. Not a
	// request budget; this is the upper bound on input the model
	// can accept. Applications that implement automatic
	// compaction triggers may read this field to decide when to
	// invoke compaction.Summarize; the compaction package itself
	// does not consult it (compaction today is explicit-only via
	// /compact).
	Window int

	// MaxOutputTokens is the per-invocation output token budget.
	// Zero or negative values mean "no opinion" — adapters omit
	// the wire field and use their own default.
	MaxOutputTokens int64

	// Temperature sets the sampling temperature. nil means "use
	// the model's default" (typically 1.0 for most models). 0.0
	// (when not nil) requests greedy decoding.
	Temperature *float64

	// ThinkingLevel asks the model to spend the given amount of
	// reasoning effort. Adapters translate the level to their
	// wire format. The empty value is treated as "off" by
	// adapters; the framework does not normalize the field.
	ThinkingLevel ThinkingLevel

	// TopP is the nucleus sampling cutoff. nil means "use the
	// model's default".
	TopP *float64

	// TopK restricts sampling to the top-K tokens. nil means
	// "use the model's default".
	TopK *int

	// Seed pins the random seed for reproducible outputs. nil
	// means "no seed" (non-deterministic).
	Seed *int64

	// StopSequences are strings at which the model halts
	// generation. nil or empty means "use the model's default".
	StopSequences []string

	// FrequencyPenalty discourages token repetition based on
	// frequency. nil means "use the model's default".
	FrequencyPenalty *float64

	// PresencePenalty discourages token repetition based on
	// presence. nil means "use the model's default".
	PresencePenalty *float64
}

Spec is a value object combining a model identity with its inference configuration. It is the canonical argument to provider invocations.

Identity

A Spec's Name is the model identifier as understood by the provider that serves it. The same Name may be portable across providers (e.g. "claude-opus-4-5" served by Anthropic, Vertex, or OpenRouter) — the framework does not enforce a 1:1 mapping; that is the application's wiring concern.

Inference configuration

The fields below constrain or influence how the model runs. Pointer fields distinguish "explicitly set" from "use the framework / model default"; zero-value fields are framework defaults where applicable (MaxOutputTokens == 0 means "no opinion" — adapters omit the wire field and let their default apply). Adapters that do not recognize a given field leave it on the floor; the Spec is permissive by design.

Not in Spec

The following are *data within the inference call*, not inference configuration, and live elsewhere (per-call InvokeOptions or arguments):

• System prompt
• Tools (function definitions available to the model)
• Message history (in ledger.State)
• User-visible content

Capabilities (audio, video, image, tool support) are a separate axis and are out of scope for this version. When a concrete consumer emerges, add a Capabilities() method on Spec.

type ThinkingLevel

type ThinkingLevel string

ThinkingLevel is a portable, qualitative description of how much reasoning effort a model should spend on a turn. Adapters translate the level into their provider's wire format at request time.

Levels are case-sensitive lowercase strings. The level is the user's intent; the adapter is the translator. The empty string is not a valid level — callers should substitute their own default (commonly ThinkingLevelOff) before calling ParseThinkingLevel.

This type was moved from the provider package so that Spec can carry it as a field without creating a cycle (the provider package depends on models for the Spec type).

const (
	// ThinkingLevelOff disables extended thinking. Adapters must not
	// send a `thinking` field (or equivalent) when this level is
	// requested; the request is identical to a non-thinking request.
	ThinkingLevelOff ThinkingLevel = "off"

	// ThinkingLevelMinimal asks for the smallest amount of thinking
	// the provider supports. Useful as a low-cost pipeline probe.
	ThinkingLevelMinimal ThinkingLevel = "minimal"

	// ThinkingLevelLow asks for a small amount of thinking.
	ThinkingLevelLow ThinkingLevel = "low"

	// ThinkingLevelMedium asks for a moderate amount of thinking.
	// Recommended default for reasoning-capable models when the
	// application or user opts in.
	ThinkingLevelMedium ThinkingLevel = "medium"

	// ThinkingLevelHigh asks for a substantial amount of thinking.
	ThinkingLevelHigh ThinkingLevel = "high"

	// ThinkingLevelMax asks for the maximum amount of thinking the
	// provider allows, while still leaving room for the visible
	// response. Adapters may clamp this to their maximum.
	ThinkingLevelMax ThinkingLevel = "max"
)

func ParseThinkingLevel

func ParseThinkingLevel(s string) (ThinkingLevel, error)

ParseThinkingLevel parses a string into a ThinkingLevel. The empty string is treated as a parse error — callers should substitute their own default (commonly ThinkingLevelOff) before calling. Levels are case-sensitive lowercase.

func (ThinkingLevel) Valid

func (l ThinkingLevel) Valid() bool

Valid reports whether the level is one of the defined constants. The empty string is not valid.