arena

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 27, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package arena runs Chatwright's actor-model comparison matrix: the same Scenario (a goal plus a platform environment), the same budgets, across a declared set of provider/model configurations, N repeats each — see spec/ideas/actor-model-arena.md in the chatwright/chatwright standard repository for the arena this package implements (mandatory warm-up with cold-start as its own metric, right-sized context windows, a full retry breakdown, evidence over claims).

Ported from the scratchpad harness that produced the first actor-model arena report (chatwright/backstage research/model-arena-2026-07-23): the same warm-up/right-sizing/retry-breakdown mechanics, restructured as a reusable public API rather than a pair of throwaway cmd/ binaries driven by shell scripting. The harness wrote a run bundle plus a JSON "sidecar" per cell (call-level detail a run bundle has no field for — see CallRecord) and read them back from disk to build a report; this package never writes anything to disk itself. Run returns everything — including every cell's full sdk.Bundle — as in-memory Results; a caller (the chatwright arena CLI subcommand) persists whatever it needs (bundles, the report, a machine-readable Results dump) into its own chosen directory layout. See Run and WriteReport.

Host stability on Apple Silicon

Rapid large-model Metal load/unload cycling has triggered macOS GPU kernel panics during arena reruns on this hardware (chatwright/backstage research/model-arena-2026-07-23/crash-analysis.md): two panics, both an identical IOGPU/Metal driver signature ("Memory object unexpectedly not found in fPendingMemorySet" @IOGPUGroupMemory.cpp:219), observed 2/2 on macOS 26.5.2, Apple M5 Max 36GB unified memory, both during large-model (26B/27B) blocks. A third panic then occurred at idle, forty minutes after model activity had already finished cleanly — so model size alone is not a reliable predictor, and the machine is not safe to treat as "done" the moment inference stops. Until this is root-caused as a driver bug rather than a workload trigger: run any large-model matrix on Apple Silicon only attended, with RunOptions.FlightLog set to a persistent path (see FlightLog) — a kernel panic erases RAM, including whatever the model server itself was logging, so the flight log's fsync'd last line is what tells you which phase, and which model, was running when the machine went down.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultLoaders 

func DefaultLoaders() map[ProviderKind]Loader

DefaultLoaders returns the built-in Loader for each ProviderKind this package has a specific mechanism for: LMStudioLoader{} for KindLMStudio, OllamaLoader{} for KindOllama. RunOptions.Loaders starts from this map when left nil; a caller overrides one entry (e.g. to inject a fake Loader in a test) by setting RunOptions.Loaders itself. A ProviderKind with no entry gets NoopLoader.

func WriteReport 

func WriteReport(w io.Writer, results Results) error

WriteReport writes results as a markdown comparison report to w: an environment block (hardware, context lengths, load-hook outcomes, date — spec/ideas/actor-model-arena.md), a headline table (completion, cold-start, latency p50/p95, wall, tokens, steps, stop reasons), the required retry breakdown, the structured-output-mode split, a per-cell detail table naming every cell's bundle, and a short per-model narrative. Every declared provider in results.Models gets a row in every table, however its block went (a ProviderErr or a per-cell Err renders as an explicit error, never a silently dropped row) — the spec's exclusion policy: a model leaves a report only with a recorded, evidence-linked reason, never silently.

Types

type CallRecord 

type CallRecord struct {
	Index int
	At    time.Time
	Wall  time.Duration
	// Mode is the response_format mode that served this call (see
	// openai.ResponseFormatMode) when the wrapped Provider reports one,
	// empty otherwise (e.g. a ScriptedProvider in a test).
	Mode         string
	TaskID       string
	ProposalKind string
	InputTokens  int
	OutputTokens int
	// Error is the Propose call's own error text, empty on success.
	Error string
}

CallRecord is one Provider.Propose call, successful or not — the detail a run bundle's LoopEvent has no field for. actor.Loop only appends a LoopEvent after a successful Propose call (a Propose error aborts the task immediately, before any event is recorded — see actor.Loop.RunTask), so a failed call is otherwise invisible to a bundle entirely; the required retry breakdown's transport-errors count (spec/ideas/ actor-model-arena.md) depends on CallRecord for exactly that reason. Ported from the scratchpad harness's internal/sidecar.CallRecord, moved from a JSON sidecar file into this typed in-memory record — see the package doc comment.

type CellResult 

type CellResult struct {
	Repeat int

	// BundleName is a suggested, collision-free filename for Bundle — the
	// spec's "per-cell bundle names" report requirement. A caller
	// persisting bundles should use it verbatim, e.g.
	// filepath.Join(outDir, "bundles", cell.BundleName).
	BundleName string
	// Bundle is this cell's complete sdk.Bundle. Run never writes it to
	// disk — see the package doc comment.
	Bundle sdk.Bundle
	Wall   time.Duration

	StopReason   string
	PartStatus   string
	TaskStatus   string
	Steps        int
	InputTokens  int
	OutputTokens int

	// Calls is every Propose call this cell made, successful or not — the
	// retry breakdown's transport-error source (see CallRecord).
	Calls []CallRecord
	// ActionCounts tallies actor.ActionOutcomeKind values (as their JSON
	// string form, e.g. "skipped-invalid") across this cell's LoopEvents.
	ActionCounts map[string]int
	// ModeCounts tallies each successful Propose call's response_format
	// mode (see CallRecord.Mode); a call with no reported mode is not
	// counted.
	ModeCounts map[string]int
	// Latencies is each LoopEvent's Usage.Latency, in the loop's own
	// order — the p50/p95 proposal-latency metric's raw material.
	Latencies []time.Duration

	// Verified/VerifyDetail carry Scenario.Verify's verdict, when the
	// Scenario declares one (see Scenario.Verify) — both zero value
	// (false, "") when it does not.
	Verified     bool
	VerifyDetail string

	// Err is set when this cell's own setup or run.Run.Execute call
	// returned a hard, Run-level configuration error (see
	// run.Run.Execute's own doc comment on what that error is reserved
	// for) — a harness/arena bug, never a model's own data point. Every
	// other field is zero when Err is set: this cell contributed no
	// evidence.
	Err error
}

CellResult is one timed repeat's complete outcome.

type Environment 

type Environment struct {
	Hardware  string
	OS, Arch  string
	GoVersion string
	Date      time.Time
	Providers []ProviderEnvironment
}

Environment is Results' declared hardware/software/scenario context — the spec's "entries carry a declared hardware/environment block".

type FlightLog added in v0.3.0 

type FlightLog struct {
	// contains filtered or unexported fields
}

FlightLog is an append-only, fsync-per-line log of an arena Run's phase boundaries — the record the founder ordered after repeated macOS kernel panics during arena reruns (chatwright/runtime-go#8; chatwright/backstage research/model-arena-2026-07-23/crash-analysis.md): a kernel panic erases every RAM-resident buffer, including a provider server's own logs — LM Studio's lost the fatal minutes across two of those panics. Logf writes one line, fsync'd to disk, before returning, so after any crash — however abrupt — whatever the flight log's last line names is provably the last thing arena.Run started, and how far it got.

See RunOptions.FlightLog for how a Run wires one in; nil (the default) disables flight logging entirely with zero behavioural change.

Give it a persistent path — never /tmp

OpenFlightLog exists to survive a crash that wipes RAM. A path under an OS temp directory can itself be cleared by the same reboot/crash recovery that follows a panic, or never even be backed by durable storage — defeating the entire point. Point path at somewhere the caller actually owns and expects to persist (a run's own output directory, a fixed ~/.chatwright/arena/flight-logs file, ...) — anywhere durable, never /tmp.

func OpenFlightLog added in v0.3.0 

func OpenFlightLog(path string) (*FlightLog, error)

OpenFlightLog opens (or creates) path for append-only writing — os.O_APPEND|os.O_CREATE, deliberately never os.O_TRUNC: re-running arena.Run against the same path preserves whatever a prior run, crashed mid-flight or not, already wrote, rather than erasing it. See FlightLog's own doc comment for why path must be a persistent location.

func (*FlightLog) Close added in v0.3.0 

func (fl *FlightLog) Close() error

Close closes the underlying file. Call once, after a Run (or matrix of Runs) finishes.

func (*FlightLog) Logf added in v0.3.0 

func (fl *FlightLog) Logf(format string, args ...any) error

Logf formats one line as "<RFC3339Nano timestamp> <message>\n", appends it to the file, and calls (*os.File).Sync() — before returning. By the time Logf returns, the line is durable: any reader of path, including a freshly-opened, independent *os.File in another process, is guaranteed to see it, even if this process is killed (or the kernel panics) on the very next instruction. Safe for concurrent use.

type LMStudioLoader 

type LMStudioLoader struct {
	// Exec runs name with args, returning combined output — defaults to
	// os/exec (exec.CommandContext(ctx, name, args...).CombinedOutput());
	// overridden in tests so this package's unit tests never shell out for
	// real.
	Exec func(ctx context.Context, name string, args ...string) ([]byte, error)
	// LookPath resolves name to an executable path — defaults to
	// exec.LookPath; overridden in tests alongside Exec.
	LookPath func(name string) (string, error)
}

LMStudioLoader right-sizes and evicts via the `lms` CLI — LM Studio's own command-line tool (spec/ideas/actor-model-arena.md: "lms load --context-length"): `lms unload --all` (the evict-others hook) then `lms load <model> --context-length <n> --yes`. Both steps are best-effort: a missing `lms` binary, or either subcommand failing, degrades to LoadResult{Performed: false} rather than returning an error — the arena keeps running against whatever the server already has loaded.

func (LMStudioLoader) Load 

func (l LMStudioLoader) Load(ctx context.Context, spec ProviderSpec) (LoadResult, error)

Load implements Loader.

type LoadResult 

type LoadResult struct {
	// Performed is true when eviction plus a right-sized load actually
	// ran. False means Run degrades to a JIT load, relying on a long
	// warm-up timeout instead (the spec's documented fallback) — never a
	// hard failure.
	Performed bool
	// Note is a short, human-readable explanation: what ran, or why it
	// didn't (tooling absent, a non-fatal error).
	Note string
}

LoadResult is one Loader.Load call's outcome — always recorded in the environment block (spec/ideas/actor-model-arena.md's right-sizing rule: "Recorded in the report's environment block so entries stay comparable") regardless of whether right-sizing actually happened, so a report reader can tell "loaded at a 4k context" apart from "server defaulted to whatever it felt like" at a glance.

type Loader 

type Loader interface {
	Load(ctx context.Context, spec ProviderSpec) (LoadResult, error)
}

Loader evicts whatever else a server has loaded and pre-loads a ProviderSpec's model at its declared ContextLength, when the server's tooling supports it — see LMStudioLoader, OllamaLoader. Deliberately optional (spec/ideas/actor-model-arena.md: "both optional (degrade to JIT with a long warm-up timeout when tooling is absent)"): a Loader that cannot do this degrades to LoadResult{Performed: false, ...} rather than failing the run — the mandatory warm-up call (see Run) still measures cold-start either way, just against a JIT load path instead of a pre-sized one.

type Matrix 

type Matrix struct {
	// Scenario is the goal/environment every provider/repeat runs
	// identically — see Scenario, GreetbotScenario.
	Scenario Scenario
	// Providers is this matrix's declared column set, run in this order —
	// see ProviderSpec.
	Providers []ProviderSpec
	// Repeats is how many timed campaigns each provider runs, on top of
	// its own mandatory untimed warm-up. Must be >= 1.
	Repeats int
	// Budgets overrides every cell's goal.Budgets when non-zero (compared
	// against the zero value); the zero value leaves Scenario.Goal.Budgets
	// as declared. Applies uniformly across the whole matrix — the spec's
	// "identical budgets" requirement.
	Budgets goal.Budgets
}

Matrix declares one arena run: the Scenario every cell attempts identically, the provider/model columns to compare, how many timed repeats each gets, and the Budgets every cell runs under.

type ModelResult 

type ModelResult struct {
	Spec   ProviderSpec
	Warmup *WarmupResult
	Cells  []CellResult
	// ProviderErr is set when Run could not even construct this
	// provider's actor.Provider (a ProviderFactory failure) — recorded as
	// a data point rather than aborting the whole matrix, per the spec's
	// exclusion policy: a model leaves the report only with a recorded,
	// evidence-linked reason, "never silently".
	ProviderErr error
}

ModelResult is one matrix column's complete outcome: its declared spec, its mandatory warm-up, and every timed repeat it ran.

type NoopLoader 

type NoopLoader struct{}

NoopLoader performs no eviction/right-sizing at all — the default for KindOpenAICompat, and any ProviderKind a caller does not supply a Loader for.

func (NoopLoader) Load 

func (NoopLoader) Load(context.Context, ProviderSpec) (LoadResult, error)

Load implements Loader: always degraded, never an error.

type OllamaLoader 

type OllamaLoader struct {
	HTTPClient *http.Client
	// KeepAlive is sent as the pre-load request's "keep_alive" — how long
	// Ollama keeps the model resident after this call. Defaults to "10m".
	KeepAlive string
}

OllamaLoader right-sizes via Ollama's native API (spec/ideas/ actor-model-arena.md: "a native-API pre-load with num_ctx options (Ollama)"): it POSTs an empty-prompt /api/generate request carrying options.num_ctx and a long keep_alive, which loads the model without generating anything. Ollama itself evicts whatever else it had resident (its default is a single resident model), so — unlike LM Studio, which can cohabit several loaded models — no separate evict step is needed. A transport error (server unreachable, an old Ollama build with no native API) degrades rather than failing the run.

func (OllamaLoader) Load 

func (l OllamaLoader) Load(ctx context.Context, spec ProviderSpec) (LoadResult, error)

Load implements Loader.

type ProviderEnvironment 

type ProviderEnvironment struct {
	Spec       ProviderSpec
	LoadResult LoadResult
}

ProviderEnvironment records one matrix column's declared configuration plus whatever its per-block Loader actually managed to do.

type ProviderKind 

type ProviderKind string

ProviderKind names which local/hosted server family a ProviderSpec targets — the seam DefaultLoaders and the default provider factory key off to pick the right eviction/right-sizing mechanism (see Loader) and display label. It does not change how Propose calls are made: every kind speaks the OpenAI-compatible wire format (see actor/openai's own package doc comment), so a server this package has no specific mechanism for still runs, as KindOpenAICompat — only the optional eviction/ right-sizing pre-load step is skipped; the mandatory warm-up call still applies. 

const (
	// KindOllama: a local Ollama server — right-sized/pre-loaded via its
	// native API (see OllamaLoader).
	KindOllama ProviderKind = "ollama"
	// KindLMStudio: a local LM Studio server — right-sized/pre-loaded via
	// the `lms` CLI when present (see LMStudioLoader).
	KindLMStudio ProviderKind = "lmstudio"
	// KindOpenAICompat: any other OpenAI-compatible endpoint (a hosted
	// vendor, OpenRouter, vLLM, ...) — no eviction/right-sizing mechanism.
	KindOpenAICompat ProviderKind = "openai-compat"
)

Provider kinds. See ProviderKind.

type ProviderSpec 

type ProviderSpec struct {
	// Kind selects the eviction/right-sizing mechanism (see ProviderKind,
	// Loader) and is recorded in the report; it does not change how
	// Propose calls are made.
	Kind ProviderKind
	// Label overrides this column's display name in the report (default:
	// "<Kind>/<Model>", matching the scratchpad harness's report rows).
	Label string
	// BaseURL is the OpenAI-compatible server's base URL, e.g.
	// "http://localhost:11434/v1" for Ollama or "http://localhost:1234/v1"
	// for LM Studio — see actor/openai.Config.BaseURL.
	BaseURL string
	// Model is the model id as the server expects it.
	Model string
	// ContextLength is the context window to load the model with, where
	// the server allows it — the spec's right-sizing rule (spec/ideas/
	// actor-model-arena.md, "founder rule 2026-07-23"). Zero means "let
	// the server pick its own default"; recorded either way in the
	// environment block so entries stay comparable.
	ContextLength int
	// APIKey optionally authenticates every request — see
	// actor/openai.Config.APIKey.
	APIKey string
	// MaxTokens bounds each reply; <= 0 uses actor/openai.DefaultMaxTokens.
	MaxTokens int
}

ProviderSpec declares one matrix column: a provider/model configuration every repeat runs identically.

type Results 

type Results struct {
	Scenario    ScenarioInfo
	Environment Environment
	// Models is one ModelResult per Matrix.Providers entry, in that same
	// declared order — every declared provider always gets an entry here,
	// however its block went (see ModelResult.ProviderErr,
	// CellResult.Err).
	Models []ModelResult
}

Results is everything one arena Run produced.

func Run 

func Run(ctx context.Context, matrix Matrix, opts RunOptions) (Results, error)

Run executes matrix's cells sequentially: one per-model block per Matrix.Providers entry, in declared order — a Loader evict/right-size hook, then one untimed mandatory warm-up, then Matrix.Repeats timed campaigns. Sequential by design (spec/ideas/actor-model-arena.md, "Not Doing": "Parallel arena execution — sequential first; timing fidelity beats speed").

A per-provider failure (ProviderFactory) is recorded on that ModelResult and Run moves on to the next provider; a per-cell setup or run.Run.Execute configuration error is recorded on that CellResult the same way. Run's own returned error is reserved for a Matrix that cannot be executed at all (no Scenario.Setup, no Providers, Repeats < 1) or a Loader's own hard error — never for a model's own data point: every declared provider that Run does start always gets a ModelResult, however it went, per the spec's exclusion policy ("never silently").

type RunOptions 

type RunOptions struct {
	// Now supplies Run's notion of the current time, for every clock this
	// package's own logic needs: cold-start/wall timing, each cell's
	// run.Environment clock, and Results.Environment.Date. Nil uses
	// time.Now.
	Now func() time.Time

	// HTTPTimeout bounds one timed-repeat Propose call. <= 0 defaults to
	// 150s.
	HTTPTimeout time.Duration
	// WarmupTimeout bounds the mandatory untimed warm-up call — longer
	// than HTTPTimeout by default because a cold/JIT load can run long
	// (spec/ideas/actor-model-arena.md: "LM Studio loading a 27B exceeded
	// a 60s call timeout"). <= 0 defaults to 300s.
	WarmupTimeout time.Duration

	// Loaders maps a ProviderKind to the Loader Run uses for that kind's
	// per-model-block evict-others-then-right-size step — see Loader. Nil
	// uses DefaultLoaders(); a kind missing from the map gets NoopLoader.
	Loaders map[ProviderKind]Loader

	// ProviderFactory builds the actor.Provider Run drives for one
	// ProviderSpec — called exactly once per matrix column, and the
	// result reused for that column's mandatory warm-up and every one of
	// its timed repeats (a real actor/openai.Provider is a thin, stateless
	// HTTP wrapper — safe to reuse across calls; see
	// actor/openai.Provider.LastResponseFormatMode's own concurrency-safe
	// doc comment). Nil uses a default that builds an actor/openai.Provider
	// from the spec's BaseURL/Model/APIKey/MaxTokens. Tests override this
	// to substitute actor.NewScriptedProvider — see the package's e2e
	// test — so a whole matrix runs at zero cost and zero tokens, with no
	// real network or CLI dependency.
	ProviderFactory func(spec ProviderSpec) (actor.Provider, error)

	// Hardware is a free-text label for the report's environment block
	// (spec/ideas/actor-model-arena.md: entries "carry a declared
	// hardware/environment block") — e.g. "Apple M5 Max, 36GB unified
	// memory". Never auto-detected: left blank unless the caller supplies
	// it.
	Hardware string

	// ProgressWriter, when non-nil, receives one formatted stage line
	// (FormatProgressLine) per run.ProgressSnapshot emitted while running
	// each cell — spec/ideas/campaign-progress-reporting.md's "the arena
	// harness (per-cell progress and matrix position, e.g. 'model 2/4 ·
	// repeat 1/3 · task 1/2 · steps 5/12')". Nil (the zero value) means no
	// progress output — Run behaves exactly as before this field existed.
	ProgressWriter io.Writer

	// FlightLog, when non-nil, makes Run write one fsync'd line to it
	// before/around every consequential phase boundary — matrix start,
	// each model block's loader invocation and its actual outcome, warm-up
	// start/end (with cold-start seconds), each cell's start/end (repeat
	// index, outcome summary), block end (with a best-effort host-memory
	// snapshot), and matrix end. Nil (the default) disables flight logging
	// entirely — zero behavioural change, and Run never allocates or opens
	// anything on its account. See FlightLog and, in particular, its own
	// doc comment on why the path backing it must be persistent, never
	// /tmp: this exists to survive a kernel panic (chatwright/runtime-go#8).
	FlightLog *FlightLog
}

RunOptions configures Run beyond what Matrix declares: clock injection, timeouts, and the seams tests use to substitute a fake Provider/Loader for the real network/CLI ones.

type Scenario 

type Scenario struct {
	// ID is this scenario's stable identity, e.g.
	// "greetbot-language-onboarding".
	ID string
	// Version is this scenario's own revision, independent of this
	// package's version — bump it whenever Goal, Setup or Verify changes
	// in a way that makes an old and a new run no longer comparable.
	Version string
	// Title is a human-readable one-line summary for the report header.
	Title string

	// Goal is the goal.Goal every provider/repeat attempts, identically —
	// including its own Budgets, used unless Matrix.Budgets overrides them
	// (see Matrix.Budgets).
	Goal goal.Goal

	// Setup boots a fresh platform environment (an emulator plus a
	// bot-under-test wired to it) for exactly one cell — called once per
	// timed repeat, and once more (untimed) for the mandatory warm-up.
	// Every call gets fresh state: no cell, and no warm-up call, ever
	// observes another call's conversation history.
	Setup func() (*ScenarioSession, error)

	// Verify optionally re-derives completion from the chat's raw journal
	// after a cell finishes, independent of whatever the actor itself
	// proposed (ProposeTaskDone) — see VerifyResult. Nil means a cell's
	// only completion signal is its own Report TaskOutcome.Status (the
	// model's self-declared claim, unverified).
	Verify func(entries []platform.JournalEntry) VerifyResult
}

Scenario declares one benchmarkable task: a Goal every provider/repeat attempts identically, the platform environment it runs against, and an optional deterministic re-check of completion, independent of the model's own task-done claim ("evidence over claims" — spec/ideas/ actor-model-arena.md's "Quality stays objective" rule). ID and Version are recorded in every Results — the spec's groundwork for a future canonical-scenario registry ("no registry yet"): two Results sharing the same ID+Version ran the identical scenario and are provably apples-to-apples comparable; anything else is not.

func GreetbotScenario 

func GreetbotScenario() Scenario

GreetbotScenario is the arena's built-in first scenario (spec/ideas/ actor-model-arena.md's MVP scope), ported from the scratchpad harness that produced the first actor-model arena report (chatwright/backstage research/model-arena-2026-07-23): send "/start", read the observed actions and click the one labelled exactly "English" among three choices, recognise the greeting has changed (an in-place edit, not a new message — that first run's own most consistent finding was a model re-clicking the button after it already worked because a naive non-progress detector cannot tell "the bot re-edited its own message" in place apart from "genuinely new progress"), and acknowledge it with free text before declaring done. One task, three distinct actor capabilities, identical across every provider in the matrix.

type ScenarioInfo 

type ScenarioInfo struct {
	ID, Version, Title string
}

ScenarioInfo records which scenario (and version) a Results was run against — spec/ideas/actor-model-arena.md's groundwork for a canonical- scenario registry: two Results sharing the same ID+Version are apples-to-apples comparable; anything else is not.

type ScenarioSession 

type ScenarioSession struct {
	Emulator platform.Emulator
	ChatID   int64
	User     platform.User
	// BotActor is the roster entry for the bot-under-test this session
	// wired up — Run copies it into every cell's bundle roster verbatim,
	// alongside the ai-agent actor Run itself constructs per provider.
	BotActor sdk.Actor
	// Close tears down everything Setup started (the emulator's server,
	// the bot's own HTTP server). Always called by Run once the
	// cell/warm-up finishes, even on error.
	Close func()
}

ScenarioSession is one Scenario.Setup call's live environment: an isolated platform.Emulator with a fresh bot-under-test wired to it, ready to run exactly one campaign against.

type VerifyResult 

type VerifyResult struct {
	// Verified is true only when the journal itself shows the scenario's
	// discriminating steps happened — never merely because the model
	// declared the task done.
	Verified bool
	// Detail is a short, human-readable explanation for the report's
	// per-model narrative: what the journal shows (or doesn't), in
	// evidence terms.
	Detail string
}

VerifyResult is one Scenario.Verify call's deterministic verdict.

type WarmupResult 

type WarmupResult struct {
	ColdStart time.Duration
	Call      CallRecord
	// Err is set when the warm-up call itself errored (e.g. an
	// empty-reply model) — still recorded, never silently dropped:
	// cold-start is measured either way, up to the point of the error.
	Err error
}

WarmupResult is one provider's mandatory untimed warm-up call — the spec's own required metric: "the measured cold-start/load time is reported as its own metric, never mixed into proposal latency".

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.