conformance

Documentation

Overview

Package conformance runs a scripted scenario against a wasm game through the REAL gameabi adapter (limits ON) and reports per-callback latency, exit codes, frames, and peak linear memory, plus budget verdicts that name the breached limit, the measured value, and the step that breached it. It is the shared engine behind `shellcade-kit check` and is importable by internal/catalog for release-intake gating.

The harness drives the handler's callbacks SYNCHRONOUSLY against an instrumented Room (not the live actor goroutine): synchronous control is what lets it sample guest memory after each callback, advance a virtual clock in fixed steps, and snapshot/restore mid-script for the hibernation-determinism check — none of which the async RoomCtl surface exposes. The adapter, limits, virtualized WASI, and host functions exercised are the production ones.

Index

• type DurationMS
• type Report
  • func Run(path string, opts gameabi.Options, script Script) (Report, error)
  • func (r Report) Pass() bool
• type Script
• type SeatFrames
  • func RunShots(path string, opts gameabi.Options, req SmokeRun) ([]SeatFrames, error)
• type SmokeRun
• type Step
  • func Advance(ms int64) Step
  • func Input(seat int, r rune) Step
  • func Join(seat int) Step
  • func Key(seat int, key uint8) Step
  • func Leave(seat int) Step
  • func Shot(name string, seats []int) Step
  • func SnapshotRestore() Step
  • func Wake() Step
• type StepKind
• type StepMetric
• type Verdict
  • func LeaderboardVerdict(meta sdk.GameMeta) Verdict

Types

type DurationMS

type DurationMS int64

DurationMS is a millisecond duration carried in the script (kept simple so a script literal reads cleanly and so JSON/flag wiring stays trivial later).

type Report

type Report struct {
	ABIVersion uint32
	Meta       sdk.GameMeta

	Steps    []StepMetric
	PeakMem  uint64 // bytes, peak guest linear memory sampled after each callback
	MemCap   uint64 // bytes, the load-time memory cap
	Deadline time.Duration

	Verdicts []Verdict

	// HibernationChecked is true if the script contained a snapshot/restore
	// checkpoint and the determinism comparison ran; HibernationOK is its result.
	HibernationChecked bool
	HibernationOK      bool
}

Report is the outcome of a conformance run: the artifact's declared ABI/meta, a metric row per executed step, the peak guest linear memory, and the budget verdicts. Pass reports true only when every verdict passed.

func Run

func Run(path string, opts gameabi.Options, script Script) (Report, error)

Run executes the script against the artifact at path through the real adapter with the given Options (limits ON) and returns a Report. An error is returned only for setup failures (load/instantiate); per-step budget breaches are reported as failing Verdicts, not errors.

func (Report) Pass

func (r Report) Pass() bool

Pass reports whether every verdict passed.

type Script

type Script []Step

Script is an ordered list of steps.

type SeatFrames

type SeatFrames struct {
	Name   string
	Seats  []int
	Frames []sdk.Frame
}

SeatFrames is one captured shot: the latest frame of each captured seat at the moment the Shot step ran.

func RunShots

func RunShots(path string, opts gameabi.Options, req SmokeRun) ([]SeatFrames, error)

RunShots executes a smoke script against the artifact at path through the real adapter (limits ON) and returns the frames captured at each Shot step. Unlike Run it does not produce a Report: a guest fault or a shot with no frame is an error — smoke wants screens or a reason there are none.

type SmokeRun

type SmokeRun struct {
	Seed   int64
	Seats  int               // seats joined before the first step (seat-0..)
	Config map[string]string // per-game config values
	Epoch  time.Time         // virtual clock start: kit smoke.SeedEpoch(Seed)
	Script Script            // Input/Wake/Advance/Shot steps (no joins)
}

SmokeRun configures RunShots: the deterministic inputs from a game's smoke.yaml. The clock contract (Epoch) and the room shape (mode, capacity, identities) mirror the kit smoke runner exactly — that is what makes the wasm path's shot bytes identical to `go run . -smoke`.

type Step

type Step struct {
	Kind StepKind

	Seat int        // seat index for Join/Leave/Input (0-based)
	Rune rune       // for StepInput (rune input; 0 means a key input)
	Key  uint8      // for StepInput (key code when Rune == 0)
	Dur  DurationMS // for StepAdvance
	Note string     // human-readable label echoed in the report

	Name  string // for StepShot: the shot name
	Seats []int  // for StepShot: captured seats, ascending (nil = all members)
}

Step is one scripted action. Construct steps with the helpers below rather than building this struct directly.

func Advance

func Advance(ms int64) Step

Advance moves the virtual clock forward by ms milliseconds (the next callback reads the new clock as its CallContext time).

func Input

func Input(seat int, r rune) Step

Input delivers a rune to the player at seat.

func Join

func Join(seat int) Step

Join admits the player at seat into the room.

func Key

func Key(seat int, key uint8) Step

Key delivers a key code to the player at seat.

func Leave

func Leave(seat int) Step

Leave removes the player at seat.

func Shot

func Shot(name string, seats []int) Step

Shot marks a capture point for RunShots: the latest frame of each listed seat (nil = every member) is recorded under name. Run ignores shot steps.

func SnapshotRestore

func SnapshotRestore() Step

SnapshotRestore checkpoints the room: snapshot the current state, restore into a fresh handler, and continue the script against the restored room. The runner also verifies the post-checkpoint frames match an uninterrupted control (hibernation determinism).

func Wake

func Wake() Step

Wake fires one host-heartbeat wake.

type StepKind

type StepKind uint8

StepKind tags a scripted step.

const (
	StepJoin StepKind = iota
	StepLeave
	StepInput
	StepWake
	StepAdvance
	StepSnapshotRestore
	StepShot // capture marker for RunShots; a no-op under Run
)

type StepMetric

type StepMetric struct {
	Index    int
	Desc     string        // human-readable ("input rune 'p' -> seat 0")
	Callback string        // the ABI callback this step drove ("input", "wake", …) or "" for non-callbacks
	Latency  time.Duration // wall time of the driven callback
	Exit     uint32        // wasm exit code (0 = clean)
	Faulted  bool          // the callback trapped or hit the deadline
	Frames   int           // frames the guest pushed during this step
	MemBytes uint32        // guest linear memory sampled after the step
}

StepMetric is one executed step's measurements.

type Verdict

type Verdict struct {
	Name     string // "callback deadline", "linear memory", "guest fault"
	OK       bool
	Limit    string // the limit, formatted ("100ms", "5 MiB")
	Measured string // the measured value that breached, formatted
	Step     int    // breaching step index (-1 if not step-specific)
	Detail   string
}

Verdict is a named budget check. On a breach, Limit/Measured/Step name the rule, the offending value, and the step index that breached it.

func LeaderboardVerdict

func LeaderboardVerdict(meta sdk.GameMeta) Verdict

LeaderboardVerdict is the publishing-policy gate that every PUBLISHED game must declare a leaderboard so its results are recorded and ranked. It is NOT part of the generic ABI conformance verdicts (which also run against minimal test fixtures that legitimately declare no board): callers that enforce the catalog publishing policy — `shellcade-kit check --require-leaderboard`, used by the games-repo CI — append it to the Report before checking Pass().

It is deliberately STATIC (a check on GameMeta), not a behavioral "posts on leave" assertion. A mid-play disconnect in a round-based multiplayer game is recorded at round SETTLEMENT (the leaver ranked dnf), not necessarily on the leave callback, and the remaining player may keep playing — so a behavioral gate false-fails correct games. The disconnect-save and continuous periodic-save behavior is specified in game-sdk and verified by each game's own tests.