testgate

Documentation

Overview

Package testgate is the engine behind `dockyard test` — the contract + compliance gate (RFC §9.1, §9.4).

Run executes, against one Dockyard project, every test category Dockyard's quality bar is built on:

• go-test — the project's own Go unit tests (`go test ./...`).
• contract — the contract-first assertions: the generated JSON Schema and TypeScript still match the Go contract structs (P1).
• golden — the project's fixture / golden snapshots are current.
• spec-compliance — the project's Apps/Tasks constructs conform to the vendored MCP specs (CLAUDE.md §11 — vendored specs, never a live host).
• capability — the project degrades gracefully across host capability sets (RFC §7.5; no hardcoded host matrix, CLAUDE.md §6).

A category is either gating — a regression exits the process non-zero — or informational. Run composes the existing seams (internal/validate.Run, internal/generate, internal/codegen, runtime/apps) rather than reimplementing them: the gate is defined once.

The cobra `dockyard test` command (internal/cli) is a thin wrapper over Run — the orchestration is a testable package, not logic buried in a RunE closure.

Index

• Variables
• type Category
• type Options
• type Report
  • func Run(opts Options) (*Report, error)
  • func (r *Report) Failed() bool
  • func (r *Report) Passed() bool
• type Result
  • func (r Result) String() string

Variables

var ErrTestGate = errors.New("dockyard/internal/testgate: the test gate could not run")

ErrTestGate is the sentinel wrapping a failure that is not itself a category regression — a missing project, an unloadable manifest, an I/O fault. A category regression is never a returned error: it is a failed Result in the Report. Run returns a nil error and a Report carrying the verdicts. Callers branch with errors.Is(err, ErrTestGate).

Types

type Category

type Category string

Category names one test category `dockyard test` runs.

const (
	// CategoryGoTest is the project's own Go unit tests (`go test ./...`).
	CategoryGoTest Category = "go-test"
	// CategoryContract is the contract-first assertion: the generated schema
	// and TypeScript still match the Go contract structs (P1).
	CategoryContract Category = "contract"
	// CategoryGolden is the project's fixture / golden snapshot check.
	CategoryGolden Category = "golden"
	// CategorySpecCompliance is MCP spec compliance against the vendored specs.
	CategorySpecCompliance Category = "spec-compliance"
	// CategoryCapability is the capability-degradation category: the project
	// degrades gracefully across host capability sets (RFC §7.5).
	CategoryCapability Category = "capability"
)

type Options

type Options struct {
	// ProjectDir is the root of the Dockyard project — the directory holding
	// dockyard.app.yaml. Required.
	ProjectDir string
	// SkipGoTest skips the go-test category. `go test` is the slowest category;
	// a fast non-interactive run (a smoke script) skips it while still
	// exercising the contract, golden, spec, and capability gates. A real
	// `dockyard test` invocation never sets it.
	SkipGoTest bool
}

Options configures one test-gate run.

type Report

type Report struct {
	// Results is one entry per category that ran, in categoryOrder.
	Results []Result
}

Report is the structured outcome of a test-gate run: every category's Result, in category order.

func Run

func Run(opts Options) (*Report, error)

Run executes every test category against the Dockyard project rooted at opts.ProjectDir and returns a Report.

A non-nil error means the gate could not run at all (a missing project, an unloadable manifest) — it wraps ErrTestGate. A category regression is never a returned error: it is a failed Result in the Report. The caller decides the exit code from Report.Failed.

Run composes the existing seams — internal/validate.Run, internal/generate, internal/codegen, runtime/apps — it does not reimplement them. It builds fresh state per call and holds no shared mutable state, so it is safe to invoke concurrently.

func (*Report) Failed

func (r *Report) Failed() bool

Failed reports whether any gating category failed — the exit-code seam: `dockyard test` exits non-zero exactly when this is true. A failed non-gating (informational) category never flips it.

func (*Report) Passed

func (r *Report) Passed() bool

Passed reports whether every gating category passed — the inverse of Failed.

type Result

type Result struct {
	// Category is which category produced this Result.
	Category Category
	// Passed reports whether the category found no regression.
	Passed bool
	// Gating reports whether a failure of this category exits the process
	// non-zero. Every V1 category is gating (RFC §9.4 — these are build
	// blockers); the field is explicit so a future informational category can
	// be added without changing the exit-code logic.
	Gating bool
	// Detail is the human-facing, actionable description: on a pass, a one-line
	// summary; on a fail, the specific regression and how to fix it.
	Detail string
}

Result is one category's verdict.

func (Result) String

func (r Result) String() string

String renders a Result as one actionable line.