runner

Documentation

Overview

Package runner orchestrates docker-compose-backed system tests using two YAML config files (with legacy JSON fallback): a systems.yaml (compose + health probes) and a tests.yaml (setup commands + suites). The unmarshaller is picked from the file extension — `.yaml` / `.yml` use YAML, anything else uses JSON. Struct keys are identical in both formats (camelCase composeFile etc.), so one struct round-trips either source. The runner has no concept of architecture, language, or suite flavor — those identities live in shop's filenames and directory names. This package treats every input as opaque.

Index

• func Build(sys *SystemConfig, cwd string, opts BuildOptions) error
• func Clean(sys *SystemConfig, cwd string) error
• func Down(sys *SystemConfig, cwd string) error
• func IsAnyURLUp(sys SystemEntry, opts HealthOptions) bool
• func RunSetup(tests *TestsConfig, testsCwd string) error
• func RunTests(sys *SystemConfig, tests *TestsConfig, systemCwd, testsCwd string, ...) error
• func Up(sys *SystemConfig, cwd string, opts SystemOptions) error
• func WaitForSystem(sys SystemEntry, opts HealthOptions) error
• func WaitForURL(url string, opts HealthOptions) error
• type BuildOptions
• type Component
• type HealthOptions
• type SetupCommand
• type Suite
• type SuiteResult
• type SystemConfig
  • func LoadSystem(path string) (*SystemConfig, error)
• type SystemEntry
• type SystemOptions
• type TestOptions
• type TestsConfig
  • func LoadTests(path string) (*TestsConfig, error)
  • func (t *TestsConfig) FindSuite(id string) *Suite
  • func (t *TestsConfig) SuiteIDs() []string

Functions

func Build

func Build(sys *SystemConfig, cwd string, opts BuildOptions) error

Build runs `docker compose -f <composeFile> build` for every entry in sys. cwd is the working directory the compose-file paths are resolved against (typically the system-test directory holding systems.json). When opts.Rebuild is true, `--no-cache` is appended so every layer is rebuilt.

func Clean

func Clean(sys *SystemConfig, cwd string) error

Clean tears down every system and removes its named volumes plus locally- built images (`docker compose down -v --rmi local`). External images pulled from registries are left alone — same scope as `./gradlew clean`, which deletes build outputs but not the dependency cache.

Like Down, errors are logged per-system but do not short-circuit the loop.

func Down

func Down(sys *SystemConfig, cwd string) error

Down brings down every system in sys (compose down + container cleanup). Errors per-system are reported but do not short-circuit the loop, so a partial cleanup still proceeds.

func IsAnyURLUp

func IsAnyURLUp(sys SystemEntry, opts HealthOptions) bool

IsAnyURLUp returns true if any of the URLs in sys responds with 200 OK inside a single short attempt. Used as a fast "is the system already running?" probe so we can skip a docker-compose restart when not needed.

func RunSetup

func RunSetup(tests *TestsConfig, testsCwd string) error

RunSetup runs every entry in tests.SetupCommands in testsCwd. Used by the `gh optivem test setup` verb. Each failure halts the run with a wrapped error naming the failing command.

func RunTests

func RunTests(sys *SystemConfig, tests *TestsConfig, systemCwd, testsCwd string, opts TestOptions) error

RunTests iterates suites in tests:

1. If sys is non-nil, probes every entry in sys.Systems and errors out if any isn't responding to its health probe (caller must have already started the system via `gh optivem system start`).
2. Filters suites per opts.Suite (a set; empty means all). Errors out with the available ids if any requested id doesn't match.
3. Runs each remaining suite. After the last suite (or first failure), prints a summary table.

testsCwd is tests.json's dir (setupCommands and suite.path resolve against it). systemCwd is unused today but retained in the signature so callers don't need a per-call branch on whether sys is supplied.

setupCommands are not run by this verb — invoke `gh optivem test setup` (or runner.RunSetup) explicitly before tests. This mirrors mainstream service-lifecycle CLIs where each phase is a separate verb.

Returns the first failure or nil. The summary table is printed regardless, so the user always sees per-suite status.

func Up

func Up(sys *SystemConfig, cwd string, opts SystemOptions) error

Up brings up every system in sys. For each entry, behavior depends on opts:

• opts.Restart=true → unconditional `down` then `up -d`, then health-wait.
• opts.Restart=false → if IsAnyURLUp returns true, skip the entry; else `down` + `up -d` + health-wait. (Mirrors the PS1 runner's behavior so local re-runs are fast when the stack is already healthy.)

`up -d` is wrapped in a small retry loop for transient network errors.

func WaitForSystem

func WaitForSystem(sys SystemEntry, opts HealthOptions) error

WaitForSystem polls every component+externalSystem URL in sys until each returns 200 OK. Components without a URL are skipped. Returns the first failure (with the failed URL) or nil on success.

func WaitForURL

func WaitForURL(url string, opts HealthOptions) error

WaitForURL polls url until it returns 200 OK or attempts is exhausted. Returns nil on success; an error including the URL on timeout.

Types

type BuildOptions

type BuildOptions struct {
	// Rebuild, when true, forces a full rebuild from scratch (every layer
	// rebuilt, no cache reuse). Maps to `docker compose build --no-cache`
	// under the hood. Analog of dotnet's `build --no-incremental` and
	// gradle's `--rerun-tasks` — outcome-oriented ("rebuild") rather than
	// mechanism-oriented ("skip cache").
	Rebuild bool
}

BuildOptions tunes behavior of Build. Zero-values are safe defaults.

type Component

type Component struct {
	Name          string `json:"name" yaml:"name"`
	URL           string `json:"url" yaml:"url"`
	ContainerName string `json:"containerName" yaml:"containerName"`
}

Component is one service within a system (a SUT component or external sim). URL is optional — components without one are skipped during health probes.

type HealthOptions

type HealthOptions struct {
	Attempts int           // max attempts per URL
	Interval time.Duration // sleep between attempts
	Timeout  time.Duration // per-request timeout
}

HealthOptions tunes the polling loop. Zero-values fall back to defaults.

type SetupCommand

type SetupCommand struct {
	Name    string            `json:"name" yaml:"name"`
	Command string            `json:"command" yaml:"command"`
	Env     map[string]string `json:"env,omitempty" yaml:"env,omitempty"`
}

SetupCommand is one test-runner-side setup step — npm ci, dotnet build, gradle compileTestJava. NOT a SUT image build (use `build system` for that).

type Suite

type Suite struct {
	ID                  string            `json:"id" yaml:"id"`
	Name                string            `json:"name" yaml:"name"`
	Command             string            `json:"command" yaml:"command"`
	Env                 map[string]string `json:"env,omitempty" yaml:"env,omitempty"`
	Path                string            `json:"path" yaml:"path"`
	TestReportPath      string            `json:"testReportPath" yaml:"testReportPath"`
	SampleTest          string            `json:"sampleTest,omitempty" yaml:"sampleTest,omitempty"`
	TestInstallCommands []string          `json:"testInstallCommands,omitempty" yaml:"testInstallCommands,omitempty"`
}

Suite is one runnable test suite. Env vars are set on the test process, not interpolated into Command. TestInstallCommands run once before the suite if non-empty (e.g. installing playwright browsers per-suite).

type SuiteResult

type SuiteResult struct {
	Name     string
	Status   string // "PASSED" | "FAILED"
	Duration time.Duration
}

SuiteResult records the outcome of one suite — used to print the summary table at the end of a run, even when one suite failed mid-way.

type SystemConfig

type SystemConfig struct {
	Systems []SystemEntry `json:"systems" yaml:"systems"`
}

SystemConfig describes one or more docker-compose stacks ("systems") to build, bring up, and probe for health. Loaded from systems.{yaml,json}.

func LoadSystem

func LoadSystem(path string) (*SystemConfig, error)

LoadSystem reads and validates systems.{yaml,json} from path. The format is chosen by extension (`.yaml` / `.yml` → YAML, anything else → JSON).

type SystemEntry

type SystemEntry struct {
	Label           string      `json:"label" yaml:"label"`
	ComposeFile     string      `json:"composeFile" yaml:"composeFile"`
	ContainerName   string      `json:"containerName" yaml:"containerName"`
	Components      []Component `json:"components" yaml:"components"`
	ExternalSystems []Component `json:"externalSystems" yaml:"externalSystems"`
}

SystemEntry is one compose stack. Label is a free-form log string; the runner never interprets it (typical values in shop: "real", "stub").

type SystemOptions

type SystemOptions struct {
	// LogLines controls how many trailing lines of compose logs are dumped on
	// a health-probe failure during Up. Zero defaults to 50.
	LogLines int
	// Restart, when true, makes Up tear down the system before starting it
	// again. When false, Up first checks IsAnyURLUp; if the system already
	// responds, Up is a no-op.
	Restart bool
	// Health overrides default polling parameters.
	Health HealthOptions
	// UpTimeout caps a single `docker compose up -d` attempt. Zero uses
	// defaultUpTimeout. The Up retry loop treats a timeout as transient and
	// retries, so the effective ceiling is roughly UpTimeout * maxAttempts.
	UpTimeout time.Duration
}

SystemOptions tunes behavior of Up/Down. Zero-values are safe defaults.

type TestOptions

type TestOptions struct {
	// Suite, when non-empty, limits the run to the suites with these ids.
	// Order in the run follows tests.json declaration order, not slice
	// order, so two invocations with the same ids run in the same order
	// regardless of how the user typed them.
	Suite []string
	// Test, when non-empty, narrows execution to the given test names.
	// Injected into the suite's Command via TestsConfig.TestFilter and
	// joined per TestsConfig.TestFilterJoin.
	Test []string
	// Sample, when true, uses each suite's sampleTest field as the test
	// name (if both Sample is set and Test is non-empty, Test wins).
	Sample bool
	// Health overrides default HTTP-probe parameters used by the pre-run
	// probe (every entry in systems.yaml must be responding before any suite
	// runs).
	Health HealthOptions
}

TestOptions narrows or modifies a tests run.

type TestsConfig

type TestsConfig struct {
	SetupCommands  []SetupCommand `json:"setupCommands" yaml:"setupCommands"`
	TestFilter     string         `json:"testFilter" yaml:"testFilter"`
	TestFilterJoin string         `json:"testFilterJoin,omitempty" yaml:"testFilterJoin,omitempty"`
	Suites         []Suite        `json:"suites" yaml:"suites"`
}

TestsConfig describes test-runner setup + suites. Loaded from tests.{yaml,json}.

TestFilter is a template containing the literal "<test>" — the runner substitutes the user-supplied --test value(s) (or a suite's sampleTest field when --sample is set). Two forms are supported:

"--grep '<test>'"       — full flag; appended as a new argument
"&Category=<test>"      — expression fragment beginning with "&"; injected
                           into an existing --filter '...' argument

TestFilterJoin controls how multiple --test values are combined:

"" / "or" (default) — join names with "|" and substitute once. Covers
                       dotnet (`&DisplayName~T1|T2`) and playwright/jest
                       (`--grep 'T1|T2'`) where the runner already treats
                       "|" as alternation.
"repeat"            — substitute the whole TestFilter once per name and
                       concatenate. Covers gradle (`--tests T1 --tests T2`)
                       where the *flag itself* must repeat.

func LoadTests

func LoadTests(path string) (*TestsConfig, error)

LoadTests reads and validates tests.{yaml,json} from path. The format is chosen by extension (`.yaml` / `.yml` → YAML, anything else → JSON).

func (*TestsConfig) FindSuite

func (t *TestsConfig) FindSuite(id string) *Suite

FindSuite returns the suite with the given id, or nil if not found.

func (*TestsConfig) SuiteIDs

func (t *TestsConfig) SuiteIDs() []string

SuiteIDs returns all suite ids in declaration order. Used in error messages when the user asks for a suite that doesn't exist.