testmatrix

Documentation

Overview

Package testmatrix provides a multi-environment test harness. A plugin lifecycle test wrapped in ForEachEnvironment runs once per available execution backend (native, docker, nix), with t.Run sub-tests so failures are attributed to a specific mode.

This is the guardrail for the "every plugin tests all three execution environments" architectural rule: plugin authors write one test, and parity across modes becomes a CI-enforced guarantee rather than a claim.

Index

• func ForEachEnvironment(t *testing.T, dir string, fn func(t *testing.T, env runners.RunnerEnvironment), ...)
• type Case
• type EnvFactory
• type Option
  • func Only(names ...string) Option
  • func WithDockerImage(img *resources.DockerImage) Option

Functions

func ForEachEnvironment

func ForEachEnvironment(t *testing.T, dir string, fn func(t *testing.T, env runners.RunnerEnvironment), opts ...Option)

ForEachEnvironment runs fn against each supported backend as a sub-test. A backend unavailable on the host (Docker not running, Nix not installed, missing flake.nix) is skipped in the default matrix. Passing Only(...) is an explicit assertion that those backends must exist and turns absence into a failure. This keeps local/default suites portable without weakening CI jobs that deliberately require a backend.

`dir` is the workspace or source directory the plugin expects to operate on. For native/nix it's used as cwd; for docker it's bind-mounted.

Types

type Case

type Case struct {
	Name    string
	Factory EnvFactory
}

Case wraps a (name, factory) pair for a single backend.

type EnvFactory

type EnvFactory func(ctx context.Context, dir string) (runners.RunnerEnvironment, error)

EnvFactory is a constructor that returns a RunnerEnvironment bound to dir. Returning (nil, nil) means the backend is unavailable on this host. The default matrix skips it; an explicit Only(...) selection makes it required.

type Option

type Option func(*options)

Option tweaks the set of backends used by ForEachEnvironment.

func Only

func Only(names ...string) Option

Only restricts the matrix to the named backends (e.g. "native", "nix"). Useful when a plugin genuinely cannot support one — prefer Skip-in-factory over Only when the backend is merely unavailable on the host.

func WithDockerImage

func WithDockerImage(img *resources.DockerImage) Option

WithDockerImage overrides the Docker image for the `docker` case. Most plugins want their language's canonical image (golang:1.26, python:3.12, node:22-alpine) rather than the default alpine:3.20 which has no toolchains beyond /bin/sh.