mutate

package
v0.0.0-...-3d49dda Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 19, 2026 License: MIT Imports: 36 Imported by: 0

Documentation

Overview

Persistent mutant verdict cache.

turango has no resume capability without this: a hard kill (SIGKILL, an OOM reaper, a killed background job) or even a graceful Ctrl+C throws away every verdict already computed, and re-running the same sweep starts over from mutant zero. cache.go is the on-disk half of the fix — a JSON-Lines file of (key, verdict) records, appended to as soon as each verdict is produced and loaded once, read-only, before a run's workers start.

The central risk this file exists to close, not just to speed things up, is a *wrong* cache hit: serving a stale verdict for a mutation that looks the same by mutantID but is not, in fact, the same code. See [cacheFingerprint]'s doc comment for the concrete example (a same-width literal edit at the same file/line/column) that rules out mutantID alone as a safe key.

Package mutate implements turango's mutation-testing engine: it walks the AST of every target package, asks each registered operator what it would change, and runs the package's tests against each individual change in an isolated copy of the module to find out whether the suite notices.

The engine is split across three files: 

report.go   the result types a run produces, and how they are reported
engine.go   orchestration: package resolution, the AST walk, the mutant loop
runner.go   one mutant: apply, print, copy the module, run `go test`, classify

Index

Constants

View Source 
const MinBaselineTimeout = 10 * time.Second

MinBaselineTimeout is the floor applied to a derived timeout.

The baseline measures a suite whose build cache is already warm, so it barely pays for compilation — but every mutant recompiles its package from changed source, and that cost is not in the measurement. On a fast, small suite the derived product can land below the time a single mutant needs just to build, which would report perfectly good mutants as killed-by-timeout. The floor only ever raises a derived value; it never lowers one, and it never applies to an explicit Options.TestTimeout.

Variables

This section is empty.

Functions

This section is empty.

Types

type EquivalentResult 

type EquivalentResult struct {
	// File is the absolute path of the mutated source file, matching
	// [MutantResult.File].
	File string

	// Line is the line the mutated node starts on.
	Line int

	// Operator is the registry name of the mutator that produced the
	// mutation, e.g. "statement/remover".
	Operator string

	// Description is the operator's human-readable summary of the edit.
	Description string
}

EquivalentResult records one mutation Trivial Compiler Equivalence (TCE) found semantically identical to the unmutated package: the mutant's compiled output exactly matched the baseline's, so it was never run against the test suite — there was nothing behavioral for the suite to have a chance of catching.

It is not a mutant and never becomes one, the same reasoning SuppressionResult is built on: there is no verdict to record, only the fact that nothing was attempted. Unlike a suppression, an equivalent mutation *was* generated and *did* get compiled — TCE is a filter applied after generation, not a directive that stops the walk before it.

type EstimateResult 

type EstimateResult struct {
	// Total is the mutant count the walk found — the same number
	// len(Result.Mutants) + len(Result.Equivalents) would report from a
	// real Run with the same Options, since this estimate deliberately
	// ignores TCE (see TCE below) and counts every matching mutation as if
	// it will run.
	Total int

	// Packages holds one entry per package the walk found at least one
	// mutant in, in the order the walk encountered them. A package with
	// zero matching mutants (every node suppressed, or none matched
	// -mutate's FuncPattern) is not listed: there is nothing to time or
	// extrapolate for it.
	Packages []PackageEstimate

	// TCE reports whether the run this estimate previews would have
	// Trivial Compiler Equivalence enabled ([Options.TCE]). It has no
	// effect on Total or any Baseline — this estimate does not run TCE's
	// compile-and-compare step, since that would cost real time per
	// mutant, working directly against being a fast preview — it exists
	// only so the console/JSON output can print the
	// "the real run may filter some of these and finish faster" caveat
	// when, and only when, it is actually relevant.
	TCE bool

	// SerialEstimate is Σ over Packages of (mutant count × baseline time):
	// the naive lower bound if every mutant ran one after another, on one
	// worker.
	SerialEstimate time.Duration

	// Workers is the worker count ParallelEstimate was divided by — the
	// run's [Options.Parallel] as resolved by Options.parallel(), recorded
	// so the output can state the assumption plainly rather than leaving
	// the reader to guess where the divisor came from.
	Workers int

	// ParallelEstimate is SerialEstimate divided by Workers. It is
	// explicitly an optimistic lower bound, not a promise: real wall-clock
	// speedup under -mutateparallel is sub-linear once CPU contention and
	// shared GOCACHE pressure kick in — this project directly measured
	// roughly 8 mutants/minute against a raw per-mutant cost that should
	// have supported far more. Always report both
	// numbers together; never present ParallelEstimate alone as if it were
	// a confident prediction.
	ParallelEstimate time.Duration
}

EstimateResult is the outcome of a walk-only, execution-free Estimate run: how many mutants a real Run would produce, and a rough, intentionally-hedged prediction of how long running them would take.

It is deliberately not a *Result: an estimate never classifies anything (no `go test` ever runs against a real mutant), so Status/Output — and every score/suppression/equivalent computation built on them — would be fields nobody ever set, on mutants that never actually ran.

Every duration here comes from a single timing sample per package, not [baselineRuns]' three-run average a real run uses to derive its timeout: cold-cache variance alone was measured at roughly 5x for an identical invocation (0.75s warm vs. 3.95s cold GOCACHE) during this gap's own validation. Treat every number on this type as a rough estimate to decide whether to commit to a real run, not a promise about what that run will actually measure.

func Estimate 

func Estimate(ctx context.Context, opts Options) (*EstimateResult, error)

Estimate performs a walk-only preview of what -mutate would produce: how many mutants a real Run would generate, broken down per package, and a rough, honestly-hedged prediction of how long running them would take — without ever writing a mutation to disk or spawning a single `go test` subprocess to classify one.

It is a separate entry point from Run, not an Options flag, deliberately — see gap 11a: the two results answer structurally different questions. Run's *Result carries Status/Output fields a mutant that was never executed could never populate honestly (a zero Status would even print as "killed", the iota's zero value); EstimateResult's fields — a count, a single timing sample — are exactly what an unexecuted walk actually knows and nothing it doesn't.

Estimate reuses exactly the same package/operator/type resolution and AST walk Run does — load, needsTypes/loadTyped, plan/planPackage, mutateFile/ visitNode — via [walkForEstimate]. The only behavioural difference is one branch inside visitNode's per-mutation loop (guarded by a non-nil tally) that tallies a package's count instead of calling [runner.run]. Dependency- closure resolution and per-package coverage maps (ScopeImpact) are both execution-time concerns with nothing to contribute to a count, so planPackage skips building them for an estimate-only job.

Per-package baseline timing intentionally runs *after* the count-only walk finishes, not precomputed alongside it the way ScopeImpact's coverage map or TCE's baseline compile are for a real run: only a package that the walk actually found at least one mutant in is worth timing at all, and that is only known once the walk is done.

func (*EstimateResult) WriteEstimate 

func (e *EstimateResult) WriteEstimate(w io.Writer)

WriteEstimate prints the human-readable preview a -mutateestimate run produces: how many mutants a real run would generate, per package, and two honestly-hedged time predictions.

Unlike Result.WriteSummary, there is no score, no suppression ratio, and no survivor listing — none of those exist until mutants actually run (see EstimateResult's own doc comment for why this is a different type entirely, not a partially-filled *Result).

type MutantResult 

type MutantResult struct {
	// ID is this mutant's stable, content-hashed identifier — see
	// [mutantID]. Stable across re-runs of unchanged source; not stable
	// across edits to the file above the mutated line. Reproduce this exact
	// mutant with -mutatemutant=<ID>.
	ID string

	// File is the absolute path of the mutated source file. Reporting layers
	// are expected to relativise it for display; the engine keeps it absolute
	// so results stay unambiguous across modules.
	File string

	// Line is the line the mutated node starts on, in the original file.
	Line int

	// Operator is the registry name of the mutator that produced the mutation,
	// e.g. "operator/binary".
	Operator string

	// Description is the operator's human-readable summary of the edit, e.g.
	// "== -> !=".
	Description string

	// Before and After are the mutated node's printed source text,
	// immediately before and immediately after the mutation's Apply — the
	// actual diff Description only summarises. Populated unconditionally
	// (so they're always in a JSON report), but the console survivor
	// listing shows Description only; dumping full before/after source
	// into a scannable table would defeat the table's own point.
	//
	// After is empty specifically when the mutated node's own printed text
	// is identical before and after Apply: the node itself wasn't edited
	// in place (its containing list's slot was repointed elsewhere, e.g. a
	// deleted statement), so the only way the removal is visible is by the
	// node's absence, not by any diff in its own printed form. Before is
	// never empty for a real mutant — the syntactic no-op check upstream
	// already filters mutations that changed nothing about the file at
	// all.
	Before, After string

	// Status is the verdict.
	Status Status

	// Output is the captured output of the mutant's `go test` run. It is the
	// only way to explain a Survived or NotViable verdict to a user, so it is
	// retained even though it is by far the largest field here.
	Output string
}

MutantResult is the outcome of running the test suite against exactly one applied mutation.

A mutation whose printed source is byte-identical to the unmutated source never becomes a MutantResult: it is not a real mutant and the engine skips it silently (see [runner.run]).

type Options 

type Options struct {
	// Packages holds the package patterns to mutate, in `go test` syntax
	// ("./...", "./internal/...", an import path). Empty means "." — the
	// package in Dir.
	//
	// This is package *selection*, deliberately kept separate from
	// FuncPattern below — the same separation -run/-bench/-fuzz all have
	// between "which package(s)" (their trailing positional args) and
	// "which named target within them" (the flag's own regexp value).
	Packages []string

	// Dir is the directory patterns are resolved relative to. Empty means the
	// process working directory.
	Dir string

	// FuncPattern is a regular expression matched against the name of every
	// top-level function and method declaration in the selected packages.
	// Only functions whose name matches — and everything nested in their
	// bodies — are mutated. Package-level declarations outside any function
	// (a var/const block, say) are not "in" a function for this pattern to
	// match against, so this filter never affects them.
	//
	// Empty means every function matches: Go's regexp package treats an
	// empty pattern as matching everywhere, so the zero value naturally
	// means "no narrowing," the same convention Packages' own zero value
	// uses for package selection.
	FuncPattern string

	// Operators names the mutation operators to apply, using their registry
	// names ("operator/binary", "control/if"). Empty means every registered
	// operator. An unknown name fails the run rather than being skipped: a
	// typo'd operator that silently reduced the mutant set would quietly
	// inflate the score.
	Operators []string

	// Scope selects the tests each mutant is judged by. The zero value is
	// [ScopeFull].
	Scope Scope

	// Parallel bounds how many *files* are mutated concurrently. Zero or less
	// means one.
	//
	// The unit is a file, not a mutant, and that is a correctness constraint
	// rather than a tuning choice: every mutant of a file shares that file's
	// AST, which the runner mutates in place and reverts, so two mutants of one
	// file can never run at the same time. Different files hold entirely
	// separate trees.
	Parallel int

	// TestTimeout bounds each individual mutant's `go test` run. A bound is not
	// optional: mutation is very good at producing infinite loops — turning an
	// `i++` into an `i--` hangs the suite forever — and without one such a
	// mutant stalls the whole run until `go test`'s own 10-minute default
	// fires.
	//
	// Zero means "derive it": the engine times the unmutated suite before
	// mutating anything and scales that (see [baselineTimeout]). Set it
	// explicitly to skip the baseline entirely, which is what phase 5's
	// -mutatetimeout flag does.
	TestTimeout time.Duration

	// MutantID replays exactly one mutant by its [MutantResult.ID] instead of
	// running every mutation the selected operators offer. Empty means "every
	// mutant," the ordinary run.
	//
	// The walk still visits every file and node — that part is a cheap AST
	// walk regardless — but a mutation whose computed ID does not match this
	// one is never handed to the runner, so the resulting [Result] holds at
	// most one [MutantResult].
	MutantID string

	// TCE enables Trivial Compiler Equivalence: a mutant whose compiled
	// output exactly matches a per-package baseline (see [planPackage]) is
	// filtered before it reaches the test suite, and reported under
	// [Result.Equivalents] instead of [Result.Mutants]. Off by default —
	// the zero value matches every other Options field's "safe default"
	// philosophy, and unlike a narrower scope (which can only under-report
	// kills, never mis-report one), a false positive in the compiled-output
	// comparison would silently discard a real mutant.
	TCE bool

	// Workspace selects how each mutant's throwaway execution copy is built.
	// The zero value is [WorkspaceCopy] — today's existing filesystem-copy
	// behaviour, unchanged.
	Workspace Workspace

	// CacheDir persists every mutant's verdict to a JSON-Lines file
	// (mutate-cache.jsonl) inside this directory, and reuses those verdicts
	// on a later run against unchanged source instead of re-executing
	// `go test` for every mutant that already has a trustworthy cached
	// verdict. Empty means disabled — the zero-value-is-safe convention
	// every other opt-in feature in this project already follows (TCE,
	// dependency-closure copying, git-worktree execution).
	//
	// The cache key is not MutantID alone: a same-position, same-width
	// literal/identifier edit produces an identical MutantID for genuinely
	// different code, so a scope-appropriate content fingerprint (reusing
	// the dependency-closure machinery), Scope, TCE and a toolchain
	// identifier are all folded in too — see [cacheKey]'s own doc comment
	// for the full reasoning and the concrete example that rules out
	// MutantID alone.
	CacheDir string
}

Options configures a mutation run.

Every field has a usable zero value: the zero Options mutates the package in the working directory, with every registered operator, at ScopeFull, one file at a time, with a derived per-mutant timeout.

type PackageEstimate 

type PackageEstimate struct {
	// Package is the package's import path, e.g.
	// "example.com/fixture/mathx" — not a file path, since one package's
	// mutants are always reported together regardless of which of its
	// files they came from.
	Package string

	// Mutants is how many mutants a real [Run] would produce for this
	// package — the same walk that would populate [Result.Mutants], just
	// never executed. Equivalent to how many of them Trivial Compiler
	// Equivalence might filter (see [EstimateResult.TCE]): this count is
	// the raw total regardless of TCE, deliberately.
	Mutants int

	// Baseline is one sample of how long this package's own tests take to
	// run, under the scope the estimate was asked about: under
	// [ScopePackage]/[ScopeImpact], this package's own `go test` pattern;
	// under [ScopeFull], the whole module's baseline (identical across
	// every package, since every mutant really does run `go test ./...`
	// under that scope). A single sample, not
	// [baselineRuns]' three-run average, so treat it as rough, not
	// authoritative — see [EstimateResult]'s own doc comment.
	Baseline time.Duration
}

PackageEstimate is one package's contribution to an EstimateResult — how many mutants Estimate's walk found in it, and one rough timing sample of its own tests.

type Result 

type Result struct {
	Mutants []MutantResult

	// Suppressions holds every //nomutant hit, in the same walk order as
	// Mutants. It is deliberately outside the scoring path: a suppressed node
	// produced no mutant, so it appears in neither [Result.Counts] nor
	// [Result.Score]. Reporting surfaces it separately, as
	// [Result.SuppressionRatio], so that liberal suppression is visible rather
	// than quietly inflating the score.
	Suppressions []SuppressionResult

	// Equivalents holds every mutation Trivial Compiler Equivalence filtered
	// out, in the same walk order as Mutants. Deliberately outside the
	// scoring path for the same reason Suppressions is: an equivalent
	// mutation produced no verdict, so it appears in neither [Result.Counts]
	// nor [Result.Score].
	Equivalents []EquivalentResult
}

Result accumulates every mutant a run produced, in walk order: files in package order, nodes in AST order, operators sorted by name. The order is deterministic so two runs over unchanged sources produce identical reports.

Later phases extend this with scoring thresholds; the counts below are derived on demand rather than maintained incrementally so that a partially-filled Result (from a cancelled run) is always self consistent.

func Run 

func Run(ctx context.Context, opts Options) (*Result, error)

Run executes a full mutation run and reports every mutant it produced.

Files are mutated concurrently, up to Options.Parallel at a time, so the order in which results *arrive* is not deterministic. The report is: Run sorts Result.Mutants and Result.Suppressions by file, line, operator and description before returning, so two runs over unchanged sources still produce identical reports regardless of scheduling. Within one file the walk is strictly sequential — a file's mutants share one AST that is mutated in place and reverted between mutants, so the whole run reuses one parse per file.

Run is cancellation-aware between mutants: when ctx is done, every in-flight file stops at its next mutant rather than finishing its walk, and Run returns the mutants completed so far together with ctx.Err(), so a caller interrupting a long run still gets a partial report rather than nothing.

func (*Result) Counts 

func (r *Result) Counts() (killed, survived, notViable int)

Counts reports how many mutants landed in each status.

func (*Result) EquivalentCount 

func (r *Result) EquivalentCount() int

EquivalentCount reports how many mutations Trivial Compiler Equivalence filtered out before they reached the test suite.

func (*Result) Relativize 

func (r *Result) Relativize(base string) *Result

Relativize returns a copy of r with every File expressed relative to base.

The engine records absolute paths so a mutant is unambiguous across the module copies it is tested in, but an absolute path is the wrong thing to *publish*: a report generated in a CI container names directories that exist nowhere else, and the same run on two machines produces two different reports. Both the console summary and the JSON report are relativised — against the working directory — so a report can be downloaded and read anywhere, and so a path in the summary is the same string as the path in the JSON.

A path that cannot be expressed relative to base (a different volume, or an already-relative path) is kept exactly as it is: a correct absolute path beats a mangled relative one. Relativize never modifies r.

func (*Result) Score 

func (r *Result) Score() (float64, bool)

Score is the mutation score: killed / (killed + survived).

NotViable mutants are excluded from both halves of the ratio — they measure the mutation operators' precision, not the test suite's. Score reports 0 when no viable mutant ran at all, and the second return value reports whether the score is meaningful, so a caller can distinguish "scored zero" from "nothing to score".

func (*Result) SuppressedCount 

func (r *Result) SuppressedCount() int

SuppressedCount reports how many nodes were skipped because of a //nomutant directive.

func (*Result) SuppressionRatio 

func (r *Result) SuppressionRatio() (float64, bool)

SuppressionRatio is suppressed / (killed + survived + suppressed): the share of the code turango was asked to judge that a //nomutant directive put out of reach.

It is the counterweight to Result.Score. A suppressed node is excluded from the score's denominator, so suppressing the parts of a package the tests do not cover raises the score without a single new assertion being written — exactly the way a `// nocoverage`-style pragma games a coverage number. The two numbers are only trustworthy together, which is why reporting prints them side by side.

NotViable mutants are left out of the denominator for the same reason Result.Score leaves them out: they measure the operators, not the suite. The second return value reports whether the ratio is meaningful, so a caller can tell "nothing was suppressed" from "there was nothing to suppress".

func (*Result) WriteSummary 

func (r *Result) WriteSummary(w io.Writer, base string)

WriteSummary prints the human-readable summary of a run to w, with file paths relativised against base (see Result.Relativize; an empty base leaves them absolute).

The layout answers three questions in the order a user asks them: how much was attempted, how it scored, and what to go and fix. Only surviving mutants are listed individually, because they are the only actionable ones — a killed mutant is the suite working, and a not-viable one is an operator producing code that does not compile, which says nothing about the tests.

type Scope 

type Scope int

Scope selects which tests are run to decide whether a mutant was caught.

The three modes trade run time against confidence, and they can disagree: a mutant only a neighbouring package's tests exercise is killed under ScopeFull and survives under ScopePackage. Narrower scopes never produce *more* kills than wider ones, so a narrow scope's score is a lower bound on the full one. 

const (
	// ScopeFull runs the whole module's tests (`go test ./...`) against every
	// mutant. It is the default because it is the only scope that cannot miss a
	// kill: a package's behaviour is frequently only asserted on by its
	// callers' tests, and those live in other packages.
	ScopeFull Scope = iota

	// ScopePackage runs only the tests of the package holding the mutated file.
	// Much cheaper than [ScopeFull] on a large module, at the cost of reporting
	// cross-package kills as survivors.
	ScopePackage

	// ScopeImpact runs only the tests that actually execute the mutated line,
	// derived from a per-test coverage map built once per package before any
	// mutant runs (see impact.go). A line no test covers is not tested at all,
	// so its mutants are reported as survived without running anything.
	ScopeImpact
)

func ParseScope 

func ParseScope(s string) (Scope, error)

ParseScope converts a -mutatescope flag value to a Scope.

It lives beside the type rather than in the command so that any caller constructing Options from user input — the CLI today, a config file later — agrees on the spellings.

func (Scope) String 

func (s Scope) String() string

String reports the scope's -mutatescope spelling.

type Status 

type Status int

Status is the verdict for a single mutant. 

const (
	// Killed means the mutated code compiled and at least one test failed: the
	// suite noticed the change.
	Killed Status = iota

	// Survived means the mutated code compiled and every test still passed:
	// nothing in the suite asserts on the behaviour that was changed.
	Survived

	// NotViable means the mutated code did not compile. It says nothing about
	// the test suite, so these mutants are excluded from the score entirely
	// rather than counted as kills — a compile error is not a caught bug.
	NotViable
)

func (Status) MarshalJSON 

func (s Status) MarshalJSON() ([]byte, error)

MarshalJSON encodes the status as its Status.String name.

Status is an iota'd int, and the default encoding — 0, 1, 2 — makes the JSON report unreadable without a copy of this file, and silently reinterprets every existing report the day a status is inserted in the middle of the list. The names are the stable, self-describing wire form; the numbers are an implementation detail that stops at the package boundary.

func (Status) String 

func (s Status) String() string

String reports the status name used in reports.

func (*Status) UnmarshalJSON 

func (s *Status) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes the name form written by Status.MarshalJSON.

The pair exists so a report round-trips: turango's own tests read reports back in, and a consumer decoding into these types deserves the same. An unknown name is an error rather than a silent Killed — mapping a status turango does not recognise onto the one that means "the suite caught it" would inflate whatever the consumer computes from it.

type SuppressionResult 

type SuppressionResult struct {
	// File is the absolute path of the file holding the suppressed node,
	// matching [MutantResult.File].
	File string

	// Line is the line the suppressed node starts on. For a node suppressed by
	// a directive on a compound statement — where the whole subtree is skipped
	// — this is the compound statement's line, which is also where the
	// directive that caused it can be found.
	Line int

	// Reason is the text following the directive's colon
	// (`//nomutant: known flaky`), or empty for a bare `//nomutant`.
	Reason string
}

SuppressionResult records one node the walk refused to mutate because of a //nomutant directive.

It is not a mutant and never becomes one: the walk stops at the suppressed node, so which mutations the operators would have offered inside it is never discovered. That is why suppressions are tracked separately from MutantResult rather than as a fourth Status — there is no verdict to record, only the fact that nothing was attempted.

type Workspace 

type Workspace int

Workspace selects how a mutant's throwaway execution copy of the module is built. See runner.go's copyModule/copyWorktree for the two strategies. 

const (
	// WorkspaceCopy recursively copies the module into a fresh temp directory
	// per mutant. It has no dependency on git and works against any module,
	// git-tracked or not — the default, and the only strategy available
	// before git-worktree execution was added.
	WorkspaceCopy Workspace = iota

	// WorkspaceWorktree uses `git worktree add` instead of a filesystem copy.
	// Strictly opt-in and never a hard requirement: it is only ever attempted
	// when the target module is inside a clean git working tree (see
	// runner.go's gitWorktreeClean), falling back to [WorkspaceCopy]
	// automatically otherwise — so requesting it is always safe, even
	// against a directory (a corpus fixture's own module/, say) that turns
	// out not to be a clean git checkout, or not a git repo at all.
	WorkspaceWorktree
)

func ParseWorkspace 

func ParseWorkspace(s string) (Workspace, error)

ParseWorkspace converts a -mutateworkspace flag value to a Workspace.

func (Workspace) String 

func (w Workspace) String() string

String reports the workspace's -mutateworkspace spelling.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.