fuzz

Documentation

Overview

Package fuzz drives a terminal program with randomised but structured input and watches for the ways a TUI breaks: crashing, hanging, corrupting the screen model, growing without bound, or exiting without restoring the terminal. When it finds something it minimises the input that caused it and writes a tape file that reproduces it, because a fuzzer that reports a failure without a reproduction is not actionable.

Everything the fuzzer sends is expressible as a tape command, and it replays candidates through the same tape player that runs a tape file by hand. That is what makes a generated repro trustworthy: it is not a description of what the fuzzer did, it is the same execution path.

Index

• func LoadCorpusEntry(path string) ([]tape.Command, error)
• func TapeFor(f *Failure) string
• type Config
• type Failure
  • func (f *Failure) Error() string
• type FailureKind
• type Invariant
• type Limits
  • func DefaultLimits() Limits
• type Options
  • func DefaultOptions(argv []string) Options
• type Result
  • func Run(ctx context.Context, opts Options) (*Result, error)

Functions

func LoadCorpusEntry

func LoadCorpusEntry(path string) ([]tape.Command, error)

LoadCorpusEntry reads a corpus entry the way a fuzzing session replays it: the reproduction only, with the trailing assertion dropped. It is exported so a test can check that the two halves of an entry stay separate.

func TapeFor

func TapeFor(f *Failure) string

TapeFor renders a failure as a runnable tape file. The header records what broke and how to replay it; the body is the minimised input followed by an assertion that the failure did not happen. The result is an ordinary tape, so it runs under "tuitest run" with no fuzz-specific tooling and can be committed as a regression test.

The trailing assertion is what makes the file a test rather than a transcript. Without it a reproduction passes even while the program still crashes, since replaying the input is not the same as checking the outcome, and "rerun with the same -corpus to check a fix" would be a promise the file could not keep.

Types

type Config

type Config struct {
	// Cols and Rows are the size the program is spawned at.
	Cols, Rows int
	// ActionsPerRun bounds how many commands one iteration sends.
	ActionsPerRun int
	// ExcludeKeys lists key tokens the generator must never emit, for programs
	// where a key legitimately quits and would cut every run short.
	ExcludeKeys []string
	// NoHostile disables malformed and oversized escape sequences, leaving only
	// well-formed input.
	NoHostile bool
	// NoResize disables resize actions.
	NoResize bool
	// NoMouse disables mouse actions.
	NoMouse bool
}

Config controls how input is generated.

type Failure

type Failure struct {
	Kind FailureKind
	// Detail explains the specific violation, for the tape header and report.
	Detail string
	// Commands is the input that produced it, in tape form.
	Commands []tape.Command
	// Screen is the plain-text screen at the moment of detection.
	Screen string
	// Seed and Iteration identify the run that found it.
	Seed      uint64
	Iteration int
	// Shrunk records how many commands the original failing input had, so a
	// report can show what minimisation achieved.
	Original int
	// Invariant names the user-supplied invariant that failed, for
	// FailInvariant. It is empty for every other kind.
	Invariant string
	// Onset is the index, into Commands, of the command after which the
	// failure was first observed. It is not the command at which it was
	// reported: an invariant is only reported once it has survived a wait, so
	// the two differ, and the earlier one is the one that points at the bug.
	// Zero means the failure carries no onset.
	Onset int
	// Verified records whether the minimised input reproduced on a final
	// confirmation run. A repro that does not re-verify is still reported, but
	// labelled, because a flaky finding is worth less than a solid one.
	Verified bool
}

Failure is one reproducible finding.

func (*Failure) Error

func (f *Failure) Error() string

type FailureKind

type FailureKind string

FailureKind classifies what went wrong. Shrinking compares kinds to decide whether a reduced input still reproduces "the same" bug, so kinds must be coarse enough to survive minimisation but fine enough to keep distinct bugs apart.

const (
	// FailCrash is the program dying from a fault signal or exiting non-zero.
	FailCrash FailureKind = "crash"
	// FailHang is the program going unresponsive: still alive, but producing no
	// output for longer than the hang bound while input keeps arriving.
	FailHang FailureKind = "hang"
	// FailScreenInconsistent is the screen model contradicting itself, such as
	// the cursor sitting outside the grid or the grid changing size on its own.
	FailScreenInconsistent FailureKind = "screen-inconsistent"
	// FailDirtyExit is the program exiting without restoring the terminal.
	FailDirtyExit FailureKind = "dirty-terminal"
	// FailMemoryGrowth is resident memory climbing past the configured ceiling.
	FailMemoryGrowth FailureKind = "memory-growth"
	// FailReplacementChar is U+FFFD reaching the screen, meaning the program
	// mangled a byte sequence somewhere between reading it and drawing it.
	FailReplacementChar FailureKind = "replacement-char"
	// FailInvariant is a user-supplied invariant returning an error. The
	// invariant's name is in Failure.Invariant, which keeps two different
	// invariants apart during shrinking and deduplication.
	FailInvariant FailureKind = "invariant"
)

type Invariant

type Invariant struct {
	// Name identifies the invariant in reports and, more importantly, to the
	// shrinker: minimisation only accepts a candidate that violates the same
	// named invariant, so two invariants failing in one session stay two
	// separate findings rather than collapsing into one.
	Name string
	// Check reports whether the property holds for this screen.
	Check func(tuitest.Screen) error
}

Invariant is a property of the screen the program is expected to maintain, supplied by the caller. Check returns nil while the property holds and an error describing the violation when it does not; the error text goes into the report, so it should say what was expected rather than only that something was wrong.

An invariant is evaluated after every command but only reported once it has survived a wait, so a screen caught mid-redraw does not produce a finding. See monitor.reportInvariants.

type Limits

type Limits struct {
	// HangAfter is how long the program may produce no output at all, while
	// input keeps arriving and it is still alive, before it counts as hung.
	HangAfter time.Duration
	// MaxRSSGrowth is the factor resident memory may grow by, relative to the
	// reading taken once the program settled, before it counts as a leak. Zero
	// disables the check.
	MaxRSSGrowth float64
	// MinRSSBytes is the floor below which growth is ignored, so a program
	// starting at a few hundred kilobytes does not trip the ratio on noise.
	MinRSSBytes uint64
	// AllowDirtyExit suppresses the terminal-restoration check, for programs
	// that are not full-screen and never claimed to restore anything.
	AllowDirtyExit bool
	// DetectReplacementChars enables the U+FFFD check. It is off by default
	// because the default generator sends malformed UTF-8, and against
	// malformed input a replacement character is the correct thing to draw
	// rather than a bug; see checkReplacementChars for the whole argument.
	DetectReplacementChars bool
}

Limits bounds what the detectors treat as acceptable.

func DefaultLimits

func DefaultLimits() Limits

DefaultLimits are tuned to be quiet on well-behaved programs. The hang bound is generous because a TUI legitimately sits silent when it ignores input; only total silence for seconds, with input still arriving, is evidence.

type Options

type Options struct {
	// Argv is the program under test.
	Argv []string
	// Seed makes a session reproducible. Two sessions with the same seed,
	// options, and program generate the same input.
	Seed uint64
	// Iterations bounds how many programs are spawned and driven. Zero means
	// unlimited, in which case Duration must bound the run.
	Iterations int
	// Duration bounds wall-clock time. Zero means unlimited, in which case
	// Iterations must bound the run.
	Duration time.Duration
	// Gen configures input generation.
	Gen Config
	// Limits bounds what the detectors accept.
	Limits Limits
	// Invariants are properties of the screen the program is expected to
	// maintain, checked alongside the built-in detectors. They are a Go-API
	// feature: a tape file cannot express a Go closure, so the CLI has no way
	// to set them.
	Invariants []Invariant
	// StopOnFirst ends the session as soon as one failure is found, instead of
	// continuing to look for distinct ones.
	StopOnFirst bool
	// Shrink enables minimisation of a failing input. On by default via
	// DefaultOptions; disabling it makes a session faster but its output much
	// less useful.
	Shrink bool
	// ShrinkBudget bounds how many candidate replays minimisation may spend on
	// one failure.
	ShrinkBudget int
	// Corpus is a directory where reproductions are written and from which
	// known cases are replayed as regressions first. Empty disables both.
	Corpus string
	// Out receives progress lines. Nil discards them.
	Out io.Writer
	// SettleTimeout bounds each individual wait inside an iteration.
	SettleTimeout time.Duration
}

Options configures a fuzzing session.

func DefaultOptions

func DefaultOptions(argv []string) Options

DefaultOptions returns options that are safe to run against an unknown program: bounded, shrinking enabled, memory checking off.

type Result

type Result struct {
	// Iterations is how many programs were actually spawned and driven.
	Iterations int
	// Failures holds one entry per distinct failure kind found.
	Failures []*Failure
	// Elapsed is the wall-clock duration of the session.
	Elapsed time.Duration
	// Seed echoes the seed used, so a session can be replayed.
	Seed uint64
}

Result summarises a session.

func Run

func Run(ctx context.Context, opts Options) (*Result, error)

Run fuzzes the program until the budget is exhausted or ctx is cancelled.