scupper

README

scupper

Enforce a reachability floor for Go projects: every line was executed at least once by some test, or is explicitly dismissed with a reasoned directive in the source (reviewable in a diff). Aim it at 100%.

Reachability is a floor, not a quality signal. "Executed at least once" is strictly weaker than "tested" — a line an e2e flow merely walks past counts as reached. The value is catching code that has never run under any test: dead branches, an unreachable-in-practice path, a stray 1/0 — code that is unobserved until a user hits it in production. Test quality is a separate axis you build on top of this floor. scupper never claims a reached line is tested; its output says so out loud.

A scupper is a deck drain that lets water run off deliberately. This tool lets explicitly-marked lines drain out of the reachability count.

It's the Go analogue of PHP's @codeCoverageIgnore (pcov / phpunit) or Python's # pragma: no cover (coverage.py) — a layer Go's built-in tooling doesn't provide — plus a merge step so coverage from every test source counts.

How it works

Go produces coverage profiles (go test -coverprofile, and binary coverage via go build -cover + GOCOVERDIR). For a true reachability floor you want a generous profile — unit + cross-package (-coverpkg) + e2e + build-tagged, merged with go tool covdata so "executed by ANY test" is measured fully. scupper sits between that merged profile and your threshold check: it reads the profile, drops any block your source explicitly dismissed, then reports reachability and optionally fails if it's below a threshold.

unit + -coverpkg + e2e + tagged  →  covdata merge  →  scupper  →  pass/fail

Because it only removes dismissed blocks, it can never hide a genuine gap: a never-executed, non-dismissed line still counts against you. And it merges duplicate blocks from -coverpkg correctly (covered by any run = reached), so cross-package execution is credited rather than read as 0%.

Directives

Put these in ordinary source comments. A directive must be the first token of its comment — prose that merely mentions the keyword is not a directive.

Directive	Effect
//scupper:ignore	ignore the line the comment sits on
//scupper:ignore-func	ignore the whole function declared just below (uses the Go AST to find its span)
//scupper:ignore-start … //scupper:ignore-end	ignore every line in the block
//scupper:ignore-file	ignore the whole file

A trailing human note is allowed and encouraged, e.g. //scupper:ignore unreachable: alwaysOK never errors.

func Load(path string) Config {
	v, err := parse(path)
	if err != nil { //scupper:ignore-start
		panic(fmt.Sprintf("config parsed at build time, cannot fail: %v", err))
	} //scupper:ignore-end
	return v
}

//scupper:ignore-file
// Code generated by protoc. DO NOT EDIT.
package pb

Usage

go install github.com/sourcehaven-bv/scupper/cmd/scupper@latest

go test ./... -coverprofile=cover.out -covermode=set
scupper -i cover.out -o cover.filtered.out -threshold 100

Flags:

-i, --in FILE          input coverage profile (default: stdin; "-" also means stdin)
-o, --out FILE         write the filtered profile here (feed to go tool cover, covdata, etc.)
-t, --threshold N      fail (exit 1) if reachability is below N percent
-d, --directive S      directive base keyword (default: "scupper:ignore";
                       use "coverage-ignore" to read go-test-coverage comments)
    --dir PATH         directory to resolve import paths from (default: cwd)
    --require-reason   a directive without a trailing ": <reason>" is an error
-q, --quiet            suppress the reachability report on stdout

Exit codes: 0 met threshold · 1 below threshold (some line never executed and not dismissed) · 2 usage/processing error.

The filtered profile is a standard Go profile, so it works with everything downstream:

go tool cover -html=cover.filtered.out    # visualize what is still unreached
go tool cover -func=cover.filtered.out    # per-function breakdown

CI (GitHub Actions)

For a real reachability floor, feed scupper a generous profile — merge every test source with go tool covdata first (see the pipeline below). The minimal unit-only form:

- run: go test ./... -coverprofile=cover.out -covermode=set
- run: go run github.com/sourcehaven-bv/scupper/cmd/scupper@latest \
         -i cover.out -threshold 100 --require-reason

The second step fails the build if any non-dismissed line was never executed. --require-reason additionally fails if any dismissal lacks an explanation, so every exclusion stays reviewable.

Generous profiles (counting execution from every source)

Default go test -coverprofile measures only unit tests of the package under test. A reachability floor wants every execution to count. Collect each source as binary coverage and union them:

# unit + cross-package: code run by ANY package's unit tests
go test -cover -covermode=set -coverpkg=./... ./... -args -test.gocoverdir=DIR

# e2e / integration: build an instrumented binary, set GOCOVERDIR when spawning it
go build -cover -covermode=set -o bin/app ./cmd/app     # then GOCOVERDIR=DIR bin/app ...

# merge everything, convert to a profile, enforce
go tool covdata merge   -i=DIR1,DIR2,... -o=MERGED
go tool covdata textfmt -i=MERGED -o=cover.generous.out
scupper -i cover.generous.out -threshold 100 --require-reason

Note: a -cover binary only writes coverage on a clean exit — a server must handle SIGTERM/SIGINT and shut down gracefully, or its coverage is lost.

Statement coverage & multi-line code

Go's coverage unit is the basic block, not the physical line — and a block carries a full [startLine, endLine] span. //scupper:ignore drops every block whose span overlaps the ignored line. This composes cleanly in most cases and has one sharp edge worth knowing (all six behaviors below are covered by integration_test.go, which asserts against the real toolchain):

• A multi-line single statement is one block. A multi-line return, a fmt.Sprintf(...) call across lines, or a multi-line composite literal is a single block spanning all its lines. //scupper:ignore on any of those lines drops the whole statement. Intuitive.

• Several statements on one line collapse into one block (a := 1; b := 2; return). That block has numStmt > 1, so a line ignore drops all of them — you can't ignore just one. Rare in idiomatic Go.

• Branches share the condition line — use the block form for whole branches. An if emits multiple blocks (condition, then-body, continuation), and the condition line is shared. A line ignore on if err != nil { drops the blocks overlapping that line but not the else/continuation. To exclude an entire impossible branch, wrap it:

  //scupper:ignore-start
  if err != nil {
  	panic(fmt.Sprintf("cannot happen: %v", err))
  }
  //scupper:ignore-end
  

  That's the reason the block form exists.

• Dismissals can only remove, never mask. A genuinely never-executed, non-dismissed statement still counts against reachability — you cannot accidentally hide a real gap by dismissing a different line.

Notes & limits

• Statement coverage. Go coverage is statement-based, so "100% line coverage" is really "100% of statements" (see the section above).
• Custom keyword. -d nocov switches the directives to //nocov, //nocov-start, //nocov-end, //nocov-file.
• Path resolution. Profile filenames are import-path based; scupper runs go list (cached per package) to map them to files on disk. Run it from inside the module, or pass --dir.
• Block directives are strictly balanced. Every misuse is a hard error (exit 2) that names the offending line, never a silent swallow — because a mistyped directive must not quietly change what counts toward coverage:
  • -end before -start (swapped) → "ignore-end with no open block"
  • a stray -end with no matching -start → same
  • a second -start while one is open (blocks don't nest) → "ignore-start with a block already open"
  • -start with no -end (unterminated at EOF) → "unterminated ignore-start"

Testing

go test ./...                          # unit + integration (real toolchain)
go test -run x -fuzz FuzzParseProfile  # fuzz the profile parser
go test -run x -fuzz FuzzScanFile      # fuzz the directive scanner

The two hand-written parsers (ParseProfile, ScanFile) have native Go fuzz targets asserting: never panic on arbitrary input, and never mis-accept — the parser round-trips anything it accepts; the scanner errors on an unterminated block rather than emitting a bogus range. Their seed corpora run as normal tests in CI. (Everything else — Filter, path resolution, the CLI — shells out to go list and touches the filesystem, so it's covered by integration tests, not fuzzing.)

License

MIT — see LICENSE.

Documentation

Overview

Package scupper filters a Go coverage profile, removing blocks that the source has explicitly marked as excluded via comment directives. This lets a project set a 100% line-coverage target where "100%" means "all code that should be covered is covered" — build wiring, impossible error branches, and generated code are excluded visibly in the source itself.

A scupper is a deck drain that lets water run off deliberately; here it lets explicitly-marked lines drain out of the coverage count.

Four directive styles are supported:

//scupper:ignore            — ignore the line it appears on (trailing or own-line)
//scupper:ignore-start      — begin an ignored block
//scupper:ignore-end        — end an ignored block
//scupper:ignore-file       — ignore the entire file

The default directive keyword is "scupper:ignore"; it is configurable so a project can adopt its own convention.

Index

• type Block
• type Directives
  • func DefaultDirectives(base string) Directives
• type FileIgnore
  • func ScanFile(path string, d Directives) (FileIgnore, error)
  • func (fi FileIgnore) Covers(n int) bool
  • func (fi FileIgnore) OverlapsRange(lo, hi int) bool
• type FilterResult
  • func Filter(p *Profile, res *Resolver, d Directives) (*FilterResult, error)
• type Profile
  • func ParseProfile(r io.Reader) (*Profile, error)
  • func (p *Profile) Merge() *Profile
  • func (p *Profile) Write(w io.Writer) error
• type Range
• type Resolver
  • func NewResolver(dir string) *Resolver
  • func (r *Resolver) Resolve(token string) (string, error)
• type Stats
  • func Compute(p *Profile) Stats
  • func (s Stats) Percent() float64

Types

type Block

type Block struct {
	File      string // profile file token (import-path based), verbatim
	StartLine int
	StartCol  int
	EndLine   int
	EndCol    int
	NumStmt   int
	Count     int
}

Block is one entry in a Go coverage profile: a half-open span of source with a statement count and a hit count.

file.go:startLine.startCol,endLine.endCol numStmt count

type Directives

type Directives struct {
	Line  string // e.g. "scupper:ignore"
	Start string // e.g. "scupper:ignore-start"
	End   string // e.g. "scupper:ignore-end"
	File  string // e.g. "scupper:ignore-file"
	Func  string // e.g. "scupper:ignore-func"

	// RequireReason makes a directive without a trailing explanation a hard
	// error. This enforces the "every dismissal is explicit and reviewable"
	// rule — a bare `//coverage-ignore` is rejected; `//coverage-ignore: why`
	// is accepted. The -end directive is exempt (it only closes a block whose
	// -start already carries the reason).
	RequireReason bool
}

Directives holds the configured comment keywords. Zero value is not usable; call DefaultDirectives.

func DefaultDirectives

func DefaultDirectives(base string) Directives

DefaultDirectives returns the standard directive keywords built from base, e.g. base "scupper:ignore" yields "scupper:ignore-start" etc. Reasons are not required; set RequireReason on the result to enforce them.

type FileIgnore

type FileIgnore struct {
	WholeFile bool
	// Ranges are inclusive [start,end] 1-based line numbers.
	Ranges []Range
}

FileIgnore describes the ignored line ranges within a single source file.

func ScanFile

func ScanFile(path string, d Directives) (FileIgnore, error)

ScanFile reads path and returns the ignored ranges implied by its directive comments.

Block directives must be balanced and correctly ordered. Every misuse is a hard error rather than a silent swallow — because a tool whose value is "exclusions are visible and reviewable" must not quietly mis-handle a mistyped directive:

• an ignore-end with no open block (a stray or swapped end);
• a second ignore-start while a block is already open (blocks do not nest);
• an ignore-start with no matching ignore-end (unterminated at EOF).

func (FileIgnore) Covers

func (fi FileIgnore) Covers(n int) bool

Covers reports whether line n falls within any ignored range (or the whole file is ignored).

func (FileIgnore) OverlapsRange

func (fi FileIgnore) OverlapsRange(lo, hi int) bool

OverlapsRange reports whether the inclusive line span [lo,hi] intersects any ignored range (or the whole file is ignored). Used to match a profile block, which may span several lines, against a directive that sits on any one of them.

type FilterResult

type FilterResult struct {
	Kept         *Profile
	RemovedStmts int // statements dropped because they were in ignored ranges
	IgnoredFiles []string
}

FilterResult reports what Filter did.

func Filter

func Filter(p *Profile, res *Resolver, d Directives) (*FilterResult, error)

Filter removes profile blocks that fall within ignored ranges of their source file. A block is removed if its start line is within an ignored range; this matches the intent of marking a statement or branch as ignored. Whole-file ignores drop every block for that file.

scanCache memoizes FileIgnore per resolved path so each source file is read once.

type Profile

type Profile struct {
	Mode   string
	Blocks []Block
}

Profile is a parsed coverage profile.

func ParseProfile

func ParseProfile(r io.Reader) (*Profile, error)

ParseProfile reads a Go coverage profile (the output of `go test -coverprofile`).

func (*Profile) Merge

func (p *Profile) Merge() *Profile

Merge collapses duplicate blocks — blocks sharing a span and statement count that appear more than once — into a single block whose count is the max of the duplicates. This is required to read a profile produced with `go test -coverpkg=./...`, where a package's blocks are emitted once per test binary that instruments it (count 0 from a binary that never runs them, count >0 from one that does). Taking the max means "covered by ANY run", matching how `go tool cover` and go-test-coverage merge such profiles. Without this, the same statement is counted several times — once covered, once not — producing a coverage number that is both wrong and below reality.

Block order is preserved by first appearance. A profile without duplicates is returned unchanged in effect (every block maps to itself).

func (*Profile) Write

func (p *Profile) Write(w io.Writer) error

Write serializes a profile back to the standard textual format.

type Range

type Range struct{ Start, End int }

Range is an inclusive 1-based line range.

type Resolver

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver maps a profile file token (import-path based, e.g. "example.com/mod/pkg/file.go") to an absolute path on disk. Resolution uses `go list` for the packages it encounters, cached per package directory.

func NewResolver

func NewResolver(dir string) *Resolver

NewResolver returns a Resolver that resolves packages relative to dir (the module root or any dir inside the module).

func (*Resolver) Resolve

func (r *Resolver) Resolve(token string) (string, error)

Resolve returns the absolute filesystem path for a profile file token. The token is "<import path of package>/<basename>.go". If the file already exists as given (relative or absolute), that path is returned directly — this covers profiles written with real paths.

type Stats

type Stats struct {
	TotalStmts   int
	CoveredStmts int
}

Stats summarizes coverage over a profile.

func Compute

func Compute(p *Profile) Stats

Compute returns coverage statistics for a profile.

func (Stats) Percent

func (s Stats) Percent() float64

Percent returns line/statement coverage as a percentage. An empty profile (no statements) is defined as 100% covered — there is nothing left uncovered.