testenv

Documentation

Overview

Package testenv gives a test suite that starts shells an environment of its own, assembled from nothing rather than inherited from whoever ran `go test`.

The reason a suite needs one is that a shell reads a person's startup files, writes a person's history file and appends to a person's block store, and it does all three by reading the environment it was handed. A test that hands a shell the developer's environment has aimed those reads and writes at the developer's home — so the suite is measuring the developer's dotfiles rather than the shell, and the same test is green on a runner whose home is empty and red on the machine the work is done on. That is the worst shape a gate can take, because the failure looks like the branch under it (#890, #1984, #1987).

The guard has three parts and all three are load-bearing:

1. **Assemble from an allowlist**, never remove names. Removing HISTFILE and ZDOTDIR fixes the two variables that were caught; it says nothing about the next variable a dialect learns to read, and the startup surface has at least four — Semantics.NonInteractiveStartupVariable, Semantics.StartupDirectoryVariable, $ENV, and $HOME behind them all.
2. **Hand out a named scratch home** and put its name in the environment too, so a test that re-executes the binary as a shell can tell it is the child and leave the environment its parent aimed alone.
3. **Require that home to be empty when the run ends.** A sweep for writes outside an allowed root is the tempting alternative and it does not work: it has to exempt $TMPDIR, which is where t.TempDir and every scratch file a suite makes already live, so it passes while the hole is wide open.

It lives here rather than in one suite's own file because it was written twice already — once in driver and once in repl, as near-copies — and the third package that needed it is the one that found #1984 by failing on a developer's machine with an rc file in it. A guard that has to be copied to reach a package is a guard the next package will not have.

Index

• Constants
• func Assembled() bool
• func EntriesUnder(home string) []string
• func HomeTouchedMessage(suite string, left []string) string
• func Run(suite string, m interface{ ... }) int
• func Scratch(t *testing.T) string
• func WriteExecutable(path string, content []byte, mode fs.FileMode) error

Constants

const TripwireHome = "SH_TEST_HOME"

TripwireHome names the home a suite runs under, and doubles as the mark that says this process had its environment assembled by Run.

The second job is what makes it worth a name. Several suites re-execute their own test binary to *be* the shell — the only way to ask what happens to a process that gets killed — and the environment such a child is handed is the assertion that test is making. So a child takes what it was given untouched, and this name arriving already set is how it knows it is one.

SH_TEST_-prefixed because the assembly below keeps that prefix, which is the channel every re-executing test in this module already uses.

Functions

func Assembled

func Assembled() bool

Assembled reports whether this process is already running inside an environment Run built — which is what a re-executed copy of a test binary is. Such a process leaves the environment alone: its parent aimed it.

func EntriesUnder

func EntriesUnder(home string) []string

EntriesUnder lists everything under a home, relative to it.

A named directory that has to stay empty, rather than a sweep for files written outside some allowed root — see the package comment for why the sweep does not work.

Directories count, not only files. A shell that created ~/.local/state and wrote nothing into it still reached into a person's home.

func HomeTouchedMessage

func HomeTouchedMessage(suite string, left []string) string

HomeTouchedMessage says what was found and what to do about it, because the person who trips this will be reading it a year from now with no memory of #890.

func Run

func Run(suite string, m interface{ Run() int }) int

Run assembles the environment, runs m in it, and then checks that the home it handed out is still untouched. suite names the package in the messages, since the person reading one will be reading it from a `go test ./...` line that says nothing about which suite tripped.

func Scratch

func Scratch(t *testing.T) string

Scratch gives one test a home of its own, and a history file under it.

Both, because HISTFILE outranks HOME: a test that sets only the home still writes its history wherever the environment already pointed, and a test that sets only the history file still appends to the block store under the home. That pair is the whole of #890, and a helper that did one of them would have left half of it standing.

Returned so a test can look at what the session left there, which is the other reason to want one.

func WriteExecutable

func WriteExecutable(path string, content []byte, mode fs.FileMode) error

WriteExecutable writes a file a test is about to run, and is a drop-in for os.WriteFile everywhere a suite makes a fixture it then executes.

Why the plain write is not enough

Linux refuses to exec a file that any process still holds open for writing: ETXTBSY, `text file busy`. `os.WriteFile` closes its own descriptor before it returns, so a test is never racing itself — the holder is somebody else's *child*. This is golang/go#22315. Go opens files O_CLOEXEC, so a process forked anywhere else in the same program loses the descriptor at its own exec — but it holds it, open for writing, for the whole window between its fork and that exec. A test suite full of `t.Parallel` tests that start external commands has that window open more or less continuously, and an exec of a freshly written fixture inside it fails.

The failure is a flake with a plausible wrong story attached: the shell reports `text file busy` and exits 126, so the test reads as the shell having refused a file it should have run (#2787, and #1623 in internal/startupcost before it).

Why holding ForkLock is the whole of the fix

syscall.ForkLock is held for writing across a forkExec, and on Linux os/exec's child is started with CLONE_VFORK — so the parent, and with it the lock, is suspended until the child has reached its own exec and dropped every close-on-exec descriptor it inherited. While this holds the lock there is therefore no child anywhere in the process sitting between a fork and an exec, and none can start. The window is not narrowed; it is closed.

Measured, in a golang container on Linux, with eight goroutines starting `/bin/sh -c :` in a loop and 400 rounds of write-then-exec alongside them: 34 of the 400 execs failed with ETXTBSY with the plain write, and 0 of 400 failed with the write under this lock. TestAFreshlyWrittenExecutableRuns is that measurement, kept.

macOS was measured not to enforce the rule at all — a file with a writer still open on it executes there without complaint — which is why only the ubuntu job has ever seen this. The lock is taken on every platform anyway: it costs a mutex against a file write, and a guard that is only taken where the failure has already been seen is a guard the next platform will not have.

It lives in this package for the reason the package itself exists: a helper a suite has to copy to reach a package is a helper the next package will not have, and the copy is where the fix goes missing.