Documentation
¶
Overview ¶
Package fsutil holds the durable-write and path-safety primitives shared by the ~/.abcd and repo .abcd store writers. It is transport-agnostic: no stdout, no os.Exit, no CLI knowledge.
It is the single home for the atomic temp-file+fsync+rename write and the "is this a real directory, not a symlink" check: the ahoy, capture, and memory store writers all route through WriteFileAtomic / WriteFileAtomicPreserveMode / IsRealDir rather than keep divergent copies (the one-canonical-primitive invariant, guarded by TestNoNonCanonicalAtomicWritePrimitives).
Directory validation is the same story one level up. IsRealDir answers the question; EnsureRealDir and EnsureRealDirAll are the create-then-prove sequence built on it, and the lifeboat voyage log, the intent record store and the transcript store all call them rather than each carrying the sequence (iss-2609091128479544).
Each read/write primitive comes in two forms. The plain form takes a path and guards the LEAF only; the …InRoot form takes an *os.Root and resolves every component inside it, which is what contains a path whose ANCESTOR is a symlink planted by untrusted content. Any path that arrives as data — a committed configuration value, a packed manifest entry — belongs in the …InRoot form.
Index ¶
- Variables
- func CaseFoldingFS() bool
- func CreateExclusiveIn(root *os.Root, rel string, data []byte, perm os.FileMode) error
- func DirHasEntries(path string) (bool, error)
- func EnsureRealDir(dir string, perm os.FileMode) error
- func EnsureRealDirAll(base, rel string, perm os.FileMode) error
- func Exists(path string) (bool, error)
- func ExistsNoFollow(path string) (bool, error)
- func FoldPath(p string, fold bool) string
- func InsideGitDir(p string) bool
- func IsDir(path string) (bool, error)
- func IsRealDir(path string) bool
- func ModuleRoot(start string) (string, error)
- func OwnerUID(path string) (uint32, error)
- func PathWithin(child, parent string, fold bool) bool
- func PathsOverlap(a, b string, fold bool) bool
- func ReadGuarded(path string, limit int64) ([]byte, error)
- func ReadGuardedInRoot(root *os.Root, rel string, limit int64) ([]byte, error)
- func RealExistingPath(p string) string
- func RedactHome(s string) string
- func RedactRoot(s, root, repl string) string
- func RepoRel(base, target string) string
- func ValidRelPath(p string) bool
- func WithFileLock(lockPath string, timeout time.Duration, fn func() error) error
- func WriteFileAtomic(path string, data []byte, perm os.FileMode) error
- func WriteFileAtomicInRoot(root *os.Root, rel string, data []byte, perm os.FileMode) error
- func WriteFileAtomicPreserveMode(path string, data []byte) error
- func WriteFileAtomicPreserveModeInRoot(root *os.Root, rel string, data []byte) error
Constants ¶
This section is empty.
Variables ¶
var ( ErrLockContention = errors.New("fsutil: lock contention") ErrLockPathUnsafe = errors.New("fsutil: lock path unsafe") )
ErrLockContention and ErrLockPathUnsafe are the file-lock sentinels: the exclusive lock could not be acquired within the caller's timeout, and the lock path is a symlink or not a regular file.
var ( ErrNotRegular = errors.New("fsutil: not a regular file") ErrTooBig = errors.New("fsutil: file exceeds size cap") )
ErrNotRegular and ErrTooBig are the guarded-read sentinels: a non-regular leaf (symlink/FIFO/device/directory) and a file over the caller's byte cap.
var ErrNotRealDir = errors.New("fsutil: not a real directory")
ErrNotRealDir is the directory-validation sentinel: the path exists, and it is a symlink or a non-directory rather than a real directory. Callers map it onto their own typed refusal with errors.Is.
var ErrOwnerUnknown = errors.New("fsutil: owning uid unavailable")
ErrOwnerUnknown is OwnerUID's sentinel for a path that exists but whose owning uid the platform did not report. abcd builds only darwin and linux, where os.FileInfo.Sys() always carries a *syscall.Stat_t, so it cannot arise there — it exists so a fail-closed caller has a named condition to refuse on rather than an "ok" it never earned.
Functions ¶
func CaseFoldingFS ¶ added in v0.6.7
func CaseFoldingFS() bool
CaseFoldingFS reports whether the platform's default filesystem folds case. macOS (APFS/HFS+ default) and Windows do; abcd assumes this default rather than probing each volume, and the only cost of a false assumption is a stricter comparison.
func CreateExclusiveIn ¶
CreateExclusiveIn writes data to rel INSIDE root, failing if rel already exists. It is the canonical primitive for a durable write that must (a) stay contained under a directory even against a symlinked ancestor, and (b) never replace an existing file.
It is deliberately not WriteFileAtomic. That primitive's contract is atomic REPLACEMENT — a temp file renamed over the target — which is the right shape when a file is rewritten in place, and the wrong one here: the rename would happily clobber, and the containment would depend on the caller having resolved the directory safely rather than on the Root. Here the exclusive create IS both guarantees at once. There is no partial-overwrite hazard to protect against, because the file cannot already exist; a crash mid-write leaves a short file, which the caller's own no-overwrite refusal surfaces on the next run rather than silently completing.
rel is slash-separated and relative to root. os.ErrExist is returned unwrapped enough for errors.Is, so a caller can map it to its own typed refusal. A write that fails after the create removes the partial file.
func DirHasEntries ¶
DirHasEntries reports whether path is a directory holding at least one entry, dotfiles included — a directory kept alive by a lone .gitkeep is not empty.
An absent path is false with no error: "missing" and "empty" are distinct conditions, and pairing this with Exists lets a presence rule and a non-empty rule report independently rather than one masking the other. A path that exists but is not a directory is likewise false with no error.
func EnsureRealDir ¶ added in v0.8.0
EnsureRealDir creates dir at perm if it is absent — a single, non-following Mkdir, never MkdirAll — and then PROVES the result is a real directory rather than a symlink or a non-directory. It is the canonical directory-validation primitive: exactly one place knows that "create it" and "prove it" are two steps and that the second one is not optional.
The proof is the part a bare os.Mkdir cannot give. Mkdir on an existing symlink returns EEXIST whether the link points at a directory the caller meant or at somewhere it never did, so the ErrExist branch on its own accepts a planted redirect; the Lstat afterwards is what refuses it. Creating one level at a time is the other half — MkdirAll on a leaf follows a symlinked ancestor and creates directories under its target — which is why EnsureRealDirAll walks rather than delegating to MkdirAll.
perm is the mode a NEW directory is created with, and it is a parameter because callers differ legitimately: 0o700 for a private store under the caller's home, 0o755 for a record directory in a shared worktree. An already-existing dir keeps whatever mode it has — Mkdir returns ErrExist without touching it — so this never widens or narrows a directory the caller made themselves, and consolidating callers onto it changes no directory's mode.
A create that fails for any other reason returns os.Mkdir's error unwrapped enough for errors.Is and os.IsPermission. A path occupied by the wrong kind of thing returns ErrNotRealDir inside an *os.PathError, so the offending path is in the message and the class is testable.
func EnsureRealDirAll ¶ added in v0.8.0
EnsureRealDirAll is EnsureRealDir over a chain: it creates every missing level of rel under base and proves EACH level real, so a symlinked ANCESTOR is refused rather than followed. That is the guarantee os.MkdirAll cannot make, and the one a caller's private copy carried as a documented hole before this primitive absorbed it.
base must already exist and is itself proved real; nothing above it is created or inspected. The bound is deliberate rather than incidental — a walk that ran to the filesystem root would refuse the platform's own symlinked prefixes (/tmp and /var are symlinks on macOS, and the test temp directory lives under one) — so the caller's trusted root is where the proof starts.
rel is a slash-separated relative path held to ValidRelPath: a path that arrives as data has no business creating directories, and a caller that joins an untrusted segment onto its root gets a refusal rather than an escape.
func Exists ¶
Exists reports whether path exists, following symlinks — so a link to a real file exists and a dangling link does not. A stat error other than not-exist is returned rather than swallowed, so a caller checking a convention fails closed on a permission error instead of silently reporting "absent".
func ExistsNoFollow ¶
ExistsNoFollow reports whether a filesystem entry occupies path, NEVER following a symlink at the leaf — so a dangling symlink still exists: the name is taken, and whatever occupies it (including a link whose target string is itself sensitive) is what a placement check must see. Ancestor components are resolved by the kernel as usual. Error handling mirrors Exists: not-present is false with no error, anything else fails closed.
func FoldPath ¶ added in v0.6.7
FoldPath returns p in the spelling a path comparison uses: lower-cased AND Unicode-normalised (NFC) when fold is set, unchanged otherwise. It is the single place a case-folding comparison key is minted, so a containment gate and a duplicate-target map fold identically rather than drifting apart.
A case-folding filesystem (APFS/HFS+, Windows) is ALSO normalisation- insensitive: the NFC and NFD spellings of one name address the same file, so a key that folds only case leaves the NFC/NFD half of the equivalence class open — the payload-destination gate, the pack overlap gate and embark's claimed key all then read two spellings of one directory as distinct (iss-2608270926030978). NFC is applied AFTER the lower-casing so the key is always in composed form. When fold is false the host is case-sensitive and normalisation-sensitive (Linux), so the key stays byte-exact and two distinct spellings remain distinct.
fold is a parameter rather than a call to CaseFoldingFS because a fail-closed gate's case-folding branch cannot be provoked on a case-sensitive host: the caller passes its own predicate, and a test passes the branch it means to prove.
func InsideGitDir ¶ added in v0.8.0
InsideGitDir reports whether the repo-relative path p names the git directory or anything under it.
ValidRelPath accepts ".git/config" — it is clean, relative, and inside the containment root — so a configuration value that arrives as data and is then READ AND PUBLISHED needs this second gate on top of it: .git/config carries a credential-bearing remote URL (on a CI runner, the checkout token in an `http.…extraheader` line), and nothing under .git is ever a legitimate source of rendered text. The first segment is compared case-insensitively, because a case-folding filesystem reaches the same .git through ".GIT" (iss-150).
Only the leading segment is examined, so ".github/workflows/ci.yml" and ".gitignore" — different directories that merely start with the same bytes — are not caught.
func IsDir ¶
IsDir reports whether path exists and is a directory. An absent path is false with no error; any other stat error is returned (fail closed).
It follows symlinks: use IsRealDir where a symlinked directory must read as false (the owned-store guard).
func IsRealDir ¶
IsRealDir reports whether path is a directory and NOT a symlink. It lstats (never following) so a symlink pointing at a directory reads as false — the owned-directory guard the store re-runs on every mutating call.
It checks the LEAF only: every ancestor component is resolved by the kernel, so a symlinked parent passes. Where a write must be contained against a symlinked ancestor too, route it through CreateExclusiveIn under an os.Root opened at the trust boundary, which refuses symlink traversal at every level.
func ModuleRoot ¶
ModuleRoot walks up from start until it finds the directory holding go.mod — the module root.
It exists for the repo's code generators, which must write to the same file whether they are invoked by `go generate` (the working directory is the package being generated) or run directly from the repo root. Anchoring on go.mod rather than on .git means a generator also works inside a worktree or an export where the git directory is not where the walk expects it.
A start outside any module is an error rather than a fallback to the working directory: a generator that silently wrote its artefact into an unrelated directory would be worse than one that refuses.
func OwnerUID ¶ added in v0.8.0
OwnerUID reports the uid that owns path, following symlinks (the caller has usually resolved the path already, and a link's own uid is not the uid that controls its content). It is the canonical ownership lookup: code asking "is this directory mine?" compares this against os.Getuid() instead of reaching for syscall.Stat_t itself, so exactly one place knows the platform's stat shape.
The error is returned rather than folded into a boolean because ownership is a fail-closed question: a caller that cannot learn the owner must be able to tell that from learning the owner is itself, and a (uid, bool) shape makes the two look alike at the call site.
func PathWithin ¶ added in v0.6.7
PathWithin reports whether child is equal to or nested inside parent. With fold set the comparison is case-insensitive: otherwise a destination like ".../REPO/dist" computes as an out-of-tree sibling of ".../repo" and slips a containment gate, even though the two name the SAME directory on a case-folding filesystem — and the write then lands inside the tree the gate exists to protect. Erring toward "within" is the safe direction for a destructive-write gate (iss-2608270500192615).
It is the canonical containment comparison: the launch payload-destination gate and the lifeboat pack overlap gate both route through it rather than keeping divergent prefix tests.
func PathsOverlap ¶ added in v0.6.7
PathsOverlap reports whether a and b are the same directory or one contains the other — either way a write under one touches the other.
func ReadGuarded ¶
ReadGuarded opens path once, read-only, with O_NOFOLLOW (refuse a symlinked leaf) and O_NONBLOCK (a FIFO/device leaf returns immediately instead of blocking the open forever), then validates on the SAME descriptor that it is a regular file within limit bytes before reading through a LimitReader — so no symlink swap between stat and read, no non-regular leaf, and no size overrun can reach the caller. It is the shared trust-boundary read primitive for any file inside a repo working tree that untrusted content could have replaced with a symlink to /dev/zero or an endless device. The raw open error is returned so callers can test os.IsNotExist / syscall.ELOOP; a non-regular or oversize file returns ErrNotRegular / ErrTooBig.
func ReadGuardedInRoot ¶
ReadGuardedInRoot is ReadGuarded resolved inside an os.Root containment scope. rel is a slash-separated path relative to root; every component is resolved by the OS within root, so a symlinked ANCESTOR directory — the shape a hostile clone commits as git mode 120000, `etclink -> /etc` or `home -> ../../..` — cannot walk the read outside the root. ValidRelPath is lexical and cannot see a symlink, so containment must rest here, not on it.
It keeps every guarantee ReadGuarded gives at the leaf: a symlinked leaf is refused (lstat, never followed), the descriptor is confirmed to be the very file that was vetted (os.SameFile closes the lstat→open swap), a non-regular leaf returns ErrNotRegular, O_NONBLOCK stops a FIFO or device blocking the open, and the caller's byte cap is enforced against both the fstat size and the bytes actually read.
Errors are returned raw where the OS raised them, so a caller can still test os.IsNotExist; a path that escapes the root is NOT an os.IsNotExist error, so a caller skipping absent files fails closed on the escape rather than treating it as "the file is not there".
func RealExistingPath ¶ added in v0.6.9
RealExistingPath returns p made absolute with its longest EXISTING prefix symlink-resolved and the absent remainder rejoined lexically. A destination that does not exist yet still resolves through its real parent, so a containment gate compares real locations rather than lexical strings a symlinked ancestor could disguise — and on a platform whose temp root is itself a symlink (/var -> /private/var on macOS) an unresolved comparison would silently miss an overlap. An empty p is returned empty rather than resolved to the working directory.
It is the canonical existing-prefix resolver: the launch payload-destination gate, the lifeboat pack destination gate and the site output-directory gate all route through it rather than keeping divergent copies. It does not case-canonicalise (Go's darwin EvalSymlinks keeps the caller's spelling for every non-symlink component), so compare its results with PathWithin under CaseFoldingFS.
func RedactHome ¶
RedactHome replaces the user's home directory in s with "~" wherever it appears — the home-polarity shortcut over RedactRoot for a success or receipt envelope that carries an absolute path the CLI error scrub (which runs only on the error value) never sees. A home-resolution failure or a root-level home leaves s unchanged, so the primitive is never worse than the raw string.
func RedactRoot ¶
RedactRoot replaces every occurrence of the absolute directory root in s with repl — both a path UNDER the root (root + separator + …) and the BARE root itself when it sits at a right boundary (end of string or a non-path character). The bare-root case matters because a message that names exactly $HOME (e.g. "cannot access /Users/alex") would otherwise leak the developer abcd-audit:allow identity — its base segment IS the username. The filesystem root ("/") and empty or relative roots are skipped so a message is never mangled.
It is the one statement of what a developer-identity root looks like in rendered output: the CLI's error scrub redacts the working directory and the home directory out of every command error, and `ahoy install`'s receipt redacts the same two roots out of every write it reports (iss-177). Callers pass the root and the replacement because the polarity differs — "." for the repo, "~" for home — but the boundary rule must not.
On a case-folding filesystem the match is case-insensitive: a message that echoes $HOME in a case variant (as a shell or a syscall may on macOS/APFS) names the same directory, so the identity must still be scrubbed. The original casing of any UNREDACTED text is preserved, and on a case-sensitive host the match stays byte-exact so two distinct paths are never merged (iss-2608270908341622).
func RepoRel ¶
RepoRel renders target as a path relative to base — the repo root, or the working directory — so machine output never carries an absolute developer-identity path (iss-81). A target outside base yields a "../…" form, which is acceptable: the contract is only that the result is never an absolute /Users/<name>/… path. It falls back to the base name when a relative form cannot be computed (a different volume, a relative target, or an empty base) and to target unchanged when target is already relative. A non-path value that merely looks absolute-free (e.g. a URL) is returned untouched.
func ValidRelPath ¶
ValidRelPath reports whether p is a safe repo-relative slash path: non-empty, not absolute, already clean, free of empty, ".", ".." or control-character segments, and containing no backslash. It is the canonical lexical guard for a path that arrives as data — a committed configuration value, a packed lifeboat's manifest entry — before it is joined onto a trusted root. Lexical only: it says nothing about symlinks, so a caller that then follows the path still needs the guarded read/write primitives.
A backslash is refused outright, on every host. path.Clean is slash-only, so it leaves a backslash untouched: `..\..\x` is "clean" and slash-traversal-free, yet on a Windows target (abcd cross-compiles Windows binaries) filepath.FromSlash turns those backslashes into separators and the join walks out of the intended root. A legitimate committed repo-relative path is always a forward-slash path and never contains a backslash, so refusing it everywhere closes the Windows escape while keeping the gate identical across platforms (iss-2608280807166510).
func WithFileLock ¶
WithFileLock acquires an exclusive advisory (flock) lock on lockPath, holds it across fn, and releases it when fn returns. The lock file is opened O_NOFOLLOW (a symlinked lock path is refused) and verified on the same descriptor to be a regular file, both reported as ErrLockPathUnsafe; the exclusive flock is polled for up to timeout, after which ErrLockContention is returned. fn's own error is returned unwrapped.
It is the single inter-process load-modify-write lock primitive: capture's ledger allocator and ahoy's ~/.abcd history registry both route through it rather than keep divergent flock loops (the one-canonical-primitive invariant). The lock is advisory — every writer of the guarded state must take it — and its scope is one lock file; callers that must not deadlock keep their acquisitions unnested.
func WriteFileAtomic ¶
WriteFileAtomic writes data to path durably: a temp file in the target directory is written, chmod'd to perm on its open descriptor, flushed, fsync'd, then renamed over the target, and finally the parent directory is fsync'd best-effort so the rename survives a crash. Parent directories are created as needed.
The rename is the commit point: a reader sees either the old file or the complete new one, never a half-written file. os.Rename does not follow a symlink at the leaf — a pre-planted symlink at path is replaced by the real file, not written through. The mode is set with fchmod on the open descriptor (never chmod-by-name on the closed temp), so the enumerable temp name cannot be swapped for a symlink between close and chmod and have the mode applied to an attacker-chosen target.
func WriteFileAtomicInRoot ¶
WriteFileAtomicInRoot is WriteFileAtomic resolved inside an os.Root containment scope: rel is a slash-separated path relative to root, and the missing parents, the temp file, the rename, and the parent fsync all happen through root. A symlinked ancestor committed by a hostile repo therefore cannot land the write outside the root — the counterpart of ReadGuardedInRoot, and the reason a write path validated only lexically is not contained.
The commit point and the ordering are WriteFileAtomic's: write, fchmod on the open descriptor (never chmod-by-name), fsync, close, rename, parent fsync. The temp name is unpredictable so the enumerable-name swap WriteFileAtomic guards against has nothing to aim at.
func WriteFileAtomicPreserveMode ¶
WriteFileAtomicPreserveMode is WriteFileAtomic that keeps the target's existing permission bits when it already exists, defaulting to 0644 for a new file. It is the canonical form for the store writers that rewrite a file in place and must not silently reset its mode.
func WriteFileAtomicPreserveModeInRoot ¶
WriteFileAtomicPreserveModeInRoot is WriteFileAtomicInRoot that keeps the target's existing permission bits when it already exists, defaulting to 0644 for a new file — the contained counterpart of WriteFileAtomicPreserveMode.
Types ¶
This section is empty.
Source Files