README
¶
go-safefs
Bounded, crash-safe file I/O for untrusted paths: capped reads, atomic writes, rooted copies, case-folded lookup and a small disk cache. Package safefs, module github.com/Rethunk-Tech/go-safefs.
For a program that writes into folders it does not own and reads files it did not write. Each call either does what it says or fails without leaving a half-written file, a followed link, or an unbounded read behind.
Quick start
go get github.com/Rethunk-Tech/go-safefs
A worked example and the contracts: HUMANS.md.
Features
- Opens and reads through the parent folder, so a symlinked last path element cannot lead out of it.
- Reads capped in bytes, failing with
ErrTooLargewhereio.LimitReaderwould end short. - Rename and remove that retry on Windows while a scanner or indexer holds the file.
- Writes, copies and clones that a crash leaves whole or absent, and a materialiser that picks the cheapest sharing.
ResolveFold, which finds a path written for Windows on a case-sensitive file system.diskcache, typed JSON cache files where a crash or a stale shape only costs a refetch.
Documentation
| Doc | Audience |
|---|---|
| HUMANS.md | Running and using the package |
| AGENTS.md | Internals, invariants, and why |
| CONTRIBUTING.md | Process — commits, tests, docs |
| SECURITY.md | Vulnerability disclosure |
| CHANGELOG.md | Release notes |
License
MIT — see LICENSE. © 2026 Rethunk.Tech, LLC.
Documentation
¶
Overview ¶
Package safefs is file I/O for a program that writes into folders it does not own and reads files it did not write: opens that a symlinked last element cannot walk out of a folder, byte-capped reads, renames and removes that ride out Windows antivirus locks, writes and copies a crash leaves whole or absent, links and clones, and path lookup across Windows case. Subpackage diskcache keeps fetched answers on disk.
Index ¶
- Variables
- func CapReader(r io.Reader, limit int64) io.Reader
- func CheckWritable(dir string) error
- func Chmod(path string, perm fs.FileMode) error
- func Clone(src, dst string) error
- func CopyFile(src, dst string, opts CopyOptions) (err error)
- func CopyTree(src, dst string, opts CopyOptions) (skipped []string, err error)
- func CopyTreeFollowingLinks(src, dst string, opts CopyOptions) error
- func Create(path string) (*os.File, error)
- func CreateExcl(path string, perm fs.FileMode) (*os.File, error)
- func Device(path string) (uint64, error)
- func EvalSymlinks(p string) (string, error)
- func ExtendedPath(abs string) string
- func FoldCase(path string) string
- func FreeBytes(path string) (int64, error)
- func HashFile(path string, h hash.Hash) (string, error)
- func IsDir(path string) bool
- func IsFile(path string) bool
- func LinkDir(link, target string) error
- func LinkUnsupported(err error) bool
- func LinkedDir(p string, info os.FileInfo) bool
- func Open(path string) (*os.File, error)
- func OpenFile(path string, flag int, perm fs.FileMode) (*os.File, error)
- func OpenRegular(path string) (*os.File, error)
- func ReadCapped(r io.Reader, limit int64) ([]byte, error)
- func ReadDirIfExists(path string) ([]os.DirEntry, bool, error)
- func ReadFile(path string) ([]byte, error)
- func ReadFileIfExists(path string) ([]byte, bool, error)
- func RealDirUnder(root, p string) bool
- func Remove(path string) error
- func RemoveAll(path string) error
- func Rename(from, to string) error
- func ResolveFold(fsys fs.FS, rel string) (string, error)
- func SamePath(a, b string) bool
- func Shared(path string) bool
- func Size(dir string) (int64, error)
- func SizeEachName(dir string) (int64, error)
- func Stat(path string) (fs.FileInfo, error)
- func SyncDir(dir string) error
- func Trash(path string) error
- func UnderRoot(root, p string) bool
- func Unshare(path string) error
- func WalkFiles(root string, fn func(rel, path string, size int64) error) error
- func WalkRegular(root string, fn func(path string, d fs.DirEntry) error) error
- func Within[T any](path string, fn func(dir *os.Root, name string) (T, error)) (T, error)
- func WriteFile(path string, data []byte, perm fs.FileMode) error
- func WriteFileAtomic(path string, data []byte, perm os.FileMode) error
- func WriteStreamAtomic(path string, perm os.FileMode, write func(io.Writer) error) error
- func WriteStreamNoSync(path string, perm os.FileMode, write func(io.Writer) error) error
- type CopyOptions
- type CopyProgress
- type DanglingLinkError
- type FileID
- type Materializer
- type Tier
Constants ¶
This section is empty.
Variables ¶
var ErrNotRegular = errors.New("not a regular file")
ErrNotRegular is OpenRegular's refusal of a directory, device, pipe or other non-regular file.
var ErrTooLarge = errors.New("larger than its size cap")
ErrTooLarge is what ReadCapped returns for input over its limit.
Functions ¶
func CapReader ¶
CapReader reads r up to limit bytes and fails with ErrTooLarge once r holds more, where io.LimitReader would end as if the input were complete.
func CheckWritable ¶
CheckWritable fails fast when dir refuses new files, as a folder under Program Files does for a non-elevated process, instead of letting the first real write fail halfway through. A missing dir is nil: creating it is the caller's job. Any other failure is the probe's *fs.PathError, which wraps fs.ErrPermission when that is why.
func Clone ¶
Clone copies src to a new dst with FICLONE, sharing extents on a filesystem that supports it. dst is created exclusively with mode 0o600 and removed again on failure.
func CopyFile ¶
func CopyFile(src, dst string, opts CopyOptions) (err error)
CopyFile copies the regular file src to dst, replacing what is there. The bytes go to a temporary file beside dst, are flushed to disk, and only then renamed into place, so a crash leaves dst whole or absent, never truncated.
func CopyTree ¶
func CopyTree(src, dst string, opts CopyOptions) (skipped []string, err error)
CopyTree copies the regular files and folders under src into dst (which may exist), file by file. Directory junctions, mount points and symlink directories are not followed, and are returned in skipped. Symlink files are copied by content when they still resolve under src, and are an error when they escape.
func CopyTreeFollowingLinks ¶
func CopyTreeFollowingLinks(src, dst string, opts CopyOptions) error
CopyTreeFollowingLinks copies src to dst like CopyTree but writes the file a file link points to wherever it lives. It is for a folder the user chose on their own disk, filled with links by another tool; linked folders are still skipped and anything that is not a regular file is refused. A link to a missing file is a *DanglingLinkError.
func CreateExcl ¶
CreateExcl creates path with perm and fails if it already exists.
func EvalSymlinks ¶
EvalSymlinks is filepath.EvalSymlinks; only Windows needs help with long paths.
func ExtendedPath ¶
ExtendedPath is the \\?\ form of an absolute Windows path of drive-letter form that is too long for the classic API, which filepath.EvalSymlinks and raw CreateFile calls use without any prefixing of their own. Short, UNC and already-extended paths come back unchanged.
func FreeBytes ¶
FreeBytes returns the bytes available to an unprivileged writer on the volume holding path.
func IsDir ¶
IsDir is true when path resolves to a directory. Like IsFile it follows a final symlink out of path's folder, so a symlinked folder still counts.
func IsFile ¶
IsFile is true when path resolves to a regular file. Unlike IsDir it uses os.Stat, so a symlink whose target is outside path's folder (an executable linked from another library) still counts.
func LinkUnsupported ¶
LinkUnsupported reports an error that means the target cannot clone or link, so the caller falls back to a copy.
func LinkedDir ¶
LinkedDir reports an entry that leads to a directory without being one: a symlink to a folder, or on Windows a junction or mount point, which Go reports as an irregular file.
func OpenRegular ¶
OpenRegular opens path for reading, refusing anything but a regular file.
func ReadCapped ¶
ReadCapped reads r whole, failing with ErrTooLarge when it holds more than limit bytes. It is the bound on every read of input the program did not write: at most limit+1 bytes are ever held.
func ReadDirIfExists ¶ added in v0.5.0
ReadDirIfExists lists the directory path, sorted by name like os.ReadDir, through the parent root. A missing path gives nil, false, nil, and any other failure is returned.
func ReadFileIfExists ¶ added in v0.4.0
ReadFileIfExists is ReadFile for a file that may not be there: a missing path gives nil, false, nil, and any other failure is returned.
func RealDirUnder ¶
RealDirUnder reports whether p is a directory whose own path still resolves under root, not a symlink directory or a junction/reparse that points somewhere else.
func RemoveAll ¶
RemoveAll is os.RemoveAll with the same antivirus/indexer retry as Rename; a repeat pass removes whatever the first one could not.
func Rename ¶
Rename is os.Rename that rides out the short Windows window in which an antivirus scanner or the search indexer holds a freshly written file open. Elsewhere it is os.Rename.
func ResolveFold ¶
ResolveFold resolves the slash path rel below fsys one element at a time, backtracking. At each element it tries the entry of exactly that name first, then each entry strings.EqualFold matches, in fs.ReadDir order (sorted by name), and returns the first whose remaining path resolves. It finds a file whenever some path in fsys matches rel element by element under case folding, which is how a path written for Windows reads on a case-sensitive file system.
rel must pass fs.ValidPath ("." returns "."), else the error is a *fs.PathError wrapping fs.ErrInvalid. No match, or a walk past its ReadDir budget (8 per element of rel), is a *fs.PathError wrapping fs.ErrNotExist. In a tree each directory is read at most once; only a linked directory, reachable under several names, can make the walk long.
func SamePath ¶
SamePath reports whether a and b name the same location once cleaned and case-folded where the file system ignores case (FoldCase).
func Shared ¶
Shared reports whether path is a symlink or a file with other hard links: writing it would change the file another folder holds too.
func Size ¶
Size returns the total size in bytes of the regular files under dir, counting hard-linked files once. Unreadable entries below dir are skipped, so the size is a best-effort total; only a failure to read dir itself is an error.
func SizeEachName ¶
SizeEachName is Size counting every name in full, for a target that refuses hard links.
func Trash ¶
Trash moves path into the user's freedesktop trash (an absolute XDG_DATA_HOME, else ~/.local/share, then Trash/), where the file manager can restore it. A path on another filesystem (another drive, or another btrfs subvolume such as /var/tmp beside /home) is copied there and then deleted.
func UnderRoot ¶
UnderRoot reports whether p is root or a path still inside it after both have been cleaned.
func Unshare ¶
Unshare replaces a shared file with a private copy of its content, leaving the other folder's file as it was.
func WalkFiles ¶
WalkFiles calls fn with the slash path below root, the full path and the size of every regular file under root, in walk order.
func WalkRegular ¶ added in v0.4.0
WalkRegular calls fn with the path and entry of every regular file under root, in walk order. A missing root is no error. Symlinks are not followed and not reported, so a link cannot lead the walk out of root.
func Within ¶
Within runs fn with an os.Root on path's parent directory and path's final element, so a symlinked final element cannot lead out of the parent. A trailing separator is dropped first: Dir keeps it as the last element while Base strips it, which would name that element twice.
func WriteFileAtomic ¶
WriteFileAtomic replaces path with data through a temp file and rename, so a crash never leaves it truncated.
func WriteStreamAtomic ¶
WriteStreamAtomic replaces path with write's output through a temp file, fsync, and rename, so a crash never leaves it truncated. write seeing a failure, or returning one, removes the temp file and leaves path unchanged.
func WriteStreamNoSync ¶
WriteStreamNoSync is WriteStreamAtomic without the fsyncs, for a file that only memoises a computation: a crash may leave it torn, so the reader must treat an unreadable file as absent. Thousands of small fsyncs cost seconds.
Types ¶
type CopyOptions ¶
type CopyOptions struct {
// TempSuffix names CopyFile's temp file, dst+TempSuffix. CopyFile removes a stale file of that name before
// writing, so the suffix must be specific enough never to name a file anyone keeps. Zero means ".safefs-tmp".
TempSuffix string
// First names a top-level entry of src that CopyTree copies before the rest.
First string
// Progress is called after each file with the running totals.
Progress func(CopyProgress)
// Put writes one file; from is where its bytes live, to where they go, rel its slash path below src. Nil means
// CopyFile, which reads through the walk's root; a Put opens from by path, outside that guarantee.
Put func(from, to, rel string) error
// Skip names entries CopyTreeFollowingLinks leaves out.
Skip func(name string) bool
}
CopyOptions tunes CopyFile, CopyTree and CopyTreeFollowingLinks. The zero value is the default for every field.
type CopyProgress ¶
CopyProgress counts files and bytes while CopyTree runs.
type DanglingLinkError ¶
DanglingLinkError is CopyTreeFollowingLinks' refusal of a file link whose target is gone.
func (*DanglingLinkError) Error ¶
func (e *DanglingLinkError) Error() string
func (*DanglingLinkError) Unwrap ¶
func (e *DanglingLinkError) Unwrap() error
type FileID ¶
type FileID [3]uint64
FileID identifies one inode on one device; the third word is the high half of a 128-bit Windows file id.
type Materializer ¶
type Materializer struct {
// Tiers is the order to try; zero means clone, hardlink, symlink, copy. Copy is the tier that never fails over.
Tiers []Tier
// tiers. rel is slash-separated on every OS, so one policy reads the same paths everywhere. Nil means every
// file.
Shareable func(rel string) bool
// Copy is the options of the copy tier.
Copy CopyOptions
}
Materializer puts files at a destination by the cheapest sharing the destination allows, falling through its tiers on LinkUnsupported errors. It remembers per destination device which tiers that device refused; that memory is process-wide, shared by every Materializer.
func (Materializer) File ¶
func (m Materializer) File(src, dst, rel string) error
File puts the one file src at dst. rel is its path inside the tree, and decides Shareable.
func (Materializer) Tree ¶
func (m Materializer) Tree(src, dst string) error
Tree puts src's files into dst.
Source Files
¶
- clone_linux.go
- copy.go
- doc.go
- evalsymlinks.go
- evalsymlinks_other.go
- foldcase_other.go
- foldpath.go
- free_statfs.go
- free_unix.go
- inode_unix.go
- link_unix.go
- links.go
- links_unix.go
- materialize.go
- readcapped.go
- rename.go
- rename_other.go
- rooted.go
- samepath.go
- size.go
- syncdir_unix.go
- trash_freedesktop.go
- walkfiles.go
- writable.go
- write.go
Directories