filesystem_ext4

package module
v0.0.0-...-6cebbb6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 15, 2026 License: BSD-3-Clause Imports: 22 Imported by: 0

README

Package ext4 test helpers and journal

This package contains an in-memory ext4 implementation and test helpers used by the unit tests. The recent change shortens the critical section in Transaction.Commit() and adds a regression test exercising concurrent commits against the sidecar journal.

See the tests under pkg/go-filesystems/ext4/test for usage examples.

ext4 filesystem

Overview

Pure-Go read/write access to ext4 filesystem images.

  • No root privileges required.
  • No external tools required for normal operations (some cross-check tests use mke2fs).
  • No CGO.

This package supports extents, 64-bit block numbers, flex_bg, directory htree indexing, CRC32c metadata checksums, and automatic partition detection.

Production-readiness improvements (recent)

  • Re-entrant locking moved from a getGID() hack to explicit owner-token reentrantMutex APIs, enabling safe reentrancy without relying on runtime internals.
  • Metadata serialization for block-group descriptors (BGD), block and inode bitmaps and inode-table writes is implemented; metadata_csum support is available for checksum-protected metadata.
  • Online resize: Grow() implements expanding images and adding block groups.
  • IOCTLs: basic GetFlags/SetFlags ioctl paths are implemented where useful for tooling.
  • Filesystem inspection and repair hooks: CheckImage() and RepairImage() provide programmatic fsck/repair entry points for integration tests.
  • Concurrency hardening: bitmap/BGD/inode-table updates are serialized with per-group locks to avoid concurrent write corruption.
  • Tests: unit and integration tests were added for Grow, repair paths and concurrent bitmap updates; many code paths are covered under the test build tag.

Important notes

  • Production builds compile without the test build tag; test-only tracing and write hooks are guarded by //go:build test and are excluded from production binaries.
  • The previous getGID() test-only helper has been replaced with an owner-token based reentrantMutex API. Code should prefer LockOwner/UnlockOwner with an explicit token (see package docs and tests).
  • Test wrappers used by //go:build test expose an UnderlyingFile() helper (for example the loggingRW wrapper) so that lock keying can share the underlying descriptor without using unsafe reflection into unexported fields.

References

https://docs.kernel.org/filesystems/ext4/

Support summary

Feature Status Notes
Open / Close ✅ Supports partitioned images (MBR/GPT auto-detect)
Format ✅ Creates ext4 images; some cross-check tests require mke2fs
ReadFile / WriteFile ✅ Full file I/O supported (extents, sparse writes)
MkDir / Delete / Rename ✅ Directory and rename operations implemented
ReadLink / Symlinks ✅ Supported
Metadata serialization (BGD/bitmaps/inode-table) ✅ Writes + metadata_csum supported
Online resize (Grow) ✅ Adds block groups and updates superblock/BGD
IOCTLs (GetFlags/SetFlags) ✅ Tooling-level ioctl support implemented
CheckImage / RepairImage hooks ✅ Programmatic fsck/repair entry points
Concurrency hardening ✅ Per-group locks for bitmap/BGD/inode-table
Test-only tracing ✅ Heavy tracing under //go:build test for diagnostics

Limitations & cautions

  • This package is intended primarily for tooling and tests (image creation, inspection, repair, and offline modification). It is not a kernel filesystem driver and does not aim to be a drop-in replacement for the in-kernel ext4 implementation.
  • Not every kernel ioctl or obscure ext4 feature is implemented; check tests and source for the supported surface.
  • Integration tests that cross-validate behavior against e2fsprogs (mke2fs, e2fsck) are provided but may be skipped on platforms lacking those tools.
  • Performance and large-scale concurrency behavior are still subjects for benchmarking; see the Next steps section below.

Module

github.com/go-filesystems/ext4

Volume label

The driver implements the optional filesystem.Labeller interface so callers can read and update the ext2/3/4 superblock's s_volume_name in place:

fs, _ := ext4.Open("disk.img", -1)
defer fs.Close()
if l, ok := fs.(filesystem.Labeller); ok {
    fmt.Println("label:", l.Label())
    _ = l.SetLabel("cloudimg-rootfs")
}

SetLabel is offline-safe only — it takes the FS write lock, re-reads the on-disk superblock, zero-fills bytes [120:136], writes the new label, recomputes the superblock CRC-32C if metadata_csum is enabled, and WriteAt+Syncs the 1024-byte block back. Concurrent writers may produce a torn superblock; use it on a filesystem no other process is actively mutating.

The superblock checksum uses the kernel-canonical CRC-32C convention — crc32c(~0, sb[:0x3FC]), no final XOR, no seed — distinct from every other metadata structure in ext4 (which is seeded from s_checksum_seed or crc32c(~0, uuid)). Mismatching the convention produces "Found ext4 filesystem with invalid superblock checksum" on the next Linux mount.

Quick start — validation and best practices

1. Static analysis
go vet ./...
# staticcheck ./...  # run if staticcheck is installed
2. Run tests with the race detector (test-only tracing requires -tags test)

Recommended quick run (ext4 packages only):

go test -race -tags test ./pkg/go-filesystems/ext4/... -v

Run the test harness package (shorter):

go test -race -tags test ./pkg/go-filesystems/ext4/test -v

For the whole repository (long):

go test -race ./... 2>&1 | tee /tmp/all_tests_race.log

To check coverage for packages you modify (useful for diskimage package work):

go test -tags test -coverprofile=cover.out ./pkg/go-filesystems/ext4 && go tool cover -html=cover.out
3. Build without test tags to ensure no test-only dependencies leak
go build ./pkg/go-filesystems/ext4

What's changed (high level)

  • Replaced getGID()-based reentrancy with owner-token reentrantMutex APIs.

  • Implemented metadata serialization for BGD / bitmaps / inode-table and metadata_csum support.

  • Added Grow() to perform online image expansions (adds block groups, updates superblock/BGD tables).

  • Added basic ioctl support for common tooling flags.

  • Exposed CheckImage() / RepairImage() hooks for programmatic fsck/repair.

  • Hardened concurrency around bitmap/BGD/inode-table updates and added tests that exercise concurrent bitmap updates.

  • Local fixes (developer notes):

    • Cached debug/lock flags to avoid hot-path os.Getenv calls.
    • Added a configurable idle timeout for commitWorker goroutines to prevent unbounded worker proliferation under stress (EXT4_COMMIT_WORKER_IDLE_SEC).
    • Fixed commit-sequence stall: Transaction.Commit() now guarantees applySeq always advances (via mustAdvanceApplySeq deferred flag) even when apply returns an error, preventing all goroutines from blocking indefinitely on dirML.Lock().
    • Stress harness adjusted: removed inter-test t.Parallel() and reduced default workload (workers, ops, write_hammer) to complete within the standard go test 10-minute timeout.
    • Verified targeted and package-level tests under -race -tags test after changes.

Next steps

supported platforms.

Backoff tuning summary

We performed a small parameter sweep to reduce lock-order contention and blocking fallbacks observed under the TestStress_Concurrent harness. The table below summarizes the long-run (32 workers × 500 ops) fallback counts for a few candidate (attempts, base_us) values. Results are noisy and depend on workload timing; treat them as guidance for tuning.

Attempts Base (µs) Total fallbacks (32×500) Top hot-group Top count Notes
4 250 435 fd:4:inode:g:3:block:false 265 Lower peak but higher total than 6,250
6 250 425 fd:4:inode:g:3:block:false 294 Recommended: lowest total fallbacks
8 250 437 fd:4:inode:g:3:block:false 341 Higher total, higher peak

Tradeoffs:

  • Lower attempts can reduce contention bursts (lower peak on hot-group), but may increase total blocking fallbacks across the run.
  • Higher attempts with jitter spreads retries but can increase aggregate attempts and CPU; balance to minimize total fallbacks for your workload.

Reproduce the experiments locally (examples):

# short grid (scripts are in /tmp in the test workspace)
bash /tmp/backoff_grid.sh

# long run for a candidate (example: attempts=6 base=250µs)
EXT4_BACKOFF_MAX_ATTEMPTS=6 EXT4_BACKOFF_BASE_US=250 \
  EXT4_LOCK_DEBUG=1 EXT4_STRESS_WORKERS=32 EXT4_STRESS_OPS_PER_WORKER=500 \
  go test -count=1 -run 'TestStress_Concurrent/debian/concurrent_rw' -tags test ./pkg/go-filesystems/ext4/test -v

Artifacts and logs created during the sweep (on the machine that ran the experiments): /tmp/backoff_grid/results.csv, /tmp/backoff_long_three/, /tmp/backoff_grid/ and other logs under /tmp (see the test scripts).

Next tuning options:

  • Run a denser grid around (6,250) to refine the minimum.
  • Increase allocation dispersion (allocGroupCursor) or subdivide BGD locks to reduce serialization on very hot groups.

Contact

  • For questions or to request additional test runs (benchmarks, long-running integration), open an issue or ask in the project chat and include the exact test command you want executed.

Documentation

Overview

Package ext4 provides read/write access to ext4 filesystem images without requiring root privileges. It supports raw disk images (MBR/GPT) as well as bare filesystem images and handles modern ext4 features: extents, 64-bit block numbers, flex_bg, dir_htree and metadata_csum (CRC32c checksums).

Package ext4 provides read/write access to ext4 filesystem images. See ext4.go for exported types and constants.

tryMutex implements a mutex with TryLock semantics using a buffered channel of size 1. It provides Lock/Unlock (blocking) and TryLock

Index

Constants

View Source 
const (
	FeatIncompatFiletype   = 0x0002
	FeatIncompatExtents    = 0x0040
	FeatIncompat64bit      = 0x0080
	FeatIncompatFlexBg     = 0x0200
	FeatIncompatCsumSeed   = 0x2000
	FeatIncompatInlineData = 0x8000
)

Incompatible feature flags (must understand to read/write).

View Source 
const (
	FeatROCompatSparseSuper  = 0x0001
	FeatROCompatLargeFile    = 0x0002
	FeatROCompatGdtCsum      = 0x0010
	FeatROCompatMetadataCsum = 0x0400
)

Read-only compatible feature flags.

View Source 
const (
	InodeFlagHashIndex  = 0x00001000 // directory uses htree index
	InodeFlagExtents    = 0x00080000 // inode uses extent tree
	InodeFlagInlineData = 0x10000000 // data stored inline in inode
)

Inode flags.

View Source 
const (
	FtUnknown = 0
	FtRegFile = 1
	FtDir     = 2
	FtSymlink = 7
	FtDirTail = 0xDE // fake tail entry carrying checksum
)

Directory entry file types.

View Source 
const ExtentMagic uint16 = 0xF30A

Extent tree magic number.

View Source 
const MaxLabelLen = 16

MaxLabelLen is the on-disk size of the ext4 volume label (s_volume_name, at superblock offset 0x78).

View Source 
const RootIno uint32 = 2

Well-known inode numbers.

Variables

View Source 
var (
	InodeOffMode       = inodeOffMode
	InodeOffSizeLo     = inodeOffSizeLo
	InodeOffLinksCount = inodeOffLinksCount
	InodeOffFlags      = inodeOffFlags
	InodeOffBlock      = inodeOffBlock
	InodeOffGeneration = inodeOffGeneration
	InodeOffFilACLLo   = inodeOffFilACLLo
	InodeOffSizeHi     = inodeOffSizeHi
	InodeOffBlocksHi   = inodeOffBlocksHi
	InodeOffCsumLo     = inodeOffCsumLo
	InodeOffExtraIsize = inodeOffExtraIsize
	InodeOffCsumHi     = inodeOffCsumHi
)

Export inode offset constants used by tests.

View Source 
var CommitHook func(seq uint64, entries int)

CommitHook, when set (tests only), is invoked with the transaction sequence number each time a transaction is committed. This allows tests to observe/measure commit behavior without parsing debug logs. CommitHook, when set (tests only), is invoked with the transaction sequence number and the number of entries each time a transaction is committed. This allows tests to observe/measure commit behavior without parsing debug logs.

View Source 
var LinuxPartTypeGPT = linuxPartTypeGPT

Expose partition constants for tests.

View Source 
var SectorSize = sectorSize

Functions

func AddDirEntry 

func AddDirEntry(f ReaderWriterAt, fsOffset int64, sb *Superblock, dir *Inode, child uint32, name string, fileType uint8) error

AddDirEntry exposes addDirEntry. 

func AddFastSymlink(t *testing.T, fs *Ext4FS, parentPath, name, target string)

AddFastSymlink creates a symlink inode and dir entry in the live filesystem (used by tests that need a quick symlink creation helper).

func AddRangeToTx 

func AddRangeToTx(tx *Transaction, f ReaderWriterAt, fsOffset int64, sb *Superblock, startAbs int64, data []byte) error

AddRangeToTx exposes addRangeToTx so external tests can prepare transaction entries using the same locking semantics as production callers. This prevents test-only preparations from bypassing per-block locking and triggering lost-update races.

func AllocBlocks 

func AllocBlocks(f ReaderWriterAt, fsOffset int64, sb *Superblock, n uint32) ([]uint64, error)

AllocBlocks exposes allocBlocks.

func AllocInode 

func AllocInode(f ReaderWriterAt, fsOffset int64, sb *Superblock, isDir bool) (uint32, error)

AllocInode exposes allocInode.

func BgdOffset 

func BgdOffset(sb *Superblock, g uint32) int64

BgdOffset exposes bgdOffset.

func BgdRaw 

func BgdRaw(d *Bgd) []byte

func BgdTableBlock 

func BgdTableBlock(sb *Superblock) uint64

func CRC32c 

func CRC32c(crc uint32, data []byte) uint32

CRC32c exposes crc32c.

func ClearBit 

func ClearBit(b []byte, i int)

func CloneFSImageFromPath 

func CloneFSImageFromPath(path string) ([]byte, error)

CloneFSImageFromPath reads the image file into a buffer copy for in-memory tests.

func ComputeInodeCsum 

func ComputeInodeCsum(sb *Superblock, in *Inode)

computeInodeCsum wrapper

func CreateFullWorker 

func CreateFullWorker(f ReaderWriterAt, fsOffset int64, sb *Superblock, group uint32, capacity int) error

CreateFullWorker inserts a commitWorker for the provided file/group with a tasks channel of the given capacity and pre-fills it so enqueue attempts will hit the queue-full fallback path. Returns an error if a worker already exists for the computed key.

func CsumSeed 

func CsumSeed(sb *Superblock) uint32

func DecrementAndFreeInode 

func DecrementAndFreeInode(f ReaderWriterAt, fsOffset int64, sb *Superblock, in *Inode) error

DecrementAndFreeInode exposes decrementAndFreeInode.

func DirBlockForPath 

func DirBlockForPath(t *testing.T, rw ReaderWriterAt, sb *Superblock, path string) uint64

DirBlockForPath returns the first data block for the given path.

func DumpBlockingFallbacks 

func DumpBlockingFallbacks()

func DumpCommitTrace 

func DumpCommitTrace()

func DumpDebugLog 

func DumpDebugLog(w io.Writer)

DumpDebugLog writes the accumulated debug buffer to the provided writer.

func EncodeBGD 

func EncodeBGD(d *Bgd, sb *Superblock) []byte

EncodeBGD encodes a decoded BGD into its raw descriptor bytes and returns the raw byte slice for inspection by external tests.

func EnqueueRawCommitOps 

func EnqueueRawCommitOps(f ReaderWriterAt, fsOffset int64, sb *Superblock, group uint32, starts []int64, datas [][]byte) (chan error, uint64, error)

EnqueueRawCommitOps exposes a thin wrapper allowing tests to enqueue a multi-op commit where each op is specified by a start offset (relative to fsOffset) and a data slice. This is useful for constructing a single commit that contains both a bitmap write and a BGD write so the worker's `lastBitmap` optimization can be exercised in tests.

func FindFreeBit 

func FindFreeBit(b []byte, maxBits int) (int, bool)

func FindFreeRun 

func FindFreeRun(b []byte, maxBits, n int) ([]int, bool)

func Format 

func Format(path string, sizeBytes int64, cfg FormatConfig) (filesystem.Filesystem, error)

Format creates a new ext4 filesystem in the file at path. The file is created (or truncated) and formatted. sizeBytes must be a multiple of 4096 and large enough to hold filesystem metadata plus at least one data block.

On success the newly formatted filesystem is opened and returned; the caller must Close it when done.

func FreeBlock 

func FreeBlock(f ReaderWriterAt, fsOffset int64, sb *Superblock, block uint64) error

FreeBlock exposes freeBlock.

func FreeInodeBlocks 

func FreeInodeBlocks(f ReaderWriterAt, fsOffset int64, sb *Superblock, in *Inode) error

FreeInodeBlocks exposes freeInodeBlocks.

func FreeInodeSlot 

func FreeInodeSlot(f ReaderWriterAt, fsOffset int64, sb *Superblock, inodeNum uint32) error

FreeInodeSlot exposes freeInodeSlot.

func GetCommitWorkerIdleSecs 

func GetCommitWorkerIdleSecs() int

Get/Set commit worker idle duration (seconds) for tests.

func GetSeqForAck 

func GetSeqForAck(ack chan error) (uint64, bool)

Test-only helpers to observe and control ack->seq mapping behavior. GetSeqForAck returns the registered seq for the provided ack channel and a boolean indicating whether the mapping exists.

func HookMemFileBuf 

func HookMemFileBuf(h *HookMemFile) []byte

HookMemFileBuf returns the underlying buffer for tests that need to mutate or inspect the raw bytes inside a HookMemFile.

func InodeNum 

func InodeNum(in *Inode) uint32

func InodeOffsetFor 

func InodeOffsetFor(r ReaderWriterAt, sb *Superblock, inodeNum uint32) int64

InodeOffsetFor computes inode byte offset (helper for tests).

func InodeRaw 

func InodeRaw(in *Inode) []byte

func InodeSizeVal 

func InodeSizeVal(in *Inode) uint64

func IsDir 

func IsDir(in *Inode) bool

func IsRegular 

func IsRegular(in *Inode) bool

func IsSeqAcked 

func IsSeqAcked(seq uint64) bool
func IsSymlink(in *Inode) bool

func MakeDir 

func MakeDir(f ReaderWriterAt, fsOffset int64, sb *Superblock, path string, perm os.FileMode) error

MakeDir exposes the internal makeDir helper for external tests.

func MarkSeqAcked 

func MarkSeqAcked(seq uint64)

Test helpers for commit dispatcher internals.

func MinDirentSize 

func MinDirentSize(nameLen int) int

Dirent helpers.

func NewOwner 

func NewOwner() uint64

NewOwner returns a new unique owner token for locking purposes.

func NumBlockGroups 

func NumBlockGroups(sb *Superblock) uint32

Exported superblock convenience accessors for external tests.

func Open 

func Open(imagePath string, partIndex int) (filesystem.Filesystem, error)

Open opens an ext4 filesystem image at imagePath, automatically detecting the partition table (MBR / GPT) and using the first Linux partition. Pass partIndex = -1 for auto-detection (first Linux partition).

func OpenFromDevice 

func OpenFromDevice(dev blockDevice, partIndex int) (filesystem.Filesystem, error)

OpenFromDevice opens an ext4 filesystem backed by an arbitrary block device. dev must remain valid until the returned Filesystem is closed.

func OwnerFrom 

func OwnerFrom(ctx context.Context) (uint64, bool)

OwnerFrom extracts an owner token from context.

func PartitionOffset 

func PartitionOffset(r io.ReaderAt, partIndex int) (int64, error)

func ReadBitmap 

func ReadBitmap(f ReaderWriterAt, fsOffset int64, sb *Superblock, bitmapBlock uint64) ([]byte, error)

ReadBitmap exposes readBitmap.

func ReadFileData 

func ReadFileData(f ReaderWriterAt, fsOffset int64, sb *Superblock, in *Inode) ([]byte, error)

ReadFileData exposes readFileData.

func ReadRawBlock 

func ReadRawBlock(f ReaderWriterAt, fsOffset int64, sb *Superblock, blockNum uint64) ([]byte, error)

ReadRawBlock exposes readRawBlock. 

func ReadSymlink(f ReaderWriterAt, fsOffset int64, sb *Superblock, in *Inode) (string, error)

ReadSymlink exposes readSymlink.

func RemoveDir 

func RemoveDir(f ReaderWriterAt, fsOffset int64, sb *Superblock, path string) error

RemoveDir exposes removeDir for tests that operate on low-level images.

func RemoveFile 

func RemoveFile(f ReaderWriterAt, fsOffset int64, sb *Superblock, path string) error

RemoveFile exposes removeFile for tests that operate on low-level images.

func RemoveWorkerFor 

func RemoveWorkerFor(f ReaderWriterAt, fsOffset int64, sb *Superblock, group uint32)

RemoveWorkerFor removes any injected worker for the given file/group.

func Rename 

func Rename(f ReaderWriterAt, fsOffset int64, sb *Superblock, oldPath, newPath string) error

Rename exposes the internal rename helper for external tests.

func RootDirBlockFor 

func RootDirBlockFor(t *testing.T, rw ReaderWriterAt, sb *Superblock) uint64

RootDirBlockFor returns the first data block for the root directory.

func SetAckToSeqDeleteGraceMS 

func SetAckToSeqDeleteGraceMS(ms int)

SetAckToSeqDeleteGraceMS sets the test-only ack->seq deletion grace duration in milliseconds. Useful for exercising TTL behavior in tests.

func SetAddDirEntryAllocBlocks 

func SetAddDirEntryAllocBlocks(fn func(ReaderWriterAt, int64, *Superblock, uint32) ([]uint64, error)) func(ReaderWriterAt, int64, *Superblock, uint32) ([]uint64, error)

Setters for addDirEntry package-level hooks so external tests can inject failures or behavior.

func SetAddDirEntrySetInlineExtents 

func SetAddDirEntrySetInlineExtents(fn func(*Inode, []extentLeaf) error) func(*Inode, []extentLeaf) error

func SetAddDirEntryWriteBlock 

func SetAddDirEntryWriteBlock(fn func(ReaderWriterAt, int64, *Superblock, uint64, []byte) error) func(ReaderWriterAt, int64, *Superblock, uint64, []byte) error

func SetBit 

func SetBit(b []byte, i int)

Bitmap/bit helpers.

func SetCommitWorkerIdleSecs 

func SetCommitWorkerIdleSecs(s int)

func SetFormatOpenFS 

func SetFormatOpenFS(fn func(string, int) (filesystem.Filesystem, error)) func(string, int) (filesystem.Filesystem, error)

func SetFormatOpenFile 

func SetFormatOpenFile(fn func(string) (FormatFile, error)) func(string) (FormatFile, error)

Format hook setters - allow external tests to inject file/open/rand behavior.

func SetFormatRandRead 

func SetFormatRandRead(fn func([]byte) (int, error)) func([]byte) (int, error)

func SetInlineExtents 

func SetInlineExtents(in *Inode, exts []ExtentLeaf) error

SetInlineExtents sets inline extents on an inode (wrapped method).

func SetMakeDirAllocBlocks 

func SetMakeDirAllocBlocks(fn func(ReaderWriterAt, int64, *Superblock, uint32) ([]uint64, error)) func(ReaderWriterAt, int64, *Superblock, uint32) ([]uint64, error)

func SetMakeDirAllocInode 

func SetMakeDirAllocInode(fn func(ReaderWriterAt, int64, *Superblock, bool) (uint32, error)) func(ReaderWriterAt, int64, *Superblock, bool) (uint32, error)

func SetMode 

func SetMode(in *Inode, mode uint16, links uint16)

func SetReadLinkReadInode 

func SetReadLinkReadInode(fn func(ReaderWriterAt, int64, *Superblock, uint32) (*Inode, error)) func(ReaderWriterAt, int64, *Superblock, uint32) (*Inode, error)

func SetRemoveDirChildDir 

func SetRemoveDirChildDir(fn func(ReaderWriterAt, int64, *Superblock, string) error) func(ReaderWriterAt, int64, *Superblock, string) error

Setter helpers for package-level hooks used by tests.

func SetRemoveDirChildFile 

func SetRemoveDirChildFile(fn func(ReaderWriterAt, int64, *Superblock, string) error) func(ReaderWriterAt, int64, *Superblock, string) error

func SetRemoveDirFreeBlocks 

func SetRemoveDirFreeBlocks(fn func(ReaderWriterAt, int64, *Superblock, *Inode) error) func(ReaderWriterAt, int64, *Superblock, *Inode) error

func SetRemoveDirFreeSlot 

func SetRemoveDirFreeSlot(fn func(ReaderWriterAt, int64, *Superblock, uint32) error) func(ReaderWriterAt, int64, *Superblock, uint32) error

func SetSize 

func SetSize(in *Inode, size uint64)

func SetWriteFileSetInlineExtents 

func SetWriteFileSetInlineExtents(fn func(*Inode, []extentLeaf) error) func(*Inode, []extentLeaf) error

func TryInsertDirEntry 

func TryInsertDirEntry(buf []byte, childIno uint32, name string, fileType uint8, needed int) bool

func UpdateDirBlockCsum 

func UpdateDirBlockCsum(buf []byte, sb *Superblock, dir *Inode)

UpdateDirBlockCsum exposes updateDirBlockCsum for tests.

func UpdateDotDot 

func UpdateDotDot(f ReaderWriterAt, fsOffset int64, sb *Superblock, dirIno *Inode, newParentIno uint32) error

UpdateDotDot exposes the internal updateDotDot helper for tests that need to exercise updating '..' directory entries.

func WithOwner 

func WithOwner(ctx context.Context) (context.Context, uint64)

WithOwner returns a new context that carries a unique owner token and the token itself. Useful for propagating an ownership token through call chains.

func WorkerExistsFor 

func WorkerExistsFor(f ReaderWriterAt, fsOffset int64, sb *Superblock, group uint32) bool

WorkerExistsFor returns true if a worker is registered for the file/group.

func WriteBGD 

func WriteBGD(f ReaderWriterAt, fsOffset int64, sb *Superblock, g uint32, d *Bgd) error

WriteBGD exposes writeBGD.

func WriteBitmapBuf 

func WriteBitmapBuf(f ReaderWriterAt, fsOffset int64, sb *Superblock, g uint32, d *Bgd, isBlock bool, bitmapBlock uint64, bmap []byte) error

WriteBitmapBuf exposes writeBitmapBuf for low-level bitmap testing.

func WriteBitmapWithCsum 

func WriteBitmapWithCsum(f ReaderWriterAt, fsOffset int64, sb *Superblock, g uint32, d *Bgd, isBlockBitmap bool) error

WriteBitmapWithCsum exposes writeBitmapWithCsum.

func WriteDirEntry 

func WriteDirEntry(buf []byte, off int, ino uint32, recLen uint16, name string, fileType uint8)

func WriteFileRaw 

func WriteFileRaw(f ReaderWriterAt, fsOffset int64, sb *Superblock, path string, data []byte, perm os.FileMode) error

WriteFileRaw exposes the internal writeFile helper used by package tests.

func WriteInode 

func WriteInode(f ReaderWriterAt, fsOffset int64, sb *Superblock, in *Inode) error

WriteInode exposes writeInode.

func WriteRawBlock 

func WriteRawBlock(f ReaderWriterAt, fsOffset int64, sb *Superblock, blockNum uint64, data []byte) error

WriteRawBlock exposes writeRawBlock.

func WriteSuperblock 

func WriteSuperblock(f ReaderWriterAt, fsOffset int64, sb *Superblock) error

func ZeroDirEntry 

func ZeroDirEntry(f ReaderWriterAt, fsOffset int64, sb *Superblock, dirIno *Inode, name string) error

ZeroDirEntry exposes zeroDirEntry.

func ZeroEntryInBlock 

func ZeroEntryInBlock(buf []byte, name string, le binary.ByteOrder) bool

ZeroEntryInBlock exposes zeroEntryInBlock.

Types

type Bgd 

type Bgd = bgd

func DecodeBGD 

func DecodeBGD(raw []byte, sb *Superblock) *Bgd

DecodeBGD exposes decodeBGD.

func ReadBGD 

func ReadBGD(f ReaderWriterAt, fsOffset int64, sb *Superblock, g uint32) (*Bgd, error)

ReadBGD exposes readBGD.

type BlockDevice 

type BlockDevice = blockDevice

BlockDevice is the exported alias of blockDevice, lets external packages satisfy the interface and pass instances to OpenFromDevice. Without it, OpenFromDevice's signature `(dev blockDevice, …)` references an unexported type — only code inside this package could call it. Use cases for implementing BlockDevice externally:

  • LUKS / dm-crypt: wrap a github.com/go-fde/luks.Device so ext4.OpenFromDevice reads the plaintext payload.
  • In-memory testing: feed a *bytes.Reader-backed fixture into the FS without writing a tempfile.
  • qcow2 / DMG / other image formats from the go-diskimages family: surface the unpacked block view as a BlockDevice.

type DirEntry 

type DirEntry struct {
	Inode    uint32
	Name     string
	FileType uint8
}

DirEntry is a parsed directory entry.

func ParseDirBlock 

func ParseDirBlock(buf []byte) []DirEntry

func ReadDir 

func ReadDir(f ReaderWriterAt, fsOffset int64, sb *Superblock, dirIno *Inode) ([]DirEntry, error)

ReadDir exposes readDir for tests that need to enumerate directory entries from a single inode.

type Ext4FS 

type Ext4FS = ext4FS

Expose the concrete FS type as an exported alias for tests.

func NewTempFS 

func NewTempFS(t *testing.T) (*Ext4FS, func())

NewTempFS creates and returns a temporary formatted ext4 filesystem for tests. The returned cleanup function should be deferred by the caller.

func NewTempFSWithSize 

func NewTempFSWithSize(t *testing.T, sizeBytes int64) (*Ext4FS, func())

NewTempFSWithSize creates and returns a temporary formatted ext4 filesystem for tests with a custom image size in bytes. The returned cleanup function should be deferred by the caller.

type ExtentLeaf 

type ExtentLeaf = extentLeaf

func BuildExtents 

func BuildExtents(phys []uint64) []ExtentLeaf

BuildExtents converts phys block list into exported ExtentLeaf slice.

func ParseExtentNode 

func ParseExtentNode(f ReaderWriterAt, fsOffset int64, sb *Superblock, buf []byte, inodeNum uint32, inodeRaw []byte) ([]ExtentLeaf, error)

ParseExtentNode exposes parseExtentNode.

type FS 

type FS = ext4FS

FS is a public alias for the concrete ext4 filesystem type. Used by downstream packages (apple-vz/grub) that need to refer to the concrete type for method receivers. The Ext4FS alias (same target) is gated behind `//go:build test` for test-only helpers; FS is always available so production code can hold a typed pointer.

type FormatConfig 

type FormatConfig struct {
	// UUID is the filesystem UUID. A random v4 UUID is generated when all bytes are zero.
	UUID [16]byte
	// Label is the volume label stored in the superblock (trimmed to 16 bytes).
	Label string
}

FormatConfig holds optional parameters for Format. All fields are optional; sensible defaults are used when left at their zero value.

type FormatFile 

type FormatFile = formatFile

FormatFile is an alias to the internal formatFile interface used by Format.

type HookMemFile 

type HookMemFile struct {

	// Exported hook fields are convenient for external tests.
	ReadHook  func(off int64, p []byte) error
	WriteHook func(off int64, p []byte) error
	// contains filtered or unexported fields
}

HookMemFile is a small in-memory reader/writer used by tests.

func CloneFSImage 

func CloneFSImage(t *testing.T, fs *Ext4FS) *HookMemFile

CloneFSImage reads the backing file for an open FS into a HookMemFile copy.

func NewHookMemFile 

func NewHookMemFile(buf []byte) *HookMemFile

NewHookMemFile returns a HookMemFile backed by the provided buffer. This is a convenience for external tests that need a HookMemFile with an initial backing store.

func (*HookMemFile) ReadAt 

func (m *HookMemFile) ReadAt(p []byte, off int64) (int, error)

func (*HookMemFile) WriteAt 

func (m *HookMemFile) WriteAt(p []byte, off int64) (int, error)

type Inode 

type Inode = inode

func LookupParent 

func LookupParent(f ReaderWriterAt, fsOffset int64, sb *Superblock, path string) (*Inode, string, error)

LookupParent exposes lookupParent for tests validating parent resolution.

func LookupPath 

func LookupPath(f ReaderWriterAt, fsOffset int64, sb *Superblock, path string) (*Inode, error)

LookupPath exposes lookupPath.

func NewTestInode 

func NewTestInode(num uint32, inodeSize uint16) *Inode

Inode helpers and accessors for tests.

func ReadInode 

func ReadInode(f ReaderWriterAt, fsOffset int64, sb *Superblock, inodeNum uint32) (*Inode, error)

ReadInode exposes readInode.

type Journal 

type Journal struct {
	// contains filtered or unexported fields
}

Journal represents an on-disk sidecar journal attached to a filesystem image. When enabled it persists transactions to the sidecar and can replay committed transactions at Open time.

func OpenJournal 

func OpenJournal(f readerWriterAt, fsOffset int64, blockSize int, sb *superblock) (*Journal, error)

OpenJournal opens/creates a sidecar journal for the provided backing reader/writer. It returns a Journal that is enabled when a sidecar file can be created; otherwise it returns a disabled journal (no-op).

func (*Journal) Close 

func (j *Journal) Close() error

Close cleanly shuts down the journal including sync workers and closes the underlying journal file. It also unregisters the journal.

Each Transaction.Commit applies its entries to the backing image synchronously before returning, so at Close all committed entries are already persisted on disk. Truncate the journal file so the next Open does not replay a growing log of already-applied transactions (which would otherwise turn ReplayOnOpen into an O(N) tax on every reopen).

func (*Journal) ReplayOnOpen 

func (j *Journal) ReplayOnOpen() error

ReplayOnOpen scans the sidecar journal for committed transactions and applies them to the backing image. Partial or uncommitted transactions are ignored.

func (*Journal) StartTx 

func (j *Journal) StartTx() (*Transaction, error)

StartTx creates a new transaction associated with the journal.

type ReaderWriterAt 

type ReaderWriterAt = readerWriterAt

Exported aliases to internal types so test packages can refer to them

type Stat 

type Stat struct {
	Mode  uint16 // Unix permission bits
	Size  uint64
	Inode uint32
}

Stat holds basic metadata about a filesystem path.

type Superblock 

type Superblock = superblock

func CloneSuperblock 

func CloneSuperblock(sb *Superblock) *Superblock

CloneSuperblock returns a deep copy of a superblock.

func CloneSuperblockFromFS 

func CloneSuperblockFromFS(fs *Ext4FS) *Superblock

CloneSuperblockFromFS returns a copy of the open filesystem's superblock.

func NewTestSuperblock 

func NewTestSuperblock(inodesPerGroup, blocksPerGroup, numGroups uint32, blockSize uint32) *Superblock

NewTestSuperblock builds an in-memory superblock for tests.

func ReadSuperblock 

func ReadSuperblock(f ReaderWriterAt, fsOffset int64) (*Superblock, error)

Superblock helpers.

type Transaction 

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction collects metadata blocks to be journaled and committed.

func (*Transaction) Abort 

func (tx *Transaction) Abort() error

Abort cancels the transaction. For this implementation this is a no-op unless the transaction has been committed.

func (*Transaction) AddBlock 

func (tx *Transaction) AddBlock(blockNum uint64, data []byte) error

AddBlock appends a metadata block to the transaction. The data is copied. AddBlock adds or updates the entry for blockNum in the transaction. If an entry for blockNum already exists it is replaced so each block appears at most once in the journal record. Callers that want to build on a previous in-tx modification of the same block must first call GetBlock to retrieve the current in-tx state before overlaying their changes and calling AddBlock.

func (*Transaction) AddCommitCallback 

func (tx *Transaction) AddCommitCallback(cb func(uint64)) error

AddCommitCallback registers a callback to be invoked when this transaction is applied. The callback receives the transaction sequence number. It is an error to register callbacks after the transaction has been committed.

func (*Transaction) Commit 

func (tx *Transaction) Commit() error

Commit writes the transaction to the sidecar journal and then applies the blocks to the backing filesystem image (write-through). This provides a straightforward durability model and simplifies replay semantics.

func (*Transaction) GetBlock 

func (tx *Transaction) GetBlock(blockNum uint64) []byte

GetBlock returns the current in-transaction data for blockNum, or nil if the block has not been prepared into this transaction yet. The returned slice is a copy and safe to modify.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.