filesystem_zfs

README

zfs

A read/write ZFS implementation for bare disk images, supporting a single pool with a single ZPL dataset. Designed for embedded tooling that needs to create, inspect, and modify ZFS filesystems programmatically.

Support summary

Feature	Status	Notes
Open / Close	✅	Opens single-pool images
Format	✅	Creates new pool images via Format
ReadFile / WriteFile	✅	Basic file I/O supported (ZPL dataset)
MkDir / Delete / Rename	✅	Directory operations supported
Snapshots / Clones	⚠️ No	Not implemented (test-oriented subset)
Compression / Checksums	⚠️ No	Not implemented on data blocks

Module

github.com/go-filesystems/zfs

API

Opening / creating

// Open opens an existing ZFS image. partIndex=-1 uses the whole image.
func Open(imagePath string, partIndex int) (*FS, error)

// Format creates a new ZFS image of sizeBytes at path and opens it.
func Format(path string, sizeBytes int64, cfg FormatConfig) (*FS, error)

Metadata

func (fs *FS) Info() Info   // pool name, GUID, version, TXG, timestamp

File operations

func (fs *FS) ReadFile(path string) ([]byte, error)
func (fs *FS) WriteFile(path string, data []byte, perm os.FileMode) error
func (fs *FS) DeleteFile(path string) error

Directory operations

func (fs *FS) ListDir(path string) ([]filesystem.DirEntry, error)
func (fs *FS) MkDir(path string, perm os.FileMode) error
func (fs *FS) DeleteDir(path string) error

Rename

func (fs *FS) Rename(oldPath, newPath string) error

Closing

func (fs *FS) Close() error

Implements

This package implements the filesystem.Filesystem interface defined in github.com/go-filesystems/interface. Example usage:

import (
	filesystem "github.com/go-filesystems/interface"
	fsz "github.com/go-filesystems/zfs"
)

f, _ := fsz.Open("pool.img", -1)
defer f.Close()
var fs filesystem.Filesystem = f
_, _ = fs.ReadFile("/hello.txt")

Image layout

Offset	Size	Content
0	256 KiB	Vdev label L0
256 KiB	256 KiB	Vdev label L1
512 KiB	varies	Pool data (MOS, ZPL objset, object arrays, ZAP blocks)
end−512 KiB	256 KiB	Vdev label L2
end−256 KiB	256 KiB	Vdev label L3

Pool data starts at offset 0x080000. The ZPL object array has 32 slots (objects 0–31), giving a maximum of 28 user files/directories per pool image.

Supported ZAP type

Only micro-ZAP is supported for directory writes. Directory entries use a 4 KiB block with 63 name slots of up to 49 bytes each.

Limitations

• Single vdev, single pool, single dataset
• No compression or checksums on data blocks
• No snapshots, clones, or ACLs
• Maximum 28 objects (files + directories) per pool image
• Directory names limited to 49 bytes

Test coverage

100% statement coverage.

Documentation

Index

• Constants
• type BlockBackend
• type DSLCryptoKey
• type FS
  • func Format(path string, sizeBytes int64, cfg FormatConfig) (FS, error)
  • func Open(imagePath string, partIndex int) (FS, error)
  • func OpenDataset(imagePath string, partIndex int, datasetPath string) (FS, error)
  • func OpenFromDevice(dev BlockBackend, partIndex int) (FS, error)
  • func OpenFromDeviceDataset(dev BlockBackend, partIndex int, datasetPath string) (FS, error)
  • func OpenFromDeviceDatasetWithKey(dev BlockBackend, partIndex int, datasetPath string, ...) (FS, error)
  • func OpenFromDevices(devs []BlockBackend, partIndex int, datasetPath string) (FS, error)
  • func OpenSnapshot(imagePath string, partIndex int, datasetPath, snapName string) (FS, error)
• type FormatConfig
• type Info
  • func (info Info) LabelOffset(partOffset int64) int64
  • func (info Info) TimestampUnix() uint64
  • func (info Info) UberblockRegionOffset(partOffset int64) int64
• type LabelInfo
  • func ProbeLabel(r io.ReaderAt, partOff int64) (*LabelInfo, error)
• type ShrinkMode
  • func (m ShrinkMode) String() string

Constants

const (
	// DSLMasterKeyMaxLen is the length of the wrapped master encryption
	// key blob (matches MASTER_KEY_MAX_LEN in OpenZFS).
	DSLMasterKeyMaxLen = 32
	// DSLHMACKeyMaxLen is the length of the wrapped HMAC key blob.
	DSLHMACKeyMaxLen = 32
	// DSLWrappingIVLen is the length of the wrap-time IV (13 bytes —
	// OpenZFS uses a 13-byte IV for the unwrap AEAD, distinct from the
	// 12-byte per-block IVs).
	DSLWrappingIVLen = 13
	// DSLWrappingMACLen is the length of the wrap-time MAC tag.
	DSLWrappingMACLen = 16
	// DSLSaltLen is the length of the PBKDF2 salt.
	DSLSaltLen = 8
	// DSLCryptoKeyPhysSize is the on-wire packed size of a
	// `dsl_crypto_key_phys`-style record:
	//   suite(8) + guid(8) + version(8) + iters(8) +
	//   iv(13) + pad(3) + mac(16) +
	//   wrapped MEK(32) + wrapped HMAC(32) + salt(8) = 136 bytes.
	DSLCryptoKeyPhysSize = 8 + 8 + 8 + 8 + 13 + 3 + 16 + DSLMasterKeyMaxLen + DSLHMACKeyMaxLen + DSLSaltLen
)

Types

type BlockBackend

type BlockBackend = blockBackend

BlockBackend is the exported alias of blockBackend — lets external packages satisfy the interface and pass instances to OpenFromDevice / OpenFromDeviceDataset.

type DSLCryptoKey

type DSLCryptoKey struct {
	// Suite is the data-encryption AEAD chosen for this dataset.
	Suite zfscrypt.Suite
	// GUID is the dataset key's unique identifier, used as part of the
	// unwrap AAD so a wrapped blob can't be silently swapped between
	// datasets.
	GUID uint64
	// Version is the on-disk format version of the DSL_CRYPTO_KEY
	// record (OpenZFS shipped version 0 historically; later format
	// changes bumped this).
	Version uint64
	// Iters is the PBKDF2-HMAC-SHA1 iteration count used to derive the
	// wrapping key from a passphrase. Zero means a raw key was supplied
	// directly.
	Iters uint64
	// IV is the 13-byte wrap-time IV passed to AES-CCM/GCM during the
	// MEK unwrap step.
	IV []byte
	// MAC is the 16-byte authentication tag emitted by the wrap step.
	MAC []byte
	// WrappedMasterKey is the AEAD ciphertext of the master encryption
	// key, exactly DSLMasterKeyMaxLen bytes.
	WrappedMasterKey []byte
	// WrappedHMACKey is the AEAD ciphertext of the per-dataset HMAC
	// key, exactly DSLHMACKeyMaxLen bytes.
	WrappedHMACKey []byte
	// Salt is the PBKDF2 salt; meaningful only when Iters > 0.
	Salt []byte
}

DSLCryptoKey is the parsed form of a ZFS dataset's DSL_CRYPTO_KEY object. It carries everything the unwrap step needs: the AEAD suite, the wrap-time IV+MAC, the wrapped (MEK||HMAC) blob, and the PBKDF2 salt+iters used to derive the wrapping key from a passphrase.

Field sizes are validated at parse time; downstream code (Unwrap, DeriveWrappingKey) can therefore assume they are correct.

type FS

type FS interface {
	filesystem.Filesystem
	Info() Info
	PartitionOffset() int64
	GrowTo(newSizeBytes int64) error
	Grow(newSizeBytes int64) error
	Resize(newSize int64) error
	Shrink(newSize int64) error
	ShrinkWithMode(newSize int64, mode ShrinkMode) error
	// Snapshot creates a frozen, isolated snapshot of the currently-open
	// dataset. The snapshot is unaffected by later writes to the live
	// dataset and is readable via OpenSnapshot. See snapshot.go.
	Snapshot(snapName string) error
}

FS is the public interface returned by Open. Extends filesystem.Filesystem with ZFS-specific operations (Info, PartitionOffset, GrowTo / Grow / Resize / Shrink / ShrinkWithMode).

GrowTo, Grow are the grow-only entry points; they reject shrink targets with a wrapped filesystem.ErrShrinkUnsupported so callers branching on grow-only contracts still see the sentinel. Resize is bidirectional — newSize > current routes to Grow, newSize < current routes to Shrink (in Auto mode). Shrink and ShrinkWithMode expose the shrink path with explicit mode control (see ShrinkMode in resize.go for the Rebuild / InPlace / Auto contract).

func Format

func Format(path string, sizeBytes int64, cfg FormatConfig) (FS, error)

Format creates a new empty ZFS pool in the file at path. sizeBytes must be at least 4 MiB. On success, the newly created pool is opened and returned; the caller must call Close when done.

func Open

func Open(imagePath string, partIndex int) (FS, error)

Open opens imagePath, optionally selecting a partition, and scans for the freshest uberblock. If the pool contains a valid ZPL object set it is opened; otherwise only uberblock metadata is available and read/write operations will return errors.

func OpenDataset

func OpenDataset(imagePath string, partIndex int, datasetPath string) (FS, error)

OpenDataset is like Open but navigates to a NESTED dataset rather than the pool's root DSL dir. datasetPath is the dataset name minus the pool prefix — e.g. for "rpool/ROOT/pve-1", pass "ROOT/pve-1". The pool prefix is implicit (every pool has exactly one root_dataset and we always start there).

Use case: chaining into a Proxmox VE or Ubuntu Server ZSYS install whose / lives on a non-root dataset. The pool root itself is typically empty in those layouts; the bootloader needs the leaf dataset's filesystem to find /boot/vmlinuz.

datasetPath="" is equivalent to Open() — the pool root dataset is opened.

func OpenFromDevice

func OpenFromDevice(dev BlockBackend, partIndex int) (FS, error)

OpenFromDevice opens a ZFS pool backed by an arbitrary blockBackend (LUKS plaintext, qcow2 unpacked view, in-memory fixture, …) and lands on the pool's root dataset.

partIndex is honoured the same way as Open: -1 for whole-image mode, >= 0 to select a partition from a GPT/MBR-partitioned backing store.

func OpenFromDeviceDataset

func OpenFromDeviceDataset(dev BlockBackend, partIndex int, datasetPath string) (FS, error)

OpenFromDeviceDataset is OpenFromDevice + dataset navigation. Use to open "<pool>/<dataset>/<…>" against a layered block source; datasetPath has the same semantics as OpenDataset (pool name implicit, "" for the pool root).

func OpenFromDeviceDatasetWithKey

func OpenFromDeviceDatasetWithKey(dev BlockBackend, partIndex int, datasetPath string, wrappingKeyOrPassphrase []byte) (FS, error)

OpenFromDeviceDatasetWithKey is the encryption-aware twin of OpenFromDeviceDataset. wrappingKeyOrPassphrase is either:

• a 32-byte raw wrapping key (already derived by the caller via zfscrypt.DeriveWrappingKey, or supplied by an external key store), or
• a passphrase shorter or longer than 32 bytes, in which case this function derives the wrapping key on the fly using the salt + iter count stored in the dataset's DSL_CRYPTO_KEY object.

On success the returned FS reads encrypted blocks transparently — every cleartext consumer (Stat / ListDir / ReadFile / …) works unchanged.

Status: crypto primitives (github.com/go-encryptions/ccm + .../zfscrypt) and the DSL_CRYPTO_KEY on-disk parser (parseDSLCryptoKeyPhys / parseDSLCryptoKeyFromZAP) are in place. The remaining piece is the dataset-walker that locates the DSL_CRYPTO_KEY object for a given dataset and feeds its bytes through the parser; until that lands loadCryptKey surfaces a clear "locator not wired" error so callers don't silently get an undecrypted FS.

func OpenFromDevices

func OpenFromDevices(devs []BlockBackend, partIndex int, datasetPath string) (FS, error)

OpenFromDevices opens a multi-vdev pool. The first device is the "primary" — its label 0 is read to discover the vdev tree (mirror / raidz / disk) and the GUIDs of every leaf in declaration order. `devs` must contain one backend per leaf, in the SAME id order as the on-disk vdev_tree.children array (= zpool-create argument order). Mismatches are detected via dev_item.guid check.

For a SINGLE-vdev pool the slice has 1 element and the call is equivalent to OpenFromDeviceDataset. For mirror / raidz the additional legs are required: mirror uses any leg, raidz needs ALL data legs present for healthy reads (one missing leg falls back to the parity reconstruction path, which is not yet implemented — see memory:userland-fs-drivers).

func OpenSnapshot

func OpenSnapshot(imagePath string, partIndex int, datasetPath, snapName string) (FS, error)

OpenSnapshot opens a snapshot of a dataset for reading. datasetPath has the same semantics as OpenDataset (pool prefix implicit, "" for the pool root); snapName is the snapshot name created by FS.Snapshot. The returned FS is a read-only view of the frozen snapshot — write operations are not meaningful against a snapshot and are not supported.

Equivalent to OpenDataset(imagePath, partIndex, datasetPath+"@"+snapName).

type FormatConfig

type FormatConfig struct {
	// PoolName is stored in the vdev label nvlist; defaults to "data".
	PoolName string
	// PoolGUID is the 64-bit pool GUID; a time-derived value is used when zero.
	PoolGUID uint64
}

FormatConfig holds optional parameters for Format.

type Info

type Info struct {
	Version          uint64
	TransactionGroup uint64
	GUIDSum          uint64
	RawTimestamp     uint64
	Timestamp        time.Time
	Label            int
	Slot             int
	Offset           int64
	Endian           string
}

Info holds the fields decoded from a ZFS uberblock.

func (Info) LabelOffset

func (info Info) LabelOffset(partOffset int64) int64

LabelOffset returns the absolute offset of the label that contained the uberblock.

func (Info) TimestampUnix

func (info Info) TimestampUnix() uint64

TimestampUnix returns the raw uberblock timestamp in seconds since the Unix epoch.

func (Info) UberblockRegionOffset

func (info Info) UberblockRegionOffset(partOffset int64) int64

UberblockRegionOffset returns the absolute offset of the uberblock ring.

type LabelInfo

type LabelInfo struct {
	PoolName     string
	PoolGUID     uint64
	TopGUID      uint64 // GUID of the top-level vdev (mirror/raidz parent)
	ThisGUID     uint64 // GUID of THIS device (this leaf)
	VdevChildren uint64 // number of top-level vdevs in the pool
	Type         string // "file"/"disk" for single-vdev, "mirror"/"raidz"/... for multi-vdev
	NParity      uint64 // raidz nparity (0 for non-raidz)
	Ashift       uint64
	// LeafGUIDs is the ordered list of guids from vdev_tree.children
	// (matching vdev id order, which is the order OpenFromDevices
	// requires for its devs slice). For single-vdev pools the slice
	// is empty (the pool root IS a leaf).
	LeafGUIDs []uint64
}

LabelInfo is the subset of a vdev label's top-level NVList that cloud-boot needs to assemble a multi-vdev pool: pool identity, this-leaf identity, top-vdev topology, and the leaf-guid list of the top vdev's children in vdev-id order.

func ProbeLabel

func ProbeLabel(r io.ReaderAt, partOff int64) (*LabelInfo, error)

ProbeLabel reads label 0 from `r` (positioned at partition start) and decodes the subset of fields LabelInfo describes. No data blocks are read; this is purely a label/NVList parse and is safe to call against random devices (returns an error for non-ZFS media without side effects).

type ShrinkMode

type ShrinkMode int

ShrinkMode selects the on-disk relocation strategy used by Shrink. See the package comment in resize.go for the full per-mode contract.

const (
	// ShrinkMode_Auto picks InPlace when the pool is snapshot-free (the
	// only state our writer ever produces) and falls back to Rebuild
	// otherwise. Resize() always uses Auto.
	ShrinkMode_Auto ShrinkMode = iota
	// ShrinkMode_Rebuild always runs the rebuild path. Use this when
	// the caller wants the deterministic, fully-replayed image even on
	// a pool that the InPlace path could have handled.
	ShrinkMode_Rebuild
	// ShrinkMode_InPlace always runs the in-place relocation path and
	// errors out if any snapshot is present. The check is conservative:
	// any non-zero ds_prev_snap_obj / ds_next_snap_obj on the head
	// dataset rejects the operation.
	ShrinkMode_InPlace
)

func (ShrinkMode) String

func (m ShrinkMode) String() string