detect

README

detect

Pure-Go (CGO=0), blkid-style filesystem-type prober and driver-opener registry for the go-filesystems family — no root, no external tools, no libblkid.

detect probes on-disk magic signatures to identify the filesystem on an image, then dispatches to a registered driver to open it. It mirrors the standard library's image.Decode design: the core package depends only on github.com/go-filesystems/interface plus the standard library. It never imports a concrete driver, so it composes cleanly as a meta-importer — the drivers' replace github.com/go-filesystems/interface => ../interface sibling directive does not survive transitive importing, which is exactly the problem this split avoids.

Drivers are wired in at the call site, either directly via Register or by blank-importing a thin per-driver adapter sub-package.

Install

go get github.com/go-filesystems/detect

Usage

Probe only

import "github.com/go-filesystems/detect"

t, err := detect.Detect(r, size) // r is an io.ReaderAt, size in bytes (-1 if unknown)
if err == nil {
    fmt.Println("filesystem:", t) // e.g. "fat32"
}

Detect performs a small number of bounded reads and returns one of the recognised Type values, or (Unknown, ErrUnknown) when no signature matches.

Probe and open

Register the drivers you need, then dispatch with Open / OpenFile:

import (
    "github.com/go-filesystems/detect"
    _ "github.com/go-filesystems/detect/fat32reg" // registers the FAT32 driver
)

fs, t, err := detect.OpenFile("disk.img")
if err != nil {
    log.Fatal(err)
}
defer fs.Close()
data, _ := fs.ReadFile("/hello.txt")
fmt.Println(t, len(data))

Open returns ErrUnknown if nothing matches, or ErrUnsupported if a type was detected but no Opener is registered for it.

Registering a driver directly

detect.Register(detect.XFS, func(r io.ReaderAt, size int64) (filesystem.Filesystem, error) {
    return xfs.Open(r, size)
})

Register is idempotent and safe for concurrent use, so adapter packages call it from init.

Adapter sub-packages

Thin adapters import exactly one driver and register it in init, so a consumer can wire a driver in with a single blank import:

Adapter	Driver	Type
detect/fat32reg	go-filesystems/fat32	fat32

Each adapter is its own module: it carries the replace github.com/go-filesystems/interface => ../../interface sibling directive that the driver needs, while keeping the core package free of any driver dependency.

Detected types

fat32, exfat, ntfs, ext4, xfs, btrfs, zfs, apfs, hfsplus, iso9660, squashfs, ufs — and unknown when nothing matches.

Magic-signature table

Every signature below was verified against the corresponding go-filesystems driver's own on-disk reader (offset, value and byte order). Probes run most-specific first so nested or ambiguous layouts never produce a false positive.

Type	Offset	Magic	Width / order	Notes
btrfs	0x10040 (65600)	_BHRfS_M (0x4D5F53665248425F)	u64 LE	superblock at 64 KiB + 0x40
squashfs	0	hsqs (0x73717368) / sqsh (0x68737173)	u32, LE or BE image	both byte orders recognised
xfs	0	XFSB (0x58465342)	u32 BE	XFS is big-endian on disk
ext4	0x438 (1080)	0xEF53	u16 LE	ext2/3/4 share the magic; reported as ext4
apfs	32	NXSB	4 bytes	NX container superblock
hfsplus	1024	H+ (0x482B) / HX (0x4858)	u16 BE	HFS+ and HFSX
iso9660	0x8001 (32769)	CD001	5 bytes	volume descriptor, logical sector 16
ufs	65536+1372 / 8192+1372	0x19540119 / 0x00011954	u32 LE	UFS2 then UFS1 superblock
zfs	uberblock ring (label 0)	0x00bab10c	u64, LE or BE	best-effort — see caveat
exfat	3	EXFAT	8 bytes	OEM identifier
ntfs	3 + 510	NTFS + 0xAA55	8 bytes + u16 LE	both required
fat32	0x52 + 510	FAT32 + 0xAA55	8 bytes + u16 LE	label at 0x52 distinguishes FAT32 from FAT12/16 (whose label sits at 0x36)

ZFS detection caveat

ZFS recognition is best-effort. detect scans the uberblock ring of the first front vdev label (offset 131072, 128 slots) for the uberblock magic 0x00bab10c in either byte order. This reliably recognises a ZFS-labelled vdev without reading the whole device, but it does not validate the full label NVlist or checksum, and it does not search the rear labels. A device whose front label has been wiped but whose rear labels survive will not be detected.

Design notes

• Bounded reads. Every probe reads through go-volumes/safeio with a 1 MiB per-read ceiling, so a truncated or hostile image never triggers a large allocation, and probing past the end of a small image is not fatal.
• No false positives. Container/whole-disk magics are checked before the FAT-family boot-sector heuristics, and FAT32 / NTFS require multiple corroborating fields.
• Pure Go, six arches. CI builds and tests on amd64, arm64, riscv64, loong64, ppc64le and s390x (big-endian), exercising the explicit byte-order handling.

License

BSD-3-Clause — see LICENSE.

Documentation

Overview

Package detect is a blkid-style, pure-Go (CGO=0) filesystem-type prober and driver-opener registry for the go-filesystems family.

It mirrors the standard library's image.Decode design: the core package depends only on github.com/go-filesystems/interface plus the standard library (and a bounded-read helper). Detect probes on-disk magic signatures and returns a Type; it never imports a concrete driver, so it composes cleanly as a meta-importer despite the drivers' use of a `replace github.com/go-filesystems/interface => ../interface` sibling directive that does not survive transitive importing.

Drivers are wired in at the call site. A consumer registers an Opener for each Type it wants to open — either directly via Register, or by blank-importing a thin per-driver adapter sub-package (for example `import _ "github.com/go-filesystems/detect/fat32reg"`), each of which calls Register in its init function.

t, err := detect.Detect(r, size)            // magic probe only
fs, t, err := detect.Open(r, size)          // probe then dispatch
fs, t, err := detect.OpenFile("disk.img")   // open path, probe, dispatch

Index

• Variables
• func Register(t Type, o Opener)
• func Registered(t Type) bool
• type Opener
• type Type
  • func Detect(r io.ReaderAt, size int64) (Type, error)
  • func Open(r io.ReaderAt, size int64) (filesystem.Filesystem, Type, error)
  • func OpenFile(path string) (filesystem.Filesystem, Type, error)

Variables

var (
	// ErrUnknown is returned by Detect (and wrapped by Open) when no known
	// magic signature matches the probed image.
	ErrUnknown = errors.New("detect: unknown filesystem")

	// ErrUnsupported is returned by Open when Detect recognised the
	// filesystem type but no Opener has been registered for it.
	ErrUnsupported = errors.New("detect: no opener registered for filesystem type")
)

Sentinel errors. Callers should test with errors.Is.

Functions

func Register

func Register(t Type, o Opener)

Register associates an Opener with a filesystem Type. It is idempotent (re-registering the same Type replaces the previous Opener) and safe for concurrent use, so a driver adapter may call it from init without coordinating with others. A nil Opener, or the Unknown type, is rejected to keep the dispatch table well-formed.

func Registered

func Registered(t Type) bool

Registered reports whether an Opener is currently registered for t.

Types

type Opener

type Opener func(r io.ReaderAt, size int64) (filesystem.Filesystem, error)

Opener constructs a filesystem.Filesystem from an image. It mirrors the concrete drivers' Open(rs io.ReaderAt, size int64) entry points. An Opener is responsible for its own validation; Open only calls it after Detect has already matched the corresponding Type.

type Type

type Type string

Type is a filesystem-type identifier as reported by Detect. The set of recognised values is closed and matches the go-filesystems driver family.

const (
	Unknown  Type = "unknown"
	FAT32    Type = "fat32"
	ExFAT    Type = "exfat"
	NTFS     Type = "ntfs"
	Ext4     Type = "ext4"
	XFS      Type = "xfs"
	Btrfs    Type = "btrfs"
	ZFS      Type = "zfs"
	APFS     Type = "apfs"
	HFSPlus  Type = "hfsplus"
	ISO9660  Type = "iso9660"
	SquashFS Type = "squashfs"
	UFS      Type = "ufs"
)

The recognised filesystem types. Unknown is the zero value returned alongside ErrUnknown when no signature matches.

func Detect

func Detect(r io.ReaderAt, size int64) (Type, error)

Detect probes the on-disk magic signatures of r and returns the matching Type. size is the logical size of the image in bytes; pass a negative value if unknown (probes are then bounded only by maxProbeRead and the reader's own extent). Detection is read-only and performs a small number of bounded reads. When no signature matches, Detect returns (Unknown, ErrUnknown).

Probes run most-specific first so that nested or ambiguous layouts never produce a false positive: container/whole-disk magics (btrfs, squashfs, xfs, ext, apfs, ufs, zfs) are checked before the FAT-family boot-sector heuristics, and FAT32 is only reported when both its boot signature and its "FAT32 " filesystem-type label are present.

func Open

func Open(r io.ReaderAt, size int64) (filesystem.Filesystem, Type, error)

Open detects the filesystem type of r and dispatches to the Opener registered for that type. The detected Type is always returned, even on error, so callers can report it.

• If no signature matches, Open returns (nil, Unknown, ErrUnknown).
• If a type is detected but no Opener is registered for it, Open returns (nil, <type>, ErrUnsupported) wrapped with the type name.
• Otherwise the Opener's result is returned with the detected Type.

func OpenFile

func OpenFile(path string) (filesystem.Filesystem, Type, error)

OpenFile opens the file at path, detects its filesystem type and dispatches to the registered Opener. The underlying *os.File is closed only on error; on success it is retained by the returned Filesystem (whose Close must be called by the caller) — drivers read lazily through the io.ReaderAt, so the descriptor must outlive the returned value.