pkg2go

README

pkg2go

Read Linux package artifacts — and the archive container formats they are built from — as a Go io/fs.FS.

Idea

rpm and deb are just compressed container archives plus a metadata blob. The reusable primitives are the container formats, so this module implements fs.FS for those and lets the package formats fall out as thin compositions (a file inside one FS is the source for the next):

• deb = an ar archive of control.tar.* and data.tar.* → rootfs = ar FS → open data.tar.* → decompress → tar FS
• rpm = header metadata + a compressed cpio payload → rootfs = decompress payload → cpio FS

Compressed inner members are decompressed to a temp file and served through an io.ReaderAt, so every container reader is pure random-access indexing over a ReaderAt regardless of the codec.

Packages

Package	Status	Description
cpio	✅ initial	newc (SVR4) cpio archives as fs.FS
ar	✅ initial	Unix ar archives as fs.FS
compress	✅ initial	decompress a stream to a random-access io.ReaderAt
deb	✅ initial	.deb packages (ar + tar)
rpm	✅ initial	.rpm packages (header + cpio)

tar is read via github.com/cpuguy83/tar2go rather than a reader in this module.

Autodetection

The top-level pkg2go.NewFS sniffs the archive magic and dispatches to the right container reader:

f, _ := os.Open("archive")
info, _ := f.Stat()
fsys, _ := pkg2go.NewFS(f, info.Size()) // ar or cpio, chosen by magic

It recognizes ar archives, newc cpio archives, and tar archives (via tar2go). Compressed streams are not detected — decompress first and pass the resulting io.ReaderAt.

For packages, pkg2go.OpenPackage sniffs the .deb (ar) and .rpm (lead) magics and returns a Package whose Rootfs() yields the extracted filesystem:

f, _ := os.Open("pkg.deb") // or pkg.rpm
info, _ := f.Stat()
pkg, _ := pkg2go.OpenPackage(f, info.Size())
defer pkg.Close()
rootfs, _ := pkg.Rootfs()

cpio

f, _ := os.Open("payload.cpio")
info, _ := f.Stat()
fsys, _ := cpio.NewFS(f, info.Size())

data, _ := fs.ReadFile(fsys, "usr/bin/hello")

FS implements fs.FS, fs.StatFS, fs.ReadDirFS, and fs.ReadLinkFS. Per-entry metadata beyond fs.FileInfo (uid/gid, device numbers, link count) is available via info.Sys().(cpio.Header).

Only the newc formats (070701 and 070702) are supported — the formats produced by cpio -H newc and used by rpm payloads.

ar

f, _ := os.Open("package.deb")
info, _ := f.Stat()
fsys, _ := ar.NewFS(f, info.Size())

control, _ := fs.ReadFile(fsys, "control.tar.gz")

FS implements fs.FS, fs.StatFS, and fs.ReadDirFS. ar is a flat archive, so every member is a regular file directly under the root. Per-member metadata (uid/gid, mode) is available via info.Sys().(ar.Header).

The GNU (// name table with /N references) and BSD (#1/N) extended-name variants are supported. Archive symbol and name tables are consumed for indexing but not exposed as members.

tar

tar is read through github.com/cpuguy83/tar2go, which indexes a tar over an io.ReaderAt and serves it as an fs.FS:

f, _ := os.Open("data.tar")
fsys := tar2go.NewIndex(f).FS()

data, _ := fs.ReadFile(fsys, "usr/bin/app")

pkg2go.NewFS wires this up automatically when it detects a tar stream.

compress

f, _ := os.Open("data.tar.zst")
info, _ := f.Stat()
dec, _ := compress.Decompress(f, info.Size()) // sniffs the codec by magic
defer dec.Close()

fsys := tar2go.NewIndex(io.NewSectionReader(dec, 0, dec.Size())).FS()

Decompress takes an io.ReaderAt and its size, detects the codec from the leading magic (gzip, bzip2, xz, zstd) and expands the member to a temporary file, returned as a *File that is an io.ReaderAt — giving the random access the container readers need. An uncompressed member is passed through directly, without copying or a temp file (and Close is then a no-op). DecompressAs selects the codec explicitly (needed for lzma, which has no reliable magic; rpm names its payload codec in the header).

deb

f, _ := os.Open("hello.deb")
info, _ := f.Stat()
pkg, _ := deb.Open(f, info.Size())
defer pkg.Close()

rootfs, _ := pkg.Rootfs() // fs.FS over the extracted data.tar.*

A .deb is an ar archive of debian-binary, control.tar.* and data.tar.*. deb.Open indexes only the outer ar; Rootfs decompresses the data.tar.* member on first use and memoizes it. The payload codec is detected from the member's magic, so the file extension is irrelevant. Temp files created for compressed members are released by Close.

rpm

f, _ := os.Open("hello.rpm")
info, _ := f.Stat()
pkg, _ := rpm.Open(f, info.Size())
defer pkg.Close()

rootfs, _ := pkg.Rootfs() // fs.FS over the decompressed cpio payload

An .rpm is a 96-byte lead, a signature header, a main header, and a compressed cpio payload. rpm.Open parses the lead and headers only far enough to locate the payload and read its RPMTAG_PAYLOADCOMPRESSOR; Rootfs decompresses the payload on first use and memoizes it. The named compressor is honored (lzma in particular has no reliable magic), falling back to sniffing when the tag is absent. Close releases the temp file backing the payload.

Symlinks

The extracted filesystem reproduces the package faithfully: symlinks keep the exact target recorded in the archive, exposed through ReadLink/Lstat (io/fs.ReadLinkFS). Package symlinks are frequently absolute and assume the root filesystem lives at / — e.g. /usr/bin/foo -> /usr/lib/foo/foo.

Absolute targets are resolved by the kernel against the reading process's root, not against wherever the tree happens to be rooted. So if you mount or extract the rootfs under a subdirectory (say /mnt/pkg), an absolute symlink still points at the host's /... and escapes the tree. This is ordinary path resolution, not a quirk of pkg2go — extracting the package to /mnt/pkg on disk behaves identically. To consume the tree as a real root filesystem, root it at / (a mount namespace with pivot_root, or chroot).

Documentation

Overview

Package pkg2go reads Linux package artifacts and the container archive formats they are built from as an io/fs.FS.

The subpackages implement individual formats: github.com/cpuguy83/pkg2go/ar for Unix ar archives (the outer container of a .deb) and github.com/cpuguy83/pkg2go/cpio for newc cpio archives (used by rpm payloads). NewFS sniffs the archive magic and dispatches to the right one.

Index

• func NewFS(ra io.ReaderAt, size int64) (fs.FS, error)
• type Package
  • func OpenPackage(ra io.ReaderAt, size int64) (Package, error)

Functions

func NewFS

func NewFS(ra io.ReaderAt, size int64) (fs.FS, error)

NewFS identifies the archive format of ra, which must contain exactly size bytes, and returns an io/fs.FS serving its contents. It recognizes ar, newc cpio, and tar archives by their magic and delegates to ar.NewFS, cpio.NewFS, or github.com/cpuguy83/tar2go respectively.

It returns an error if the format is not recognized. Compressed streams are not detected; decompress first and pass the resulting io.ReaderAt.

Types

type Package

type Package interface {
	// Rootfs returns the files the package installs as an fs.FS rooted at the
	// filesystem root.
	Rootfs() (fs.FS, error)
	// Close releases any temporary files created while reading the package.
	Close() error
}

Package is an opened OS package whose installed files can be read as an io/fs.FS. Both *deb.Package and *rpm.Package satisfy it.

func OpenPackage

func OpenPackage(ra io.ReaderAt, size int64) (Package, error)

OpenPackage identifies whether ra, which must contain exactly size bytes, is a Debian or RPM package by its magic and returns it as a Package. A .deb is recognized by its ar magic and read by deb.Open; a .rpm is recognized by its lead magic and read by rpm.Open.

It returns an error if the format is not recognized. The returned Package must be closed.