archive

README

go-archive

---

One reader for untrusted archives, as a single Go package: archive, module github.com/Rethunk-Tech/go-archive. Mortar is its one consumer.

Magic bytes pick the format, never the file extension. Eleven are read: zip, RAR 4, RAR 5, 7z, tar, and gzip, xz, lzma, zstd, bzip2 or lz4, each alone or wrapped around a tar. A zip entry may be stored or compressed with Deflate, bzip2, LZMA, xz or zstd. Archives arrive from the internet with no trust attached, so every reader here is bounded in memory and in bytes before it produces output.

Quick start

go get github.com/Rethunk-Tech/go-archive

Rooting, caps, and a worked Extract and Lister example: HUMANS.md.

Features

• Extract enforces policy: path traversal, unsafe names, case collisions, symlinks and hardlinks, device and pipe entries, encryption, and the entry, entry-size, total-size and ratio caps.
• Lister walks entries forward-only and applies no extraction policy. Decompression is its only bound (ListerOptions.MaxEntryBytes).
• The destination must already exist. Writes travel through the caller's *os.Root, so no entry resolves outside it.
• Decoder windows are capped independently of entry sizes, so a crafted header cannot force a multi-gigabyte allocation.
• Fuzzed against every format, with the crash corpus committed.

Documentation

Doc	Audience
HUMANS.md	Running and using the package
AGENTS.md	Internals, invariants, and why
CONTRIBUTING.md	Process — commits, tests, docs
SECURITY.md	Vulnerability disclosure
CHANGELOG.md	Release notes

License

MIT — see LICENSE. © 2026 Rethunk Technologies

Documentation

Overview

Package archive reads untrusted archives and writes what Extract unpacks under a root the caller supplies.

Format comes from the magic bytes, never the extension: zip, RAR 4 and 5, 7z, tar, and gzip, xz, lzma, zstd, bzip2 or lz4, each alone or wrapped around a tar. Every reader is bounded in memory and in bytes before it produces output.

Extract enforces the name, kind and size policy. Lister walks the same entries and only bounds decompression.

Index

• Constants
• Variables
• func DeclaredSize(src Source, size int64) (_ int64, err error)
• func Extract(src Source, size int64, baseName string, dest *os.Root, opts Options) (err error)
• func RecoverMalformed(err *error)
• func WindowsSegment(seg string) bool
• type Entry
• type Error
  • func (e *Error) Error() string
  • func (e *Error) Unwrap() error
• type Lister
  • func NewLister(src Source, size int64, baseName string, opts ListerOptions) (*Lister, error)
  • func (l *Lister) Close() error
  • func (l *Lister) Entries() iter.Seq2[Entry, error]
  • func (l *Lister) Next() (Entry, error)
  • func (l *Lister) Open() (_ io.ReadCloser, err error)
• type ListerOptions
• type Options
• type Source

Constants

const (
	DefaultMaxEntryBytes int64 = 32 << 30
	DefaultMaxTotalBytes int64 = 64 << 30
	DefaultMaxEntries          = 20000

	// Below DefaultRatioFloor an archive may unpack to any size within the caps;
	// above it, to at most DefaultMaxRatio times its own size. Deflate tops out
	// near 1030:1, so real archives stay well inside.
	DefaultMaxRatio   int64 = 1024
	DefaultRatioFloor int64 = 2 << 30
)

Default caps, applied to every zero field of Options. The byte caps sit far above any real archive, so the ratio guard rather than these stops a small archive claiming them.

Variables

var (
	ErrUnsupportedFormat = errors.New("not a zip, RAR, 7z, tar, gzip, xz, lzma, zstd, bzip2 or lz4 archive")
	ErrEncrypted         = errors.New("archive is encrypted")
	ErrTraversal         = errors.New("path escapes the destination")
	ErrUnsafeName        = errors.New("name is not allowed")
	ErrCaseCollision     = errors.New("name collides with another entry")
	ErrLink              = errors.New("symlinks and hardlinks are not allowed")
	ErrSpecialFile       = errors.New("device, pipe and socket entries are not allowed")
	ErrEntryTooLarge     = errors.New("entry exceeds the size cap")
	ErrArchiveTooLarge   = errors.New("archive exceeds the extracted size cap")
	ErrTooManyEntries    = errors.New("archive exceeds the entry cap")
	ErrRatio             = errors.New("archive unpacks to far more than its size allows")
	ErrChecksum          = errors.New("checksum mismatch")
	ErrMalformed         = errors.New("archive is malformed")
	ErrDecoderMemory     = errors.New("decoder memory exceeds the extraction cap")
)

Reasons carried by Error. A disk-full write is reported as the underlying error, so errors.Is(err, syscall.ENOSPC) holds.

Functions

func DeclaredSize

func DeclaredSize(src Source, size int64) (_ int64, err error)

DeclaredSize sums the uncompressed sizes the archive's headers declare. It is informational (an untrusted archive can lie); Extract's caps count bytes. A stream declares no size, so it is read within the default caps and one over them reports the cap.

func Extract

func Extract(src Source, size int64, baseName string, dest *os.Root, opts Options) (err error)

Extract unpacks the archive src (size bytes) into dest, detecting the format from its magic bytes. dest must exist and is rooted by the caller. baseName names the single file a bare compressed stream yields. Extraction stops at the first failure and leaves what it wrote in dest, so the caller passes a temp directory and discards it on error. A zero Options field means its default.

func RecoverMalformed

func RecoverMalformed(err *error)

RecoverMalformed turns a panic in a format reader, which a crafted archive can cause, into ErrMalformed for *err, so an untrusted download fails its item instead of ending the process.

func WindowsSegment

func WindowsSegment(seg string) bool

WindowsSegment reports whether one path segment is a name Windows stores as written. Windows drops a trailing dot or space, reads the characters in windowsUnsafe as separators or invalid, and hands a device name to any file whose stem matches one.

Types

type Entry

type Entry struct {
	Name  string
	Size  int64
	IsDir bool
}

Entry is one archive entry.

type Error

type Error struct {
	Entry  string
	Reason error
}

Error names the entry that failed and why. Entry is empty for a failure of the archive as a whole.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type Lister

type Lister struct {
	// contains filtered or unexported fields
}

Lister walks an archive's entries once, forward.

Next reports each entry in order and returns io.EOF at the end. Open gives the current entry's bytes, valid only until the next Next; a caller wanting names and sizes alone calls Next without Open and the decoder skips that entry's bytes. Beyond the 7z header check every reader runs, nothing here counts entries, checks a name against a destination or refuses a link: those are extraction's decisions.

func NewLister

func NewLister(src Source, size int64, baseName string, opts ListerOptions) (*Lister, error)

NewLister detects the archive's format from its magic bytes and returns a Lister positioned before the first entry. It decompresses nothing until Next.

func (*Lister) Close

func (l *Lister) Close() error

Close releases the decoders the walk holds. It never closes the caller's Source, and calling it twice does nothing.

func (*Lister) Entries

func (l *Lister) Entries() iter.Seq2[Entry, error]

Entries adapts the same walk to range-over-func. A size comes from the entry's header, like DeclaredSize's; a bare stream declares none, so its one entry is drained to learn it, which Next without Open would leave at -1.

func (*Lister) Next

func (l *Lister) Next() (Entry, error)

Next advances to the next entry, normalising its name. It returns io.EOF once the archive is exhausted.

func (*Lister) Open

func (l *Lister) Open() (_ io.ReadCloser, err error)

Open returns the current entry's bytes, valid until the next Next. A directory holds no bytes, so it reads empty rather than failing. A format reader that panics on crafted bytes here or in Read fails with ErrMalformed.

type ListerOptions

type ListerOptions struct {
	// MaxEntryBytes bounds one entry's decompressed bytes. Zero means 256 MiB, which is
	// stricter than an extraction cap: callers that list streams must raise it.
	MaxEntryBytes int64
}

ListerOptions bounds what a Lister may decompress. A zero field means the default.

type Options

type Options struct {
	MaxEntryBytes int64
	MaxTotalBytes int64
	MaxEntries    int
	MaxRatio      int64
	RatioFloor    int64
	// Segment reports whether one path segment is a name the destination platform
	// will store as written. nil means WindowsSegment.
	Segment func(seg string) bool
}

Options lowers or raises the extraction caps; a zero field means its default.

type Source

type Source interface {
	io.ReaderAt
}

Source is what a reader needs. *os.File, *bytes.Reader and *io.SectionReader satisfy it. Every read is at an offset, so the caller's file position is never moved.