archive

README

go-archive

---

One reader for untrusted archives, as a single Go package: archive, module github.com/Rethunk-Tech/go-archive.

Magic bytes pick the format, never the file extension. Eleven are read: zip, RAR 4, RAR 5, 7z, tar, and gzip, xz, lzma, zstd, bzip2 or lz4, each alone or wrapped around a tar. A zip entry may be stored or compressed with Deflate, bzip2, LZMA, xz or zstd. Archives arrive from the internet with no trust attached, so every reader here is bounded in memory and in bytes before it produces output.

Quick start

go get github.com/Rethunk-Tech/go-archive

Rooting, caps, and a worked Extract and Lister example: HUMANS.md.

Features

• Extract enforces policy: path traversal, unsafe names, case collisions, symlinks and hardlinks, device and pipe entries, encryption, and the entry, entry-size, total-size and ratio caps.
• ExtractFunc applies the same policy and hands each entry to a callback, for in-memory or non-filesystem sinks. ExtractReader extracts RAR and the streams from a plain io.Reader.
• Extensions, HasExtension, StripExtension, IsStream and IsMediaType name the formats by file ending and by Content-Type.
• Lister walks entries forward-only and applies no extraction policy. Decompression is its only bound (ListerOptions.MaxEntryBytes).
• The destination must already exist. Writes travel through the caller's *os.Root, so no entry resolves outside it.
• Decoder windows are capped independently of entry sizes, so a crafted header cannot force a multi-gigabyte allocation.
• Fuzzed against every format, with the crash corpus committed.

Documentation

Doc	Audience
HUMANS.md	Running and using the package
AGENTS.md	Internals, invariants, and why
CONTRIBUTING.md	Process — commits, tests, docs
SECURITY.md	Vulnerability disclosure
CHANGELOG.md	Release notes

License

MIT — see LICENSE. © 2026 Rethunk.Tech, LLC. The vendored internal/rardecode keeps its BSD 2-Clause licence; see NOTICE.

Documentation

Overview

Package archive reads untrusted archives and writes what Extract unpacks under a root the caller supplies.

Format comes from the magic bytes, never the extension: zip, RAR 4 and 5, 7z, tar, and gzip, xz, lzma, zstd, bzip2 or lz4, each alone or wrapped around a tar. Every reader is bounded in memory and in bytes before it produces output.

Extract enforces the name, kind and size policy. Lister walks the same entries and only bounds decompression.

Index

• Constants
• Variables
• func DeclaredSize(src Source, size int64) (_ int64, err error)
• func Extensions() []string
• func Extract(src Source, size int64, baseName string, dest *os.Root, opts Options) (err error)
• func ExtractFunc(src Source, size int64, baseName string, opts Options, ...) (err error)
• func ExtractReader(r io.Reader, baseName string, dest *os.Root, opts Options) (err error)
• func HasExtension(name string) bool
• func IsMediaType(contentType string) bool
• func IsStream(name string) bool
• func RecoverMalformed(err *error)
• func StripExtension(name string) string
• func WindowsSegment(seg string) bool
• type Entry
• type Error
  • func (e *Error) Error() string
  • func (e *Error) Unwrap() error
• type Lister
  • func NewLister(src Source, size int64, baseName string, opts ListerOptions) (*Lister, error)
  • func (l *Lister) Close() error
  • func (l *Lister) Entries() iter.Seq2[Entry, error]
  • func (l *Lister) Next() (Entry, error)
  • func (l *Lister) Open() (_ io.ReadCloser, err error)
• type ListerOptions
• type Options
• type Source

Constants

const (
	DefaultMaxEntryBytes int64 = 32 << 30
	DefaultMaxTotalBytes int64 = 64 << 30
	DefaultMaxEntries          = 20000

	// Below DefaultRatioFloor an archive may unpack to any size within the caps;
	// above it, to at most DefaultMaxRatio times its own size. Deflate tops out
	// near 1030:1, so real archives stay well inside.
	DefaultMaxRatio   int64 = 1024
	DefaultRatioFloor int64 = 2 << 30
)

Default caps, applied to every zero field of Options. The byte caps sit far above any real archive, so the ratio guard rather than these stops a small archive claiming them.

Variables

var (
	ErrUnsupportedFormat = errors.New("not a zip, RAR, 7z, tar, gzip, xz, lzma, zstd, bzip2 or lz4 archive")
	ErrEncrypted         = errors.New("archive is encrypted")
	ErrTraversal         = errors.New("path escapes the destination")
	ErrUnsafeName        = errors.New("name is not allowed")
	ErrCaseCollision     = errors.New("name collides with another entry")
	ErrLink              = errors.New("symlinks and hardlinks are not allowed")
	ErrSpecialFile       = errors.New("device, pipe and socket entries are not allowed")
	ErrEntryTooLarge     = errors.New("entry exceeds the size cap")
	ErrArchiveTooLarge   = errors.New("archive exceeds the extracted size cap")
	ErrTooManyEntries    = errors.New("archive exceeds the entry cap")
	ErrRatio             = errors.New("archive unpacks to far more than its size allows")
	ErrChecksum          = errors.New("checksum mismatch")
	ErrMalformed         = errors.New("archive is malformed")
	ErrDecoderMemory     = errors.New("decoder memory exceeds the extraction cap")
)

Reasons carried by Error. A disk-full write is reported as the underlying error, so errors.Is(err, syscall.ENOSPC) holds.

Functions

func DeclaredSize

func DeclaredSize(src Source, size int64) (_ int64, err error)

DeclaredSize sums the uncompressed sizes the archive's headers declare. It is informational (an untrusted archive can lie); Extract's caps count bytes. A stream declares no size, so it is read within the default caps and one over them reports the cap.

func Extensions

func Extensions() []string

Extensions are the file name endings of the formats this package reads, longest first. Detection never trusts them; they are for naming and picking files. Each call returns a fresh slice.

func Extract

func Extract(src Source, size int64, baseName string, dest *os.Root, opts Options) (err error)

Extract unpacks the archive src (size bytes) into dest, detecting the format from its magic bytes. dest must exist and is rooted by the caller. baseName names the single file a bare compressed stream yields. Extraction stops at the first failure and leaves what it wrote in dest, so the caller passes a temp directory and discards it on error. A zero Options field means its default.

func ExtractFunc

func ExtractFunc(src Source, size int64, baseName string, opts Options,
	fn func(name string, isDir bool, r io.Reader) error,
) (err error)

ExtractFunc walks src under Extract's admission rules and hands each admitted entry to fn instead of writing it. name is the cleaned slash path; r reads nothing for a directory. The bytes fn reads count against the caps, and a file's checksum is verified only when fn reads it to the end. fn's own error is returned as it is.

No filesystem judges the tree, so a file "a" followed by "a/b" passes here where Extract's mkdir would refuse it.

func ExtractReader

func ExtractReader(r io.Reader, baseName string, dest *os.Root, opts Options) (err error)

ExtractReader is Extract for a stream with no random access and no known size: RAR, tar, and gzip, xz, lzma, zstd, bzip2 or lz4, alone or around a tar. Zip and 7z need a Source and fail with ErrUnsupportedFormat. With the size unknown no ratio applies, so MaxTotalBytes is the bound. A read error from r ends extraction and is returned in the chain; a body that ends before its format can be told apart fails as io.ErrUnexpectedEOF, so a caller can treat it as a dropped connection.

func HasExtension

func HasExtension(name string) bool

HasExtension reports whether name ends in one of Extensions, ignoring case.

func IsMediaType

func IsMediaType(contentType string) bool

IsMediaType reports whether contentType names a format this package reads, under the types uploaders and MIME databases give them. Parameters and case do not matter. application/octet-stream is not one: it means the sender did not know.

func IsStream

func IsStream(name string) bool

IsStream reports whether name's archive ending is a bare compressed stream (.gz, .xz, …) rather than a tar or a container. Such a file holds one unnamed payload, which Extract names from baseName.

func RecoverMalformed

func RecoverMalformed(err *error)

RecoverMalformed turns a panic in a format reader, which a crafted archive can cause, into ErrMalformed for *err, so an untrusted download fails its item instead of ending the process.

func StripExtension

func StripExtension(name string) string

StripExtension is name without its archive ending, or name itself when it has none.

func WindowsSegment

func WindowsSegment(seg string) bool

WindowsSegment reports whether one path segment is a name Windows stores as written. Windows drops a trailing dot or space, reads the characters in windowsUnsafe as separators or invalid, and hands a device name to any file whose stem matches one.

Types

type Entry

type Entry struct {
	Name  string
	Size  int64
	IsDir bool
}

Entry is one archive entry.

type Error

type Error struct {
	Entry  string
	Reason error
}

Error names the entry that failed and why. Entry is empty for a failure of the archive as a whole.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type Lister

type Lister struct {
	// contains filtered or unexported fields
}

Lister walks an archive's entries once, forward.

Next reports each entry in order and returns io.EOF at the end. Open gives the current entry's bytes, valid only until the next Next; a caller wanting names and sizes alone calls Next without Open and the decoder skips that entry's bytes. Beyond the 7z header check every reader runs, nothing here counts entries, checks a name against a destination or refuses a link: those are extraction's decisions.

func NewLister

func NewLister(src Source, size int64, baseName string, opts ListerOptions) (*Lister, error)

NewLister detects the archive's format from its magic bytes and returns a Lister positioned before the first entry. It decompresses nothing until Next.

func (*Lister) Close

func (l *Lister) Close() error

Close releases the decoders the walk holds. It never closes the caller's Source, and calling it twice does nothing.

func (*Lister) Entries

func (l *Lister) Entries() iter.Seq2[Entry, error]

Entries adapts the same walk to range-over-func. A size comes from the entry's header, like DeclaredSize's; a bare stream declares none, so its one entry is drained to learn it, which Next without Open would leave at -1.

func (*Lister) Next

func (l *Lister) Next() (Entry, error)

Next advances to the next entry, normalising its name. It returns io.EOF once the archive is exhausted.

func (*Lister) Open

func (l *Lister) Open() (_ io.ReadCloser, err error)

Open returns the current entry's bytes, valid until the next Next. A directory holds no bytes, so it reads empty rather than failing. A format reader that panics on crafted bytes here or in Read fails with ErrMalformed.

type ListerOptions

type ListerOptions struct {
	// MaxEntryBytes bounds one entry's decompressed bytes. Zero means 256 MiB, which is
	// stricter than an extraction cap: callers that list streams must raise it.
	MaxEntryBytes int64
}

ListerOptions bounds what a Lister may decompress. A zero field means the default.

type Options

type Options struct {
	MaxEntryBytes int64
	MaxTotalBytes int64
	MaxEntries    int
	MaxRatio      int64
	RatioFloor    int64
	// Segment reports whether one path segment is a name the destination platform
	// will store as written. nil means WindowsSegment.
	Segment func(seg string) bool
	// CaseSensitive turns off the case-collision check, for an ExtractFunc sink
	// that keeps names differing only by case apart (a map, an object store). An
	// exact duplicate file is still refused. Extract and ExtractReader write to a
	// filesystem, where such names collide, so they leave it false.
	CaseSensitive bool
}

Options lowers or raises the extraction caps; a zero field means its default.

type Source

type Source interface {
	io.ReaderAt
}

Source is what a reader needs. *os.File, *bytes.Reader and *io.SectionReader satisfy it. Every read is at an offset, so the caller's file position is never moved.