Documentation
¶
Overview ¶
Package xz is a pure-Go decoder for the .xz container (xz-file-format.txt, tukaani.org) whose memory follows the decoded data and not the sizes a header declares.
Its API mirrors the decode side of github.com/ulikunitz/xz so that a consumer can swap the import. The dictionary window of the LZMA2 filter is allocated lazily and never beyond ReaderConfig.MaxDictionary; a stream that needs more fails with ErrTooLarge before the allocation is made.
Example ¶
package main
import (
"bytes"
"encoding/hex"
"fmt"
"io"
"github.com/go-compressions/xz"
)
// A complete .xz file as written by `xz` (CRC64 check, one LZMA2 block).
const helloXZ = "fd377a585a000004e6d6b4460200210116000000742fe5a301000f636875" +
"6e6b20636f6d7072657373696f00ab31da83b7defdd400012810e50b6c60" +
"1fb6f37d010000000004595a"
func main() {
data, _ := hex.DecodeString(helloXZ)
r, err := xz.NewReader(bytes.NewReader(data))
if err != nil {
fmt.Println(err)
return
}
out, err := io.ReadAll(r)
fmt.Printf("%q %v\n", out, err)
}
Output: "chunk compressio" <nil>
Index ¶
Examples ¶
Constants ¶
const DefaultMaxDictionary = 256 << 20
DefaultMaxDictionary is the window ceiling used when ReaderConfig.MaxDictionary is zero: 256 MiB, four times the largest dictionary of xz -9 (64 MiB).
const HeaderLen = 12
HeaderLen is the length of a stream header and of a stream footer.
Variables ¶
var ( // ErrCorrupt is wrapped by every error caused by malformed input. ErrCorrupt = xerr.ErrCorrupt // ErrChecksum is a mismatch of a block check (CRC32, CRC64, SHA-256). ErrChecksum = xerr.ErrChecksum // ErrTruncated is input that ends inside a structure; it is also an // ErrCorrupt and an io.ErrUnexpectedEOF. ErrTruncated = xerr.ErrTruncated // ErrTooLarge means the stream needs a dictionary window larger than // MaxDictionary. No memory was allocated for it. ErrTooLarge = xerr.ErrTooLarge // ErrUnsupported is a well-formed feature that is not implemented: an // unknown filter ID, a reserved check type, a new stream version. ErrUnsupported = xerr.ErrUnsupported )
Errors. They are shared with package lzma and work with errors.Is.
Functions ¶
This section is empty.
Types ¶
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader decodes one or several concatenated xz streams.
type ReaderConfig ¶
type ReaderConfig struct {
// MaxDictionary is the ceiling in bytes on the dictionary window the
// reader will allocate for a block; zero means DefaultMaxDictionary.
// It bounds memory, not the format: a block that declares a 4 GiB
// dictionary decodes fine as long as the data it produces fits.
MaxDictionary int
// DictCap is accepted so that code written for ulikunitz/xz compiles.
// There it is a minimum allocation; here the window follows the data,
// so the field has no effect.
//
// Deprecated: use MaxDictionary.
DictCap int
// SingleStream makes the reader expect exactly one stream, without
// padding, and report anything after it as an error.
SingleStream bool
}
ReaderConfig defines the parameters of the xz reader.
Example ¶
MaxDictionary bounds the memory a hostile file can make the reader use: the window grows with the decoded data and stops, with ErrTooLarge, at the ceiling. A header that merely declares a huge dictionary costs nothing.
package main
import (
"bytes"
"encoding/hex"
"errors"
"fmt"
"io"
"github.com/go-compressions/xz"
)
// A complete .xz file as written by `xz` (CRC64 check, one LZMA2 block).
const helloXZ = "fd377a585a000004e6d6b4460200210116000000742fe5a301000f636875" +
"6e6b20636f6d7072657373696f00ab31da83b7defdd400012810e50b6c60" +
"1fb6f37d010000000004595a"
func main() {
data, _ := hex.DecodeString(helloXZ)
cfg := xz.ReaderConfig{MaxDictionary: 64 << 20, SingleStream: true}
r, err := cfg.NewReader(bytes.NewReader(data))
if err != nil {
fmt.Println(err)
return
}
_, err = io.Copy(io.Discard, r)
fmt.Println(err, errors.Is(err, xz.ErrTooLarge))
}
Output: <nil> false
func (ReaderConfig) NewReader ¶
func (c ReaderConfig) NewReader(xz io.Reader) (*Reader, error)
NewReader creates a reader using the configuration. Unless SingleStream is set it processes concatenated streams and stream padding.
func (*ReaderConfig) Verify ¶
func (c *ReaderConfig) Verify() error
Verify checks the parameters and replaces zero values by defaults.
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
benchxz
command
Command benchxz measures single-core decode throughput of this library, of github.com/ulikunitz/xz and of the C reference (xz -dc) on a 64 MiB realistic corpus: the Go source tree and the Go toolchain binaries of the running Go installation, concatenated, then compressed once by xz.
|
Command benchxz measures single-core decode throughput of this library, of github.com/ulikunitz/xz and of the C reference (xz -dc) on a 64 MiB realistic corpus: the Go source tree and the Go toolchain binaries of the running Go installation, concatenated, then compressed once by xz. |
|
bombdemo
command
Command bombdemo prints how much memory this library and github.com/ulikunitz/xz allocate to decode a few bytes of data stored in a block whose header declares a large LZMA2 dictionary.
|
Command bombdemo prints how much memory this library and github.com/ulikunitz/xz allocate to decode a few bytes of data stored in a block whose header declares a large LZMA2 dictionary. |
|
corpus
Package corpus generates the deterministic inputs used by the tests and by the fixture generator (internal/mkfixtures).
|
Package corpus generates the deterministic inputs used by the tests and by the fixture generator (internal/mkfixtures). |
|
covgaps
command
Command covgaps prints the blocks of a Go coverage profile that were never executed (summing the counts a block has in every test binary), and the overall statement coverage.
|
Command covgaps prints the blocks of a Go coverage profile that were never executed (summing the counts a block has in every test binary), and the overall statement coverage. |
|
filter
Package filter implements the non-last filters of the xz format in the decoding direction: the Delta filter and the Branch/Call/Jump (BCJ) filters for executables, ported from liblzma/simple/*.c and liblzma/delta/delta_decoder.c (0BSD) of XZ Utils.
|
Package filter implements the non-last filters of the xz format in the decoding direction: the Delta filter and the Branch/Call/Jump (BCJ) filters for executables, ported from liblzma/simple/*.c and liblzma/delta/delta_decoder.c (0BSD) of XZ Utils. |
|
lzmacore
Package lzmacore is the LZMA decoder shared by the xz and lzma packages.
|
Package lzmacore is the LZMA decoder shared by the xz and lzma packages. |
|
mkfixtures
command
Command mkfixtures regenerates the .xz and .lzma files under internal/testfiles/data with the xz command line tool (the C reference, xz-utils).
|
Command mkfixtures regenerates the .xz and .lzma files under internal/testfiles/data with the xz command line tool (the C reference, xz-utils). |
|
testfiles
Package testfiles embeds the fixtures made by the xz command line tool (see internal/mkfixtures).
|
Package testfiles embeds the fixtures made by the xz command line tool (see internal/mkfixtures). |
|
xerr
Package xerr holds the error values shared by the xz and lzma packages, so that both re-export the very same variables and errors.Is works across them.
|
Package xerr holds the error values shared by the xz and lzma packages, so that both re-export the very same variables and errors.Is works across them. |
|
Package lzma decodes the .lzma ("LZMA alone") format and raw LZMA2 streams.
|
Package lzma decodes the .lzma ("LZMA alone") format and raw LZMA2 streams. |