xz

README

Package xz

This Go language package supports the reading and writing of xz compressed streams. It includes also a gxz command for compressing and decompressing data. The package is completely written in Go and doesn't have any dependency on any C code.

The package is currently under development. There might be bugs and APIs are not considered stable. At this time the package cannot compete with the xz tool regarding compression speed and size. The algorithms there have been developed over a long time and are highly optimized. However there are a number of improvements planned and I'm very optimistic about parallel compression and decompression. Stay tuned!

Using the API

The following example program shows how to use the API.

package main

import (
    "bytes"
    "io"
    "log"
    "os"

    "github.com/ulikunitz/xz"
)

func main() {
    const text = "The quick brown fox jumps over the lazy dog.\n"
    var buf bytes.Buffer

    // compress text
    w := xz.NewWriter(&buf)
    if _, err := io.WriteString(w, text); err != nil {
        log.Fatalf("WriteString error %s", err)
    }
    if err := w.Close(); err != nil {
        log.Fatalf("w.Close error %s", err)
    }

    // decompress buffer and write output to stdout
    r, err := xz.NewReader(&buf)
    if err != nil {
        log.Fatalf("NewReader error %s", err)
    }
    if _, err = io.Copy(os.Stdout, r); err != nil {
        log.Fatalf("io.Copy error %s", err)
    }
}

Using the gxz compression tool

The package includes a gxz command line utility for compression and decompression.

Use following command for installation:

$ go get github.com/ulikunitz/xz/cmd/gxz

To test it call the following command.

$ gxz bigfile

After some time a much smaller file bigfile.xz will replace bigfile. To decompress it use the following command.

$ gxz -d bigfile.xz

Documentation

Overview

Package xz supports the compression and decompression of xz files.

Example

const text = "The quick brown fox jumps over the lazy dog."
var buf bytes.Buffer

// compress text
w := NewWriter(&buf)
if _, err := io.WriteString(w, text); err != nil {
	log.Fatalf("WriteString error %s", err)
}
if err := w.Close(); err != nil {
	log.Fatalf("w.Close error %s", err)
}

// decompress buffer and write result to stdout
r, err := NewReader(&buf)
if err != nil {
	log.Fatalf("NewReader error %s", err)
}
if _, err = io.Copy(os.Stdout, r); err != nil {
	log.Fatalf("io.Copy error %s", err)
}

Output:
The quick brown fox jumps over the lazy dog.

Index

• Constants
• Variables
• func ValidHeader(data []byte) bool
• type Reader
  • func NewReader(xz io.Reader) (r *Reader, err error)
  • func NewReaderParams(xz io.Reader, p *ReaderParams) (r *Reader, err error)
  • func (r *Reader) Read(p []byte) (n int, err error)
• type ReaderParams
  • func (p *ReaderParams) Verify() error
• type Writer
  • func NewWriter(xz io.Writer) *Writer
  • func NewWriterParams(xz io.Writer, p *WriterParams) (w *Writer, err error)
  • func (w *Writer) Close() error
  • func (w *Writer) Write(p []byte) (n int, err error)
• type WriterParams
  • func (p *WriterParams) Verify() error

Examples

• Package

Constants

const (
	CRC32  byte = 0x1
	CRC64       = 0x4
	SHA256      = 0xa
)

Constants for the checksum methods supported by xz.

const HeaderLen = 12

HeaderLen provides the length of the xz file header.

Variables

var ReaderDefaults = ReaderParams{
	Reader2Params: lzma.Reader2Defaults,
}

ReaderDefaults defines the defaults for the xz reader.

var WriterDefaults = WriterParams{
	Writer2Params: lzma.Writer2Defaults,
	BlockSize:     maxInt64,
	CheckSum:      CRC64,
}

WriterDefaults defines the defaults for the Writer parameters.

Functions

func ValidHeader

func ValidHeader(data []byte) bool

ValidHeader checks whether data is a correct xz file header. The length of data must be HeaderLen.

Types

type Reader

type Reader struct {
	ReaderParams
	// contains filtered or unexported fields
}

Reader decodes xz files.

func NewReader

func NewReader(xz io.Reader) (r *Reader, err error)

NewReader creates a new xz reader using the default parameters. NewReader reads and checks the header of the XZ stream.

func NewReaderParams

func NewReaderParams(xz io.Reader, p *ReaderParams) (r *Reader, err error)

NewReaderParams creates a new xz reader using the given parameters. NewReaderParams reads and checks the header of the XZ stream.

func (*Reader) Read

func (r *Reader) Read(p []byte) (n int, err error)

Read reads actual data from the xz stream.

type ReaderParams

type ReaderParams struct {
	lzma.Reader2Params
}

ReaderParams defines the parameters for the xz reader. The defaults are defined by ReaderDefaults.

func (*ReaderParams) Verify

func (p *ReaderParams) Verify() error

Verify checks the Reader parameters for errors.

type Writer

type Writer struct {
	WriterParams
	// contains filtered or unexported fields
}

Writer compresses data written to it. It is an io.WriteCloser.

func NewWriter

func NewWriter(xz io.Writer) *Writer

NewWriter creates a new Writer using the default writer parameters.

func NewWriterParams

func NewWriterParams(xz io.Writer, p *WriterParams) (w *Writer, err error)

NewWriterParams creates a new Writer using the given parameters.

func (*Writer) Close

func (w *Writer) Close() error

Close closes the writer and adds the footer to the Writer. Close doesn't close the underlying writer.

func (*Writer) Write

func (w *Writer) Write(p []byte) (n int, err error)

Write compresses the uncompressed data provided.

type WriterParams

type WriterParams struct {
	lzma.Writer2Params
	BlockSize int64
	// checksum method: CRC32, CRC64 or SHA256
	CheckSum byte
}

WriterParams describe the parameters for a writer. The defaults are provided by WriterDefaults.

func (*WriterParams) Verify

func (p *WriterParams) Verify() error

Verify checks the writer parameters for invalid values.