compress

package module
Version: v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 6, 2019 License: BSD-3-Clause Imports: 3 Imported by: 0

README

Collection of compression libraries for Go

GoDoc Build Status Report Card

Introduction

NOTE: This library is in active development. As such, there are no guarantees about the stability of the API. The author reserves the right to arbitrarily break the API for any reason.

This repository hosts a collection of compression related libraries. The goal of this project is to provide pure Go implementations for popular compression algorithms beyond what the Go standard library provides. The goals for these packages are as follows:

  • Maintainable: That the code remains well documented, well tested, readable, easy to maintain, and easy to verify that it conforms to the specification for the format being implemented.
  • Performant: To be able to compress and decompress within at least 80% of the rates that the C implementations are able to achieve.
  • Flexible: That the code provides low-level and fine granularity control over the compression streams similar to what the C APIs would provide.

Of these three, the first objective is often at odds with the other two objectives and provides interesting challenges. Higher performance can often be achieved by muddling abstraction layers or using non-intuitive low-level primitives. Also, more features and functionality, while useful in some situations, often complicates the API. Thus, this package will attempt to satisfy all the goals, but will defer to favoring maintainability when the performance or flexibility benefits are not significant enough.

Library Status

For the packages available, only some features are currently implemented:

Package Reader Writer
brotli
bzip2
flate
xflate

This library is in active development. As such, there are no guarantees about the stability of the API. The author reserves the right to arbitrarily break the API for any reason. When the library becomes more mature, it is planned to eventually conform to some strict versioning scheme like Semantic Versioning.

However, in the meanwhile, this library does provide some basic API guarantees. For the types defined below, the method signatures are guaranteed to not change. Note that the author still reserves the right to change the fields within each Reader and Writer structs.

type ReaderConfig struct { ... }
type Reader struct { ... }
  func NewReader(io.Reader, *ReaderConfig) (*Reader, error) { ... }
  func (*Reader) Read([]byte) (int, error)                  { ... }
  func (*Reader) Close() error                              { ... }

type WriterConfig struct { ... }
type Writer struct { ... }
  func NewWriter(io.Writer, *WriterConfig) (*Writer, error) { ... }
  func (*Writer) Write([]byte) (int, error)                 { ... }
  func (*Writer) Close() error                              { ... }

To see what work still remains, see the Task List.

Performance

See Performance Metrics.

Frequently Asked Questions

See Frequently Asked Questions.

Installation

Run the command:

go get -u github.com/dsnet/compress

This library requires Go1.9 or higher in order to build.

Packages

Package Description
brotli Package brotli implements the Brotli format, described in RFC 7932.
bzip2 Package bzip2 implements the BZip2 compressed data format.
flate Package flate implements the DEFLATE format, described in RFC 1951.
xflate Package xflate implements the XFLATE format, an random-access extension to DEFLATE.

Documentation

Overview

Package compress is a collection of compression libraries.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BufferedReader

type BufferedReader interface {
	io.Reader

	// Buffered returns the number of bytes currently buffered.
	//
	// This value becomes invalid following the next Read/Discard operation.
	Buffered() int

	// Peek returns the next n bytes without advancing the reader.
	//
	// If Peek returns fewer than n bytes, it also returns an error explaining
	// why the peek is short. Peek must support peeking of at least 8 bytes.
	// If 0 <= n <= Buffered(), Peek is guaranteed to succeed without reading
	// from the underlying io.Reader.
	//
	// This result becomes invalid following the next Read/Discard operation.
	Peek(n int) ([]byte, error)

	// Discard skips the next n bytes, returning the number of bytes discarded.
	//
	// If Discard skips fewer than n bytes, it also returns an error.
	// If 0 <= n <= Buffered(), Discard is guaranteed to succeed without reading
	// from the underlying io.Reader.
	Discard(n int) (int, error)
}

BufferedReader is an interface accepted by all decompression Readers. It guarantees that the decompressor never reads more data than is necessary from the underlying io.Reader. Since BufferedReader allows a decompressor to peek at bytes further along in the stream without advancing the read pointer, decompression can experience a significant performance gain when provided a reader that satisfies this interface. Thus, a decompressor will prefer this interface over ByteReader for performance reasons.

The bufio.Reader satisfies this interface.

type ByteReader

type ByteReader interface {
	io.Reader
	io.ByteReader
}

ByteReader is an interface accepted by all decompression Readers. It guarantees that the decompressor never reads more data than is necessary from the underlying io.Reader.

type Error

type Error interface {
	error
	CompressError()

	// IsDeprecated reports the use of a deprecated and unsupported feature.
	IsDeprecated() bool

	// IsCorrupted reports whether the input stream was corrupted.
	IsCorrupted() bool
}

The Error interface identifies all compression related errors.

Source Files

Directories

Path Synopsis
Package brotli implements the Brotli compressed data format, described in RFC 7932.
Package brotli implements the Brotli compressed data format, described in RFC 7932.
Package bzip2 implements the BZip2 compressed data format.
Package bzip2 implements the BZip2 compressed data format.
Package flate implements the DEFLATE compressed data format, described in RFC 1951.
Package flate implements the DEFLATE compressed data format, described in RFC 1951.
Package xflate implements the XFLATE compressed data format.
Package xflate implements the XFLATE compressed data format.
Package internal is a collection of common compression algorithms.
Package internal is a collection of common compression algorithms.
cgo/brotli
Package brotli implements the Brotli compressed data format using C wrappers.
Package brotli implements the Brotli compressed data format using C wrappers.
cgo/bzip2
Package bzip2 implements the BZip2 compressed data format using C wrappers.
Package bzip2 implements the BZip2 compressed data format using C wrappers.
cgo/flate
Package flate implements the DEFLATE compressed data format, described in RFC 1951, using C wrappers.
Package flate implements the DEFLATE compressed data format, described in RFC 1951, using C wrappers.
cgo/lzma
Package lzma implements the LZMA2 compressed data format using C wrappers.
Package lzma implements the LZMA2 compressed data format using C wrappers.
cgo/zstd
Package zstd implements the Zstandard compressed data format using C wrappers.
Package zstd implements the Zstandard compressed data format using C wrappers.
errors
Package errors implements functions to manipulate compression errors.
Package errors implements functions to manipulate compression errors.
prefix
Package prefix implements bit readers and writers that use prefix encoding.
Package prefix implements bit readers and writers that use prefix encoding.
testutil
Package testutil is a collection of testing helper methods.
Package testutil is a collection of testing helper methods.
tool/bench
Benchmark tool to compare performance between multiple compression implementations.
Benchmark tool to compare performance between multiple compression implementations.
tool/bitgen
BitGen to generate a binary from a BitGen formatting input.
BitGen to generate a binary from a BitGen formatting input.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL