README
¶
zlib for go
This ultra fast go zlib library wraps the original zlib library written in C by Jean-loup Gailly and Mark Adler using cgo.
It offers considerable performance benefits compared to the standard go zlib library - especially on older and lower-end hardware, as the benchmarks show.
This library is designed to be completely and easily interchangeable with the go standard zlib library. You won't have to rewrite or modify a single line of code! Checking if this library works for you is as easy as changing imports!
This library also offers blazing fast convenience methods that can be used as a clean, alternative interface to that provided by the go standard library. (See usage).
With this library you may also choose from different compression strategies on top of different compression levels.
Table of Contents
Features
- zlib compression / decompression
- A variety of different
compression strategiesandcompression levelsto choose from - Seamless interchangeability with the go standard zlib library
- Alternative, super fast convenience methods for compression / decompression
- Benchmarks with comparisons to the go standard zlib library
- Custom, user-defined dictionaries
Installation
Prerequisites
In order to use this library with your go source code, you must be able to use the go tool cgo, which in turn requires a GCC compiler.
If you are on Linux, there is a good chance you already have GCC installed, otherwise just get it with your favorite package manager.
If you are on MacOS, Xcode - for instance - supplies the required tools.
If you are on Windows, you will need to install a GNU environment. I can recommend tdm-gcc which is based off of MinGW. Please note that cgo requires the 64-bit version (as stated here).
Download and Installation
To get the most recent stable version just type:
$ go get github.com/4kills/zlib
You may also use go modules (available since go 1.11) to get the version of a specific branch or tag if you want to try out or use experimental features. However, beware that these versions are not necessarily guaranteed to be stable or thoroughly tested.
Import
This library is designed in a way to make it easy to swap it out for the go standard zlib library. Therefore you should only need to change imports and not a single line of your written code.
Just remove:
import compress/zlib
and use instead:
import "github.com/4kills/zlib"
If there are any problems with your existing code after this step, please let me know.
Usage
This library can be used exactly like the go standard zlib library but it also adds additional methods to make your life easier.
Compress
Like with the standard library:
var b bytes.Buffer // use any writer
w := zlib.NewWriter(&b) // create a new zlib.Writer, compressing to b
defer w.Close() // don't forget to close this
w.Write([]byte("uncompressed")) // put in any data as []byte
Or alternatively:
w := zlib.NewWriter(nil) // requires no writer if WriteBytes is used
defer w.Close() // always close when you are done with it
c, _ := w.WriteBytes([]byte("uncompressed")) // compresses input & returns compressed []byte
Decompress
Like with the standard library:
b := bytes.NewBuffer(compressed) // reader with compressed data
r, err := zlib.NewReader(&b) // create a new zlib.Reader, decompressing from b
defer r.Close() // don't forget to close this either
io.Copy(os.Stdout, r) // read all the decompressed data and write it somewhere
// or:
// r.Read(someBuffer) // can also be done directly
Or alternatively:
r := zlib.NewReader(nil) // requires no reader if ReadBytes is used
defer r.Close() // always close or bad things will happen
dc, _ := r.ReadBytes(compressed) // decompresses input & returns decompressed []byte
Notes
-
Do NOT use the same Reader / Writer across multiple threads simultaneously. You can do that if you sync the read/write operations, but you could also create as many readers/writers as you like - for each thread one, so to speak. This library is generally considered thread-safe.
-
Always
Close()your Reader / Writer when you are done with it - especially if you create a new reader/writer for each decompression/compression you undertake (which is generally discouraged anyway). As the C-part of this library is not subject to the go garbage collector, the memory allocated by it must be released manually (by a call toClose()) to avoid memory leakage. -
HuffmanOnlydoes NOT work as with the standard library. This is the only exception from the philosophy to make this library interchangeable with the standard library. If you want to useHuffmanOnly, refer to theNewWriterLevelStrategy()constructor function. However, your existing code won't break by leavingHuffmanOnlyas argument toNewWriterLevel(), it will just use the default compression strategy and compression level 2. -
You are strongly encouraged to use the same Reader / Writer for multiple Decompressions / Compressions as it is not required nor beneficial in any way, shape or form to create a new one every time. The contrary is true: It is more performant to reuse a reader/writer. Of course, if you use the same reader/writer multiple times, you do not need to close them until you are completely done with them (perhaps only at the very end of your program).
Benchmarks
License
Copyright (c) 1995-2017 Jean-loup Gailly and Mark Adler
Copyright (c) 2020 Dominik Ochs
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
Links
Documentation
¶
Index ¶
- Constants
- type Reader
- type Resetter
- type Writer
- func NewWriter(w io.Writer) *Writer
- func NewWriterLevel(w io.Writer, level int) (*Writer, error)
- func NewWriterLevelDict(w io.Writer, level int, dict []byte) (*Writer, error)
- func NewWriterLevelStrategy(w io.Writer, level, strategy int) (*Writer, error)
- func NewWriterLevelStrategyDict(w io.Writer, level, strategy int, dict []byte) (*Writer, error)
Constants ¶
const ( //NoCompression does not compress given input NoCompression = 0 //BestSpeed is fastest but with lowest compression BestSpeed = 1 //BestCompression is slowest but with best compression BestCompression = 9 //DefaultCompression is a compromise between BestSpeed and BestCompression. //The level might change if algorithms change. DefaultCompression = -1 // Filtered is more effective for small (but not all too many) randomly distributed values. // It forces more Huffman encoding. Use it for filtered data. It's between Default and Huffman only. Filtered = 1 //HuffmanOnly only uses Huffman encoding to compress the given data HuffmanOnly = 2 //RLE (run-length encoding) limits match distance to one, thereby being almost as fast as HuffmanOnly // but giving better compression for PNG data. RLE = 3 //Fixed disallows dynamic Huffman codes, thereby making it a simpler decoder Fixed = 4 // DefaultStrategy is the default compression strategy that should be used for most appliances DefaultStrategy = 0 )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader decompresses data from an underlying io.Reader or via the ReadBytes method, which should be preferred
func NewReader ¶
NewReader returns a new reader, reading from r. It decompresses read data. r may be nil if you only plan on using ReadBytes
func NewReaderDict ¶
NewReaderDict does exactly like NewReader as of NOW. This will change once custom dicionaries are implemented. This function has been added for compatibility with the std lib.
func (*Reader) Close ¶
Close closes the Reader by closing and freeing the underlying zlib stream. You should not forget to call this after being done with the writer.
func (*Reader) Read ¶
Read reads compressed data from the provided Reader into the provided buffer p. Please consider using ReadBytes instead, as it is slightly faster and generally easier to use
func (*Reader) ReadBytes ¶
ReadBytes takes compressed data p, decompresses it and returns it as new byte slice. It also returns the number n of bytes that were processed from the compressed slice. If n < len(compressed) and err == nil then only the first n compressed bytes were in a suitable zlib format and as such decompressed. This method is generally slightly faster than Read.
func (*Reader) Reset ¶
Reset resets the Reader to the state of being initialized with zlib.NewX(..), but with the new underlying reader instead. It allows for reuse of the same reader. AS OF NOW dict IS NOT USED. It's just there to implement the Resetter interface to allow for easy interchangeablility with the std lib. Just pass nil.
type Resetter ¶
type Resetter interface {
// Reset resets the Reader to the state of being initialized with zlib.NewX(..),
// but with the new underlying reader and dict instead. It allows for reuse of the same reader.
Reset(r io.Reader, dict []byte) error
}
Resetter resets the zlib.Reader returned by NewReader by assigning a new underyling reader, discarding any buffered data from the previous reader. This interface is mainly for compatibility with the std lib
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer compresses and writes given data to an underlying io.Writer
func NewWriter ¶
NewWriter returns a new Writer with the underlying io.Writer to compress to. w may be nil if you only plan on using WriteBytes. Panics if the underlying c stream cannot be allocated which would indicate a severe error not only for this library but also for the rest of your code.
func NewWriterLevel ¶
NewWriterLevel performs like NewWriter but you may also specify the compression level. w may be nil if you only plan on using WriteBytes.
func NewWriterLevelDict ¶
NewWriterLevelDict does exactly like NewWriterLevel as of NOW. This will change once custom dicionaries are implemented. This function has been added for compatibility with the std lib.
func NewWriterLevelStrategy ¶
NewWriterLevelStrategy performs like NewWriter but you may also specify the compression level and strategy. w may be nil if you only plan on using WriteBytes.
func NewWriterLevelStrategyDict ¶
NewWriterLevelStrategyDict does exactly like NewWriterLevelStrategy as of NOW. This will change once custom dicionaries are implemented. This function has been added mainly for completeness' sake.
func (*Writer) Close ¶
Close closes the writer by flushing any unwritten data to the underlying writer. You should not forget to call this after being done with the writer.
func (*Writer) Flush ¶
Flush does NOTHING. Write always flushes content to the underlying writer. This method just exists for easy interchangeability with the go std zlib library
func (*Writer) Reset ¶
Reset resets the Writer to the state of being initialized with zlib.NewX(..), but with the new underlying writer instead. This will panic if the writer has already been closed
Source Files
Directories