bm

README

bm

A Golang implementation of the Bentley/McIlroy compression algorithm

Documentation

Index

• func DeserializeDictionary(o []byte, m map[uint32]uint32) error
• type Compressor
  • func NewCompressor() *Compressor
  • func (c *Compressor) Close() error
  • func (c *Compressor) CompressedSize() int
  • func (c *Compressor) GetDictionary() *Dictionary
  • func (c *Compressor) InputSize() int
  • func (c *Compressor) Ratio() int
  • func (c *Compressor) SerializeDictionary() ([]byte, error)
  • func (c *Compressor) SetDictionary(dict *Dictionary)
  • func (c *Compressor) SetWriter(w io.Writer)
  • func (c *Compressor) Write(p []byte) (int, error)
• type Dictionary
• type Expander
  • func NewExpander(r io.Reader, dict []byte) *Expander
  • func (e *Expander) Expand(p []byte) (q []byte, err error)

Functions

func DeserializeDictionary

func DeserializeDictionary(o []byte, m map[uint32]uint32) error

DeserializeDictionary reads the H part of the Dictionary from a []byte previously created with SerializeDictionary

Types

type Compressor

type Compressor struct {
	// contains filtered or unexported fields
}

A Compressor is a complete instance of the compressor

func NewCompressor

func NewCompressor() *Compressor

NewCompressor creates a new compressor. The Compressor implements io.Writer and so calling Write() compress and writes to the actual output. Note that you must call SetWriter and SetDictionary before doing any compression to set the output writer.

func (*Compressor) Close

func (c *Compressor) Close() error

Close tells the compressor that all the data has been written. This does not close the underlying io.Writer. This is where the Bentley/McIlroy and Rabin/Karp algorithms are implemented. Reference those papers for a full explanation.

func (*Compressor) CompressedSize

func (c *Compressor) CompressedSize() int

Get the size in bytes of the last compressed output. Only makes sense after Close() has been called.

func (*Compressor) GetDictionary

func (c *Compressor) GetDictionary() *Dictionary

GetDictionary retrieves the dictionary structure for serialization

func (*Compressor) InputSize

func (c *Compressor) InputSize() int

Get the size in bytes of the last uncompressed input. Only makes sense after Close() has been called.

func (*Compressor) Ratio

func (c *Compressor) Ratio() int

Ratio retrieves the compression ratio of the last compression performed. Only makes sense after Close() has been called. The returned value is an integer representing the size of the output as a percentage of the input * 100. If the return value is -1 then it indicates that there was no input.

func (*Compressor) SerializeDictionary

func (c *Compressor) SerializeDictionary() ([]byte, error)

SerializeDictionary turns H (the map part of the Dictionary) into a []byte for easy storage in memcached or elsewhere.

func (*Compressor) SetDictionary

func (c *Compressor) SetDictionary(dict *Dictionary)

SetDictionary sets a dictionary. When a dictionary has been loaded references are made to the dictionary (rather than internally in the compressed data itself)

func (*Compressor) SetWriter

func (c *Compressor) SetWriter(w io.Writer)

SetWriter sets the writer to which the compressed output will be written. This must be called otherwise an error will occur.

func (*Compressor) Write

func (c *Compressor) Write(p []byte) (int, error)

Write implements the io.Writer interface. To compress data Write repeatedly and it will be compressed. When done it is necessary to call Close() where the actual compression occurs.

type Dictionary

type Dictionary struct {
	Dict []byte // Bytes to compress against
	H    map[uint32]uint32
}

A Dictionary contains both the raw data being compressed against and the hash table built using the Rabin/Karp procedure

type Expander

type Expander struct {
	// contains filtered or unexported fields
}

An Expander is the complete state of the expander returned by NewExpander

func NewExpander

func NewExpander(r io.Reader, dict []byte) *Expander

NewExpander creates a new decompressor. Pass in an io.Reader that can be used to read the raw compressed data. The Expander implements io.Reader and so calling Read() decompress data and reads the actual input.

func (*Expander) Expand

func (e *Expander) Expand(p []byte) (q []byte, err error)

Expand expands the compressed data into a buffer