binpack

README

binpack

Package binpack implements a compact binary encoding format.

Documentation

Overview

Package binpack implements a compact binary encoding format.

Structured data are represented in binpack as a concatenated sequence of tag-value records. A tag is an unsigned integer value, a value is an array of bytes. The tags and values are opaque to the encoding; the caller must provide additional structure as needed. For example, the application may encode type information in some low-order bits of the tag.

Tags are encoded as 1, 2, or 4 bytes, having values up to 2^30-1. Values are length-prefixed byte arrays up to 2^29-1 bytes in length.

The encoding of a tag is as follows:

Byte 0 (index)
+---------------+
|0|   7 bits    | + 0 bytes  : values 0..127 (7 bits)
+---------------+
|1|0| 6 bits    | + 1 byte   : values 0..16383 (14 bits)
+---------------+
|1|1| 6 bits    | + 3 bytes  : values 0..1073741823 (30 bits)
+---------------+

The first byte of the tag is called the index, and its high-order two bits determine the size of the tag in bytes (0_=1, 01=2, 11=4).

The encoding of a value is as follows:

Byte 0 (index)
+---------------+
|0|   7 bits    | + 0 bytes         : length 1, value 0..127
+---------------+
|1|0| 6 bits    | + 0 bytes + data  : length 0..63
+---------------+
|1|1|0| 5 bits  | + 1 bytes + data  : length 0..8191
+---------------+
|1|1|1| 5 bits  | + 3 bytes + data  : length 0..536870911
+---------------+

The first byte of the value is called the index, and its high-order three bits determine the size of the length prefix. Small single-byte values are encoded directly with a prefix of 0; otherwise the length is 1, 2, or 4 bytes.

Primitive Types

Integer types are sign-extended to 64 bits and encoded using PackUint64 or PackInt64 as appropriate.

Floating-point values are converted to binary using math.Float64bits or math.Float32bits as appropriate, and the resulting bits are encoded as integers.

Index

• func Marshal(v interface{}) ([]byte, error)
• func PackInt64(z int64) []byte
• func PackUint64(z uint64) []byte
• func Unmarshal(data []byte, v interface{}) error
• func UnpackInt64(data []byte) int64
• func UnpackUint64(data []byte) uint64
• type Decoder
  • func NewDecoder(r io.Reader) *Decoder
  • func (d *Decoder) Decode() (int, []byte, error)
• type Encoder
  • func NewEncoder(buf *bytes.Buffer) *Encoder
  • func (e *Encoder) Encode(tag int, value []byte) error

Functions

func Marshal

func Marshal(v interface{}) ([]byte, error)

Marshal encodes v as a binary value for a binpack tag-value pair. if v implements encoding.BinaryMarshaler, that method is called.

For struct types, Marshal uses field tags to select which exported fields should be included and to assign them tag values. The tag format is:

binpack:"tag=n"

where n is an unsigned integer value. Fields without tags are skipped, and zero-valued fields are not encoded.

Slices are marshaled as the concatenation of their contents. A struct field of slice type other than []byte is encoded inline, meaning each slice element is written as a separate tag-value pair within the struct.

Note that map values are encoded in iteration order, which means that marshaling a value that is or contains a map may not be deterministic. Other than maps, however, the output is deterministic.

func PackInt64

func PackInt64(z int64) []byte

PackInt64 encodes z as a slice in big-endian order with zigzag encoding, omitting leading zeroes. The encoding of 0 is a slice of length 1.

Zigzag encoding represents a signed value as the bitwise complement of its 2s complement value, with its sign in the least-significant bit.

func PackUint64

func PackUint64(z uint64) []byte

PackUint64 encodes z as a slice in big-endian order, omitting leading zeroes. The encoding of 0 is a slice of length 1.

func Unmarshal

func Unmarshal(data []byte, v interface{}) error

Unmarshal decodes data from binpack format into v. If v implements encoding.BinaryUnmarshaler, that method is called.

Because the binpack format does not record type information, unmarshaling into an untyped interface will produce the input data unmodified.

func UnpackInt64

func UnpackInt64(data []byte) int64

UnpackInt64 decodes z from a big-endian slice with zigzag encoding.

func UnpackUint64

func UnpackUint64(data []byte) uint64

UnpackUint64 decodes z from a big-endian slice.

Types

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

A Decoder decodes tag-value pairs from an io.Reader.

func NewDecoder

func NewDecoder(r io.Reader) *Decoder

NewDecoder constructs a Decoder that reads records from r.

func (*Decoder) Decode

func (d *Decoder) Decode() (int, []byte, error)

Decode returns the next tag-value record from the reader. At the end of the input, it returns io.EOF.

type Encoder

type Encoder struct {
	Data *bytes.Buffer
}

An Encoder encodes tag-value records to a buffer. Call the Encode method to add values. The buffer can be recovered from the Data field.

func NewEncoder

func NewEncoder(buf *bytes.Buffer) *Encoder

NewEncoder constructs an Encoder that writes data to buf. If buf == nil, a new empty buffer is allocated and can be retrieved from the Data field of the Encoder.

func (*Encoder) Encode

func (e *Encoder) Encode(tag int, value []byte) error

Encode appends a single tag-value pair to the output.