binary

README

Generic and Fast Binary Serializer for Go

This repository contains a fast binary packer for Golang, this allows to encode/decode arbtitrary golang data structures of variable size. Documentation can be found on https://godoc.org/github.com/Kelindar/binary.

This package extends support to arbitrary, variable-sized values by prefixing these values with their varint-encoded size, recursively. This was originally inspired by Alec Thomas's binary package, but I've reworked the serialization format and improved the performance and size. Here's a few notable features/goals of this binary package:

• Zero-allocation encoding. I'm hoping to make the encoding to be as fast as possible, simply writing binary to the io.Writer without unncessary allocations.
• Support for maps, arrays, slices, structs, primitive and nested types.
• This is essentially a json.Marshal and json.Unmarshal drop-in replacement, I wanted this package to be simple to use and leverage the power of reflect package of golang.
• The ints and uints are encoded using varint, making the payload small as possible.
• Fast-paths encoding and decoding of []byte, as I've designed this package to be used for inter-broker message encoding for emitter.
• Support for custom BinaryMarshaler and BinaryUnmarshaler for tighter packing control and built-in types such as time.Time.

Usage

To serialize a message, simply Marshal:

v := &message{
    Name:      "Roman",
    Timestamp: 1242345235,
    Payload:   []byte("hi"),
    Ssid:      []uint32{1, 2, 3},
}

encoded, err := binary.Marshal(v)

To deserialize, Unmarshal:

var v message
err := binary.Unmarshal(encoded, &v)

Disclaimer

This is not intended as a replacement for JSON or protobuf, this codec does not maintain any versioning or compatibility - and not intended to become one. The goal of this binary codec is to efficiently exchange binary data of known format between systems where you control both ends and both of them are written in Go.

Documentation

Index

• Variables
• func Marshal(v interface{}) (output []byte, err error)
• func MarshalTo(v interface{}, dst io.Writer) (err error)
• func ToBytes(v string) []byte
• func ToString(b *[]byte) string
• func Unmarshal(b []byte, v interface{}) (err error)
• type Codec
• type Decoder
  • func NewDecoder(r io.Reader) *Decoder
  • func (d *Decoder) Decode(v interface{}) (err error)
  • func (d *Decoder) Read(b []byte) (int, error)
  • func (d *Decoder) ReadBool() (bool, error)
  • func (d *Decoder) ReadFloat32() (out float32, err error)
  • func (d *Decoder) ReadFloat64() (out float64, err error)
  • func (d *Decoder) ReadSlice() (b []byte, err error)
  • func (d *Decoder) ReadString() (out string, err error)
  • func (d *Decoder) ReadUint16() (out uint16, err error)
  • func (d *Decoder) ReadUint32() (out uint32, err error)
  • func (d *Decoder) ReadUint64() (out uint64, err error)
  • func (d *Decoder) ReadUvarint() (uint64, error)
  • func (d *Decoder) ReadVarint() (int64, error)
  • func (d *Decoder) Slice(n int) ([]byte, error)
• type Encoder
  • func NewEncoder(out io.Writer) *Encoder
  • func (e *Encoder) Buffer() io.Writer
  • func (e *Encoder) Encode(v interface{}) (err error)
  • func (e *Encoder) Reset(out io.Writer)
  • func (e *Encoder) Write(p []byte)
  • func (e *Encoder) WriteFloat32(v float32)
  • func (e *Encoder) WriteFloat64(v float64)
  • func (e *Encoder) WriteString(v string)
  • func (e *Encoder) WriteUint16(v uint16)
  • func (e *Encoder) WriteUint32(v uint32)
  • func (e *Encoder) WriteUint64(v uint64)
  • func (e *Encoder) WriteUvarint(x uint64)
  • func (e *Encoder) WriteVarint(v int64)
• type Reader

Variables

var (
	LittleEndian = binary.LittleEndian
	BigEndian    = binary.BigEndian
)

Constants

Functions

func Marshal

func Marshal(v interface{}) (output []byte, err error)

Marshal encodes the payload into binary format.

func MarshalTo

func MarshalTo(v interface{}, dst io.Writer) (err error)

MarshalTo encodes the payload into a specific destination.

func ToBytes

func ToBytes(v string) []byte

ToBytes converts a string to a byte slice without allocating.

func ToString

func ToString(b *[]byte) string

ToString converts byte slice to a string without allocating.

func Unmarshal

func Unmarshal(b []byte, v interface{}) (err error)

Unmarshal decodes the payload from the binary format.

Types

type Codec

type Codec interface {
	EncodeTo(*Encoder, reflect.Value) error
	DecodeTo(*Decoder, reflect.Value) error
}

Codec represents a single part Codec, which can encode and decode something.

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder represents a binary decoder.

func NewDecoder

func NewDecoder(r io.Reader) *Decoder

NewDecoder creates a binary decoder.

func (*Decoder) Decode

func (d *Decoder) Decode(v interface{}) (err error)

Decode decodes a value by reading from the underlying io.Reader.

func (*Decoder) Read

func (d *Decoder) Read(b []byte) (int, error)

Read reads a set of bytes

func (*Decoder) ReadBool

func (d *Decoder) ReadBool() (bool, error)

ReadBool reads a single boolean value from the slice.

func (*Decoder) ReadFloat32

func (d *Decoder) ReadFloat32() (out float32, err error)

ReadFloat32 reads a float32

func (*Decoder) ReadFloat64

func (d *Decoder) ReadFloat64() (out float64, err error)

ReadFloat64 reads a float64

func (*Decoder) ReadSlice

func (d *Decoder) ReadSlice() (b []byte, err error)

ReadSlice reads a varint prefixed sub-slice without copying and returns the underlying byte slice.

func (*Decoder) ReadString

func (d *Decoder) ReadString() (out string, err error)

ReadString a string prefixed with a variable-size integer size.

func (*Decoder) ReadUint16

func (d *Decoder) ReadUint16() (out uint16, err error)

ReadUint16 reads a uint16

func (*Decoder) ReadUint32

func (d *Decoder) ReadUint32() (out uint32, err error)

ReadUint32 reads a uint32

func (*Decoder) ReadUint64

func (d *Decoder) ReadUint64() (out uint64, err error)

ReadUint64 reads a uint64

func (*Decoder) ReadUvarint

func (d *Decoder) ReadUvarint() (uint64, error)

ReadUvarint reads a variable-length Uint64 from the buffer.

func (*Decoder) ReadVarint

func (d *Decoder) ReadVarint() (int64, error)

ReadVarint reads a variable-length Int64 from the buffer.

func (*Decoder) Slice

func (d *Decoder) Slice(n int) ([]byte, error)

Slice selects a sub-slice of next bytes. This is similar to Read() but does not actually perform a copy, but simply uses the underlying slice (if available) and returns a sub-slice pointing to the same array. Since this requires access to the underlying data, this is only available for a slice reader.

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder represents a binary encoder.

func NewEncoder

func NewEncoder(out io.Writer) *Encoder

NewEncoder creates a new encoder.

func (*Encoder) Buffer

func (e *Encoder) Buffer() io.Writer

Buffer returns the underlying writer.

func (*Encoder) Encode

func (e *Encoder) Encode(v interface{}) (err error)

Encode encodes the value to the binary format.

func (*Encoder) Reset

func (e *Encoder) Reset(out io.Writer)

Reset resets the encoder and makes it ready to be reused.

func (*Encoder) Write

func (e *Encoder) Write(p []byte)

Write writes the contents of p into the buffer.

func (*Encoder) WriteFloat32

func (e *Encoder) WriteFloat32(v float32)

WriteFloat32 a 32-bit floating point number

func (*Encoder) WriteFloat64

func (e *Encoder) WriteFloat64(v float64)

WriteFloat64 a 64-bit floating point number

func (*Encoder) WriteString

func (e *Encoder) WriteString(v string)

WriteString writes a string prefixed with a variable-size integer size.

func (*Encoder) WriteUint16

func (e *Encoder) WriteUint16(v uint16)

WriteUint16 writes a Uint16

func (*Encoder) WriteUint32

func (e *Encoder) WriteUint32(v uint32)

WriteUint32 writes a Uint32

func (*Encoder) WriteUint64

func (e *Encoder) WriteUint64(v uint64)

WriteUint64 writes a Uint64

func (*Encoder) WriteUvarint

func (e *Encoder) WriteUvarint(x uint64)

WriteUvarint writes a variable size unsigned integer

func (*Encoder) WriteVarint

func (e *Encoder) WriteVarint(v int64)

WriteVarint writes a variable size integer

type Reader

type Reader interface {
	io.Reader
	io.ByteReader
}

Reader represents the interface a reader should implement.