sbor

README

MessagePack serializer in Go

SBOR is a modern and straightforward MessagePack serializer written completely in Go, without the use of code that use "unsafe" package and thus preserving the cross-compilation characteristics of the language.

The aim of the project is to make a library that balances the performance with the ease of use. Its code must be easy and understandable, and the tests must be adequate, with unit test code coverage greater than 95%.

Resources

• Reference
• Examples

What is MessagePack?

MessagePack is an efficient binary serialization format that lets you exchange data among multiple languages, like JSON, but it's faster and smaller, and provide support to custom types defined by the user, called extension. In addition, it supports the transmission of raw bytes, unlike JSON. If you are curious, check its specification. MessagePack is not human-readable directly.

Meaning of the project name

SBOR is an acronym that stands for Serializer to Binary Object Representation, to use a short name for the library and highlight the fact that the format is binary. Moreover, in the Czech language this word means "choir", and this could represent the set of different functions that compose the library and that work together to provide the correct output.

Contrary to what the name might say, this project currently has no connection with CBOR format, but potential support in the future for that too, as some of its features are similar to MessagePack, should not be ruled out.

Installation

You can install this library using:

go get github.com/ErikPelli/sbor

At the moment it's still in unstable version.

Features

• Encoding of primitives, time.Time, arrays, slices, maps, structs and value contained in an interface
• Encoding of struct as an array or as a map (key value)
• Omit only specified fields using sbor:"-"
• Renaming of fields using sbor:"new_field_name"
• Support for every type as the key (it could be an integer, a map, an array, etc.), using custom keys

TODO

• Cache intermediate results to avoid repetition of certain operations when encoding
• Decode from MessagePack bytes to Go data types

Quickstart

Please read carefully the documentation contained in reference to learn more about the available functions.

package example

import (
	"fmt"
	"github.com/ErikPelli/sbor"
)

func StructMarshalExample() {
	type Test struct {
		Keys  map[string][]byte `sbor:",setcustomkeys"`
		Hello string            `sbor:",customkey"`
	}
	
	s := Test{
		Keys: map[string][]byte{"Hello" : {0xCC, 0x11, 0xAA, 0x00}},
		Hello: "world",
	}

	result, err := sbor.Marshal(s)
	if err != nil {
		panic(err)
	}
	
	// Byte slice result equivalent to:
	// {
	//      [0xCC, 0x11, 0xAA, 0x00]: "world"
	// }
	fmt.Println(result)
}

License

This project is licensed under the MIT License. See LICENSE for the full license text.

Documentation

Index

• func Marshal(v interface{}) ([]byte, error)
• type CustomEncoder
• type Encoder
  • func NewEncoder(w io.Writer) *Encoder
  • func (e *Encoder) Encode(v interface{}) error
  • func (e *Encoder) SetExternalType(id int8, value interface{}, c ...CustomEncoder) error
• type MessagePackCustom

Functions

func Marshal

func Marshal(v interface{}) ([]byte, error)

Marshal returns the MessagePack encoding of v.

External types are supported only through Encoder, to be able to separate different session, each one with its own defined external types.

Marshal uses the following type-dependent default encodings:

Boolean values encode as MessagePack boolean.

Nil values encode as MessagePack nil (for example an empty pointer).

Integer values encode as MessagePack int.

Floating point values encode as MessagePack float.

String values encode as MessagePack string.

Array, channel and slice values encode as MessagePack array, except that []byte encodes as MessagePack binary, and a nil slice encodes as an empty MessagePack array (length=0).

Struct values encode as MessagePack map. Each exported struct field becomes a member of the object, using the field name as the object key, unless the field is omitted for one of the reasons given below.

The encoding of each struct field can be customized by the format string stored under the "sbor" key in the struct field's tag. The format string gives the name of the field, possibly followed by a comma-separated list of options. The name may be empty in order to specify options without overriding the default field name.

The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.

The "structarray" option specifies that the current struct must be encoded as an array instead of a map, so all the keys will be discarded.

As a special case, if the field tag is "-", the field is always omitted. Note that a field with name "-" can still be generated using the tag "-,". Examples of struct field tags and their meanings:

// Field appears in MessagePack as key "myName".
Field int `sbor:"myName"`

// Field appears in MessagePack as key "myName" and
// the field is omitted from the object if its value is empty,
// as defined above.
Field int `sbor:"myName,omitempty"`

// Field appears in MessagePack as key "Field" (the default), but
// the field is skipped if empty.
// Note the leading comma.
Field int `sbor:",omitempty"`

// Field appears in MessagePack as key "Field" (the default).
// Whole struct is now an array.
Field int `sbor:",structarray"`

// Field is ignored by this package.
Field int `sbor:"-"`

// Field appears in MessagePack as key "-".
Field int `sbor:"-,"`

It's possible to use other types as key than string. You can use "setcustomkeys" option on an exported field which is a map that have a string as its key. In that way, with the customkey option, you can use a string as the key and change the MessagePack key with the value set with this option. If you use this option, this field won't be in the MessagePack, it's automatically skipped. This option must not contain duplicated values for different keys, or the behavior maybe bad.

The "customkey" specifies that the current name is only a key for the precedent set custom keys collection, and so the MessagePack key is the value correspondent to the field name.

// Now this map is used to get correspondent key for other fields.
Keys map[string]int `sbor:",setcustomkeys"`

// Field appears in MessagePack as key "myName".
Field int `sbor:"myName"`

// "myName" is used as key in the Keys map.
// In this case the key will be an integer.
// If "myName" doesn't exists in map, en error is returned.
Field int `sbor:"myName,customkey"`

If a struct has multiple field with the same name, an error will be returned.

Anonymous struct fields are usually marshaled as sub-maps, with the field name as key in their parent, or a custom name given in its field tag. To force ignoring of an anonymous struct field, give the field a tag of "-".

Map values encode as MessagePack maps. The map's key type can be any type supported by MessagePack (int, float, string, map, array, ...). Keys of any type are used directly.

Pointer values encode as the value pointed to. A nil pointer encodes as the nil MessagePack value.

Interface values encode as the value contained in the interface. A nil interface value encodes as the nil MessagePack value.

Complex and function values cannot be encoded in MessagePack. Attempting to encode such a value causes Marshal to return an InvalidTypeError.

Marshal handle cyclic data structures representing them only once, to avoid an infinite loop.

Types

type CustomEncoder

type CustomEncoder struct {
	Encoder func(i interface{}) ([]byte, error)
}

CustomEncoder specifies a function to encode the type when the MessagePackCustom interface is not implemented by the type. This is useful for primitive types unhandled by this library, such as complex64 and complex128.

Encoder receives as input the associated type and must return its byte slice representation, or an error if the input is invalid.

To get the type you should do a type assertion without checking for the success, because this function will be used only with the associated type. Ex: value := i.(complex64)

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

An Encoder writes MessagePack values to an output stream.

func NewEncoder

func NewEncoder(w io.Writer) *Encoder

NewEncoder returns a new encoder that writes to w.

func (*Encoder) Encode

func (e *Encoder) Encode(v interface{}) error

Encode writes the MessagePack encoding of v to the stream. See the documentation for Marshal for details about the conversion of Go values to MessagePack.

func (*Encoder) SetExternalType

func (e *Encoder) SetExternalType(id int8, value interface{}, c ...CustomEncoder) error

SetExternalType associate a type with an external type code, for this Encoder. This link between them is used when encoding a custom type in MessagePack, to give the user the freedom to do his own encoding for the current external type.

ID is the correspondent MessagePack External type, and must be a number between 0 and 127.

Value is an instance of the type, it can be zero value or can contain a value, the important thing is that it belongs to the type we have to encode.

If value implements MessagePackCustom interface, the corresponding method MarshalMsgpack will be used for encoding, else you have to provide a CustomEncoder function.

type MessagePackCustom

type MessagePackCustom interface {
	MarshalMsgpack() ([]byte, error)
	UnmarshalMsgpack([]byte) error
}

MessagePackCustom is used to encode an external type for which the user have to implement his own code for the encoding.