Back to godoc.org
github.com/tendermint/go-amino

Package amino

v0.15.1
Latest Go to latest
Published: Oct 11, 2019 | License: Apache-2.0 | Module: github.com/tendermint/go-amino

Overview

Amino is an encoding library that can handle interfaces (like protobuf "oneof") well. This is achieved by prefixing bytes before each "concrete type".

A concrete type is some non-interface value (generally a struct) which implements the interface to be (de)serialized. Not all structures need to be registered as concrete types -- only when they will be stored in interface type fields (or interface type slices) do they need to be registered.

Registering types

All interfaces and the concrete types that implement them must be registered.

amino.RegisterInterface((*MyInterface1)(nil), nil)
amino.RegisterInterface((*MyInterface2)(nil), nil)
amino.RegisterConcrete(MyStruct1{}, "com.tendermint/MyStruct1", nil)
amino.RegisterConcrete(MyStruct2{}, "com.tendermint/MyStruct2", nil)
amino.RegisterConcrete(&MyStruct3{}, "anythingcangoinhereifitsunique", nil)

Notice that an interface is represented by a nil pointer.

Structures that must be deserialized as pointer values must be registered with a pointer value as well. It's OK to (de)serialize such structures in non-pointer (value) form, but when deserializing such structures into an interface field, they will always be deserialized as pointers.

How it works

All registered concrete types are encoded with leading 4 bytes (called "prefix bytes"), even when it's not held in an interface field/element. In this way, Amino ensures that concrete types (almost) always have the same canonical representation. The first byte of the prefix bytes must not be a zero byte, so there are 2**(8*4)-2**(8*3) possible values.

When there are 4096 types registered at once, the probability of there being a conflict is ~ 0.2%. See https://instacalc.com/51189 for estimation. This is assuming that all registered concrete types have unique natural names (e.g. prefixed by a unique entity name such as "com.tendermint/", and not "mined/grinded" to produce a particular sequence of "prefix bytes").

TODO Update instacalc.com link with 255/256 since 0x00 is an escape.

Do not mine/grind to produce a particular sequence of prefix bytes, and avoid using dependencies that do so.

Since 4 bytes are not sufficient to ensure no conflicts, sometimes it is necessary to prepend more than the 4 prefix bytes for disambiguation. Like the prefix bytes, the disambiguation bytes are also computed from the registered name of the concrete type. There are 3 disambiguation bytes, and in binary form they always precede the prefix bytes. The first byte of the disambiguation bytes must not be a zero byte, so there are 2**(8*3)-2**(8*2) possible values.

// Sample Amino encoded binary bytes with 4 prefix bytes.
> [0xBB 0x9C 0x83 0xDD] [...]

// Sample Amino encoded binary bytes with 3 disambiguation bytes and 4
// prefix bytes.
> 0x00 <0xA8 0xFC 0x54> [0xBB 0x9C 0x83 0xDD] [...]

The prefix bytes never start with a zero byte, so the disambiguation bytes are escaped with 0x00.

Notice that the 4 prefix bytes always immediately precede the binary encoding of the concrete type.

Computing prefix bytes

To compute the disambiguation bytes, we take `hash := sha256(concreteTypeName)`, and drop the leading 0x00 bytes.

> hash := sha256("com.tendermint.consensus/MyConcreteName")
> hex.EncodeBytes(hash) // 0x{00 00 A8 FC 54 00 00 00 BB 9C 83 DD ...} (example)

In the example above, hash has two leading 0x00 bytes, so we drop them.

> rest = dropLeadingZeroBytes(hash) // 0x{A8 FC 54 00 00 BB 9C 83 DD ...}
> disamb = rest[0:3]
> rest = dropLeadingZeroBytes(rest[3:])
> prefix = rest[0:4]

The first 3 bytes are called the "disambiguation bytes" (in angle brackets). The next 4 bytes are called the "prefix bytes" (in square brackets).

> <0xA8 0xFC 0x54> [0xBB 0x9C 9x83 9xDD]
Example

Code:

defer func() {
	if e := recover(); e != nil {
		fmt.Println("Recovered:", e)
	}
}()

type Message interface{}

type bcMessage struct {
	Message string
	Height  int
}

type bcResponse struct {
	Status  int
	Message string
}

type bcStatus struct {
	Peers int
}

var cdc = amino.NewCodec()
cdc.RegisterInterface((*Message)(nil), nil)
cdc.RegisterConcrete(&bcMessage{}, "bcMessage", nil)
cdc.RegisterConcrete(&bcResponse{}, "bcResponse", nil)
cdc.RegisterConcrete(&bcStatus{}, "bcStatus", nil)

var bm = &bcMessage{Message: "ABC", Height: 100}
var msg = bm

var bz []byte // the marshalled bytes.
var err error
bz, err = cdc.MarshalBinaryLengthPrefixed(msg)
fmt.Printf("Encoded: %X (err: %v)\n", bz, err)

var msg2 Message
err = cdc.UnmarshalBinaryLengthPrefixed(bz, &msg2)
fmt.Printf("Decoded: %v (err: %v)\n", msg2, err)
var bm2 = msg2.(*bcMessage)
fmt.Printf("Decoded successfully: %v\n", *bm == *bm2)
Encoded: 0B740613650A034142431064 (err: <nil>)
Decoded: &{ABC 100} (err: <nil>)
Decoded successfully: true

Index

Examples

Package Files

Constants

const (
	// Typ3 types
	Typ3_Varint     = Typ3(0)
	Typ3_8Byte      = Typ3(1)
	Typ3_ByteLength = Typ3(2)
	//Typ3_Struct     = Typ3(3)
	//Typ3_StructTerm = Typ3(4)
	Typ3_4Byte = Typ3(5)
)
const (
	PrefixBytesLen = 4
	DisambBytesLen = 3
	DisfixBytesLen = PrefixBytesLen + DisambBytesLen
)

Lengths

const Version = "0.15.0"

Version

Variables

var (
	ErrOverflowInt = errors.New("encoded integer value overflows int(32)")
)

func ByteSliceSize

func ByteSliceSize(bz []byte) int

func DecodeBool

func DecodeBool(bz []byte) (b bool, n int, err error)

func DecodeByte

func DecodeByte(bz []byte) (b byte, n int, err error)

func DecodeByteSlice

func DecodeByteSlice(bz []byte) (bz2 []byte, n int, err error)

func DecodeDisambPrefixBytes

func DecodeDisambPrefixBytes(bz []byte) (db DisambBytes, hasDb bool, pb PrefixBytes, hasPb bool, n int, err error)

func DecodeFloat32

func DecodeFloat32(bz []byte) (f float32, n int, err error)

NOTE: UNSAFE

func DecodeFloat64

func DecodeFloat64(bz []byte) (f float64, n int, err error)

NOTE: UNSAFE

func DecodeInt16

func DecodeInt16(bz []byte) (i int16, n int, err error)

func DecodeInt32

func DecodeInt32(bz []byte) (i int32, n int, err error)

func DecodeInt64

func DecodeInt64(bz []byte) (i int64, n int, err error)

func DecodeInt8

func DecodeInt8(bz []byte) (i int8, n int, err error)

func DecodeString

func DecodeString(bz []byte) (s string, n int, err error)

func DecodeTime

func DecodeTime(bz []byte) (t time.Time, n int, err error)

DecodeTime decodes seconds (int64) and nanoseconds (int32) since January 1, 1970 UTC, and returns the corresponding time. If nanoseconds is not in the range [0, 999999999], or if seconds is too large, the behavior is undefined. TODO return error if behavior is undefined.

func DecodeUint16

func DecodeUint16(bz []byte) (u uint16, n int, err error)

func DecodeUint32

func DecodeUint32(bz []byte) (u uint32, n int, err error)

func DecodeUint64

func DecodeUint64(bz []byte) (u uint64, n int, err error)

func DecodeUint8

func DecodeUint8(bz []byte) (u uint8, n int, err error)

func DecodeUvarint

func DecodeUvarint(bz []byte) (u uint64, n int, err error)

func DecodeVarint

func DecodeVarint(bz []byte) (i int64, n int, err error)

func DeepCopy

func DeepCopy(o interface{}) (r interface{})

Deeply copies an object. If anything implements `.DeepCopy() <any>` along the way, the result of that function will be used. Otherwise, if it implements `.MarshalAmino() (<any>, error)` and `.UnmarshalAmino(<any>) error`, the pair will be used to copy. If .MarshalAmino() or .UnmarshalAmino() returns an error, this function will panic.

func EncodeBool

func EncodeBool(w io.Writer, b bool) (err error)

func EncodeByte

func EncodeByte(w io.Writer, b byte) (err error)

func EncodeByteSlice

func EncodeByteSlice(w io.Writer, bz []byte) (err error)

func EncodeFloat32

func EncodeFloat32(w io.Writer, f float32) (err error)

NOTE: UNSAFE

func EncodeFloat64

func EncodeFloat64(w io.Writer, f float64) (err error)

NOTE: UNSAFE

func EncodeInt16

func EncodeInt16(w io.Writer, i int16) (err error)

func EncodeInt32

func EncodeInt32(w io.Writer, i int32) (err error)

func EncodeInt64

func EncodeInt64(w io.Writer, i int64) (err error)

func EncodeInt8

func EncodeInt8(w io.Writer, i int8) (err error)

func EncodeString

func EncodeString(w io.Writer, s string) (err error)

func EncodeTime

func EncodeTime(w io.Writer, t time.Time) (err error)

EncodeTime writes the number of seconds (int64) and nanoseconds (int32), with millisecond resolution since January 1, 1970 UTC to the Writer as an UInt64. Milliseconds are used to ease compatibility with Javascript, which does not support finer resolution.

func EncodeUint16

func EncodeUint16(w io.Writer, u uint16) (err error)

func EncodeUint32

func EncodeUint32(w io.Writer, u uint32) (err error)

func EncodeUint64

func EncodeUint64(w io.Writer, u uint64) (err error)

func EncodeUint8

func EncodeUint8(w io.Writer, u uint8) (err error)

func EncodeUvarint

func EncodeUvarint(w io.Writer, u uint64) (err error)

EncodeUvarint is used to encode golang's int, int32, int64 by default. unless specified differently by the `binary:"fixed32"`, `binary:"fixed64"`, or `binary:"zigzag32"` `binary:"zigzag64"` tags. It matches protobufs varint encoding.

func EncodeVarint

func EncodeVarint(w io.Writer, i int64) (err error)

func MarshalBinaryBare

func MarshalBinaryBare(o interface{}) ([]byte, error)

func MarshalBinaryLengthPrefixed

func MarshalBinaryLengthPrefixed(o interface{}) ([]byte, error)

func MarshalBinaryLengthPrefixedWriter

func MarshalBinaryLengthPrefixedWriter(w io.Writer, o interface{}) (n int64, err error)

func MarshalJSON

func MarshalJSON(o interface{}) ([]byte, error)

func MarshalJSONIndent

func MarshalJSONIndent(o interface{}, prefix, indent string) ([]byte, error)

func MustMarshalBinaryBare

func MustMarshalBinaryBare(o interface{}) []byte

func MustMarshalBinaryLengthPrefixed

func MustMarshalBinaryLengthPrefixed(o interface{}) []byte

func MustUnmarshalBinaryBare

func MustUnmarshalBinaryBare(bz []byte, ptr interface{})

func MustUnmarshalBinaryLengthPrefixed

func MustUnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{})

func NameToDisfix

func NameToDisfix(name string) (db DisambBytes, pb PrefixBytes)

Return the DisambBytes and the PrefixBytes for a given name.

func UnmarshalBinaryBare

func UnmarshalBinaryBare(bz []byte, ptr interface{}) error

func UnmarshalBinaryLengthPrefixed

func UnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{}) error

func UnmarshalBinaryLengthPrefixedReader

func UnmarshalBinaryLengthPrefixedReader(r io.Reader, ptr interface{}, maxSize int64) (n int64, err error)

func UnmarshalJSON

func UnmarshalJSON(bz []byte, ptr interface{}) error

func UvarintSize

func UvarintSize(u uint64) int

func VarintSize

func VarintSize(i int64) int

type Codec

type Codec struct {
	// contains filtered or unexported fields
}

func NewCodec

func NewCodec() *Codec

func (*Codec) MarshalBinaryBare

func (cdc *Codec) MarshalBinaryBare(o interface{}) ([]byte, error)

MarshalBinaryBare encodes the object o according to the Amino spec. MarshalBinaryBare doesn't prefix the byte-length of the encoding, so the caller must handle framing.

func (*Codec) MarshalBinaryLengthPrefixed

func (cdc *Codec) MarshalBinaryLengthPrefixed(o interface{}) ([]byte, error)

MarshalBinaryLengthPrefixed encodes the object o according to the Amino spec, but prefixed by a uvarint encoding of the object to encode. Use MarshalBinaryBare if you don't want byte-length prefixing.

For consistency, MarshalBinaryLengthPrefixed will first dereference pointers before encoding. MarshalBinaryLengthPrefixed will panic if o is a nil-pointer, or if o is invalid.

func (*Codec) MarshalBinaryLengthPrefixedWriter

func (cdc *Codec) MarshalBinaryLengthPrefixedWriter(w io.Writer, o interface{}) (n int64, err error)

MarshalBinaryLengthPrefixedWriter writes the bytes as would be returned from MarshalBinaryLengthPrefixed to the writer w.

func (*Codec) MarshalJSON

func (cdc *Codec) MarshalJSON(o interface{}) ([]byte, error)

func (*Codec) MarshalJSONIndent

func (cdc *Codec) MarshalJSONIndent(o interface{}, prefix, indent string) ([]byte, error)

MarshalJSONIndent calls json.Indent on the output of cdc.MarshalJSON using the given prefix and indent string.

func (*Codec) MustMarshalBinaryBare

func (cdc *Codec) MustMarshalBinaryBare(o interface{}) []byte

Panics if error.

func (*Codec) MustMarshalBinaryLengthPrefixed

func (cdc *Codec) MustMarshalBinaryLengthPrefixed(o interface{}) []byte

Panics if error.

func (*Codec) MustMarshalJSON

func (cdc *Codec) MustMarshalJSON(o interface{}) []byte

MustMarshalJSON panics if an error occurs. Besides tha behaves exactly like MarshalJSON.

func (*Codec) MustUnmarshalBinaryBare

func (cdc *Codec) MustUnmarshalBinaryBare(bz []byte, ptr interface{})

Panics if error.

func (*Codec) MustUnmarshalBinaryLengthPrefixed

func (cdc *Codec) MustUnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{})

Panics if error.

func (*Codec) MustUnmarshalJSON

func (cdc *Codec) MustUnmarshalJSON(bz []byte, ptr interface{})

MustUnmarshalJSON panics if an error occurs. Besides tha behaves exactly like UnmarshalJSON.

func (*Codec) PrintTypes

func (cdc *Codec) PrintTypes(out io.Writer) error

PrintTypes writes all registered types in a markdown-style table. The table's header is:

| Type | Name | Prefix | Notes |

Where Type is the golang type name and Name is the name the type was registered with.

func (*Codec) RegisterConcrete

func (cdc *Codec) RegisterConcrete(o interface{}, name string, copts *ConcreteOptions)

This function should be used to register concrete types that will appear in interface fields/elements to be encoded/decoded by go-amino. Usage: `amino.RegisterConcrete(MyStruct1{}, "com.tendermint/MyStruct1", nil)`

func (*Codec) RegisterInterface

func (cdc *Codec) RegisterInterface(ptr interface{}, iopts *InterfaceOptions)

This function should be used to register all interfaces that will be encoded/decoded by go-amino. Usage: `amino.RegisterInterface((*MyInterface1)(nil), nil)`

func (*Codec) Seal

func (cdc *Codec) Seal() *Codec

func (*Codec) UnmarshalBinaryBare

func (cdc *Codec) UnmarshalBinaryBare(bz []byte, ptr interface{}) error

UnmarshalBinaryBare will panic if ptr is a nil-pointer.

func (*Codec) UnmarshalBinaryLengthPrefixed

func (cdc *Codec) UnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{}) error

Like UnmarshalBinaryBare, but will first decode the byte-length prefix. UnmarshalBinaryLengthPrefixed will panic if ptr is a nil-pointer. Returns an error if not all of bz is consumed.

func (*Codec) UnmarshalBinaryLengthPrefixedReader

func (cdc *Codec) UnmarshalBinaryLengthPrefixedReader(r io.Reader, ptr interface{}, maxSize int64) (n int64, err error)

Like UnmarshalBinaryBare, but will first read the byte-length prefix. UnmarshalBinaryLengthPrefixedReader will panic if ptr is a nil-pointer. If maxSize is 0, there is no limit (not recommended).

func (*Codec) UnmarshalJSON

func (cdc *Codec) UnmarshalJSON(bz []byte, ptr interface{}) error

type ConcreteInfo

type ConcreteInfo struct {

	// These fields are only set when registered (as implementing an interface).
	Registered       bool // Registered with RegisterConcrete().
	PointerPreferred bool // Deserialize to pointer type if possible.
	// NilPreferred     bool        // Deserialize to nil for empty structs if PointerPreferred.
	Name            string      // Registered name.
	Disamb          DisambBytes // Disambiguation bytes derived from name.
	Prefix          PrefixBytes // Prefix bytes derived from name.
	ConcreteOptions             // Registration options.

	// These fields get set for all concrete types,
	// even those not manually registered (e.g. are never interface values).
	IsAminoMarshaler       bool         // Implements MarshalAmino() (<ReprObject>, error).
	AminoMarshalReprType   reflect.Type // <ReprType>
	IsAminoUnmarshaler     bool         // Implements UnmarshalAmino(<ReprObject>) (error).
	AminoUnmarshalReprType reflect.Type // <ReprType>
}

func (ConcreteInfo) GetDisfix

func (cinfo ConcreteInfo) GetDisfix() DisfixBytes

type ConcreteOptions

type ConcreteOptions struct {
}

type DisambBytes

type DisambBytes [DisambBytesLen]byte

Prefix types

func (DisambBytes) Bytes

func (db DisambBytes) Bytes() []byte

func (DisambBytes) EqualBytes

func (db DisambBytes) EqualBytes(bz []byte) bool

type DisfixBytes

type DisfixBytes [DisfixBytesLen]byte // Disamb+Prefix

Prefix types

func (DisfixBytes) Bytes

func (df DisfixBytes) Bytes() []byte

func (DisfixBytes) EqualBytes

func (df DisfixBytes) EqualBytes(bz []byte) bool

type FieldInfo

type FieldInfo struct {
	Name         string        // Struct field name
	Type         reflect.Type  // Struct field type
	Index        int           // Struct field index
	ZeroValue    reflect.Value // Could be nil pointer unlike TypeInfo.ZeroValue.
	UnpackedList bool          // True iff this field should be encoded as an unpacked list.
	FieldOptions               // Encoding options
}

type FieldOptions

type FieldOptions struct {
	JSONName      string // (JSON) field name
	JSONOmitEmpty bool   // (JSON) omitempty
	BinFixed64    bool   // (Binary) Encode as fixed64
	BinFixed32    bool   // (Binary) Encode as fixed32
	BinFieldNum   uint32 // (Binary) max 1<<29-1

	Unsafe        bool // e.g. if this field is a float.
	WriteEmpty    bool // write empty structs and lists (default false except for pointers)
	EmptyElements bool // Slice and Array elements are never nil, decode 0x00 as empty struct.
}

type InterfaceInfo

type InterfaceInfo struct {
	Priority     []DisfixBytes               // Disfix priority.
	Implementers map[PrefixBytes][]*TypeInfo // Mutated over time.
	InterfaceOptions
}

type InterfaceOptions

type InterfaceOptions struct {
	Priority           []string // Disamb priority.
	AlwaysDisambiguate bool     // If true, include disamb for all types.
}

type InvalidTimeErr

type InvalidTimeErr string

func (InvalidTimeErr) Error

func (e InvalidTimeErr) Error() string

type PrefixBytes

type PrefixBytes [PrefixBytesLen]byte

Prefix types

func NewPrefixBytes

func NewPrefixBytes(prefixBytes []byte) PrefixBytes

Copy into PrefixBytes

func (PrefixBytes) Bytes

func (pb PrefixBytes) Bytes() []byte

func (PrefixBytes) EqualBytes

func (pb PrefixBytes) EqualBytes(bz []byte) bool

type StructInfo

type StructInfo struct {
	Fields []FieldInfo // If a struct.
}

type Typ3

type Typ3 uint8

func (Typ3) String

func (typ Typ3) String() string

type TypeInfo

type TypeInfo struct {
	Type      reflect.Type // Interface type.
	PtrToType reflect.Type
	ZeroValue reflect.Value
	ZeroProto interface{}
	InterfaceInfo
	ConcreteInfo
	StructInfo
}

func (TypeInfo) String

func (ti TypeInfo) String() string
Documentation was rendered with GOOS=linux and GOARCH=amd64.

Jump to identifier

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to identifier