jettison

README

Jettison

Jettison is an instructions-based JSON encoder inspired by bet365/jingo, with a richer features set, aiming at 100% compatibility with the standard library.

---

Installation

Jettison uses the new Go modules. Releases are tagged with the SemVer format, prefixed with a v, starting from 0.2.0. You can get the latest release using the following command.

$ go get github.com/wI2L/jettison

❗ Requires Go1.12+, due to the usage of io.StringWriter.

Key points

• Fast, see benchmarks
• Zero allocations on average
• Behavior identical to the standard library by default
• No code generation required
• Clear and simple API
• Options available to configure encoding
• Native support for time.Time and time.Duration
• Extensive testsuite

Overview

The goal of Jettision is to take up the idea introduced by the bet365/jingo package and build a fully-featured JSON encoder around it, that comply with the behavior of the encoding/json package. Unlike the latter, Jettison does not use reflection during marshaling, but only once to create the instruction set for a given type ahead of time. The drawback to this approach requires to instantiate an encoder once for each type that needs to be marshaled.

The package aims to have a behavior similar to that of the standard library for all types encoding and struct tags, meaning that the documentation of the json.Marshal function is applicable for Jettison, with a few exceptions described in this section. As such, most of the tests compare their output against it to guarantee that.

Implementation details

The main concept of Jettison consists of using pre-build encoders to reduce the cost of using the reflect package at runtime. When a new instance of an encoder is created for a specific type, a set of instructions is recursively generated, which defines how to iteratively encode a value. An instruction is a function or a closure, that have all the information required to read the data from memory using unsafe operations during the instruction set execution.

Usage

Basic

Starting from version 0.3.0, the Marshal and MarshalTo functions are available. The first will allocate a new bytes slice to store the encoding of the given value, while the latter will write to the given stream. These functions use a package's cache to fetch the appropriate encoder for the given value type. If an encoder does not exist, a new one is created on the fly and stored in the cache for future reuse.

import (
   "fmt"

   "github.com/wI2L/jettison"
)

type X struct {
   A string `json:"a"`
   B int64  `json:"b"`
}
b, err := jettison.Marshal(x{
   A: "Loreum",
   B: 42,
})
if err != nil {
   // handle error
}
fmt.Println(string(b))
// {"a":"Loreum","b":42}

Advanced

If more control over the encoding behavior is required, or to avoid the latency of creating a new encoder when encoding a type for the first time, an encoder can be created ahead of time, during initialization. Note that if you don't invoke the Compile method, the instruction set will be generated once, on the first call to the Encode method.

The second parameter of the Encode method is an interface that groups the io.Writer, io.StringWriter and io.ByteWriter interfaces. In the above example, we use a new bytes.Buffer instance, which implements the three interfaces previously mentioned.

import (
   "bytes"
   "fmt"
   "reflect"

   "github.com/wI2L/jettison"
)

type x struct {
    A string `json:"a,omitempty"`
    B int    `json:"b"`
}
enc, err := jettison.NewEncoder(reflect.TypeOf(x{}))
if err != nil {
    // handle error
}
err = enc.Compile()
if err != nil {
    // handle error
}
xx := x{
    A: "Loreum",
    B: 42,
}
var buf bytes.Buffer
if err := enc.Encode(&xx, &buf); err != nil {
    // handle error
}
fmt.Println(buf.String())
// {"a":"Loreum","b":42}

Options

Opt-in options are available to customize the behavior of the package. The third parameter of the Encode method is variadic and accept a list of functional options described below.

Option	Description
TimeLayout	Defines the layout used to encode time.Time values. time.RFC3339Nano is the default.
DurationFormat	Defines the format used to encode time.Duration values. DurationString is the default. See DurationFmt for the complete list of formats available.
UnixTimestamp	Encode time.Time values as JSON numbers representing Unix timestamps.
UnsortedMap	Disables map keys sort.
ByteArrayAsString	Encodes byte arrays as JSON strings rather than JSON arrays. The output is subject to the same escaping rules used for the string type, unless the option NoStringEscaping is also specified.
RawByteSlices	Disables base64 default encoding used for byte slices.
NilMapEmpty	Encodes nil maps as empty JSON objects rather than null.
NilSliceEmpty	Encodes nil slices as empty JSON arrays rather than null.
NoStringEscaping	Disables strings escaping.

Differences with encoding/json

• Strings are not coerced to valid UTF-8 by replacing invalid bytes with the Unicode replacement rune, and the brackets < and > or ampersand & are not escaped. Escaping is only applied to bytes lower than 0x20 and those described in the JSON spec.

• time.Time and Time.Duration types are handled natively by the package. For the first, the encoder doesn't invoke the MarshalJSON/MarshalText methods, but use time.AppendFormat directly. For the second, it isn't necessary to implements the json.Marshaler or encoding.TextMarshaler interfaces, the encoder uses the result of the methods Minutes, Seconds, Nanoseconds or String based on the duration format configured.

• Nil map keys values implementing the encoding.TextMarshaler interface are encoded as an empty string, while the encoding/json package currently panic because of that. See this issue for more details.^[1]

• Nil struct fields implementing the encoding.TextMarshaler interface are encoded as null, while the encoding/json package currently panic because of that. See this issue for more details.^[1]

^{1: The issues mentioned above have had their associated CL merged, and should be shipped with Go 1.14.}

Benchmarks

Ubuntu 16.04.6 LTS, Intel(R) Core(TM) i5-6600 CPU @ 3.30GHz go version go1.13 linux/amd64

Simple

Basic object with fields of type string, int and bool. source

Stats

name                      time/op
SimplePayload/standard-4    658ns ± 2%
SimplePayload/jsoniter-4    711ns ± 1%
SimplePayload/gojay-4       474ns ± 1%
SimplePayload/jettison-4    441ns ± 0%
-
name                      speed
SimplePayload/standard-4  205MB/s ± 2%
SimplePayload/jsoniter-4  190MB/s ± 1%
SimplePayload/gojay-4     285MB/s ± 1%
SimplePayload/jettison-4  306MB/s ± 0%
-
name                      alloc/op
SimplePayload/standard-4     144B ± 0%
SimplePayload/jsoniter-4     152B ± 0%
SimplePayload/gojay-4        512B ± 0%
SimplePayload/jettison-4    0.00B
-
name                      allocs/op
SimplePayload/standard-4     1.00 ± 0%
SimplePayload/jsoniter-4     2.00 ± 0%
SimplePayload/gojay-4        1.00 ± 0%
SimplePayload/jettison-4     0.00

Complex

Payload with a variety of composite Go types, such as struct, multi-dimensions array, and slice, with pointer and non-pointer value types. source

Stats

name                       time/op
ComplexPayload/standard-4   2.57µs ± 0%
ComplexPayload/jsoniter-4   2.38µs ± 1%
ComplexPayload/jettison-4   1.50µs ± 0%
-
name                       speed
ComplexPayload/standard-4  151MB/s ± 0%
ComplexPayload/jsoniter-4  163MB/s ± 1%
ComplexPayload/jettison-4  258MB/s ± 0%
-
name                       alloc/op
ComplexPayload/standard-4     416B ± 0%
ComplexPayload/jsoniter-4     472B ± 0%
ComplexPayload/jettison-4    0.00B
-
name                       allocs/op
ComplexPayload/standard-4     1.00 ± 0%
ComplexPayload/jsoniter-4     3.00 ± 0%
ComplexPayload/jettison-4     0.00

Interface

Stats

name                  time/op
Interface/standard-4     161ns ± 4%
Interface/jsoniter-4     141ns ± 1%
Interface/jettison-4    64.7ns ± 3%
-
name                  speed
Interface/standard-4  49.7MB/s ± 4%
Interface/jsoniter-4  56.8MB/s ± 1%
Interface/jettison-4   124MB/s ± 3%
-
name                  alloc/op
Interface/standard-4     8.00B ± 0%
Interface/jsoniter-4     8.00B ± 0%
Interface/jettison-4     0.00B
-
name                  allocs/op
Interface/standard-4      1.00 ± 0%
Interface/jsoniter-4      1.00 ± 0%
Interface/jettison-4      0.00

Map

Stats

name                   time/op
Map/standard-4           1.21µs ± 5%
Map/jsoniter-4           1.07µs ± 1%
Map/jettison/sort-4       860ns ± 0%
Map/jettison/nosort-4     379ns ± 1%
-
name                   speed
Map/standard-4         15.8MB/s ± 4%
Map/jsoniter-4         17.7MB/s ± 1%
Map/jettison/sort-4    22.1MB/s ± 1%
Map/jettison/nosort-4  50.1MB/s ± 1%
-
name                   alloc/op
Map/standard-4             536B ± 0%
Map/jsoniter-4             680B ± 0%
Map/jettison/sort-4        496B ± 0%
Map/jettison/nosort-4      128B ± 0%
-
name                   allocs/op
Map/standard-4             13.0 ± 0%
Map/jsoniter-4             11.0 ± 0%
Map/jettison/sort-4        6.00 ± 0%
Map/jettison/nosort-4      2.00 ± 0%

---

This package uses some portions of code of the Go encoding/json package. The associated license can be found in LICENSE.golang.

Documentation

Index

• Constants
• Variables
• func ByteArrayAsString(es *encodeState)
• func Marshal(v interface{}) ([]byte, error)
• func MarshalTo(v interface{}, w Writer) error
• func NilMapEmpty(es *encodeState)
• func NilSliceEmpty(es *encodeState)
• func NoStringEscaping(es *encodeState)
• func RawByteSlices(es *encodeState)
• func Register(typ reflect.Type) error
• func UnixTimestamp(es *encodeState)
• func UnsortedMap(es *encodeState)
• type DurationFmt
  • func (df DurationFmt) String() string
• type Encoder
  • func NewEncoder(rt reflect.Type) (*Encoder, error)
  • func (e *Encoder) Compile() error
  • func (e *Encoder) Encode(i interface{}, w Writer, opts ...Option) error
• type Instruction
• type MarshalerError
  • func (e *MarshalerError) Error() string
  • func (e *MarshalerError) Unwrap() error
• type Option
  • func DurationFormat(df DurationFmt) Option
  • func TimeLayout(layout string) Option
• type TypeMismatchError
  • func (e *TypeMismatchError) Error() string
• type UnsupportedTypeError
  • func (e *UnsupportedTypeError) Error() string
• type UnsupportedValueError
  • func (e *UnsupportedValueError) Error() string
• type Writer

Constants

const (
	DurationString = iota
	DurationMinutes
	DurationSeconds
	DurationMilliseconds
	DurationMicroseconds
	DurationNanoseconds
)

Duration formats.

Variables

var ErrInvalidWriter = errors.New("invalid writer")

ErrInvalidWriter is the error returned by an Encoder's Encode method when the given Writer is invalid.

Functions

func ByteArrayAsString

func ByteArrayAsString(es *encodeState)

ByteArrayAsString encodes byte arrays as raw JSON strings.

func Marshal

func Marshal(v interface{}) ([]byte, error)

Marshal returns the JSON encoding of v. It uses a pre-compiled encoder from the package's cache, or create and store one on the fly.

func MarshalTo

func MarshalTo(v interface{}, w Writer) error

MarshalTo writes the JSON encoding of v to w. It uses a pre-compiled encoder from the package's cache, or create and store one on the fly.

func NilMapEmpty

func NilMapEmpty(es *encodeState)

NilMapEmpty encodes a nil Go map as an empty JSON object, rather than null.

func NilSliceEmpty

func NilSliceEmpty(es *encodeState)

NilSliceEmpty encodes a nil Go slice as an empty JSON array, rather than null.

func NoStringEscaping

func NoStringEscaping(es *encodeState)

NoStringEscaping disables string escaping.

func RawByteSlices

func RawByteSlices(es *encodeState)

RawByteSlices disables the default behavior that encodes byte slices as base64-encoded strings.

func Register

func Register(typ reflect.Type) error

Register records a new compiled encoder for the given type to the cache used by the global functions Marshal and MarshalTo. This may be used during the initialization of a program to speed up the first calls to previously mentioned functions.

func UnixTimestamp

func UnixTimestamp(es *encodeState)

UnixTimestamp configures the encoder to encode time.Time value as Unix timestamps. This setting has precedence over any time layout.

func UnsortedMap

func UnsortedMap(es *encodeState)

UnsortedMap disables map keys sort.

Types

type DurationFmt

type DurationFmt int

DurationFmt represents the format used to encode a time.Duration.

func (DurationFmt) String

func (df DurationFmt) String() string

String implements the fmt.Stringer interface.

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder writes the JSON values of a specific type using a set of instructions compiled when the encoder is instantiated.

func NewEncoder

func NewEncoder(rt reflect.Type) (*Encoder, error)

NewEncoder returns a new encoder that can marshal the values of the given type. The Encoder can be explicitly initialized by calling its Compile method, otherwise the operation is done on first call to Marshal.

func (*Encoder) Compile

func (e *Encoder) Compile() error

Compile generates the encoder's instructions. Calling this method more than once is a noop.

func (*Encoder) Encode

func (e *Encoder) Encode(i interface{}, w Writer, opts ...Option) error

Encode writes the JSON encoding of i to w.

type Instruction

type Instruction func(unsafe.Pointer, Writer, *encodeState) error

Instruction represents a function that writes the JSON representation of a value to a stream.

type MarshalerError

type MarshalerError struct {
	Typ reflect.Type
	Err error
	// contains filtered or unexported fields
}

MarshalerError represents an error from calling a MarshalJSON or MarshalText method.

func (*MarshalerError) Error

func (e *MarshalerError) Error() string

Error implements the builtin error interface.

func (*MarshalerError) Unwrap

func (e *MarshalerError) Unwrap() error

type Option

type Option func(*encodeState)

Option represents an option that defines the behavior of an encoder.

func DurationFormat

func DurationFormat(df DurationFmt) Option

DurationFormat sets the format used to encode a time.Duration value.

func TimeLayout

func TimeLayout(layout string) Option

TimeLayout sets the time layout used to encode a time.Time value.

type TypeMismatchError

type TypeMismatchError struct {
	SrcType reflect.Type
	EncType reflect.Type
}

TypeMismatchError is the error returned by an Encoder's Encode method whhen the type of the input value does not match the type for which the encoder was compiled.

func (*TypeMismatchError) Error

func (e *TypeMismatchError) Error() string

Error implements the builtin error interface.

type UnsupportedTypeError

type UnsupportedTypeError struct {
	Typ     reflect.Type
	Context string
}

UnsupportedTypeError is the error returned by an Encoder's Encode method when attempting to encode an unsupported value type.

func (*UnsupportedTypeError) Error

func (e *UnsupportedTypeError) Error() string

Error implements the bultin error interface.

type UnsupportedValueError

type UnsupportedValueError struct {
	Str string
}

UnsupportedValueError is the error returned by an Encoder's Encode method when attempting to encode an unsupported value.

func (*UnsupportedValueError) Error

func (e *UnsupportedValueError) Error() string

Error implements the builtin error interface.

type Writer

type Writer interface {
	io.Writer
	io.StringWriter
	io.ByteWriter
}

Writer is an interface that groups the io.Writer, io.StringWriter and io.ByteWriter interfaces.