jettison

package module
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 22, 2019 License: MIT Imports: 18 Imported by: 26

README

Jettison

GoCaptain

Jettison is a high performance, reflection-less, and configurable JSON encoder for the Go programming language, inspired by bet365/jingo, with a richer features set, aiming at 100% compatibility with the standard library.


Installation

Jettison uses the new Go modules. Releases are tagged according to the SemVer format, prefixed with a v, starting from 0.2.0. You can get the latest release using the following command.

$ go get github.com/wI2L/jettison

⚠ Requires Go1.12+, due to the usage of the io.StringWriter interface.

Key features

  • Fast, see benchmarks
  • Efficient, zero allocations on average
  • Behavior identical to the standard library by default
  • No code generation required
  • Clear and concise API
  • Configurable with opt-in functional options
  • Native support for time.Time and time.Duration types
  • Custom Marshaler interface to work with a Writer
  • Extensive testsuite that compares its output against encoding/json

Overview

The goal of Jettision is to take up the idea introduced by the bet365/jingo package and build a fully-featured JSON encoder around it, that comply with the behavior of the encoding/json package. Unlike the latter, Jettison does not use reflection during marshaling, but only once to create the instruction set for a given type ahead of time. The drawback to this approach requires to instantiate an encoder once for each type that needs to be marshaled.

The package aims to have a behavior similar to that of the standard library for all types encoding and struct tags, meaning that the documentation of the json.Marshal function is applicable for Jettison, with a few exceptions described in this section. As such, most of the tests compare their output against it to guarantee that.

Implementation details

The main concept of Jettison consists of using pre-build encoders to reduce the cost of using the reflect package at runtime. When a new instance of an encoder is created for a specific type, a set of instructions is recursively generated, which defines how to iteratively encode a value. An instruction is a function or a closure, that have all the information required to read the data from memory using unsafe operations during the instruction set execution.

Differences with encoding/json

All notable differences with the standard library behavior are listed below. Please bote that these might evolve with future versions of the package.

Limitations
  • The JSON returned by the MarshalJSON method of types implementing the json.Marshaler interface is neither validated nor compacted.
Improvements
  • The time.Time and time.Duration types are handled natively. For time values, the encoder doesn't invoke MarshalJSON or MarshalText, but use the time.AppendFormat function instead, and write the result to the stream. Similarly, for durations, it isn't necessary to implements the json.Marshaler or encoding.TextMarshaler interfaces on a custom wrapper type, the encoder uses the result of one of the methods Minutes, Seconds, Nanoseconds or String, based on the duration format configured.
Bugs

  • Nil map keys values implementing the encoding.TextMarshaler interface are encoded as empty strings, while the encoding/json package currently panic because of that. See this issue for more details.[1]

  • Nil struct fields implementing the encoding.TextMarshaler interface are encoded as null, while the encoding/json package currently panic because of that. See this issue for more details.[1]

1: The issues mentioned above have had their associated CL merged, and should be shipped with Go 1.14.

Usage

Basic

Starting from version 0.3.0, the Marshal and MarshalTo functions are available. The first will allocate a new bytes slice to store the encoding of the given value, similar to json.Marshal, while the latter will write to the Writer. These functions use a package's cache to fetch the appropriate encoder for the given value type. If an encoder does not exist, a new one is created on the fly and stored in the cache for future reuse.

type X struct {
   A string `json:"a"`
   B int64  `json:"b"`
}
b, err := jettison.Marshal(X{
   A: "Loreum",
   B: 42,
})
if err != nil {
   log.Fatal(err)
}
os.Stdout.Write(b)

Output

{"a":"Loreum","b":42}

Advanced

If more control over the encoding behavior is required, or to avoid the latency of creating a new encoder when encoding a type for the first time, an encoder can be created ahead of time, during initialization. Note that if you don't invoke the Compile method, the instruction set will be generated once, on the first call to the Encode method.

The second parameter of the Encode method is an interface that groups the io.Writer, io.StringWriter and io.ByteWriter interfaces. In the following example, we use a new bytes.Buffer instance, which implements the three interfaces previously mentioned.

type X struct {
    A string `json:"a,omitempty"`
    B int    `json:"b"`
}
enc, err := jettison.NewEncoder(reflect.TypeOf(X{}))
if err != nil {
    log.Fatal(err)
}
err = enc.Compile()
if err != nil {
    log.Fatal(err)
}
xx := X{
    A: "Loreum",
    B: 42,
}
var buf bytes.Buffer
if err := enc.Encode(&xx, &buf); err != nil {
    log.Fatal(err)
}
os.Stdout.Write(b)

Output

{"a":"Loreum","b":42}

Options

Several opt-in options are available to customize the behavior of an encoder. The third parameter of the Encode method is variadic and accept a list of functional options described below.

  • TimeLayout
    Defines the layout used to encode time.Time values. The layout must be compatible with the AppendFormat method. The default layout is time.RFC3339Nano.
  • DurationFormat
    Defines the format used to encode time.Duration values. The default format is DurationString. See the documentation of the DurationFmt type for the complete list of formats available.
  • UnixTimestamp
    Encode time.Time values as JSON numbers representing Unix timestamps, the number of seconds elapsed since Januaray 1, 1970 UTC. It uses the time.Unix method. This option has precedence over TimeLayout.
  • UnsortedMap
    Disables map keys sort. See Map benchmark for performance difference.
  • ByteArrayAsString
    Encodes byte arrays as JSON strings rather than JSON arrays. The output is subject to the same escaping rules used for the string type, unless the option NoStringEscaping is also used.
  • RawByteSlice
    Disables the base64 default encoding used for byte slices.
  • NilMapEmpty
    Encodes nil Go maps as empty JSON objects rather than null.
  • NilSliceEmpty
    Encodes nil Go slices as empty JSON arrays rather than null.
  • NoStringEscaping
    Disables string escaping. NoHTMLEscaping and NoUTF8Coercion are ignored when this option is used.
  • NoHTMLEscaping
    Disables the escaping of special HTML characters such as &, < and > in JSON strings. This is similar to json.Encoder.SetEscapeHTML(false).
  • NoUTF8Coercion
    Disables the replacement of invalid bytes with the Unicode replacement rune in JSON strings.

Benchmarks

Ubuntu 16.04.6 LTS, Intel(R) Core(TM) i5-6600 CPU @ 3.30GHz
go version go1.13 linux/amd64
jettison v0.3.0

Simple

Basic object with fields of type string, int and bool. source

Simple Payload Benchmark Graph
Stats
name                      time/op
SimplePayload/standard-4    658ns ± 2%
SimplePayload/jsoniter-4    711ns ± 1%
SimplePayload/gojay-4       474ns ± 1%
SimplePayload/jettison-4    441ns ± 0%
-
name                      speed
SimplePayload/standard-4  205MB/s ± 2%
SimplePayload/jsoniter-4  190MB/s ± 1%
SimplePayload/gojay-4     285MB/s ± 1%
SimplePayload/jettison-4  306MB/s ± 0%
-
name                      alloc/op
SimplePayload/standard-4     144B ± 0%
SimplePayload/jsoniter-4     152B ± 0%
SimplePayload/gojay-4        512B ± 0%
SimplePayload/jettison-4    0.00B
-
name                      allocs/op
SimplePayload/standard-4     1.00 ± 0%
SimplePayload/jsoniter-4     2.00 ± 0%
SimplePayload/gojay-4        1.00 ± 0%
SimplePayload/jettison-4     0.00

Complex

Payload with a variety of composite Go types, such as struct, multi-dimensions array, and slice, with pointer and non-pointer value types. source

Complex Payload Benchmark Graph
Stats
name                       time/op
ComplexPayload/standard-4   2.57µs ± 0%
ComplexPayload/jsoniter-4   2.38µs ± 1%
ComplexPayload/jettison-4   1.50µs ± 0%
-
name                       speed
ComplexPayload/standard-4  151MB/s ± 0%
ComplexPayload/jsoniter-4  163MB/s ± 1%
ComplexPayload/jettison-4  258MB/s ± 0%
-
name                       alloc/op
ComplexPayload/standard-4     416B ± 0%
ComplexPayload/jsoniter-4     472B ± 0%
ComplexPayload/jettison-4    0.00B
-
name                       allocs/op
ComplexPayload/standard-4     1.00 ± 0%
ComplexPayload/jsoniter-4     3.00 ± 0%
ComplexPayload/jettison-4     0.00

Interface

Interface Benchmark Graph
Stats
name                  time/op
Interface/standard-4     161ns ± 4%
Interface/jsoniter-4     141ns ± 1%
Interface/jettison-4    64.7ns ± 3%
-
name                  speed
Interface/standard-4  49.7MB/s ± 4%
Interface/jsoniter-4  56.8MB/s ± 1%
Interface/jettison-4   124MB/s ± 3%
-
name                  alloc/op
Interface/standard-4     8.00B ± 0%
Interface/jsoniter-4     8.00B ± 0%
Interface/jettison-4     0.00B
-
name                  allocs/op
Interface/standard-4      1.00 ± 0%
Interface/jsoniter-4      1.00 ± 0%
Interface/jettison-4      0.00

Map

Compares Go map marshaling performances, with and without keys sort. source

Map Benchmark Graph
Stats
name                   time/op
Map/standard-4           1.21µs ± 5%
Map/jsoniter-4           1.07µs ± 1%
Map/jettison/sort-4       860ns ± 0%
Map/jettison/nosort-4     379ns ± 1%
-
name                   speed
Map/standard-4         15.8MB/s ± 4%
Map/jsoniter-4         17.7MB/s ± 1%
Map/jettison/sort-4    22.1MB/s ± 1%
Map/jettison/nosort-4  50.1MB/s ± 1%
-
name                   alloc/op
Map/standard-4             536B ± 0%
Map/jsoniter-4             680B ± 0%
Map/jettison/sort-4        496B ± 0%
Map/jettison/nosort-4      128B ± 0%
-
name                   allocs/op
Map/standard-4             13.0 ± 0%
Map/jsoniter-4             11.0 ± 0%
Map/jettison/sort-4        6.00 ± 0%
Map/jettison/nosort-4      2.00 ± 0%

License

Jettison is licensed under the MIT license. See the LICENSE file.

This package also uses some portions of code from the Go encoding/json package. The associated license can be found in LICENSE.golang.

Documentation

Overview

Example (CustomMarshaler) 
package main

import (
	"log"
	"os"
	"strconv"

	"github.com/wI2L/jettison"
)

type Animal int

const (
	Unknown Animal = iota
	Gopher
	Zebra
)

func (a Animal) WriteJSON(w jettison.Writer) error {
	var s string
	switch a {
	default:
		s = "unknown"
	case Gopher:
		s = "gopher"
	case Zebra:
		s = "zebra"
	}
	_, err := w.WriteString(strconv.Quote(s))
	return err
}

func main() {
	zoo := []Animal{
		Unknown,
		Zebra,
		Gopher,
		Zebra,
		Unknown,
		Zebra,
		Gopher,
	}
	b, err := jettison.Marshal(zoo)
	if err != nil {
		log.Fatal(err)
	}
	os.Stdout.Write(b)
}

Output:
["unknown","zebra","gopher","zebra","unknown","zebra","gopher"]

Index

Examples

Constants

View Source 
const (
	DurationString = iota
	DurationMinutes
	DurationSeconds
	DurationMilliseconds
	DurationMicroseconds
	DurationNanoseconds
)

Duration formats.

Variables

View Source 
var ErrInvalidWriter = errors.New("invalid writer")

ErrInvalidWriter is the error returned by an Encoder's Encode method when the given Writer is invalid.

Functions

func Marshal added in v0.3.0 

func Marshal(v interface{}) ([]byte, error)

Marshal returns the JSON encoding of v. It uses a pre-compiled encoder from the package's cache, or create and store one on the fly.

Example 
package main

import (
	"log"
	"os"

	"github.com/wI2L/jettison"
)

func main() {
	type X struct {
		A string   `json:"a"`
		B int64    `json:"b"`
		C []string `json:"colors"`
	}
	x := X{
		A: "Loreum",
		B: -42,
		C: []string{"blue", "green", "purple"},
	}
	b, err := jettison.Marshal(&x)
	if err != nil {
		log.Fatal(err)
	}
	os.Stdout.Write(b)
}

Output:
{"a":"Loreum","b":-42,"colors":["blue","green","purple"]}

func MarshalTo added in v0.3.0 

func MarshalTo(v interface{}, w Writer) error

MarshalTo writes the JSON encoding of v to w. It uses a pre-compiled encoder from the package's cache, or create and store one on the fly.

Example 
package main

import (
	"bytes"
	"log"
	"os"

	"github.com/wI2L/jettison"
)

func main() {
	type X struct {
		A bool              `json:"a"`
		B uint32            `json:"b"`
		C map[string]string `json:"users"`
	}
	x := X{
		A: true,
		B: 42,
		C: map[string]string{
			"bob":   "admin",
			"jerry": "user",
		},
	}
	var buf bytes.Buffer
	err := jettison.MarshalTo(&x, &buf)
	if err != nil {
		log.Fatal(err)
	}
	os.Stdout.Write(buf.Bytes())
}

Output:
{"a":true,"b":42,"users":{"bob":"admin","jerry":"user"}}

func Register added in v0.3.0 

func Register(typ reflect.Type) error

Register records a new compiled encoder for the given type to the cache used by the global functions Marshal and MarshalTo. This may be used during the initialization of a program to speed up the first calls to previously mentioned functions.

Types

type DurationFmt 

type DurationFmt int

DurationFmt represents the format used to encode a time.Duration value.

func (DurationFmt) String 

func (df DurationFmt) String() string

String implements the fmt.Stringer interface.

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder writes the JSON values of a specific type using a set of instructions compiled when the encoder is instantiated.

func NewEncoder 

func NewEncoder(rt reflect.Type) (*Encoder, error)

NewEncoder returns a new encoder that can marshal the values of the given type. The Encoder can be explicitly initialized by calling its Compile method, otherwise the operation is done on first call to Marshal.

func (*Encoder) Compile 

func (e *Encoder) Compile() error

Compile generates the encoder's instructions. Calling this method more than once is a noop.

func (*Encoder) Encode 

func (e *Encoder) Encode(i interface{}, w Writer, opts ...Option) error

Encode writes the JSON encoding of i to w.

Example 
package main

import (
	"bytes"
	"fmt"
	"log"
	"reflect"

	"github.com/wI2L/jettison"
)

func main() {
	type X struct {
		M map[string]int
		S []int
	}
	x := X{}
	enc, err := jettison.NewEncoder(reflect.TypeOf(x))
	if err != nil {
		log.Fatal(err)
	}
	var buf bytes.Buffer
	if err := enc.Encode(&x, &buf); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s\n", buf.String())

	buf.Reset()
	if err := enc.Encode(&x, &buf,
		jettison.NilMapEmpty(),
		jettison.NilSliceEmpty(),
	); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s\n", buf.String())
}

Output:
{"M":null,"S":null}
{"M":{},"S":[]}

type Instruction 

type Instruction func(unsafe.Pointer, Writer, *encodeState) error

Instruction represents a function that writes the JSON representation of a value to a stream.

type Marshaler added in v0.4.0 

type Marshaler interface {
	WriteJSON(Writer) error
}

Marshaler is a variant of the json.Marshaler interface, implemented by types that can write a valid JSON representation of themselves to a Writer. If a type implements both interfaces, Jettison will use its own interface in priority.

type MarshalerError 

type MarshalerError struct {
	Err error
	Typ reflect.Type
	// contains filtered or unexported fields
}

MarshalerError represents an error from calling a MarshalJSON or MarshalText method.

func (*MarshalerError) Error 

func (e *MarshalerError) Error() string

Error implements the builtin error interface.

func (*MarshalerError) Unwrap added in v0.3.0 

func (e *MarshalerError) Unwrap() error

Unwrap returns the wrapped error. This doesn't implement a public interface, but allow to use the errors.Unwrap function released in Go 1.13 with a MarshalerError.

type Option 

type Option func(*encodeOpts)

Option overrides a particular behavior of an encoder

func ByteArrayAsString 

func ByteArrayAsString() Option

ByteArrayAsString encodes byte arrays as raw JSON strings.

func DurationFormat 

func DurationFormat(df DurationFmt) Option

DurationFormat sets the format used to encode time.Duration values.

func NilMapEmpty 

func NilMapEmpty() Option

NilMapEmpty encodes nil Go maps as empty JSON objects, rather than null.

func NilSliceEmpty 

func NilSliceEmpty() Option

NilSliceEmpty encodes nil Go slices as empty JSON arrays, rather than null.

func NoHTMLEscaping added in v0.3.1 

func NoHTMLEscaping() Option

NoHTMLEscaping disables the escaping of HTML characters when encoding JSON strings.

func NoStringEscaping 

func NoStringEscaping() Option

NoStringEscaping disables strings escaping.

func NoUTF8Coercion added in v0.3.1 

func NoUTF8Coercion() Option

NoUTF8Coercion disables UTF-8 coercion when encoding JSON strings.

func RawByteSlice added in v0.4.0 

func RawByteSlice() Option

RawByteSlice disables the default behavior that encodes byte slices as base64-encoded strings.

func TimeLayout 

func TimeLayout(layout string) Option

TimeLayout sets the time layout used to encode time.Time values.

func UnixTimestamp 

func UnixTimestamp() Option

UnixTimestamp configures the encoder to encode time.Time values as Unix timestamps. This option has precedence over any time layout.

func UnsortedMap 

func UnsortedMap() Option

UnsortedMap disables map keys sort.

type TypeMismatchError added in v0.3.0 

type TypeMismatchError struct {
	SrcType reflect.Type
	EncType reflect.Type
}

TypeMismatchError is the error returned by an Encoder's Encode method whhen the type of the input value does not match the type for which the encoder was compiled.

func (*TypeMismatchError) Error added in v0.3.0 

func (e *TypeMismatchError) Error() string

Error implements the builtin error interface.

type UnsupportedTypeError 

type UnsupportedTypeError struct {
	Typ     reflect.Type
	Context string
}

UnsupportedTypeError is the error returned by an Encoder's Encode method when attempting to encode an unsupported value type.

func (*UnsupportedTypeError) Error 

func (e *UnsupportedTypeError) Error() string

Error implements the bultin error interface.

type UnsupportedValueError 

type UnsupportedValueError struct {
	Str string
}

UnsupportedValueError is the error returned by an Encoder's Encode method when attempting to encode an unsupported value.

func (*UnsupportedValueError) Error 

func (e *UnsupportedValueError) Error() string

Error implements the builtin error interface.

type Writer 

type Writer interface {
	io.Writer
	io.StringWriter
	io.ByteWriter
}

Writer is an interface that groups the io.Writer, io.StringWriter and io.ByteWriter interfaces.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.