jx

package module
v0.16.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 5, 2021 License: MIT Imports: 11 Imported by: 0

README

jx Go Reference codecov

Package jx implements encoding and decoding of json [RFC 7159]. Lightweight fork of jsoniter.

go get github.com/ogen-go/jx

Features

  • Directly encode and decode json values
  • No reflect or interface{}
  • Pools and direct buffer access for less (or none) allocations
  • Multi-pass decoding
  • Validation

See usage for examples. Mostly suitable for fast low-level json manipulation with high control. Used in ogen project for json (un)marshaling code generation based on json and OpenAPI schemas.

Why

Most of jsoniter issues are caused by necessity to be drop-in replacement for standard encoding/json. Removing such constrains greatly simplified implementation and reduced scope, allowing to focus on json stream processing.

  • Reduced scope
    • No reflection
    • No encoding/json adapter
    • 4x less code (8.5K to 2K SLOC)
  • Fuzzing, improved test coverage
  • Drastically refactored and simplified
    • Explicit error returns
    • No Config or API

Usage

Decode

Use jx.Decoder. Zero value is valid, but constructors are available for convenience:

To reuse decoders and their buffers, use jx.GetDecoder and jx.PutDecoder alongside with reset functions:

Decoder is reset on PutDecoder.

d := jx.DecodeStr(`{"values":[4,8,15,16,23,42]}`)

// Save all integers from "values" array to slice.
var values []int

// Iterate over each object field.
if err := d.Obj(func(d *jx.Decoder, key string) error {
    switch key {
    case "values":
        // Iterate over each array element.
        return d.Arr(func(d *jx.Decoder) error {
            v, err := d.Int()
            if err != nil {
                return err
            }
            values = append(values, v)
            return nil
        })
    default:
        // Skip unknown fields if any.
        return d.Skip()
    }
}); err != nil {
    panic(err)
}

fmt.Println(values)
// Output: [4 8 15 16 23 42]
Encode

Use jx.Encoder. Zero value is valid, reuse with jx.GetEncoder, jx.PutEncoder and jx.Encoder.Reset(). Encoder is reset on PutEncoder.

var e jx.Encoder
e.ObjStart()         // {
e.ObjField("values") // "values":
e.ArrStart()         // [
for i, v := range []int{4, 8, 15, 16, 23, 42} {
    if i != 0 {
        e.More() // ,
    }
    e.Int(v)
}
e.ArrEnd() // ]
e.ObjEnd() // }
fmt.Println(e)
fmt.Println("Buffer len:", len(e.Bytes()))
// Output: {"values":[4,8,15,16,23,42]}
// Buffer len: 28

Raw

Use jx.Decoder.Raw to read raw json values, similar to json.RawMessage.

d := jx.DecodeStr(`{"foo": [1, 2, 3]}`)

var raw jx.Raw
if err := d.Obj(func(d *jx.Decoder, key string) error {
    v, err := d.Raw()
    if err != nil {
        return err
    }
    raw = v
    return nil
}); err != nil {
    panic(err)
}

fmt.Println(raw.Type(), raw)
// Output:
// array [1, 2, 3]

Number

Use jx.Decoder.Num to read numbers, similar to json.Number. Also supports number strings, like "12345", which is common compatible way to represent uint64.

d := jx.DecodeStr(`{"foo": "10531.0"}`)

var n jx.Num
if err := d.Obj(func(d *jx.Decoder, key string) error {
    v, err := d.Num()
    if err != nil {
        return err
    }
    n = v
    return nil
}); err != nil {
    panic(err)
}

fmt.Println(n)
fmt.Println("positive:", n.Positive())

// Can decode floats with zero fractional part as integers:
v, err := n.Int64()
if err != nil {
    panic(err)
}
fmt.Println("int64:", v)
// Output:
// "10531.0"
// positive: true
// int64: 10531

Validate

Check that byte slice is valid json with jx.Valid:

fmt.Println(jx.Valid([]byte(`{"field": "value"}`))) // true
fmt.Println(jx.Valid([]byte(`"Hello, world!"`)))    // true
fmt.Println(jx.Valid([]byte(`["foo"}`)))            // false

Capture

The jx.Decoder.Capture method allows to unread everything is read in callback. Useful for multi-pass parsing:

d := jx.DecodeStr(`["foo", "bar", "baz"]`)
var elems int
// NB: Currently Capture does not support io.Reader, only buffers.
if err := d.Capture(func(d *jx.Decoder) error {
	// Everything decoded in this callback will be rolled back.
	return d.Arr(func(d *jx.Decoder) error {
		elems++
		return d.Skip()
	})
}); err != nil {
	panic(err)
}
// Decoder is rolled back to state before "Capture" call.
fmt.Println("Read", elems, "elements on first pass")
fmt.Println("Next element is", d.Next(), "again")

// Output:
// Read 3 elements on first pass
// Next element is array again

ObjBytes

The Decoder.ObjBytes method tries not to allocate memory for keys, reusing existing buffer.

d := DecodeStr(`{"id":1,"randomNumber":10}`)
d.ObjBytes(func(d *Decoder, key []byte) error {
    switch string(key) {
    case "id":
    case "randomNumber":
    }
    return d.Skip()
})

Roadmap

  • Rework and export Any
  • Support Raw for io.Reader
  • Support Capture for io.Reader

Non-goals

  • Code generation for decoding or encoding
  • Replacement for encoding/json
  • Reflection or interface{} based encoding or decoding
  • Support for json path or similar

This package should be kept as simple as possible and be used as low-level foundation for high-level projects like code generator.

License

MIT, same as jsoniter

Documentation

Overview

Package jx implements RFC 7159 json encoding and decoding.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func PutDecoder 

func PutDecoder(d *Decoder)

PutDecoder puts *Decoder into pool.

func PutEncoder 

func PutEncoder(e *Encoder)

PutEncoder puts *Encoder to pool

func Valid 

func Valid(data []byte) bool

Valid reports whether data is valid json.

Example 
package main

import (
	"fmt"

	"github.com/ogen-go/jx"
)

func main() {
	fmt.Println(jx.Valid([]byte(`{"field": "value"}`)))
	fmt.Println(jx.Valid([]byte(`"Hello, world!"`)))
	fmt.Println(jx.Valid([]byte(`["foo"}`)))
}

Output:

true
true
false

Types

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder is streaming json decoder.

Can read from io.Reader or byte slice directly.

func Decode 

func Decode(reader io.Reader, bufSize int) *Decoder

Decode creates a Decoder that reads json from io.Reader.

func DecodeBytes 

func DecodeBytes(input []byte) *Decoder

DecodeBytes creates a Decoder that reads json from byte slice.

func DecodeStr 

func DecodeStr(input string) *Decoder

DecodeStr creates a Decoder that reads string as json.

Example 
package main

import (
	"fmt"

	"github.com/ogen-go/jx"
)

func main() {
	d := jx.DecodeStr(`{"values":[4,8,15,16,23,42]}`)

	// Save all integers from "values" array to slice.
	var values []int

	// Iterate over each object field.
	if err := d.Obj(func(d *jx.Decoder, key string) error {
		switch key {
		case "values":
			// Iterate over each array element.
			return d.Arr(func(d *jx.Decoder) error {
				v, err := d.Int()
				if err != nil {
					return err
				}
				values = append(values, v)
				return nil
			})
		default:
			// Skip unknown fields if any.
			return d.Skip()
		}
	}); err != nil {
		panic(err)
	}

	fmt.Println(values)
}

Output:

[4 8 15 16 23 42]

func GetDecoder 

func GetDecoder() *Decoder

GetDecoder gets *Decoder from pool.

func (*Decoder) Arr 

func (d *Decoder) Arr(f func(d *Decoder) error) error

Arr reads array and calls f on each array element.

func (*Decoder) Base64 added in v0.16.0 

func (d *Decoder) Base64() ([]byte, error)

Base64 decodes base64 encoded data from string.

Same as encoding/json, base64.StdEncoding or RFC 4648.

func (*Decoder) Base64Append added in v0.16.0 

func (d *Decoder) Base64Append(b []byte) ([]byte, error)

Base64Append appends base64 encoded data from string.

Same as encoding/json, base64.StdEncoding or RFC 4648.

func (*Decoder) BigFloat 

func (d *Decoder) BigFloat() (*big.Float, error)

BigFloat read big.Float

func (*Decoder) BigInt 

func (d *Decoder) BigInt() (*big.Int, error)

BigInt read big.Int

func (*Decoder) Bool 

func (d *Decoder) Bool() (bool, error)

Bool reads a json object as Bool

func (*Decoder) Capture 

func (d *Decoder) Capture(f func(d *Decoder) error) error

Capture calls f and then rolls back to state before call.

Does not work with reader.

Example 
package main

import (
	"fmt"

	"github.com/ogen-go/jx"
)

func main() {
	d := jx.DecodeStr(`["foo", "bar", "baz"]`)
	var elems int
	// NB: Currently Capture does not support io.Reader, only buffers.
	if err := d.Capture(func(d *jx.Decoder) error {
		// Everything decoded in this callback will be rolled back.
		return d.Arr(func(d *jx.Decoder) error {
			elems++
			return d.Skip()
		})
	}); err != nil {
		panic(err)
	}
	// Decoder is rolled back to state before "Capture" call.
	fmt.Println("Read", elems, "elements on first pass")
	fmt.Println("Next element is", d.Next(), "again")

}

Output:

Read 3 elements on first pass
Next element is array again

func (*Decoder) Elem 

func (d *Decoder) Elem() (ok bool, err error)

Elem reads array element and reports whether array has more elements to read.

func (*Decoder) Float32 

func (d *Decoder) Float32() (float32, error)

Float32 reads float32 value.

func (*Decoder) Float64 

func (d *Decoder) Float64() (float64, error)

Float64 read float64

func (*Decoder) Int 

func (d *Decoder) Int() (int, error)

Int reads integer.

func (*Decoder) Int32 

func (d *Decoder) Int32() (int32, error)

Int32 reads int32 value.

func (*Decoder) Int64 

func (d *Decoder) Int64() (int64, error)

Int64 read int64

func (*Decoder) Next 

func (d *Decoder) Next() Type

Next gets Type of relatively next json element

func (*Decoder) Null 

func (d *Decoder) Null() error

Null reads a json object as null and returns whether it's a null or not.

func (*Decoder) Num added in v0.15.0 

func (d *Decoder) Num() (Num, error)

Num decodes number.

Do not retain returned value, it references underlying buffer.

Example 
package main

import (
	"fmt"

	"github.com/ogen-go/jx"
)

func main() {
	// Can decode numbers and number strings.
	d := jx.DecodeStr(`{"foo": "10531.0"}`)

	var n jx.Num
	if err := d.Obj(func(d *jx.Decoder, key string) error {
		v, err := d.Num()
		if err != nil {
			return err
		}
		n = v
		return nil
	}); err != nil {
		panic(err)
	}

	fmt.Println(n)
	fmt.Println("positive:", n.Positive())

	// Can decode floats with zero fractional part as integers:
	v, err := n.Int64()
	if err != nil {
		panic(err)
	}
	fmt.Println("int64:", v)
}

Output:

"10531.0"
positive: true
int64: 10531

func (*Decoder) NumAppend added in v0.15.0 

func (d *Decoder) NumAppend(v Num) (Num, error)

NumAppend appends number.

func (*Decoder) Obj 

func (d *Decoder) Obj(f func(d *Decoder, key string) error) error

Obj reads json object, calling f on each field.

Use ObjBytes to reduce heap allocations for keys.

func (*Decoder) ObjBytes 

func (d *Decoder) ObjBytes(f func(d *Decoder, key []byte) error) error

ObjBytes calls f for every key in object, using byte slice as key.

The key value is valid only until f is not returned.

func (*Decoder) Raw added in v0.14.0 

func (d *Decoder) Raw() (Raw, error)

Raw is like Skip(), but saves and returns skipped value as raw json.

Do not retain returned value, it references underlying buffer.

Example 
package main

import (
	"fmt"

	"github.com/ogen-go/jx"
)

func main() {
	d := jx.DecodeStr(`{"foo": [1, 2, 3]}`)

	var raw jx.Raw
	if err := d.Obj(func(d *jx.Decoder, key string) error {
		v, err := d.Raw()
		if err != nil {
			return err
		}
		raw = v
		return nil
	}); err != nil {
		panic(err)
	}

	fmt.Println(raw.Type(), raw)
}

Output:

array [1, 2, 3]

func (*Decoder) RawAppend added in v0.14.0 

func (d *Decoder) RawAppend(buf Raw) (Raw, error)

RawAppend is Raw that appends saved raw json value to buf.

func (*Decoder) Reset 

func (d *Decoder) Reset(reader io.Reader)

Reset resets reader and underlying state, next reads will use provided io.Reader.

func (*Decoder) ResetBytes 

func (d *Decoder) ResetBytes(input []byte)

ResetBytes resets underlying state, next reads will use provided buffer.

func (*Decoder) Skip 

func (d *Decoder) Skip() error

Skip skips a json object and positions to relatively the next json object.

func (*Decoder) Str 

func (d *Decoder) Str() (string, error)

Str reads string.

func (*Decoder) StrAppend 

func (d *Decoder) StrAppend(b []byte) ([]byte, error)

StrAppend reads string and appends it to byte slice.

func (*Decoder) StrBytes 

func (d *Decoder) StrBytes() ([]byte, error)

StrBytes returns string value as sub-slice of internal buffer.

Bytes are valid only until next call to any Decoder method.

func (*Decoder) Uint 

func (d *Decoder) Uint() (uint, error)

Uint read uint.

func (*Decoder) Uint32 

func (d *Decoder) Uint32() (uint32, error)

Uint32 read uint32

func (*Decoder) Uint64 

func (d *Decoder) Uint64() (uint64, error)

Uint64 read uint64

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder encodes json to underlying buffer.

Zero value is valid.

func GetEncoder 

func GetEncoder() *Encoder

GetEncoder returns *Encoder from pool.

func (*Encoder) ArrEmpty 

func (e *Encoder) ArrEmpty()

ArrEmpty writes empty array.

func (*Encoder) ArrEnd 

func (e *Encoder) ArrEnd()

ArrEnd writes end of array, performing indentation if needed.

func (*Encoder) ArrStart 

func (e *Encoder) ArrStart()

ArrStart writes start of array, performing indentation if needed.

func (*Encoder) Base64 added in v0.16.0 

func (e *Encoder) Base64(data []byte)

Base64 encodes data as standard base64 encoded string.

Same as encoding/json, base64.StdEncoding or RFC 4648.

func (*Encoder) Bool 

func (e *Encoder) Bool(val bool)

Bool writes boolean.

func (Encoder) Bytes 

func (e Encoder) Bytes() []byte

Bytes returns underlying buffer.

func (*Encoder) False 

func (e *Encoder) False()

False writes false.

func (*Encoder) Float32 

func (e *Encoder) Float32(v float32)

Float32 writes float32.

NB: Infinities and NaN are represented as null.

func (*Encoder) Float64 

func (e *Encoder) Float64(v float64)

Float64 writes float64.

NB: Infinities and NaN are represented as null.

func (*Encoder) Int 

func (e *Encoder) Int(val int)

Int writes int to stream.

func (*Encoder) Int32 

func (e *Encoder) Int32(nval int32)

Int32 writes int32 to stream.

func (*Encoder) Int64 

func (e *Encoder) Int64(nval int64)

Int64 writes int64 to stream

func (*Encoder) More 

func (e *Encoder) More()

More writes comma, performing indentation if needed.

func (*Encoder) Null 

func (e *Encoder) Null()

Null writes null to stream.

func (*Encoder) Num added in v0.15.0 

func (e *Encoder) Num(v Num)

Num encodes number.

func (*Encoder) ObjEmpty 

func (e *Encoder) ObjEmpty()

ObjEmpty writes empty object.

func (*Encoder) ObjEnd 

func (e *Encoder) ObjEnd()

ObjEnd writes end of object token, performing indentation if needed.

func (*Encoder) ObjField 

func (e *Encoder) ObjField(field string)

ObjField writes field name and colon.

For non-zero indentation also writes single space after colon.

func (*Encoder) ObjStart 

func (e *Encoder) ObjStart()

ObjStart writes object start, performing indentation if needed

func (*Encoder) Raw 

func (e *Encoder) Raw(v string)

Raw writes string as raw json.

func (*Encoder) RawBytes 

func (e *Encoder) RawBytes(b []byte)

RawBytes writes byte slice as raw json.

func (*Encoder) Reset 

func (e *Encoder) Reset()

Reset resets underlying buffer.

func (*Encoder) SetBytes 

func (e *Encoder) SetBytes(buf []byte)

SetBytes sets underlying buffer.

func (*Encoder) SetIdent 

func (e *Encoder) SetIdent(n int)

SetIdent sets length of single indentation step.

func (*Encoder) Str 

func (e *Encoder) Str(v string)

Str write string to stream without html escape

func (*Encoder) StrEscape 

func (e *Encoder) StrEscape(v string)

StrEscape encodes string with html special characters escaping.

func (Encoder) String 

func (e Encoder) String() string

String returns string of underlying buffer.

Example 
package main

import (
	"fmt"

	"github.com/ogen-go/jx"
)

func main() {
	var e jx.Encoder
	e.ObjStart()         // {
	e.ObjField("values") // "values":
	e.ArrStart()         // [
	for i, v := range []int{4, 8, 15, 16, 23, 42} {
		if i != 0 {
			e.More() // ,
		}
		e.Int(v)
	}
	e.ArrEnd() // ]
	e.ObjEnd() // }
	fmt.Println(e)
	fmt.Println("Buffer len:", len(e.Bytes()))
}

Output:

{"values":[4,8,15,16,23,42]}
Buffer len: 28

func (*Encoder) True 

func (e *Encoder) True()

True writes true.

func (*Encoder) Uint 

func (e *Encoder) Uint(val uint)

Uint writes uint to stream.

func (*Encoder) Uint32 

func (e *Encoder) Uint32(val uint32)

Uint32 writes uint32 to stream.

func (*Encoder) Uint64 

func (e *Encoder) Uint64(val uint64)

Uint64 writes uint64 to stream.

func (*Encoder) Write 

func (e *Encoder) Write(p []byte) (n int, err error)

Write implements io.Writer.

func (*Encoder) WriteTo 

func (e *Encoder) WriteTo(w io.Writer) (n int64, err error)

WriteTo implements io.WriterTo.

type Num added in v0.15.0 

type Num []byte

Num represents number, which can be raw json number or number string.

Same as Raw, but with number invariants.

Examples: 

123.45   // Str: false, IsInt: false
"123.45" // Str: true,  IsInt: false
"12345"  // Str: true,  IsInt: true
12345    // Str: false, IsInt: true

func (Num) Equal added in v0.15.0 

func (n Num) Equal(v Num) bool

Equal reports whether numbers are strictly equal, including their formats.

func (Num) Float64 added in v0.15.0 

func (n Num) Float64() (float64, error)

Float64 decodes number as 64-bit floating point.

func (Num) Int64 added in v0.15.0 

func (n Num) Int64() (int64, error)

Int64 decodes number as a signed 64-bit integer. Works on floats with zero fractional part.

func (Num) IsInt added in v0.15.0 

func (n Num) IsInt() bool

IsInt reports whether number is integer.

func (Num) Negative added in v0.15.0 

func (n Num) Negative() bool

Negative reports whether number is negative.

func (Num) Positive added in v0.15.0 

func (n Num) Positive() bool

Positive reports whether number is positive.

func (Num) Sign added in v0.15.0 

func (n Num) Sign() int

Sign reports sign of number.

0 is zero, 1 is positive, -1 is negative.

func (Num) Str added in v0.15.0 

func (n Num) Str() bool

Str reports whether Num is string number.

func (Num) String added in v0.15.0 

func (n Num) String() string

func (Num) Uint64 added in v0.15.0 

func (n Num) Uint64() (uint64, error)

Uint64 decodes number as an unsigned 64-bit integer. Works on floats with zero fractional part.

func (Num) Zero added in v0.15.0 

func (n Num) Zero() bool

Zero reports whether number is zero.

type Raw added in v0.15.0 

type Raw []byte

Raw json value.

func (Raw) String added in v0.15.1 

func (r Raw) String() string

func (Raw) Type added in v0.15.0 

func (r Raw) Type() Type

Type of Raw json value.

type Type 

type Type int

Type of json value. 

const (
	// Invalid invalid JSON element
	Invalid Type = iota
	// String JSON element "string"
	String
	// Number JSON element 100 or 0.10
	Number
	// Nil JSON element null
	Nil
	// Bool JSON element true or false
	Bool
	// Array JSON element []
	Array
	// Object JSON element {}
	Object
)

func (Type) String 

func (t Type) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.