jsonutils

README

jsonutils

jsonutils exposes a few tools to work with JSON:

• a fast, simple Concat to concatenate (not merge) JSON objects and arrays
• FromDynamicJSON to convert a data structure into a "dynamic JSON" data structure
• ReadJSON and WriteJSON behave like json.Unmarshal and json.Marshal, with the ability to use another underlying serialization library through an Adapter configured at runtime
• a JSONMapSlice structure that may be used to store JSON objects with the order of keys maintained

Dynamic JSON

We call "dynamic JSON" the go data structure that results from unmarshaling JSON like this:

  var value any
  jsonBytes := `{"a": 1, ... }`
  _ = json.Unmarshal(jsonBytes, &value)

In this configuration, the standard library mappings are as follows:

JSON	go
number	float64
string	string
boolean	bool
null	nil
object	map[string]any
array	[]any

Map slices

When using JSONMapSlice, the ordering of keys is ensured by replacing mappings to map[string]any by a JSONMapSlice which is an (ordered) slice of JSONMapItems.

Notice that a similar feature is available for YAML (see yamlutils), with a YAMLMapSlice type based on the JSONMapSlice.

JSONMapSlice is similar to an ordered map, but the keys are not retrieved in constant time.

Another difference with the the above standard mappings is that numbers don't always map to a float64: if the value is a JSON integer, it unmarshals to int64.

See also some examples

Adapters

ReadJSON, WriteJSON and FromDynamicJSON (which is a combination of the latter two) are wrappers on top of json.Unmarshal and json.Marshal.

By default, the adapter merely wraps the standard library.

The adapter may be used to register other JSON serialization libraries, possibly several ones at the same time.

If the value passed is identified as an "ordered map" (i.e. implements ifaces.Ordered or ifaces.SetOrdered, the adapter favors the "ordered" JSON behavior and tries to find a registered implementation that support ordered keys in objects.

Our standard library implementation supports this.

As of v0.25.0, we support through such an adapter the popular mailru/easyjson library, which kicks in when the passed values support the easyjson.Unmarshaler or easyjson.Marshaler interfaces.

In the future, we plan to add more similar libraries that compete on the go JSON serializers scene.

Registering an adapter

In package github.com/go-openapi/swag/easyjson/adapters, several adapters are available.

Each adapter is an independent go module. Hence you'll pick its dependencies only if you import it.

At this moment we provide:

• stdlib: JSON adapter based on the standard library
• easyjson: JSON adapter based on the github.com/mailru/easyyjson

The adapters provide the basic Marshal and Unmarshal capabilities, plus an implementation of the MapSlice pattern.

You may also build your own adapter based on your specific use-case. An adapter is not required to implement all capabilities.

Every adapter comes with a Register function, possibly with some options, to register the adapter to a global registry.

For example, to enable easyjson to be used in ReadJSON and WriteJSON, you would write something like:

  import (
	  "github.com/go-openapi/swag/jsonutils/adapters"
	  easyjson "github.com/go-openapi/swag/jsonutils/adapters/easyjson/json"
  )

  func init() {
	  easyjson.Register(adapters.Registry)
  }

You may register several adapters. In this case, capability matching is evaluated from the last registered adapters (LIFO).

Benchmarks

Documentation

Overview

Package jsonutils provides helpers to work with JSON.

These utilities work with dynamic go structures to and from JSON.

Index

• func ConcatJSON(blobs ...[]byte) []byte
• func FromDynamicJSON(source, target any) error
• func ReadJSON(data []byte, value any) error
• func WriteJSON(value any) ([]byte, error)
• type JSONMapItem
• type JSONMapSlice
  • func (s JSONMapSlice) MarshalJSON() ([]byte, error)
  • func (s JSONMapSlice) OrderedItems() iter.Seq2[string, any]
  • func (s *JSONMapSlice) SetOrderedItems(items iter.Seq2[string, any])
  • func (s *JSONMapSlice) UnmarshalJSON(data []byte) error

Examples

• ConcatJSON (Arrays)
• ConcatJSON (Objects)
• FromDynamicJSON (Orderedmap)
• FromDynamicJSON (Struct)
• JSONMapSlice
• ReadJSON

Functions

func ConcatJSON

func ConcatJSON(blobs ...[]byte) []byte

ConcatJSON concatenates multiple json objects or arrays efficiently.

Note that ConcatJSON performs a very simmple (and fast) concatenation operation: it does not attempt to merge objects.

Example (Arrays)

package main

import (
	"fmt"

	"github.com/go-openapi/swag/jsonutils"
)

func main() {
	blob1 := []byte(`["a","b","x"]`)
	blob2 := []byte(`["c","d"]`)
	blob3 := []byte(`["e","y"]`)

	// blobs are not merged: if common keys appear, duplicates will be created
	reunited := jsonutils.ConcatJSON(blob1, blob2, blob3)

	fmt.Println(string(reunited))

}

Output:

["a","b","x","c","d","e","y"]

Example (Objects)

package main

import (
	"fmt"

	"github.com/go-openapi/swag/jsonutils"
)

func main() {
	blob1 := []byte(`{"a": 1,"b": "x"}`)
	blob2 := []byte(`{"c": 1,"d": "z"}`)
	blob3 := []byte(`{"e": "y"}`)

	// blobs are not merged: if common keys appear, duplicates will be created
	reunited := jsonutils.ConcatJSON(blob1, blob2, blob3)

	fmt.Println(string(reunited))

}

Output:

{"a": 1,"b": "x","c": 1,"d": "z","e": "y"}

func FromDynamicJSON

func FromDynamicJSON(source, target any) error

FromDynamicJSON turns a go value into a properly JSON typed structure.

"Dynamic JSON" refers to what you get when unmarshaling JSON into an untyped any, i.e. objects are represented by map[string]any, arrays by []any, and all numbers are represented as float64.

NOTE: target must be a pointer.

Maintaining the order of keys in objects

If source and target implement ifaces.Ordered and ifaces.SetOrdered respectively, they are considered "ordered maps" and the order of keys is maintained in the "jsonification" process. In that case, map[string]any values are replaced by (ordered) JSONMapSlice ones.

Example (Orderedmap)

package main

import (
	"fmt"

	"github.com/go-openapi/swag/jsonutils"
)

func main() {
	source := jsonutils.JSONMapSlice{
		{Key: "A", Value: "x"},
		{Key: "B", Value: []int{0, 1}},
		{Key: "C", Value: jsonutils.JSONMapSlice{
			{Key: "X", Value: "y"},
			{Key: "Y", Value: true},
		}},
	}

	var target jsonutils.JSONMapSlice

	if err := jsonutils.FromDynamicJSON(source, &target); err != nil {
		panic(err)
	}

	fmt.Printf("%#v", target)

}

Output:

jsonutils.JSONMapSlice{jsonutils.JSONMapItem{Key:"A", Value:"x"}, jsonutils.JSONMapItem{Key:"B", Value:[]interface {}{0, 1}}, jsonutils.JSONMapItem{Key:"C", Value:json.MapSlice{json.MapItem{Key:"X", Value:"y"}, json.MapItem{Key:"Y", Value:true}}}}

Example (Struct)

package main

import (
	"fmt"

	"github.com/go-openapi/swag/jsonutils"
)

type A struct {
	A string
	B []int
	C struct {
		X string
		Y bool
	}
}

func main() {
	source := A{
		A: "x",
		B: []int{0, 1},
		C: struct {
			X string
			Y bool
		}{X: "y", Y: true},
	}

	var target any

	if err := jsonutils.FromDynamicJSON(source, &target); err != nil {
		panic(err)
	}

	fmt.Printf("%#v", target)

}

Output:

map[string]interface {}{"A":"x", "B":[]interface {}{0, 1}, "C":map[string]interface {}{"X":"y", "Y":true}}

func ReadJSON

func ReadJSON(data []byte, value any) error

ReadJSON unmarshals JSON data into a data structure.

The difference with json.Unmarshal is that it may check among several alternatives to do so.

See adapters.Registrar for more details about how to configure multiple serialization alternatives.

NOTE: value must be a pointer.

If the provided value implements ifaces.SetOrdered, it is a considered an "ordered map" and ReadJSON will favor an adapter that supports the ifaces.OrderedUnmarshal feature, or fallback to an unordered behavior if none is found.

NOTE: to allow types that are [easyjson.Unmarshaler] s to use that route to process JSON, you now need to register the adapter for easyjson at runtime.

Example

package main

import (
	"fmt"

	"github.com/go-openapi/swag/jsonutils"
)

func main() {
	const jazon = `{"a": 1,"b": "x"}`
	var value any

	if err := jsonutils.ReadJSON([]byte(jazon), &value); err != nil {
		panic(err)
	}

	reconstructed, err := jsonutils.WriteJSON(value)
	if err != nil {
		panic(err)
	}

	fmt.Println(string(reconstructed))

}

Output:

{"a":1,"b":"x"}

func WriteJSON

func WriteJSON(value any) ([]byte, error)

WriteJSON marshals a data structure as JSON.

The difference with json.Marshal is that it may check among several alternatives to do so.

See adapters.Registrar for more details about how to configure multiple serialization alternatives.

NOTE: to allow types that are [easyjson.Marshaler] s to use that route to process JSON, you now need to register the adapter for easyjson at runtime.

Types

type JSONMapItem

type JSONMapItem struct {
	Key   string
	Value any
}

JSONMapItem represents the value of a key in a JSON object held by JSONMapSlice.

Notice that JSONMapItem should not be marshaled to or unmarshaled from JSON directly.

Use this type as part of a JSONMapSlice when dealing with JSON bytes.

type JSONMapSlice

type JSONMapSlice []JSONMapItem

JSONMapSlice represents a JSON object, with the order of keys maintained.

It behaves like an ordered map, but keys can't be accessed in constant time.

Example

package main

import (
	"fmt"

	"github.com/go-openapi/swag/jsonutils"
)

func main() {
	const jazon = `{"a": 1,"c": "x", "b": 2}`
	var value jsonutils.JSONMapSlice

	if err := value.UnmarshalJSON([]byte(jazon)); err != nil {
		panic(err)
	}

	reconstructed, err := value.MarshalJSON()
	if err != nil {
		panic(err)
	}

	fmt.Println(string(reconstructed))
	fmt.Printf("%#v\n", value)

}

Output:

{"a":1,"c":"x","b":2}
jsonutils.JSONMapSlice{jsonutils.JSONMapItem{Key:"a", Value:1}, jsonutils.JSONMapItem{Key:"c", Value:"x"}, jsonutils.JSONMapItem{Key:"b", Value:2}}

func (JSONMapSlice) MarshalJSON

func (s JSONMapSlice) MarshalJSON() ([]byte, error)

MarshalJSON renders a JSONMapSlice as JSON bytes, preserving the order of keys.

It will pick the JSON library currently configured by the adapters.Registry (defaults to the standard library).

func (JSONMapSlice) OrderedItems

func (s JSONMapSlice) OrderedItems() iter.Seq2[string, any]

OrderedItems iterates over all (key,value) pairs with the order of keys maintained.

This implements the ifaces.Ordered interface, so that ifaces.Adapter s know how to marshal keys in the desired order.

func (*JSONMapSlice) SetOrderedItems

func (s *JSONMapSlice) SetOrderedItems(items iter.Seq2[string, any])

SetOrderedItems sets keys in the JSONMapSlice objects, as presented by the provided iterator.

As a special case, if items is nil, this sets to receiver to a nil slice.

This implements the ifaces.SetOrdered interface, so that ifaces.Adapter s know how to unmarshal keys in the desired order.

func (*JSONMapSlice) UnmarshalJSON

func (s *JSONMapSlice) UnmarshalJSON(data []byte) error

UnmarshalJSON builds a JSONMapSlice from JSON bytes, preserving the order of keys.

Inner objects are unmarshaled as ordered JSONMapSlice slices and not map[string]any.

It will pick the JSON library currently configured by the adapters.Registry (defaults to the standard library).