json

package module
v0.0.0-...-8d3c8d6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 19, 2025 License: Apache-2.0 Imports: 15 Imported by: 3

README

json

This go library processes JSON streams and buffers.

It is intended to provide a low-level toolkit for functionally demanding use-cases involving JSON data.

It brings JSON support for go-openapi packages.

It is not a replacement for the standard library encoding/json. As such, it does not marshal or unmarshal your favorite go struct to or from JSON.

It does not compete with blazingly fast JSON codec libraries, which essentially target the marshal/unmarshal feature.

Instead, it provides structures and types that keep the dynamic nature of JSON, deferring the resolution of values to native go types.

Essentially, it provides ways to work with JSON documents without the need of some pre-specified go data structures.

What's inside?

The module github.com/fredbi/core/json provides the following tools:

What is not inside?

  • functionality like json.Unmarshal or json.Marshal that shuffles JSON to/from go data structures.
  • JSON schema support is provided by github.com/fredbi/core/jsonschema. This library focuses on low-level aspects of JSON only.
  • OpenAPI support is provided by github.com/fredbi/core/spec. This is far beyond the scope of mere JSON processing.

How does this lib compare to others?

We designed this library with the goal to serve advanced use cases such as JSON schema analysis and OpenAPI spec handling.

Besides, we wanted the core of go-openapi to avoid external dependencies for this kind of work.

Depending on your goals, you might find libraries that are a better fit for your use case.

  • It does not generate code like github.com/mailru/easyjson. However, it shares a lot of ideas with the lexer and writer from easyjson.
  • It does not provide an API to extract JSON values directly like github.com/valyala/fastjson or github.com/tidwall/gjson. However, it shares a lot of ideas with fastjson such as reusable values stored in an Arena.
  • It does not support mapping to your structs like encoding/json or drop-in replacements for the standard library like github.com/go-ccy/go-json or github.com/jsoniterator/go.
  • TODO: compare github.com/go-faster/jx
  • TODO: compare github.com/francoispqt/gojay
  • TODO: compare github.com/bytedance/sonic (a JIT JSON serializing/deserializing lib)

On the other hand, it provides a few features that you won't find in the above-mentioned libraries:

  • Accurate error reporting with context when lexing JSON
  • Support for streams as well as buffers
  • Support for JSON pointers and JSONPath expressions
  • The ability to build JSON programmatically

What is "verbatim"?

A verbatim document reproduces the original JSON unaltered, that is, including non-significant blank space used for indentation. It also leaves escaped unicode sequence unchanged.

This allows to process JSON with the ability to control the original input (as in a text editor).

In contrast, a default document focuses on JSON semantics, i.e. structure and values: non-significant blanks are ignored and the annoying escaped unicode sequences are resolved as soon a possible into their UTF-8 equivalent.

Verbatim lexer, store, document and writer should be reserved to tools like linters, text editor plugins and the like.

What can I do with a JSON "Document"?

Documents are immutable, but cheap to build or amend using shallow clones. This guards against nasty bugs and allows for a concurrent processing of documents.

Documents maintain the original ordering of keys in objects.

  • Marshal / Unmarshal to/from JSON bytes
  • Encode / Decode to/from a stream of JSON bytes
  • Build or clone & amend using the Builder type
  • Walk a document using iterators
  • TODO: resolve a JSON Pointer within the document
  • TODO: Walk a document using jsonpath expressions. See github.com/fredbi/core/json/jsonpath.

Design goals

  1. accurate representation of JSON semantics[^1].
  2. accurate error reporting [^2]
  3. memory-efficient processing of JSON. The toolkit tries its best to keep a low profile in memory [^3].
  4. immutable documents, cheap to clone and build programmatically
  5. dependency-free packages, besides test dependencies
  6. extensible interfaces[^4]

Non-goals

This library favors accuracy over speed. It may not be the fastest JSON processor available for go. In particular:

  • we trade accurate error reporting for speed

At this moment, it is not a goal to support XXL documents outside of main memory, although this could be a contributed extension of the stores.Store interface.

Likewise, it is not part of the current plan to provide writers to specialized backends such as databases or message-oriented middleware. Again, such a feature may be provided in the future as a contributed extension (while keeping our dependency-free goal).

At this moment, the scope is limited to JSON. YAML data, considered as a superset of JSON, is not part of the picture for now.

Use cases

Support go-openapi v2:

Besides, here are a few examples of where this toolkit might be useful:

  • lexer/writer: JSON linter, text editor plugin
  • Document: implement an API that deals dynamically with json
  • dynamic JSON: alternative way to implement an API that deals dynamically with json

[^1]: the default lexer is strict regarding JSON semantics. JSON types allow to distinguish undefined from null.

[^2]: the lexer does its best to report informative error messages, and capture the context of the offending part, including when dealing with streams.

[^3]: this leverages the memory pooling utilities exposed by github.com/fredbi/core/pools. By and large, the toolkit favors memory scarcity over CPU, assuming that much of the CPU-intensive processing may be deferred lazily. For instance, th default implementation for stores.Store compresses long strings.

[^4]: contrib modules are proposed to host standalone go modules that may bring their own set of dependencies

Documentation

Overview

Package json deals with JSON documents.

This library provides tools to work with raw JSON data, with no attempt to convert it to native go types until the last moment.

This allows to abide strictly to the JSON standards. In particular, it supports JSON-specifics which are not well handled by the standard library, such as:

  • null
  • values with a zero value in go (e.g. 0, "", false)
  • very large or very high-precision numbers that overflow go native types (e.g. int64, float64)

Main memory usage

The library design is primarily focused on keeping a low memory profile, in terms of space as well as in terms of allocations:

  • few things are pointers or require allocations
  • most data structures that need a dynamic allocation may be recycled using pools, thus amortizing the cost of allocations
  • JSON values are isolated in a specific in-memory store, which takes care of optimizing their size
  • object keys are interned
  • lazy JSON value resolution

Immutability

Another design goal of this package is immutability: all provided objects Document, light.Node, [token.Token], stores.Value etc are all immutable, and designed to be cheap to clone instead.

Mutating or constructing a JSON Document programmatically requires a Builder to carry out a series of fluent building methods and produce a modified clone of the original Document.

Extensibility

There are tons of use-cases out there to play with.

Specific modules may be added to extend or improve the default implementations of lexers.Lexer, writers.Writer and stores.Store. As independent modules, they may bring with them their own set of dependencies without altering the dependencies of the parent module.

Index

Constants

This section is empty.

Variables

View Source 
var EmptyDocument = Make()

EmptyDocument is the JSON document of the null type.

It has no stores.Store attached to it.

Functions

func RedeemBuilder 

func RedeemBuilder(b *Builder)

func RedeemDocument 

func RedeemDocument(d *Document)

Types

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder builds or transforms JSON Document s programmatically.

You may either use it directly, starting from the EmptyDocument or clone from an existing Document using Builder.From.

Since a Document is immutable, the Builder always produces a shallow clone of the original Document.

The Builder exposes fluent building methods which may be chained to construct a JSON document. You should always check the final error state, since the building cease to be effective as soon as an error is encountered.

func BorrowBuilder 

func BorrowBuilder() *Builder

func MakeBuilder 

func MakeBuilder(s stores.Store) Builder

func NewBuilder 

func NewBuilder(s stores.Store) *Builder

func (*Builder) AppendElem 

func (b *Builder) AppendElem(value Document) *Builder

func (*Builder) AppendKey 

func (b *Builder) AppendKey(key string, value Document) *Builder

func (*Builder) Array 

func (b *Builder) Array() *Builder

Array builds a JSON array

func (*Builder) BoolValue 

func (b *Builder) BoolValue(value bool) *Builder

func (Builder) Document 

func (b Builder) Document() Document

Document returns the Document produced by the Builder.

If a build error has occured, it returns the EmptyDocument.

func (Builder) Err 

func (b Builder) Err() error

func (*Builder) FloatValue 

func (b *Builder) FloatValue(value types.Number) *Builder

func (*Builder) From 

func (b *Builder) From(d Document) *Builder

func (*Builder) Null 

func (b *Builder) Null() *Builder

func (*Builder) NumberValue 

func (b *Builder) NumberValue(value types.Number) *Builder

func (*Builder) Object 

func (b *Builder) Object() *Builder

Object builds a JSON object

func (Builder) Ok 

func (b Builder) Ok() bool

func (*Builder) Reset 

func (b *Builder) Reset()

func (*Builder) SetErr 

func (b *Builder) SetErr(err error)

func (*Builder) StringValue 

func (b *Builder) StringValue(value string) *Builder

StringValue builds a JSON string

func (*Builder) WithStore 

func (b *Builder) WithStore(s stores.Store) *Builder

type Context 

type Context struct {
	light.Context
}

Context of a node, i.e. the offset in the originally parsed JSON input.

type Document 

type Document struct {
	// contains filtered or unexported fields
}

Document represents a JSON document as a hierarchy of JSON data nodes.

A Document knows how to marshal or unmarshal bytes or decode/encode with streams.

A Document is immutable: it may be unmarshaled from JSON, from [Dynamic JSON] or built programmatically using the Builder.

Accessing nodes

Document.AtKey retrieves an individual keys in an object. Document.Elem does the same for an element in an array.

Iterators

The hierarchy of nodes defined by a Document may be explored using iterators like Document.Pairs and Document.Elems.

Iterators maintain the original order in which object keys and array elements have been provided.

Exploring a document

JSON pointers are supported within a Document using Document.GetPointer.

An implementation of JSONPath is provided in github.com/fredbi/core/json/documents.jsonpath to resolve JSONPath expressions as a Document iterator.

Working with values

TODO(fred)

Dynamic JSON

We call "dynamic JSON" (sometimes referred to as "untyped") refers to the go structures made up of "map[string]any" and "[]any" that you typically get when the go standard library unmarshals JSON into an "any" type (aka "interface{}").

A Document may unmarshal such a data structure or may be converted into one.

In that case, due to the implementation of go maps, the order of keys in objects cannot be maintained.

This package only deals with pure JSON, not schemas.

Package github.com/fredbi/core/jsonschma brings the additional logic required to deal with JSON schemas.

Package github.com/fredbi/core/spec brings the additional logic required to deal with OpenAPI documents.

func BorrowDocument 

func BorrowDocument() *Document

func Make 

func Make(opts ...Option) Document

Make an empty Document.

The empty Document marshals as "null".

func (Document) AtKey 

func (d Document) AtKey(k string) (Document, bool)

AtKey returns the value held under a key in an object, or false if not found.

Key lookup is constant-time (map index).

func (Document) Context 

func (d Document) Context() Context

func (*Document) Decode 

func (d *Document) Decode(r io.Reader) error

Decode builds a Document from a stream of JSON bytes.

func (Document) Elem 

func (d Document) Elem(i int) (Document, bool)

Elem returns the i-th element of an array.

func (Document) Elems 

func (d Document) Elems() iter.Seq[Document]

Elems returns all elements in an array.

Iteration order is stable and honors the original ordering in which elements were declared.

func (Document) Encode 

func (d Document) Encode(w io.Writer) error

Encode the Document as a JSON stream to an io.Writer.

func (Document) GetPointer 

func (d Document) GetPointer(Pointer) Document

func (Document) KeyIndex 

func (d Document) KeyIndex(k string) (int, bool)

KeyIndex returns the index of a key, of false if not found.

func (Document) Kind 

func (d Document) Kind() nodes.Kind

func (Document) Len 

func (d Document) Len() int

func (Document) LexerFactory 

func (o Document) LexerFactory() func([]byte) (lexers.Lexer, func())

func (Document) LexerFromReaderFactory 

func (o Document) LexerFromReaderFactory() func(io.Reader) (lexers.Lexer, func())

func (Document) MarshalJSON 

func (d Document) MarshalJSON() ([]byte, error)

MarshalJSON writes the Document as JSON bytes.

func (Document) Node 

func (d Document) Node() light.Node

Node low-level access to the current node in the document hierarchy.

func (Document) Pairs 

func (d Document) Pairs() iter.Seq2[string, Document]

Pairs return all (key,Node) pairs inside an object.

Iteration order is stable and honors the original ordering in which keys were declared.

func (Document) Store 

func (d Document) Store() stores.Store

func (Document) String 

func (d Document) String() string

func (*Document) UnmarshalJSON 

func (d *Document) UnmarshalJSON(data []byte) error

UnmarshalJSON builds a Document from JSON bytes.

func (Document) Value 

func (d Document) Value() (stores.Value, bool)

func (Document) WriterFactory 

func (o Document) WriterFactory() func() (writers.Writer, func())

func (Document) WriterToWriterFactory 

func (o Document) WriterToWriterFactory() func(io.Writer) (writers.Writer, func())

type DocumentCollection 

type DocumentCollection struct {
	// contains filtered or unexported fields
}

DocumentCollection is a collection of Document s, that share the same options.

It marshals as an array of [Document]s (TODO).

func NewDocumentCollection 

func NewDocumentCollection(opts ...Option) *DocumentCollection

func (*DocumentCollection) Append 

func (c *DocumentCollection) Append(d Document)

func (*DocumentCollection) Concat 

func (c *DocumentCollection) Concat(d DocumentCollection)

func (DocumentCollection) LexerFactory 

func (o DocumentCollection) LexerFactory() func([]byte) (lexers.Lexer, func())

func (DocumentCollection) LexerFromReaderFactory 

func (o DocumentCollection) LexerFromReaderFactory() func(io.Reader) (lexers.Lexer, func())

func (DocumentCollection) WriterFactory 

func (o DocumentCollection) WriterFactory() func() (writers.Writer, func())

func (DocumentCollection) WriterToWriterFactory 

func (o DocumentCollection) WriterToWriterFactory() func(io.Writer) (writers.Writer, func())

type DocumentFactory 

type DocumentFactory struct {
	// contains filtered or unexported fields
}

DocumentFactory is a factory that produces Document s with the same settings.

func NewDocumentFactory 

func NewDocumentFactory(opts ...Option) *DocumentFactory

NewDocumentFactory builds a factory for Document s

func (DocumentFactory) Clone 

func (f DocumentFactory) Clone(d Document) Document

func (DocumentFactory) Empty 

func (f DocumentFactory) Empty() Document

func (DocumentFactory) LexerFactory 

func (o DocumentFactory) LexerFactory() func([]byte) (lexers.Lexer, func())

func (DocumentFactory) LexerFromReaderFactory 

func (o DocumentFactory) LexerFromReaderFactory() func(io.Reader) (lexers.Lexer, func())

func (DocumentFactory) WriterFactory 

func (o DocumentFactory) WriterFactory() func() (writers.Writer, func())

func (DocumentFactory) WriterToWriterFactory 

func (o DocumentFactory) WriterToWriterFactory() func(io.Writer) (writers.Writer, func())

type Option 

type Option func(*options)

func WithLexer 

func WithLexer(l lexers.Lexer) Option

func WithStore 

func WithStore(s stores.Store) Option

func WithWriter 

func WithWriter(w writers.Writer) Option

type Pointer 

type Pointer struct {
}

type VerbatimDocument 

type VerbatimDocument struct {
}

VerbatimDocument holds a JSON document verbatim.

All original marks kepts, including non-significant white space and escaped unicode points.

TODO(fred)

Source Files

View all Source files

Directories

Path Synopsis
Package constrained expose constrained json.Document types.
 Package constrained expose constrained json.Document types.
Package dynamic converts a json.Document to dynamic JSON and vice versa.
 Package dynamic converts a json.Document to dynamic JSON and vice versa.
Package lexers exposes interfaces for lexing JSON.
 Package lexers exposes interfaces for lexing JSON.
default-lexer
Package lexer provides a JSON lexer.
 Package lexer provides a JSON lexer.
error-codes
Package codes exposes common lexer errors.
 Package codes exposes common lexer errors.
ld-lexer
Package lexer exposes lexer for ld-json (line-delimited JSON).
 Package lexer exposes lexer for ld-json (line-delimited JSON).
token
Package token defines common JSON token types with their kind.
 Package token defines common JSON token types with their kind.
Package nodes declares node types for JSON documents.
 Package nodes declares node types for JSON documents.
error-codes
light
Package light exposes an implementation for nodes.
 Package light exposes an implementation for nodes.
Package stores exposes the interface to work with a JSON Store.
 Package stores exposes the interface to work with a JSON Store.
default-store
Package store provides default implementations for stores.Store.
 Package store provides default implementations for stores.Store.
Package types defines common types to work with json.
 Package types defines common types to work with json.
default-writer
Package writer exposes a JSON writer.
 Package writer exposes a JSON writer.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.