purejson

package module
v0.1.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 23, 2026 License: MIT Imports: 15 Imported by: 0

README

pure-simdjson

pure-simdjson is a cgo-free Go wrapper around simdjson with a DOM API, typed number accessors, cursor-style iteration, and bootstrap support for prebuilt native libraries.

Installation

go get github.com/amikos-tech/pure-simdjson@latest

The first NewParser() call downloads the platform-native library if it is not already cached locally. Mirror, offline, and override details live in docs/bootstrap.md.

Quick Start

package main

import (
	"fmt"

	purejson "github.com/amikos-tech/pure-simdjson"
)

func main() {
	parser, err := purejson.NewParser()
	if err != nil {
		panic(err)
	}
	defer func() { _ = parser.Close() }()

	doc, err := parser.Parse([]byte(`{"name":"alice"}`))
	if err != nil {
		panic(err)
	}
	defer func() { _ = doc.Close() }()

	object, err := doc.Root().AsObject()
	if err != nil {
		panic(err)
	}

	name, err := object.GetStringField("name")
	if err != nil {
		panic(err)
	}

	fmt.Println(name)
}

This snippet is derived from example_test.go.

Supported Platforms

  • linux/amd64
  • linux/arm64
  • darwin/amd64
  • darwin/arm64
  • windows/amd64

Benchmark Snapshot

The current benchmark evidence is published in v0.1.2 results with the benchmark methodology. Comparator tables omit unsupported libraries on a given target instead of showing synthetic N/A rows.

On the linux/amd64 CI target, Tier 1 full any materialization now beats encoding/json + any by 3.15x on twitter.json, 3.39x on citm_catalog.json, and 2.47x on canada.json. Tier 2 typed extraction beats encoding/json + struct by 12.49x to 14.56x, and Tier 3 selective traversal on the current DOM API beats encoding/json + struct by 15.18x to 15.97x. Headline numbers come from linux/amd64; other platforms may differ.

The published default-install version is v0.1.4. The v0.1.2 benchmark snapshot remains the validated baseline because v0.1.3 and v0.1.4 are recovery releases targeting the publish pipeline (manylinux libstdc++ allocator behavior, Linux symbol visibility, macOS allocator-cleanup telemetry) rather than the parse or materialize hot paths. Phase 09.1 owns default-install alignment, and bootstrap details remain in docs/bootstrap.md.

Documentation

Overview

Package purejson exposes the Go wrapper for the pure-simdjson native library.

NewParser creates one reusable native parser handle. Each Parser may own only one live Doc at a time, so callers must close the current document before parsing again or before closing or pooling the parser.

Parsed documents expose typed Element accessors that preserve simdjson's int64/uint64/float64 split, copy strings into Go-owned memory, and surface arrays and objects through scanner-style iterators plus direct field lookup helpers.

Parser limits are immutable after construction:

parser, err := NewParser(
	WithMaxCapacity(8<<20),
	WithMaxDepth(128),
)

TypeBigInt values keep their exact decimal spelling. GetBigInt returns a copied Go string, so the text remains owned by Go after the document closes:

digits, err := element.GetBigInt()

Parse locations are not guessed. Check HasOffset before using Offset, including when byte zero may be the known location:

if parseErr.HasOffset() {
	log.Printf("invalid JSON at byte %d", parseErr.Offset())
}

NewParserPool hands parsers across goroutines without weakening the lifecycle rule. Kernel selection is process-global and diagnostic-only: SetKernel must run before the first parser or parser pool is created, after which it returns ErrKernelLocked. See docs/concurrency.md in the repository for the concurrency and cleanup model.

Index

Examples

Constants

This section is empty.

Variables

View Source
var (
	// ErrInvalidHandle reports that a parser, document, or element handle was not valid.
	ErrInvalidHandle = errors.New("invalid handle")
	// ErrClosed reports use of a parser, document, or document-tied
	// element/iterator after the underlying parser/doc has been closed or
	// released.
	ErrClosed = errors.New("closed")
	// ErrParserBusy reports that a parser still owns a live document.
	ErrParserBusy = errors.New("parser busy")
	// ErrNumberOutOfRange reports that a numeric conversion overflowed the target type.
	ErrNumberOutOfRange = errors.New("number out of range")
	// ErrPrecisionLoss reports that a numeric conversion would lose precision.
	ErrPrecisionLoss = errors.New("precision loss")
	// ErrCPUUnsupported reports that the loaded native library cannot run on the current CPU.
	ErrCPUUnsupported = errors.New("cpu unsupported")
	// ErrABIVersionMismatch reports that the Go wrapper and native library expose different ABI versions.
	ErrABIVersionMismatch = errors.New("abi version mismatch")
	// ErrPanic reports that the native library trapped a Rust panic at the FFI boundary.
	ErrPanic = errors.New("panic")
	// ErrCPPException reports that the native library trapped a C++ exception before it crossed the FFI boundary.
	ErrCPPException = errors.New("cpp exception")
	// ErrInvalidJSON reports invalid JSON input.
	ErrInvalidJSON = errors.New("invalid json")
	// ErrElementNotFound reports lookup of a missing element.
	ErrElementNotFound = errors.New("element not found")
	// ErrWrongType reports an accessor call on the wrong JSON value kind.
	ErrWrongType = errors.New("wrong type")
	// ErrNotImplemented reports an optional native diagnostic surface that is
	// unavailable in the loaded artifact.
	ErrNotImplemented = errors.New("not implemented")
	// ErrDepthLimitExceeded reports JSON nesting deeper than the native parser
	// or materializer depth contract can process.
	ErrDepthLimitExceeded = errors.New("depth limit exceeded")
	// ErrCapacityLimitExceeded reports input larger than the parser's immutable
	// configured capacity.
	ErrCapacityLimitExceeded = errors.New("capacity limit exceeded")
	// ErrInvalidOption reports an invalid parser construction option.
	ErrInvalidOption = errors.New("invalid option")
	// ErrKernelLocked reports kernel selection after parser or pool creation.
	ErrKernelLocked = errors.New("kernel selection locked")
	// ErrInternal reports native panics, internal failures, and any status code
	// not mapped to a dedicated sentinel.
	ErrInternal = errors.New("internal error")
)
View Source
var (
	ErrChecksumMismatch = bootstrap.ErrChecksumMismatch
	ErrAllSourcesFailed = bootstrap.ErrAllSourcesFailed
	ErrNoChecksum       = bootstrap.ErrNoChecksum
)

Bootstrap error sentinels are re-exported from internal/bootstrap. Pointer identity is preserved, so errors.Is(err, purejson.ErrChecksumMismatch) and errors.Is(err, bootstrap.ErrChecksumMismatch) both match. Canonical definitions live in internal/bootstrap/errors.go — never call errors.New for these here.

Functions

func Kernel added in v0.1.5

func Kernel() string

Kernel returns the active process-wide native implementation name. It returns an empty string until a native library has already been loaded and never triggers library resolution or bootstrap work.

func SetKernel added in v0.1.5

func SetKernel(name string) error

SetKernel selects an exact process-wide native implementation name for diagnostics. An empty name restores automatic selection. Selection becomes immutable after the first parser or parser pool is created.

Types

type Array

type Array struct {
	// contains filtered or unexported fields
}

Array wraps an Element verified to represent a JSON array. Construct via Element.AsArray; the unexported field prevents callers from creating an unverified instance.

func (Array) Iter

func (a Array) Iter() *ArrayIter

Iter returns a scanner-style iterator over the array contents in document order. Creating many descendant views or iterators on a long-lived document grows native bookkeeping proportionally; Doc.Close releases all of it at once.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`[1,2,3]`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

array, err := doc.Root().AsArray()
if err != nil {
	panic(err)
}

sum := int64(0)
iter := array.Iter()
for iter.Next() {
	value, err := iter.Value().GetInt64()
	if err != nil {
		panic(err)
	}
	sum += value
}
if err := iter.Err(); err != nil {
	panic(err)
}

fmt.Println(sum)
Output:
6

type ArrayIter

type ArrayIter struct {
	// contains filtered or unexported fields
}

ArrayIter scans array values one element at a time in document order.

func (*ArrayIter) Err

func (it *ArrayIter) Err() error

Err reports the terminal iterator error, if any.

func (*ArrayIter) Next

func (it *ArrayIter) Next() bool

Next advances the iterator and reports whether another value is available. It returns false after the iterator is exhausted or when Err reports a terminal failure.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`["first","second"]`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

array, err := doc.Root().AsArray()
if err != nil {
	panic(err)
}

iter := array.Iter()
for iter.Next() {
	value, err := iter.Value().GetString()
	if err != nil {
		panic(err)
	}
	fmt.Println(value)
}
fmt.Println(iter.Err() == nil)
Output:
first
second
true

func (*ArrayIter) Value

func (it *ArrayIter) Value() Element

Value returns the current array element after Next reports true.

type Doc

type Doc struct {
	// contains filtered or unexported fields
}

Doc wraps one live native document handle plus its cached root view.

func (*Doc) Close

func (d *Doc) Close() error

Close releases the native document and clears the owning parser's busy state. It is idempotent.

func (*Doc) Root

func (d *Doc) Root() Element

Root returns the cached root element view for the live document.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`{"name":"alice"}`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

object, err := doc.Root().AsObject()
if err != nil {
	panic(err)
}

name, err := object.GetStringField("name")
if err != nil {
	panic(err)
}

fmt.Println(name)
Output:
alice

type Element

type Element struct {
	// contains filtered or unexported fields
}

Element is the public value-view wrapper for a document root or child value.

Example (ScalarAccess)
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`{"id":7,"name":"alice","active":true}`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

object, err := doc.Root().AsObject()
if err != nil {
	panic(err)
}

idField, err := object.GetField("id")
if err != nil {
	panic(err)
}
id, err := idField.GetInt64()
if err != nil {
	panic(err)
}

nameField, err := object.GetField("name")
if err != nil {
	panic(err)
}
name, err := nameField.GetString()
if err != nil {
	panic(err)
}

activeField, err := object.GetField("active")
if err != nil {
	panic(err)
}
active, err := activeField.GetBool()
if err != nil {
	panic(err)
}

fmt.Println(id, name, active)
Output:
7 alice true

func (Element) AsArray

func (e Element) AsArray() (Array, error)

AsArray returns a typed Array view when the element represents a JSON array. Returns ErrClosed when the owning document is released and ErrWrongType when the underlying value kind is not an array.

func (Element) AsObject

func (e Element) AsObject() (Object, error)

AsObject returns a typed Object view when the element represents a JSON object. Returns ErrClosed when the owning document is released and ErrWrongType when the underlying value kind is not an object.

func (Element) GetBigInt added in v0.1.5

func (e Element) GetBigInt() (string, error)

GetBigInt reads the current element as exact copied decimal text. It accepts only TypeBigInt and returns ErrWrongType for every other element kind.

func (Element) GetBool

func (e Element) GetBool() (bool, error)

GetBool reads the current element as a bool and returns ErrClosed when the owning document has already been released.

func (Element) GetFloat64

func (e Element) GetFloat64() (float64, error)

GetFloat64 reads the current element as a float64 and returns ErrClosed when the owning document has already been released. Large int64 and uint64 values that would lose precision report ErrPrecisionLoss instead of rounding; TypeBigInt reports ErrWrongType.

func (Element) GetInt64

func (e Element) GetInt64() (int64, error)

GetInt64 reads the current element as an int64 and returns ErrClosed when the owning document has already been released. Uint64 values larger than max int64 report ErrNumberOutOfRange, while float and TypeBigInt values report ErrWrongType. Element accessors are not safe for concurrent use with Doc.Close.

func (Element) GetString

func (e Element) GetString() (string, error)

GetString reads the current element as a copied Go string and returns ErrClosed when the owning document has already been released.

func (Element) GetUint64

func (e Element) GetUint64() (uint64, error)

GetUint64 reads the current element as a uint64 and returns ErrClosed when the owning document has already been released. Negative integers report ErrNumberOutOfRange, while non-uint64 kinds, including TypeBigInt, report ErrWrongType.

func (Element) IsNull

func (e Element) IsNull() bool

IsNull reports whether the current element is a JSON null value. Closed, invalid, or tampered views return false.

func (Element) IsNullErr

func (e Element) IsNullErr() (bool, error)

IsNullErr reports whether the current element is a JSON null value while preserving native failures such as ErrClosed or ErrInvalidHandle.

func (Element) Type

func (e Element) Type() ElementType

Type reports the concrete JSON value kind for the current element. Closed, invalid, or tampered views collapse to TypeInvalid instead of returning an error.

func (Element) TypeErr

func (e Element) TypeErr() (ElementType, error)

TypeErr reports the concrete JSON value kind for the current element while preserving native failures such as ErrClosed, ErrInvalidHandle, or ErrPanic.

type ElementType

type ElementType uint32

ElementType reports the concrete JSON value kind for an Element, preserving the distinct int64, uint64, and float64 classifications from simdjson's DOM.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`18446744073709551615`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

fmt.Println(doc.Root().Type() == TypeUint64)
Output:
true
const (
	// TypeInvalid reports a closed, invalid, or otherwise unusable element view.
	TypeInvalid ElementType = ElementType(ffi.ValueKindInvalid)
	// TypeNull reports a JSON null value.
	TypeNull ElementType = ElementType(ffi.ValueKindNull)
	// TypeBool reports a JSON boolean value.
	TypeBool ElementType = ElementType(ffi.ValueKindBool)
	// TypeInt64 reports a JSON number classified as int64.
	TypeInt64 ElementType = ElementType(ffi.ValueKindInt64)
	// TypeUint64 reports a JSON number classified as uint64.
	TypeUint64 ElementType = ElementType(ffi.ValueKindUint64)
	// TypeFloat64 reports a JSON number classified as float64.
	TypeFloat64 ElementType = ElementType(ffi.ValueKindFloat64)
	// TypeString reports a JSON string value.
	TypeString ElementType = ElementType(ffi.ValueKindString)
	// TypeArray reports a JSON array value.
	TypeArray ElementType = ElementType(ffi.ValueKindArray)
	// TypeObject reports a JSON object value.
	TypeObject ElementType = ElementType(ffi.ValueKindObject)
	// TypeBigInt reports an integer outside the int64 and uint64 ranges.
	TypeBigInt ElementType = 9
)

type Error

type Error struct {
	// contains filtered or unexported fields
}

Error carries native status details while still participating in Go's sentinel-error matching via Unwrap. Status details are exposed through accessor methods so callers cannot mutate them after construction.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

_, err = parser.Parse([]byte(`{"name":`))
if err == nil {
	panic("expected parse error")
}

fmt.Println(errors.Is(err, ErrInvalidJSON))

var nativeErr *Error
fmt.Println(errors.As(err, &nativeErr))
Output:
true
true

func (*Error) Code

func (e *Error) Code() int32

Code returns the native status code returned by the FFI call.

func (*Error) Error

func (e *Error) Error() string

Error formats the native status details as a human-readable message while preserving the wrapped sentinel error semantics.

func (*Error) HasOffset added in v0.1.5

func (e *Error) HasOffset() bool

HasOffset reports whether Offset is a trustworthy native error location.

func (*Error) Message

func (e *Error) Message() string

Message returns the native error message, when available.

func (*Error) Offset

func (e *Error) Offset() uint64

Offset returns the reported byte offset for parse errors. Call HasOffset to distinguish a known location at byte zero from an unknown location.

func (*Error) Unwrap

func (e *Error) Unwrap() error

Unwrap returns the sentinel error associated with the native status code so callers can use errors.Is with the exported package errors.

type Object

type Object struct {
	// contains filtered or unexported fields
}

Object wraps an Element verified to represent a JSON object. Construct via Element.AsObject; the unexported field prevents callers from creating an unverified instance.

func (Object) GetField

func (o Object) GetField(key string) (Element, error)

GetField returns the element for the given object key. Missing fields return ErrElementNotFound, while present null fields return a valid Element whose IsNull method reports true. When duplicate keys are present, GetField returns the first matching field. An empty key performs a literal lookup for the JSON key "". Creating many descendant views or iterators on a long-lived document grows native bookkeeping proportionally; Doc.Close releases all of it at once.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`{"active":true}`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

object, err := doc.Root().AsObject()
if err != nil {
	panic(err)
}

field, err := object.GetField("active")
if err != nil {
	panic(err)
}

active, err := field.GetBool()
if err != nil {
	panic(err)
}

fmt.Println(active)
Output:
true

func (Object) GetStringField

func (o Object) GetStringField(name string) (string, error)

GetStringField returns the named field as a copied Go string using the same semantics as GetField followed by Element.GetString, including literal lookup for the JSON key "", ErrElementNotFound for missing fields, and ErrWrongType for present non-string values.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`{"name":"alice"}`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

object, err := doc.Root().AsObject()
if err != nil {
	panic(err)
}

name, err := object.GetStringField("name")
if err != nil {
	panic(err)
}

fmt.Println(name)
Output:
alice

func (Object) Iter

func (o Object) Iter() *ObjectIter

Iter returns a scanner-style iterator over the object fields in document order. Creating many descendant views or iterators on a long-lived document grows native bookkeeping proportionally; Doc.Close releases all of it at once.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`{"a":1,"b":2}`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

object, err := doc.Root().AsObject()
if err != nil {
	panic(err)
}

iter := object.Iter()
for iter.Next() {
	value, err := iter.Value().GetInt64()
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s=%d\n", iter.Key(), value)
}
if err := iter.Err(); err != nil {
	panic(err)
}
Output:
a=1
b=2

type ObjectIter

type ObjectIter struct {
	// contains filtered or unexported fields
}

ObjectIter scans object entries one field at a time in document order. Keys are exposed as copied Go strings.

func (*ObjectIter) Err

func (it *ObjectIter) Err() error

Err reports the terminal iterator error, if any.

func (*ObjectIter) Key

func (it *ObjectIter) Key() string

Key returns the current object key after Next reports true.

func (*ObjectIter) Next

func (it *ObjectIter) Next() bool

Next advances the iterator and reports whether another object entry is available. It caches the current key as a copied Go string for Key and the current value view for Value.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`{"name":"alice"}`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

object, err := doc.Root().AsObject()
if err != nil {
	panic(err)
}

iter := object.Iter()
fmt.Println(iter.Next())
fmt.Println(iter.Key())
value, err := iter.Value().GetString()
if err != nil {
	panic(err)
}
fmt.Println(value)
fmt.Println(iter.Next())
fmt.Println(iter.Err() == nil)
Output:
true
name
alice
false
true

func (*ObjectIter) Value

func (it *ObjectIter) Value() Element

Value returns the current object value after Next reports true.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser owns one live native parser handle and enforces a one-document-at-a- time lifecycle.

func NewParser

func NewParser(opts ...ParserOption) (*Parser, error)

NewParser validates immutable parser options, resolves an ABI-compatible local shared library, and allocates a reusable native parser.

func (*Parser) Close

func (p *Parser) Close() error

Close releases the native parser. While a live document still belongs to the parser, Close returns ErrParserBusy and leaves the parser usable. Subsequent calls after a successful Close return nil.

func (*Parser) Parse

func (p *Parser) Parse(data []byte) (*Doc, error)

Parse copies one JSON buffer into the native parser and returns a live Doc on success.

Example
parser, err := NewParser()
if err != nil {
	panic(err)
}
defer func() { _ = parser.Close() }()

doc, err := parser.Parse([]byte(`42`))
if err != nil {
	panic(err)
}
defer func() { _ = doc.Close() }()

value, err := doc.Root().GetInt64()
if err != nil {
	panic(err)
}

fmt.Println(value)
Output:
42

type ParserOption added in v0.1.5

type ParserOption struct {
	// contains filtered or unexported fields
}

ParserOption configures an immutable parser capacity or depth bound.

Options are created with WithMaxCapacity and WithMaxDepth. The zero value is invalid.

func WithMaxCapacity added in v0.1.5

func WithMaxCapacity(bytes int) ParserOption

WithMaxCapacity sets the maximum accepted input size in bytes. Zero selects the default 0xFFFFFFFF-byte limit.

func WithMaxDepth added in v0.1.5

func WithMaxDepth(depth int) ParserOption

WithMaxDepth sets the upstream parser's maximum nesting depth. Zero selects the default depth of 1024.

type ParserPool

type ParserPool struct {
	// contains filtered or unexported fields
}

ParserPool reuses Parser instances across goroutines while preserving the one-live-doc-per-parser invariant. There is no Close method: sync.Pool cannot be drained deterministically. Parsers left in the pool when it is discarded are reclaimed by the GC finalizer; the same leak-warning rules that apply to standalone parsers apply here.

func NewParserPool

func NewParserPool(opts ...ParserOption) (*ParserPool, error)

NewParserPool validates immutable parser options and constructs an empty parser pool. Native library resolution is deferred until the first Get miss.

func (*ParserPool) Get

func (p *ParserPool) Get() (*Parser, error)

Get returns a reusable parser or allocates a new one on a pool miss.

Example
pool, err := NewParserPool()
if err != nil {
	panic(err)
}

parser, err := pool.Get()
if err != nil {
	panic(err)
}

doc, err := parser.Parse([]byte(`{"status":"ok"}`))
if err != nil {
	panic(err)
}

object, err := doc.Root().AsObject()
if err != nil {
	panic(err)
}

status, err := object.GetStringField("status")
if err != nil {
	panic(err)
}

if err := doc.Close(); err != nil {
	panic(err)
}
if err := pool.Put(parser); err != nil {
	panic(err)
}

fmt.Println(status)
Output:
ok

func (*ParserPool) Put

func (p *ParserPool) Put(parser *Parser) error

Put returns a parser to the pool and rejects nil, closed, still-busy, or differently configured parsers instead of silently repairing misuse. The parser's mutex is held across the pool insert so a racing Close cannot stash a just-closed parser.

Directories

Path Synopsis
cmd
pure-simdjson-bootstrap command
Command pure-simdjson-bootstrap is a thin CLI wrapper around internal/bootstrap.
Command pure-simdjson-bootstrap is a thin CLI wrapper around internal/bootstrap.
internal
bootstrap
Package bootstrap — BootstrapSync orchestrator and public option surface.
Package bootstrap — BootstrapSync orchestrator and public option surface.
ffi

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL