Documentation
¶
Overview ¶
Package jsonfast provides a zero-allocation JSON builder for fixed schemas.
Designed for ultra-high-throughput pipelines (100 Gbps+) where every byte and every nanosecond matters. All methods operate on a reusable byte buffer; steady-state operation requires zero heap allocations.
Features:
- Zero-alloc Builder with Acquire/Release pool
- Pre-computed FieldKey for static field names (eliminates per-call quoting)
- NDJSON BatchWriter for batching records
- JSON scanner: IterateFields, FindField, FlattenObject (SWAR-accelerated)
- Flatten nested maps to dot-notation keys
- Word-at-a-time SWAR string scanning via unsafe for maximum throughput
Usage ¶
b := jsonfast.Acquire()
b.BeginObject()
b.AddStringField("message", "hello world")
b.AddIntField("severity", 3)
b.AddTimeRFC3339Field("timestamp", time.Now())
b.EndObject()
data := b.Bytes() // no allocation
jsonfast.Release(b) // return to pool
Pre-computed Field Keys ¶
For static field names known at init time, use FieldKey to eliminate the per-call quoting overhead:
var keyMessage = jsonfast.NewFieldKey("message")
b.AddStringFieldKey(keyMessage, "hello world")
Index ¶
- func EscapeString(s string) string
- func FindField(data []byte, key string) ([]byte, bool)
- func FlattenMap(m map[string]map[string]string, dst map[string]string) map[string]string
- func FlattenObject(b *Builder, data []byte) bool
- func IsLikelyJSON(s string) bool
- func IterateFields(data []byte, fn func(key, value []byte) bool) bool
- func IterateFieldsString(s string, fn func(key, value []byte) bool) bool
- func Release(b *Builder)
- func ReleaseBatchWriter(bw *BatchWriter)
- func SkipBracedAt(data []byte, i int, opener, closer byte) (int, bool)
- func SkipStringAt(data []byte, i int) (int, bool)
- func SkipValueAt(data []byte, i int) (int, bool)
- func SkipWS(data []byte, i int) int
- type BatchWriter
- type Builder
- func (b *Builder) AddBoolField(name string, v bool)
- func (b *Builder) AddBoolFieldKey(k FieldKey, v bool)
- func (b *Builder) AddFlattenedMapField(m map[string]map[string]string)
- func (b *Builder) AddFloat64Field(name string, v float64)
- func (b *Builder) AddFloat64FieldKey(k FieldKey, v float64)
- func (b *Builder) AddInt64Field(name string, v int64)
- func (b *Builder) AddInt64FieldKey(k FieldKey, v int64)
- func (b *Builder) AddIntField(name string, v int)
- func (b *Builder) AddIntFieldKey(k FieldKey, v int)
- func (b *Builder) AddNestedStringMapField(name string, m map[string]map[string]string)
- func (b *Builder) AddNullField(name string)
- func (b *Builder) AddNullFieldKey(k FieldKey)
- func (b *Builder) AddRawBytesField(name, value []byte)
- func (b *Builder) AddRawJSONField(name string, rawJSON []byte)
- func (b *Builder) AddRawJSONFieldKey(k FieldKey, rawJSON []byte)
- func (b *Builder) AddRawJSONFieldKeyString(k FieldKey, rawJSON string)
- func (b *Builder) AddRawJSONFieldString(name, rawJSON string)
- func (b *Builder) AddStringField(name, value string)
- func (b *Builder) AddStringFieldKey(k FieldKey, value string)
- func (b *Builder) AddStringMapObject(m map[string]string, rawJSONKey string)
- func (b *Builder) AddTimeRFC3339Field(name string, t time.Time)
- func (b *Builder) AddTimeRFC3339FieldKey(k FieldKey, t time.Time)
- func (b *Builder) AddUint8Field(name string, v uint8)
- func (b *Builder) AddUint16Field(name string, v uint16)
- func (b *Builder) AddUint32Field(name string, v uint32)
- func (b *Builder) AddUint64Field(name string, v uint64)
- func (b *Builder) AddUint64FieldKey(k FieldKey, v uint64)
- func (b *Builder) AppendEscapedString(s string)
- func (b *Builder) AppendRaw(p []byte)
- func (b *Builder) AppendRawString(s string)
- func (b *Builder) BeginObject()
- func (b *Builder) Bytes() []byte
- func (b *Builder) EndObject()
- func (b *Builder) Grow(n int)
- func (b *Builder) Len() int
- func (b *Builder) Reset()
- type FieldKey
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EscapeString ¶
EscapeString returns s with JSON special characters escaped per RFC 8259. If no escaping is needed (pure safe ASCII), it returns s unchanged (zero allocation). Invalid UTF-8 bytes are replaced with U+FFFD, consistent with Builder.escapeString.
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
escaped := jsonfast.EscapeString(`She said "hello" and left`)
fmt.Println(escaped)
}
Output: She said \"hello\" and left
func FindField ¶
FindField finds a top-level field by key name in a JSON object. Returns the raw value bytes and whether the field was found.
func FlattenMap ¶
FlattenMap flattens a nested map[string]map[string]string into a flat map[string]string using dot-notation keys: outer.inner → value. Zero allocation when the result map is provided and has sufficient capacity.
Example:
nested := map[string]map[string]string{
"sd@123": {"key1": "val1", "key2": "val2"},
}
flat := jsonfast.FlattenMap(nested, nil)
// flat = {"sd@123.key1": "val1", "sd@123.key2": "val2"}
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
m := map[string]map[string]string{
"host": {"name": "fw01", "ip": "10.0.0.1"},
}
flat := jsonfast.FlattenMap(m, nil)
fmt.Println(flat["host.name"])
fmt.Println(flat["host.ip"])
}
Output: fw01 10.0.0.1
func FlattenObject ¶
FlattenObject recursively flattens a nested JSON object into the Builder's current object. Nested objects are recursed up to 64 levels deep; leaf values are emitted as top-level fields. Non-object input is silently skipped. Returns false if the JSON is malformed or nesting exceeds the depth limit.
func IsLikelyJSON ¶
IsLikelyJSON reports whether s looks like a JSON object or array. WARNING: checks only the first and last byte — does NOT validate content. SAFETY: the caller MUST ensure upstream validation (e.g. from a trusted parser or json.Valid on ingress) before relying on this to emit raw JSON. Designed for hot-path use where []byte(s) allocation from json.Valid is unacceptable.
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
fmt.Println(jsonfast.IsLikelyJSON(`{"key":"value"}`))
fmt.Println(jsonfast.IsLikelyJSON(`not json`))
}
Output: true false
func IterateFields ¶
IterateFields calls fn for each top-level key-value pair in a JSON object. key includes quotes, value is the raw JSON bytes. Returns false if the JSON is malformed or fn returns false.
func IterateFieldsString ¶
IterateFieldsString is like IterateFields but accepts a string, avoiding the []byte(string) allocation. The callback slices share the string's backing memory and must not be mutated.
func Release ¶
func Release(b *Builder)
Release returns the Builder to the pool for reuse. The builder must not be used after calling Release. Buffers larger than 256 KB are discarded to bound memory retention.
func ReleaseBatchWriter ¶
func ReleaseBatchWriter(bw *BatchWriter)
ReleaseBatchWriter returns the BatchWriter to the pool for reuse. The writer must not be used after calling ReleaseBatchWriter.
func SkipBracedAt ¶
SkipBracedAt skips a balanced delimiter pair starting at data[i]. Delegates string scanning to SkipStringAt for SWAR throughput. For non-string, non-bracket content, uses SWAR to skip 8 bytes at a time.
func SkipStringAt ¶
SkipStringAt finds the end of a JSON string starting at data[i]. data[i] must be '"'. Returns the index past the closing quote. Rejects raw control chars (< 0x20) per RFC 8259. Uses swarSpecialSkip (see swar.go) for the detection formula.
func SkipValueAt ¶
SkipValueAt skips a complete JSON value starting at data[i]. Returns the index past the value and true, or (i, false) on error.
Types ¶
type BatchWriter ¶
type BatchWriter struct {
// contains filtered or unexported fields
}
BatchWriter accumulates newline-delimited JSON (NDJSON) records into a reusable buffer. Each record is a complete JSON object followed by '\n'.
Designed for high-throughput pipelines that batch multiple log records before sending to a message broker (Redis Streams, Kafka, NATS).
Usage:
bw := jsonfast.NewBatchWriter(4096)
b := jsonfast.Acquire()
b.BeginObject()
b.AddStringField("msg", "hello")
b.EndObject()
bw.Append(b.Bytes())
jsonfast.Release(b)
payload := bw.Bytes() // "{"msg":"hello"}\n"
bw.Reset()
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
bw := jsonfast.NewBatchWriter(256)
bw.Append([]byte(`{"line":1}`))
bw.Append([]byte(`{"line":2}`))
fmt.Print(string(bw.Bytes()))
}
Output: {"line":1} {"line":2}
func AcquireBatchWriter ¶
func AcquireBatchWriter() *BatchWriter
AcquireBatchWriter returns a BatchWriter from the pool, ready for use. Call ReleaseBatchWriter when done to return it for reuse.
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
bw := jsonfast.AcquireBatchWriter()
defer jsonfast.ReleaseBatchWriter(bw)
bw.Append([]byte(`{"pooled":true}`))
fmt.Print(string(bw.Bytes()))
}
Output: {"pooled":true}
func NewBatchWriter ¶
func NewBatchWriter(capacity int) *BatchWriter
NewBatchWriter creates a new BatchWriter with the given initial capacity.
func (*BatchWriter) Append ¶
func (w *BatchWriter) Append(record []byte)
Append adds a complete JSON record followed by a newline to the batch.
func (*BatchWriter) AppendString ¶
func (w *BatchWriter) AppendString(record string)
AppendString adds a complete JSON record (as string) followed by a newline.
func (*BatchWriter) Bytes ¶
func (w *BatchWriter) Bytes() []byte
Bytes returns the accumulated NDJSON payload.
func (*BatchWriter) Count ¶
func (w *BatchWriter) Count() int
Count returns the number of records in the batch.
func (*BatchWriter) Grow ¶
func (w *BatchWriter) Grow(n int)
Grow ensures the buffer has at least n bytes of spare capacity. Uses slices.Grow which leverages the runtime's optimized growslice.
func (*BatchWriter) Len ¶
func (w *BatchWriter) Len() int
Len returns the current byte length of the batch.
func (*BatchWriter) Reset ¶
func (w *BatchWriter) Reset()
Reset clears the batch for reuse without releasing memory.
type Builder ¶
type Builder struct {
// contains filtered or unexported fields
}
Builder is a minimal JSON builder that operates on a reusable byte slice. It avoids allocations by appending directly into the buffer. Not a fully general-purpose JSON writer; tailored for known field sets.
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
b := jsonfast.New(256)
b.BeginObject()
b.AddStringField("message", "Hello, World!")
b.AddIntField("severity", 5)
b.AddBoolField("active", true)
b.EndObject()
fmt.Println(string(b.Bytes()))
}
Output: {"message":"Hello, World!","severity":5,"active":true}
Example (WithTimestamp) ¶
package main
import (
"fmt"
"time"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
b := jsonfast.New(256)
ts := time.Date(2024, 3, 15, 14, 30, 45, 0, time.UTC)
b.BeginObject()
b.AddStringField("msg", "test")
b.AddTimeRFC3339Field("ts", ts)
b.EndObject()
fmt.Println(string(b.Bytes()))
}
Output: {"msg":"test","ts":"2024-03-15T14:30:45Z"}
func Acquire ¶
func Acquire() *Builder
Acquire returns a Builder from the pool, ready for use. Call Release when done to return it for reuse.
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
b := jsonfast.Acquire()
defer jsonfast.Release(b)
b.BeginObject()
b.AddStringField("source", "pool")
b.EndObject()
fmt.Println(string(b.Bytes()))
}
Output: {"source":"pool"}
func (*Builder) AddBoolField ¶
AddBoolField adds a "name":true/false field.
func (*Builder) AddBoolFieldKey ¶
AddBoolFieldKey adds a "name":true/false field using a pre-computed key.
func (*Builder) AddFlattenedMapField ¶
AddFlattenedMapField writes nested map data as flattened "outer.inner":"value" fields directly into the JSON object being built, without materializing an intermediate flat map. Keys are sorted for deterministic output.
Zero allocation for maps with ≤16 total entries (stack-allocated sort buffer).
func (*Builder) AddFloat64Field ¶
AddFloat64Field adds a "name":float64 field. Integer values are written without decimal point. Fractional values use strconv.AppendFloat with full precision.
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
b := jsonfast.New(64)
b.BeginObject()
b.AddFloat64Field("pi", 3.14159)
b.EndObject()
fmt.Println(string(b.Bytes()))
}
Output: {"pi":3.14159}
func (*Builder) AddFloat64FieldKey ¶
AddFloat64FieldKey adds a "name":float64 field using a pre-computed key.
func (*Builder) AddInt64Field ¶
AddInt64Field adds a "name":int64 field.
func (*Builder) AddInt64FieldKey ¶
AddInt64FieldKey adds a "name":int64 field using a pre-computed key.
func (*Builder) AddIntField ¶
AddIntField adds a "name":int field.
func (*Builder) AddIntFieldKey ¶
AddIntFieldKey adds a "name":int field using a pre-computed key.
func (*Builder) AddNestedStringMapField ¶
AddNestedStringMapField adds a "name":{"key1":{"k":"v"},"key2":{...}} field. Specifically handles map[string]map[string]string as found in RFC5424 structured data. Keys are sorted to produce deterministic JSON output for caching and deduplication.
func (*Builder) AddNullField ¶
AddNullField adds a "name":null field.
func (*Builder) AddNullFieldKey ¶
AddNullFieldKey adds a "name":null field using a pre-computed key.
func (*Builder) AddRawBytesField ¶
AddRawBytesField adds a "name":value field where name is raw bytes (without quotes) and value is raw JSON bytes. Zero allocation.
func (*Builder) AddRawJSONField ¶
AddRawJSONField adds a "name":<raw json> field without escaping. The value must be valid JSON.
func (*Builder) AddRawJSONFieldKey ¶
AddRawJSONFieldKey adds a "name":<raw json> field using a pre-computed key.
func (*Builder) AddRawJSONFieldKeyString ¶
AddRawJSONFieldKeyString adds a "name":<raw json string> field using a pre-computed key.
func (*Builder) AddRawJSONFieldString ¶
AddRawJSONFieldString adds a "name":<raw json> field where the raw JSON source is a string. Avoids the []byte(s) allocation of AddRawJSONField.
func (*Builder) AddStringField ¶
AddStringField adds a "name":"value" string field with escaping.
func (*Builder) AddStringFieldKey ¶
AddStringFieldKey adds a "name":"value" field using a pre-computed key. Eliminates the quoting overhead of fieldKey on every call.
func (*Builder) AddStringMapObject ¶
AddStringMapObject writes a map[string]string as a JSON object {...} at the current position. Keys are sorted for deterministic output. If rawJSONKey is non-empty, values for that key are embedded as raw JSON when they look like JSON (via IsLikelyJSON). This method does NOT add a field name — it writes just the object.
func (*Builder) AddTimeRFC3339Field ¶
AddTimeRFC3339Field adds a "name":"RFC3339" field without using time.Format. The output is ALWAYS in UTC (with 'Z' suffix) regardless of the input timezone, because computation uses t.Unix() which is timezone-independent. Uses civil-date arithmetic to avoid double absSec() computation that results from calling both t.Date() and t.Clock().
func (*Builder) AddTimeRFC3339FieldKey ¶
AddTimeRFC3339FieldKey adds a "name":"RFC3339" field using a pre-computed key.
func (*Builder) AddUint8Field ¶
AddUint8Field adds a "name":uint8 field without heap allocation.
func (*Builder) AddUint16Field ¶
AddUint16Field adds a "name":uint16 field without heap allocation.
func (*Builder) AddUint32Field ¶
AddUint32Field adds a "name":uint32 field without heap allocation.
func (*Builder) AddUint64Field ¶
AddUint64Field adds a "name":uint64 field without heap allocation.
func (*Builder) AddUint64FieldKey ¶
AddUint64FieldKey adds a "name":uint64 field using a pre-computed key.
func (*Builder) AppendEscapedString ¶
AppendEscapedString appends s with JSON special characters escaped directly into the builder's buffer. Unlike EscapeString, this avoids creating a temporary Builder and string allocation — zero-alloc on the hot path.
func (*Builder) AppendRaw ¶
AppendRaw appends raw bytes to the buffer without any escaping or framing.
func (*Builder) AppendRawString ¶
AppendRawString appends a raw string to the buffer without any escaping or framing.
func (*Builder) Grow ¶
Grow ensures the buffer has at least n bytes of spare capacity. Uses slices.Grow which leverages the runtime's optimized growslice, rounding up to size-classes and benefiting from PGO.
Example ¶
package main
import (
"fmt"
"github.com/ubyte-source/go-jsonfast"
)
func main() {
b := jsonfast.New(16)
b.Grow(1024)
b.BeginObject()
b.AddStringField("k", "v")
b.EndObject()
fmt.Println(string(b.Bytes()))
}
Output: {"k":"v"}
type FieldKey ¶
type FieldKey struct {
// contains filtered or unexported fields
}
FieldKey is a pre-computed JSON field key prefix. Stores `,"name":` as a single string; the non-comma variant is a substring. Total size: 16 bytes (one string header), same as passing a string argument.
Usage:
var keyMessage = jsonfast.NewFieldKey("message")
b.AddStringFieldKey(keyMessage, "hello")
func NewFieldKey ¶
NewFieldKey creates a pre-computed field key for the given name. The name must be a safe ASCII string (no escaping needed). Call this at init time, not on the hot path.