README
¶
YAML Events Binary Protocol
Go module for transporting YAML parser events across a C ABI. It is compiled into consumers and does not produce a shared library.
The root package provides a builder and a validating visitor decoder.
The glojure package encodes event sequences and decodes directly to Glojure
event vectors.
The codec itself has no Glojure dependency.
The first release targets module version v0.1.0 and wire format version 1.
Using the library
Import github.com/yamlstar/yaml-events-binary-protocol as events.
Use a zero-value events.Builder, call Add for each event, then Bytes to
obtain an owned packet.
Use events.Decode with a visitor to consume a packet.
See the executable Go example.
For Glojure, import the module's /glojure package and call Encode or
Decode to convert event sequences directly to owned vectors.
Encoding accepts keyword maps with string event names and typed fields;
it does not coerce values or silently omit unknown fields.
The C producer example compiles as C99 or C++11 and generates the same bytes as the Go golden test. It demonstrates the wire encoding, not a complete YAML document or plugin.
Run make test to check both Go packages and C/C++ interoperability.
Makes installs Go and the Zig C/C++ toolchain under .cache.
Run make test-go for Go-only tests or make fuzz for bounded fuzzing.
The integration benchmark lives in the YAMLStar checkout that consumes this
module and the json-comments plugin.
Compatibility and ownership
The module follows Go semantic versioning, with a v prefix on release tags.
The Go API may evolve during v0.x releases.
The wire version has its own compatibility contract: version 1 event numbers,
flags, and byte layout will not be repurposed in later module releases.
An incompatible encoding requires a new wire version.
Decoders reject unsupported wire versions rather than interpreting them as v1.
The codec validates the transport representation, not YAML event ordering or which fields are semantically valid for a particular event kind. An empty event sequence is a valid packet. String fields whose presence bit is clear are ignored by the builder. All bytes of the string pool must be valid UTF-8, including unreferenced bytes.
Builder.Add copies present strings and leaves the builder unchanged on error.
Builder.Bytes returns independent storage and does not reset the builder.
Use a separate builder for each concurrent producer.
Decode copies string storage; returned strings survive release or mutation
of the input packet.
Do not modify input while decoding it.
Visitor errors stop decoding immediately; discard partial results whenever
Decode returns an error.
The Glojure decoder returns no partial vector on error.
Wire format, version 1
All integers are unsigned 32-bit little-endian words. The packet is at most 4 GiB minus one byte and consists of:
- Four bytes
YEBP, followed by version (1), event count, and word count. - The event word array.
- A UTF-8 string pool occupying the remainder of the packet.
Each event begins with one word. Bits 0 through 7 contain its kind:
| Number | Event |
|---|---|
| 1 | stream_start |
| 2 | stream_end |
| 3 | document_start |
| 4 | document_end |
| 5 | mapping_start |
| 6 | mapping_end |
| 7 | sequence_start |
| 8 | sequence_end |
| 9 | scalar |
| 10 | alias |
Bits 8 through 13 indicate the presence of value, style, anchor, tag,
name, and version, respectively.
For each present field, in that order, two additional words encode its byte
offset into the string pool and its byte length.
Empty strings and embedded NUL characters are permitted.
References must be valid UTF-8 substrings.
Bit 14 means flow is present; bit 15 gives its boolean value.
Bit 16 means explicit is present; bit 17 gives its boolean value.
A value bit without its presence bit is invalid.
All other bits are reserved and must be zero.
All event words must be consumed; pool references may overlap or be reused.
The format preserves field presence and original string values, including tags and styles, without normalization. Unknown event kinds, fields, versions, or flags are rejected. This first format uses one string pool; a future format can add buffer tables if rapidyaml measurements justify them.
Shared-library extension
The optional yamlstar_plugin_v1_parse_binary symbol has the same signature
and status codes as the existing EDN parse entry point.
Status 0 returns a YEBP packet; statuses 1 and 2 return existing EDN errors.
The caller releases every returned allocation with
yamlstar_plugin_v1_free, including error responses.
Input buffers belong to the caller and must not be retained.
Output buffers belong to the plugin until that release call.
Each call must have independent output storage for concurrent use.
The decoder copies the string pool into Go-owned memory before the host
releases the foreign allocation.
No Go object or pointer crosses the shared-library boundary.
The protocol does not require Go; C and C++ producers can use the header and
golden packet in testdata/scalar.hex.
Run make release v=0.1.0 to publish a release from committed source on main.
Use d=1 to preview it or a=1 to release another branch.
See Releasing.md for the release sequence and consumer checks.
Documentation
¶
Overview ¶
Package events implements YAML event binary protocol version 1.
Index ¶
Examples ¶
Constants ¶
const ( Value = iota Style Anchor Tag Name VersionDirective StringFields )
String fields occur in this order when their presence bit is set.
const ( HasFlow uint32 = 1 << 14 Flow uint32 = 1 << 15 HasExplicit uint32 = 1 << 16 Explicit uint32 = 1 << 17 )
Boolean flags distinguish absent fields from present false values.
const HeaderSize = 16
HeaderSize is the packet header size in bytes.
const Version = 1
Version is the wire format version, independent of the Go module version.
Variables ¶
var Names = [...]string{"", "stream_start", "stream_end",
"document_start", "document_end", "mapping_start", "mapping_end",
"sequence_start", "sequence_end", "scalar", "alias"}
Names maps a valid Kind to its YAMLStar event name. Treat it as read-only.
Functions ¶
func Decode ¶
Decode validates the packet and visits events without an intermediate event slice. A visitor must discard partial results if Decode fails. The visitor must be non-nil; its first error stops decoding and is returned. Strings remain valid after packet is released or modified. Concurrent calls are safe when callers do not modify their input during decoding.
Types ¶
type Builder ¶
type Builder struct {
// contains filtered or unexported fields
}
Builder packs events without retaining their Go objects. Its zero value is ready to use. A Builder is not safe for concurrent use.
Example ¶
package main
import (
"fmt"
events "github.com/yamlstar/yaml-events-binary-protocol"
)
func main() {
var builder events.Builder
for _, event := range []events.Event{
{Kind: events.StreamStart},
{Kind: events.DocumentStart},
{Kind: events.Scalar, Flags: events.Present(events.Value),
Strings: [events.StringFields]string{"hello"}},
{Kind: events.DocumentEnd},
{Kind: events.StreamEnd},
} {
if err := builder.Add(event); err != nil {
panic(err)
}
}
packet, err := builder.Bytes()
if err != nil {
panic(err)
}
err = events.Decode(packet, func(event events.Event) error {
if event.Kind == events.Scalar {
fmt.Println(event.Strings[events.Value])
}
return nil
})
if err != nil {
panic(err)
}
}
Output: hello
Source Files
Directories