Help us shape the future of Go! Take the 2026 Go Developer Survey

omnibuf

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 25, 2026 License: MIT Imports: 19 Imported by: 0

README

OmniBuf

CI Go Reference Website

Website (omnibuf.com) · How It Works · Benchmarks · Wire Format Specification

OmniBuf is a binary wire format for Protocol Buffers (.proto schemas) built around four ideas:

  1. Keep .proto schemas and Go structs: Existing .proto files, protoc-gen-go structs, and dynamicpb messages work without changes.
  2. Fix Protobuf's CPU bottlenecks: Prefix-Varints (TZCNT) decode integers in a single load instead of a byte-by-byte continuation loop, and 2-byte length backpatching encodes messages in a single forward pass instead of walking the struct tree twice (~6x faster encode and ~8x faster zero-copy decode than google.golang.org/protobuf).
  3. Inspect packets without a schema (DecodeBlind): Explicit 4-bit wire types (0..15) replace Protobuf's overloaded Wire Type 2, and optional Field #0 metadata attaches field names on the wire when needed.
  4. Stay a flat [Tag][Value] stream: No vtables or pointer graphs. Containers carry outer byte lengths for single-step skipping (LocateField), and concatenating two packets (append(msgA, msgB...)) merges them just like Protobuf.

Language Support: OmniBuf is currently a Go-only implementation (github.com/sudorandom/omnibuf). Because the wire format is a flat [Tag][Value] stream with 4 integer widths and 16 wire types (no vtables or relative pointer graphs), it is straightforward to port. If there is interest, support for more languages will be added!

Go Library Usage

OmniBuf works directly with plain Go structs using omnibuf:"..." struct tags as well as existing protoc-gen-go generated .pb.go structs (proto.Message / protoreflect.Message):

1. Plain Go Structs (omnibuf Struct Tags)

Tag exported struct fields with a field number (1..65535) and optional key-value settings (name=<name>, wire=<type>, omitempty, or - to ignore):

package main

import (
	"fmt"

	"github.com/sudorandom/omnibuf"
)

type PeerEvent struct {
	ASN       uint32            `omnibuf:"1,name=asn"`
	RouterID  string            `omnibuf:"2,name=router_id"`
	Active    bool              `omnibuf:"3,name=active"`
	LatencyUS uint32            `omnibuf:"4,name=latency_us,wire=fixed32,omitempty"`
	Tags      []string          `omnibuf:"5,name=tags,omitempty"`
	Counters  map[string]uint64 `omnibuf:"6,name=counters,omitempty"`
	Internal  string            `omnibuf:"-"`
}

func main() {
	in := &PeerEvent{
		ASN:       64512,
		RouterID:  "edge-rtr-fra-01",
		Active:    true,
		LatencyUS: 850,
		Tags:      []string{"tier1", "ebgp"},
	}

	// Marshal (or use MarshalOptions{SelfDescribing: true} to embed Field #0 names)
	wire, err := omnibuf.MarshalOptions{SelfDescribing: true}.Marshal(in)
	if err != nil {
		panic(err)
	}

	var out PeerEvent
	if err := omnibuf.Unmarshal(wire, &out); err != nil {
		panic(err)
	}

	// Inspect without the Go struct definition (uses Field #0 names when present)
	blind, err := omnibuf.DecodeBlind(wire)
	if err != nil {
		panic(err)
	}
	fmt.Println(blind)
}
Struct Tag Reference (omnibuf:"...")
Tag Option Example Description
<fieldNum> omnibuf:"1" Field number (1..65535). Required for serialized fields.
name=<name> omnibuf:"2,name=router_id" Custom field name emitted in Field #0 (SchemaMeta) when SelfDescribing: true is enabled (defaults to the Go field name).
wire=<type> omnibuf:"4,wire=fixed32" Overrides the inferred wire encoding (uint, sint, fixed32, fixed64, string, bytes).
omitempty omnibuf:"5,omitempty" Skips encoding the field when it holds its Go zero value or an empty slice/map.
- omnibuf:"-" Ignores the field during encoding and decoding (even if a protobuf tag is present).
2. Existing .pb.go Structs (protoc-gen-go)
package main

import (
	"fmt"

	"github.com/sudorandom/omnibuf"
	"github.com/sudorandom/omnibuf/pb"
)

func main() {
	msg := &pb.PeerUpdate{
		Asn:      64512,
		RouterId: "edge-rtr-fra-01",
		Active:   true,
	}

	// Marshal any .pb.go struct or protoreflect.Message into OmniBuf wire bytes
	wire, err := omnibuf.Marshal(msg)
	if err != nil {
		panic(err)
	}

	// Unmarshal back into a struct
	var out pb.PeerUpdate
	if err := omnibuf.Unmarshal(wire, &out); err != nil {
		panic(err)
	}

	// Or inspect values without a schema (DecodeBlind)
	blind, err := omnibuf.DecodeBlind(wire)
	if err != nil {
		panic(err)
	}
	fmt.Println(blind)
}

CLI Usage (omnibuf)

Build or install the CLI tool:

go install github.com/sudorandom/omnibuf/cmd/omnibuf@latest
# or locally via just:
just build-cli

# Blind decode any OmniBuf packet to JSON without a schema
omnibuf decode testdata/golden/peer_update.field0.obuf

# Decode using a FileDescriptorSet (.json, .pb, or .pb.gz)
omnibuf decode -s testdata/golden/peer.fds.pb.gz -t PeerUpdate testdata/golden/peer_update.shared.obuf

# Encode JSON to OmniBuf wire bytes using a FileDescriptorSet (.json, .pb, or .pb.gz)
omnibuf encode -s testdata/golden/peer.fds.pb.gz -t PeerUpdate testdata/golden/peer_update.input.json -o out.obuf

# Encode with Field #0 inline field names and output hex
omnibuf encode -s testdata/golden/peer.fds.json -t PeerUpdate --field0 --hex testdata/golden/peer_update.input.json

Development Quick Start

# Install tool dependencies (buf, go, golangci-lint, just, node, pnpm) via mise
mise install

# Format and lint Go, Protobuf schemas (buf), and frontend
just fmt
just lint

# Run all unit, conformance, golden, and dynamicpb matrix tests
just test

# Run the fuzzer
just fuzz 10s

# Run the complete benchmark suite
just bench

# Start the Astro website dev server (or build into web/dist)
just web-dev
just web-build

Documentation

Index

Constants

View Source 
const (
	WireUint      uint8 = 0  // [Prefix-Varint] (unsigned int: uint32, uint64, enum)
	WireBoolFalse uint8 = 1  // 0 value bytes (false encoded directly in tag)
	WireBoolTrue  uint8 = 2  // 0 value bytes (true encoded directly in tag)
	WireString    uint8 = 3  // [ByteLen (Prefix-Varint)][UTF-8 Bytes]
	WireBytes     uint8 = 4  // [ByteLen (Prefix-Varint)][Raw Bytes]
	WireSubMsg    uint8 = 5  // [ByteLen (Prefix-Varint)][Sub-Packet Bytes]
	WireStringArr uint8 = 6  // [ByteLen (Prefix-Varint)][Count (Prefix-Varint)][Len + UTF-8 Bytes]...
	WireUintArr   uint8 = 7  // [ByteLen (Prefix-Varint)][Count (Prefix-Varint)][Prefix-Varint]...
	WireSubMsgArr uint8 = 8  // [ByteLen (Prefix-Varint)][ElemLen (Prefix-Varint) + Elem Bytes]...
	WireMap       uint8 = 9  // [ByteLen (Prefix-Varint)][MapType (1B: KeyWT<<4|ValWT)][Count (Prefix-Varint)][Key + Value]...
	WireFixed32   uint8 = 10 // [4 Bytes Little-Endian] (float32 / fixed32 / sfixed32)
	WireFixed64   uint8 = 11 // [8 Bytes Little-Endian] (float64 / fixed64 / sfixed64)
	WireSint      uint8 = 12 // [Prefix-Varint] (ZigZag signed int: int32, int64, sint32, sint64)
	WireSintArr   uint8 = 13 // [ByteLen (Prefix-Varint)][Count (Prefix-Varint)][ZigZag Prefix-Varint]...
	WireFixedArr  uint8 = 14 // [ByteLen (Prefix-Varint)][ElemType (1B: WireFixed32=10 or WireFixed64=11)][Fixed Elements...]
	WireExtended  uint8 = 15 // [ByteLen (Prefix-Varint)][ExtType (1B: 0..255)][Extended Payload...] (Reserved escape hatch)
)

Explicit 4-bit Wire Types (0..15) encoded in the low 4 bits of the field tag: Tag = (FieldNumber << 4) | WireType

View Source 
const (
	FieldPeerASN       uint16 = 1
	FieldPeerRouterID  uint16 = 2
	FieldPeerActive    uint16 = 3
	FieldPeerUptimeSec uint16 = 4
	FieldPeerRssiDbm   uint16 = 5
	FieldPeerCPULoad   uint16 = 6
	FieldPeerTags      uint16 = 7
	FieldPeerPrefixes  uint16 = 8
	FieldPeerLocation  uint16 = 9
	FieldPeerNeighbors uint16 = 10
	FieldPeerSequence  uint16 = 11

	FieldLocRegion     uint16 = 1
	FieldLocDatacenter uint16 = 2
	FieldLocCity       uint16 = 3
	FieldLocRack       uint16 = 4
	FieldLocLatitude   uint16 = 5
	FieldLocLongitude  uint16 = 6

	FieldNeighPeerASN     uint16 = 1
	FieldNeighRouter      uint16 = 2
	FieldNeighLatencyUS   uint16 = 3
	FieldNeighEstablished uint16 = 4
)

Field numbers in PeerUpdate, Location, and Neighbor.

View Source 
const MaxNestingDepth = 32

MaxNestingDepth is the maximum allowed sub-message / array nesting depth (prevents stack-overflow bombs).

Variables

This section is empty.

Functions

func AppendField0SchemaMeta 

func AppendField0SchemaMeta(dst []byte, meta *SchemaMeta) []byte

AppendField0SchemaMeta appends Field #0 (Tag = 0x0B, WireSubMsg = 5) encoding meta to dst without sorting map keys (fastest path).

func AppendField0SchemaMetaDeterministic 

func AppendField0SchemaMetaDeterministic(dst []byte, meta *SchemaMeta, deterministic bool) []byte

AppendField0SchemaMetaDeterministic appends Field #0 (Tag = 0x0B, WireSubMsg = 5) encoding meta to dst. When deterministic is true, map keys in Fields and Children are sorted in ascending order.

func AppendStringArrZeroCopy 

func AppendStringArrZeroCopy(dst []string, packet []byte, fieldNum uint16) []string

AppendStringArrZeroCopy appends elements from a WireStringArr field into dst using zero-copy strings.

func AppendUint32Arr 

func AppendUint32Arr(dst []uint32, packet []byte, fieldNum uint16) []uint32

AppendUint32Arr appends elements from a WireUintArr field into dst.

func AppendVarint 

func AppendVarint(dst []byte, v uint64) []byte

AppendVarint appends a branchless Prefix-Varint (TZCNT) to dst and returns the extended slice. Fast-pathed for 1-byte values (0..127) so the Go compiler inlines it at every call site.

func BackpatchContainerLen 

func BackpatchContainerLen(dst []byte, lenPos int) []byte

BackpatchContainerLen fills in the container byte length at dst[lenPos:lenPos+2] for the common fast path (subLen <= 16383). If subLen > 16383, it expands the reserved 2-byte slot to a 4-byte Prefix-Varint (up to 512 MB) or 9-byte Prefix-Varint by shifting the container body right in place.

func CountSubMsgArrElems 

func CountSubMsgArrElems(arrPayload []byte) (int, bool)

CountSubMsgArrElems counts and validates the length-prefixed elements inside a WireSubMsgArr payload.

func CountedArrLen 

func CountedArrLen(packet []byte, fieldNum uint16, expWT uint8) int

CountedArrLen returns the element count of a counted array container (WireStringArr, WireUintArr, WireSintArr).

func DecodeBlind 

func DecodeBlind(packet []byte) (map[string]any, error)

DecodeBlind decodes any OmniBuf packet without a compile-time schema. If Field #0 (Tag = 0x0B, WireSubMsg = 5, omnibuf.v1.SchemaMeta) is present, its field names (and any nested children schemas) are used to decode human-readable keys. Otherwise, synthetic field names ("field_1", "field_2", ...) are used.

func DecodePeerCapnpInto 

func DecodePeerCapnpInto(data []byte, out *PeerUpdate, zeroCopy bool) error

DecodePeerCapnpInto decodes a Cap'n Proto packet into out.

func DecodePeerFlatBuffersInto 

func DecodePeerFlatBuffersInto(buf []byte, out *PeerUpdate, zeroCopy bool)

DecodePeerFlatBuffersInto decodes a FlatBuffers PeerUpdate packet into out.

func DecodePeerOmniBufInto 

func DecodePeerOmniBufInto(packet []byte, out *PeerUpdate, zeroCopyStrings bool) error

DecodePeerOmniBufInto decodes an OmniBuf packet into out. If zeroCopyStrings is true, strings reference packet memory via unsafe.String (0 string allocs).

func DecodePeerProtoWireInto 

func DecodePeerProtoWireInto(b []byte, out *PeerUpdate, zeroCopy bool) error

DecodePeerProtoWireInto decodes a classic Protobuf packet into out using direct protowire.

func DecodeStr 

func DecodeStr(b []byte, zeroCopy bool) string

DecodeStr converts b into a Go string (zero-copy via unsafe.String if zeroCopy is true).

func DecodeVarint 

func DecodeVarint(b []byte) (uint64, int)

DecodeVarint decodes a Prefix-Varint from b using hardware TZCNT and returns (value, bytesRead). Fast-pathed for 1-byte values so it inlines at every call site.

func DecodeZigZag 

func DecodeZigZag(u uint64) int64

DecodeZigZag maps a ZigZag-encoded unsigned integer back to a signed int64.

func EncodePeerCapnp 

func EncodePeerCapnp(scratch []byte, p *PeerUpdate) ([]byte, error)

EncodePeerCapnp encodes PeerUpdate using capnproto.org/go/capnp/v3.

func EncodePeerFlatBuffers 

func EncodePeerFlatBuffers(b *flatbuffers.Builder, p *PeerUpdate) []byte

EncodePeerFlatBuffers builds a FlatBuffers packet for PeerUpdate using flatbuffers.Builder. Notice that FlatBuffers requires inside-out (bottom-up) serialization: all strings, nested tables, and offset vectors must be serialized BEFORE their parent table can be started.

func EncodePeerOmniBuf 

func EncodePeerOmniBuf(enc *Encoder, p *PeerUpdate, withField0 bool) []byte

EncodePeerOmniBuf encodes PeerUpdate into enc in shared-schema mode (or self-describing mode if withField0 is true).

func EncodePeerOmniBufDirect 

func EncodePeerOmniBufDirect(dst []byte, p *PeerUpdate) []byte

EncodePeerOmniBufDirect encodes PeerUpdate directly into dst []byte in a single forward pass (analogous to protowire.Append* operating on a slice in registers, without a two-pass Size() walk).

func EncodePeerProtoWire 

func EncodePeerProtoWire(dst []byte, p *PeerUpdate) []byte

EncodePeerProtoWire encodes PeerUpdate using classic Protobuf TLV + LEB128 (2-pass Size + Write).

func EncodeSintArr 

func EncodeSintArr[T intOrUint](dst []byte, items []T) []byte

EncodeSintArr appends a WireSintArr container body ([ByteLen][Count][ZigZag Varint]...) to dst.

func EncodeStringArr 

func EncodeStringArr(dst []byte, items []string) []byte

EncodeStringArr appends a WireStringArr container body ([ByteLen][Count][Len+Bytes]...) to dst.

func EncodeUintArr 

func EncodeUintArr[T intOrUint](dst []byte, items []T) []byte

EncodeUintArr appends a WireUintArr container body ([ByteLen][Count][Varint]...) to dst.

func EncodeZigZag 

func EncodeZigZag(n int64) uint64

EncodeZigZag maps signed integers to unsigned integers so small negative values stay compact.

func LocateField 

func LocateField(packet []byte, targetField uint16) (valSlice []byte, wireType uint8, ok bool)

LocateField locates targetField inside packet by hopping across field headers in O(1) time per field using TZCNT length prefixes (without parsing inside preceding strings, sub-messages, or arrays). To support raw message concatenation (append(msgA, msgB...)) with Protobuf "last write wins" semantics, it scans all field headers in the message and returns the last matching occurrence.

func LookupCapnpInPlaceNeighborRouter 

func LookupCapnpInPlaceNeighborRouter(data []byte, idx int, zeroCopy bool) (string, bool)

LookupCapnpInPlaceNeighborRouter traverses Cap'n Proto's composite list + struct pointer in-place with 0 allocations.

func LookupCapnpInPlaceNestedCity 

func LookupCapnpInPlaceNestedCity(data []byte, zeroCopy bool) (string, bool)

LookupCapnpInPlaceNestedCity traverses Cap'n Proto's 64-bit pointers directly in-place with 0 allocations.

func LookupCapnpNeighborRouter 

func LookupCapnpNeighborRouter(data []byte, idx int, zeroCopy bool) (string, bool)

LookupCapnpNeighborRouter reads peer.neighbors[idx].router from a Cap'n Proto message.

func LookupCapnpNestedCity 

func LookupCapnpNestedCity(data []byte, zeroCopy bool) (string, bool)

LookupCapnpNestedCity reads peer.location.city from a Cap'n Proto message.

func LookupFlatBuffersNeighborRouter 

func LookupFlatBuffersNeighborRouter(buf []byte, idx int, zeroCopy bool) (string, bool)

LookupFlatBuffersNeighborRouter reads peer.neighbors[idx].router via FlatBuffers vtable + vector indirection.

func LookupFlatBuffersNestedCity 

func LookupFlatBuffersNestedCity(buf []byte, zeroCopy bool) (string, bool)

LookupFlatBuffersNestedCity reads peer.location.city via FlatBuffers vtable indirection.

func LookupMapStringUint 

func LookupMapStringUint(packet []byte, fieldNum uint16, targetKey string) (uint64, bool)

LookupMapStringUint searches a WireMap field without allocating a Go map, supporting merged WireMap fields and "last write wins" across entries and concatenated packets.

func LookupMsgpackNestedCity 

func LookupMsgpackNestedCity(b []byte) (string, bool)

LookupMsgpackNestedCity uses msgpack.Decoder.Query or field skipping to read location.city.

func LookupProtoWireNeighborRouter 

func LookupProtoWireNeighborRouter(b []byte, targetIdx int, zeroCopy bool) (string, bool)

LookupProtoWireNeighborRouter scans a Protobuf packet sequentially to extract peer.neighbors[targetIdx].router.

func LookupProtoWireNestedCity 

func LookupProtoWireNestedCity(b []byte, zeroCopy bool) (string, bool)

LookupProtoWireNestedCity scans a Protobuf packet sequentially to extract peer.location.city without allocating structs (the fastest possible way to read a nested field in Protobuf).

func MapHeaderByte 

func MapHeaderByte(keyWT, valWT uint8) byte

MapHeaderByte encodes the 1-byte WireMap type header: (KeyWireType << 4) | ValWireType.

func Marshal 

func Marshal(v any) ([]byte, error)

Marshal encodes any struct pointer (including plain Go structs with `omnibuf:"..."` tags and existing *pb.PeerUpdate protoc-gen-go structs) into a newly allocated OmniBuf packet.

func MarshalAppend 

func MarshalAppend(dst []byte, v any) ([]byte, error)

MarshalAppend appends the OmniBuf encoding of struct pointer v to dst[:0] using the cached unsafe-offset table codec (0 reflection allocations on the hot path).

func MarshalDeterministic 

func MarshalDeterministic(v any) ([]byte, error)

MarshalDeterministic encodes any struct pointer or protoreflect.Message with deterministic WireMap key ordering.

func MarshalJSON 

func MarshalJSON(p *PeerUpdate) ([]byte, error)

MarshalJSON encodes PeerUpdate using standard encoding/json.

func MarshalMsgpack 

func MarshalMsgpack(p *PeerUpdate) ([]byte, error)

MarshalMsgpack encodes PeerUpdate using MessagePack.

func MarshalProtoReflect 

func MarshalProtoReflect(m protoreflect.Message, includeField0 bool) ([]byte, error)

MarshalProtoReflect encodes any protoreflect.Message (including *dynamicpb.Message) into an OmniBuf packet. When includeField0 is true, Field #0 (WireSubMsg, SchemaMeta) is emitted at byte 0 of the root message with the schema's field names and nested children schemas.

func MarshalProtoReflectAppend 

func MarshalProtoReflectAppend(dst []byte, m protoreflect.Message, includeField0 bool) ([]byte, error)

MarshalProtoReflectAppend appends the OmniBuf encoding of m to dst[:0].

func PutOverlong2BVarint 

func PutOverlong2BVarint(dst []byte, v uint16)

PutOverlong2BVarint encodes a length (0..16383) as an explicit 2-byte Prefix-Varint (TZCNT=1: uint16(v<<2)|2) for single-pass sub-message and container length backpatching.

func ReadArrNestedString 

func ReadArrNestedString(packet []byte, arrField uint16, index int, targetField uint16) (string, bool)

ReadArrNestedString reads a string inside the i-th repeated sub-message (e.g. peer.neighbors[2].router).

func ReadArrNestedStringZeroCopy 

func ReadArrNestedStringZeroCopy(packet []byte, arrField uint16, index int, targetField uint16) (string, bool)

ReadArrNestedStringZeroCopy reads a string inside the i-th repeated sub-message with 0 allocations.

func ReadBool 

func ReadBool(packet []byte, fieldNum uint16) (bool, bool)

ReadBool reads a boolean field directly from the 4-bit wire type (0 value bytes).

func ReadBytes 

func ReadBytes(packet []byte, fieldNum uint16) ([]byte, bool)

ReadBytes returns the raw byte slice of a WireBytes field.

func ReadElem 

func ReadElem(arrPayload []byte, index int) ([]byte, bool)

ReadElem slices the i-th sub-message element out of a WireSubMsgArr payload by hopping element length prefixes.

func ReadFloat32 

func ReadFloat32(packet []byte, fieldNum uint16) (float32, bool)

ReadFloat32 reads a float32 field with 0 heap allocations.

func ReadFloat32Arr 

func ReadFloat32Arr(packet []byte, fieldNum uint16) ([]float32, bool)

ReadFloat32Arr reads a WireFixedArr field (ElemType = WireFixed32, packed 4B LE float32/fixed32 values), concatenating across segments.

func ReadFloat64 

func ReadFloat64(packet []byte, fieldNum uint16) (float64, bool)

ReadFloat64 reads a float64 field with 0 heap allocations.

func ReadFloat64Arr 

func ReadFloat64Arr(packet []byte, fieldNum uint16) ([]float64, bool)

ReadFloat64Arr reads a WireFixedArr field (ElemType = WireFixed64, packed 8B LE float64/fixed64 values), concatenating across segments.

func ReadInt 

func ReadInt(packet []byte, fieldNum uint16) (int64, bool)

ReadInt reads a ZigZag-encoded signed int64 field with 0 heap allocations.

func ReadNestedString 

func ReadNestedString(packet []byte, subMsgField, targetField uint16) (string, bool)

ReadNestedString reads a nested string (e.g. peer.location.city), allocating a safe string copy.

func ReadNestedStringBytes 

func ReadNestedStringBytes(packet []byte, subMsgField, targetField uint16) ([]byte, bool)

ReadNestedStringBytes locates targetField inside subMsgField, supporting merged sub-messages across concatenated packets.

func ReadNestedStringZeroCopy 

func ReadNestedStringZeroCopy(packet []byte, subMsgField, targetField uint16) (string, bool)

ReadNestedStringZeroCopy reads a nested string (e.g. peer.location.city) with 0 allocations.

func ReadSintArr 

func ReadSintArr(packet []byte, fieldNum uint16) ([]int64, bool)

ReadSintArr reads a WireSintArr field (packed ZigZag signed integers), concatenating across segments.

func ReadString 

func ReadString(packet []byte, fieldNum uint16) (string, bool)

ReadString reads a string field, allocating a safe Go string copy.

func ReadStringArr 

func ReadStringArr(packet []byte, fieldNum uint16) ([]string, bool)

ReadStringArr reads a WireStringArr field, concatenating elements if the field appears multiple times.

func ReadStringBytes 

func ReadStringBytes(packet []byte, fieldNum uint16) ([]byte, bool)

ReadStringBytes returns the raw UTF-8 byte slice of a string field.

func ReadStringZeroCopy 

func ReadStringZeroCopy(packet []byte, fieldNum uint16) (string, bool)

ReadStringZeroCopy reads a string field with zero allocations using unsafe.String.

func ReadSubMsg 

func ReadSubMsg(packet []byte, fieldNum uint16) ([]byte, bool)

ReadSubMsg returns the inner packet slice of a nested WireSubMsg field.

func ReadSubMsgArrElem 

func ReadSubMsgArrElem(packet []byte, fieldNum uint16, index int) ([]byte, bool)

ReadSubMsgArrElem locates a WireSubMsgArr field in packet and slices its i-th element (supporting concatenated WireSubMsgArr fields across merged messages).

func ReadUint 

func ReadUint(packet []byte, fieldNum uint16) (uint64, bool)

ReadUint reads a uint64 field with 0 heap allocations.

func ReadUint32Arr 

func ReadUint32Arr(packet []byte, fieldNum uint16) ([]uint32, bool)

ReadUint32Arr reads a WireUintArr field, concatenating elements if the field appears multiple times.

func ReadVarintAt 

func ReadVarintAt(b []byte, pos int) (uint64, int)

ReadVarintAt decodes a Prefix-Varint from b[pos:] without constructing a sub-slice.

func SizeLocationProto 

func SizeLocationProto(loc *Location) int

SizeLocationProto computes the exact LEB128 Protobuf wire size of Location (Pass 1 of 2-pass Protobuf).

func SizeNeighborProto 

func SizeNeighborProto(nb *Neighbor) int

SizeNeighborProto computes the exact LEB128 Protobuf wire size of Neighbor (Pass 1 of 2-pass Protobuf).

func SkipValue 

func SkipValue(b []byte, pos int, wireType uint8) (int, error)

SkipValue advances pos past a single field payload of the given 4-bit wireType (0..15), performing full bounds validation in O(1) time without looping inside containers.

func SpliceFields 

func SpliceFields(dst, raw []byte, dirtyMask uint16, maxField uint16, appendDirty func(dst []byte, fnum uint16) []byte) []byte

SpliceFields splices unmodified contiguous byte ranges from raw and invokes appendDirty only for fields whose bit is set in dirtyMask, avoiding any decoding of untouched fields.

func SubMsgArrLen 

func SubMsgArrLen(packet []byte, fieldNum uint16) int

SubMsgArrLen returns the total number of elements in a WireSubMsgArr field across any concatenated segments.

func Unmarshal 

func Unmarshal(packet []byte, v any) error

Unmarshal decodes an OmniBuf packet into any struct pointer v (including *pb.PeerUpdate) using safe heap-allocated strings.

func UnmarshalJSON 

func UnmarshalJSON(b []byte, out *PeerUpdate) error

UnmarshalJSON decodes a JSON packet into out.

func UnmarshalMsgpack 

func UnmarshalMsgpack(b []byte, out *PeerUpdate) error

UnmarshalMsgpack decodes a MessagePack packet into out.

func UnmarshalProtoReflect 

func UnmarshalProtoReflect(data []byte, m protoreflect.Message) error

UnmarshalProtoReflect decodes an OmniBuf packet into any protoreflect.Message (including *dynamicpb.Message).

func UnmarshalZeroCopy 

func UnmarshalZeroCopy(packet []byte, v any) error

UnmarshalZeroCopy decodes an OmniBuf packet into any struct pointer v using zero-copy unsafe.String references into packet.

Types

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder encodes OmniBuf messages into a reusable contiguous byte buffer in a single forward pass.

func NewEncoder 

func NewEncoder(capacity int) *Encoder

NewEncoder creates an Encoder with the specified initial buffer capacity.

func (*Encoder) BeginSubMsg 

func (e *Encoder) BeginSubMsg(fieldNum uint16)

BeginSubMsg starts a nested sub-message (WireSubMsg) in a single pass by reserving a 2-byte Prefix-Varint length slot.

func (*Encoder) BeginSubMsgArr 

func (e *Encoder) BeginSubMsgArr(fieldNum uint16)

BeginSubMsgArr starts a repeated sub-message array (WireSubMsgArr) in a single pass.

func (*Encoder) BeginSubMsgElem 

func (e *Encoder) BeginSubMsgElem()

BeginSubMsgElem starts the next element inside an open WireSubMsgArr by reserving its 2-byte length prefix.

func (*Encoder) Bytes 

func (e *Encoder) Bytes() []byte

Bytes returns the encoded packet bytes.

func (*Encoder) EndSubMsg 

func (e *Encoder) EndSubMsg()

EndSubMsg finishes the current nested sub-message and backpatches its byte length.

func (*Encoder) EndSubMsgArr 

func (e *Encoder) EndSubMsgArr()

EndSubMsgArr backpatches the total byte length of the open WireSubMsgArr.

func (*Encoder) EndSubMsgElem 

func (e *Encoder) EndSubMsgElem()

EndSubMsgElem finishes the current element inside an open WireSubMsgArr and backpatches its length.

func (*Encoder) Finish 

func (e *Encoder) Finish() []byte

Finish returns the encoded message bytes.

func (*Encoder) Reset 

func (e *Encoder) Reset()

Reset clears the encoder buffer for zero-allocation reuse.

func (*Encoder) WriteBool 

func (e *Encoder) WriteBool(fieldNum uint16, v bool)

WriteBool writes a boolean field using 0 payload bytes (WireBoolTrue or WireBoolFalse).

func (*Encoder) WriteBytes 

func (e *Encoder) WriteBytes(fieldNum uint16, b []byte)

WriteBytes writes a raw byte slice field (WireBytes).

func (*Encoder) WriteField0Fields 

func (e *Encoder) WriteField0Fields(fields map[uint32]string)

WriteField0Fields writes Field #0 (Tag = 0x0B, WireSubMsg = 5) mapping FieldNumber -> FieldName.

func (*Encoder) WriteField0Schema 

func (e *Encoder) WriteField0Schema(names []string)

WriteField0Schema writes Field #0 (Tag = 0x0B, WireSubMsg = 5) mapping field numbers 1..len(names) to names[0..len(names)-1].

func (*Encoder) WriteField0SchemaMeta 

func (e *Encoder) WriteField0SchemaMeta(meta *SchemaMeta)

WriteField0SchemaMeta writes Field #0 (Tag = 0x0B, WireSubMsg = 5) containing meta.

func (*Encoder) WriteFixed32 

func (e *Encoder) WriteFixed32(fieldNum uint16, v uint32)

WriteFixed32 writes a 4-byte little-endian uint32 field (WireFixed32).

func (*Encoder) WriteFixed64 

func (e *Encoder) WriteFixed64(fieldNum uint16, v uint64)

WriteFixed64 writes an 8-byte little-endian uint64 field (WireFixed64).

func (*Encoder) WriteFloat32 

func (e *Encoder) WriteFloat32(fieldNum uint16, v float32)

WriteFloat32 writes a 4-byte IEEE-754 float32 field (WireFixed32).

func (*Encoder) WriteFloat32Arr 

func (e *Encoder) WriteFloat32Arr(fieldNum uint16, items []float32)

WriteFloat32Arr writes a packed float32 array (WireFixedArr with ElemType = WireFixed32).

func (*Encoder) WriteFloat64 

func (e *Encoder) WriteFloat64(fieldNum uint16, v float64)

WriteFloat64 writes an 8-byte IEEE-754 float64 field (WireFixed64).

func (*Encoder) WriteFloat64Arr 

func (e *Encoder) WriteFloat64Arr(fieldNum uint16, items []float64)

WriteFloat64Arr writes a packed float64 array (WireFixedArr with ElemType = WireFixed64).

func (*Encoder) WriteInt 

func (e *Encoder) WriteInt(fieldNum uint16, v int64)

WriteInt writes a ZigZag-encoded signed integer field (WireSint).

func (*Encoder) WriteMapStringUint 

func (e *Encoder) WriteMapStringUint(fieldNum uint16, m map[string]uint64)

WriteMapStringUint writes a map<string, uint64> (WireMap) in a single pass with zero allocations.

func (*Encoder) WriteMapStringUintDeterministic 

func (e *Encoder) WriteMapStringUintDeterministic(fieldNum uint16, m map[string]uint64)

WriteMapStringUintDeterministic writes a map<string, uint64> (WireMap) with keys sorted in lexicographical order.

func (*Encoder) WriteRaw 

func (e *Encoder) WriteRaw(raw []byte)

WriteRaw appends pre-encoded raw bytes (such as a pre-encoded Field #0 SchemaMeta block) to e.

func (*Encoder) WriteSintArr 

func (e *Encoder) WriteSintArr(fieldNum uint16, items []int64)

WriteSintArr writes a packed ZigZag-encoded signed integer array (WireSintArr).

func (*Encoder) WriteString 

func (e *Encoder) WriteString(fieldNum uint16, s string)

WriteString writes a UTF-8 string field (WireString).

func (*Encoder) WriteStringArr 

func (e *Encoder) WriteStringArr(fieldNum uint16, items []string)

WriteStringArr writes a repeated string array with an outer byte-length prefix (WireStringArr).

func (*Encoder) WriteUint 

func (e *Encoder) WriteUint(fieldNum uint16, v uint64)

WriteUint writes an unsigned integer field (WireUint).

func (*Encoder) WriteUint32Arr 

func (e *Encoder) WriteUint32Arr(fieldNum uint16, items []uint32)

WriteUint32Arr writes a packed uint32 array with a backpatched byte-skip length prefix (WireUintArr).

type Location 

type Location struct {
	Region     string  `omnibuf:"1,name=region" protobuf:"bytes,1,opt,name=region,proto3" json:"region" msgpack:"region"`
	Datacenter string  `omnibuf:"2,name=datacenter" protobuf:"bytes,2,opt,name=datacenter,proto3" json:"datacenter" msgpack:"datacenter"`
	City       string  `omnibuf:"3,name=city" protobuf:"bytes,3,opt,name=city,proto3" json:"city" msgpack:"city"`
	Rack       uint32  `omnibuf:"4,name=rack" protobuf:"varint,4,opt,name=rack,proto3" json:"rack" msgpack:"rack"`
	Latitude   float32 `omnibuf:"5,name=latitude" protobuf:"fixed32,5,opt,name=latitude,proto3" json:"latitude" msgpack:"latitude"`
	Longitude  float32 `omnibuf:"6,name=longitude" protobuf:"fixed32,6,opt,name=longitude,proto3" json:"longitude" msgpack:"longitude"`
}

type MarshalOptions 

type MarshalOptions struct {
	// Deterministic sorts WireMap keys in ascending order so identical messages
	// always produce byte-for-byte identical wire output (useful for golden files,
	// reproducible builds, and content-addressable hashing).
	Deterministic bool

	// SelfDescribing emits Field #0 (Tag = 0x0B, WireSubMsg = 5, omnibuf.v1.SchemaMeta)
	// at byte 0 of the root message with field names and nested children schemas.
	SelfDescribing bool
}

MarshalOptions configures OmniBuf wire encoding.

func (MarshalOptions) Marshal 

func (o MarshalOptions) Marshal(v any) ([]byte, error)

Marshal encodes v (any struct pointer or protoreflect.Message) using the configured options.

func (MarshalOptions) MarshalAppend 

func (o MarshalOptions) MarshalAppend(dst []byte, v any) ([]byte, error)

MarshalAppend appends the OmniBuf encoding of v to dst[:0] using the configured options.

type Neighbor 

type Neighbor struct {
	PeerASN     uint32 `omnibuf:"1,name=peer_asn" protobuf:"varint,1,opt,name=peer_asn,proto3" json:"peer_asn" msgpack:"peer_asn"`
	Router      string `omnibuf:"2,name=router" protobuf:"bytes,2,opt,name=router,proto3" json:"router" msgpack:"router"`
	LatencyUS   uint32 `omnibuf:"3,name=latency_us" protobuf:"varint,3,opt,name=latency_us,proto3" json:"latency_us" msgpack:"latency_us"`
	Established bool   `omnibuf:"4,name=established" protobuf:"varint,4,opt,name=established,proto3" json:"established" msgpack:"established"`
}

type PeerUpdate 

type PeerUpdate struct {
	ASN       uint32     `omnibuf:"1,name=asn" protobuf:"varint,1,opt,name=asn,proto3" json:"asn" msgpack:"asn"`
	RouterID  string     `omnibuf:"2,name=router_id" protobuf:"bytes,2,opt,name=router_id,proto3" json:"router_id" msgpack:"router_id"`
	Active    bool       `omnibuf:"3,name=active" protobuf:"varint,3,opt,name=active,proto3" json:"active" msgpack:"active"`
	UptimeSec uint64     `omnibuf:"4,name=uptime_sec" protobuf:"varint,4,opt,name=uptime_sec,proto3" json:"uptime_sec" msgpack:"uptime_sec"`
	RssiDbm   int32      `omnibuf:"5,name=rssi_dbm" protobuf:"zigzag32,5,opt,name=rssi_dbm,proto3" json:"rssi_dbm" msgpack:"rssi_dbm"`
	CPULoad   float64    `omnibuf:"6,name=cpu_load" protobuf:"fixed64,6,opt,name=cpu_load,proto3" json:"cpu_load" msgpack:"cpu_load"`
	Tags      []string   `omnibuf:"7,name=tags" protobuf:"bytes,7,rep,name=tags,proto3" json:"tags" msgpack:"tags"`
	Prefixes  []uint32   `omnibuf:"8,name=prefixes" protobuf:"varint,8,rep,packed,name=prefixes,proto3" json:"prefixes" msgpack:"prefixes"`
	Location  Location   `omnibuf:"9,name=location" protobuf:"bytes,9,opt,name=location,proto3" json:"location" msgpack:"location"`
	Neighbors []Neighbor `omnibuf:"10,name=neighbors" protobuf:"bytes,10,rep,name=neighbors,proto3" json:"neighbors" msgpack:"neighbors"`
	Sequence  uint64     `omnibuf:"11,name=sequence" protobuf:"varint,11,opt,name=sequence,proto3" json:"sequence" msgpack:"sequence"`
}

func SamplePeerUpdate 

func SamplePeerUpdate() PeerUpdate

SamplePeerUpdate returns a representative BGP/Telemetry PeerUpdate payload.

type SchemaMeta 

type SchemaMeta struct {
	TypeName string                 `protobuf:"bytes,1,opt,name=type_name,proto3" json:"type_name,omitempty"`
	Fields   map[uint32]string      `protobuf:"bytes,2,rep,name=fields,proto3" json:"fields,omitempty"`
	Children map[uint32]*SchemaMeta `protobuf:"bytes,3,rep,name=children,proto3" json:"children,omitempty"`
}

SchemaMeta is the optional inline schema header encoded as Field #0 (Tag = 0x0B, WireSubMsg = 5) at byte 0 of an OmniBuf message (omnibuf.v1.SchemaMeta).

func ReadField0SchemaMeta 

func ReadField0SchemaMeta(packet []byte) (*SchemaMeta, bool)

ReadField0SchemaMeta decodes and merges all Field #0 (Tag = 0x0B, WireSubMsg = 5) SchemaMeta blocks in packet.

Source Files

View all Source files

Directories

Path Synopsis
cmd
conformance-go command
Command conformance-go is the Go implementation's adapter binary for the OmniBuf conformance test runner.
 Command conformance-go is the Go implementation's adapter binary for the OmniBuf conformance test runner.
conformance-runner command
Command conformance-runner executes the language-independent OmniBuf conformance test suite against a target language adapter binary over length-prefixed stdin/stdout frames (modeled after Protobuf's conformance-test-runner).
 Command conformance-runner executes the language-independent OmniBuf conformance test suite against a target language adapter binary over length-prefixed stdin/stdout frames (modeled after Protobuf's conformance-test-runner).
omnibuf command
protoc-gen-omnibuf command
Package conformance implements the language-independent OmniBuf conformance test protocol (modeled after Protocol Buffers' conformance_test_runner) where a runner binary communicates with a language-specific adapter binary over length-prefixed stdin/stdout frames.
 Package conformance implements the language-independent OmniBuf conformance test protocol (modeled after Protocol Buffers' conformance_test_runner) where a runner binary communicates with a language-specific adapter binary over length-prefixed stdin/stdout frames.
Code generated for schemas/peer.proto.
 Code generated for schemas/peer.proto.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.