sbe

package
v0.77.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 12, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package sbe provides SBE (Simple Binary Encoding) marshaling for protobuf messages.

Messages are encoded following the FIX SBE standard. Proto descriptors with custom SBE annotations define the message layout: 

option (sbe.schema_id) = 1;
option (sbe.version) = 0;

message Order {
    option (sbe.template_id) = 1;
    uint64 order_id = 1;
    string symbol = 2 [(sbe.length) = 8];
}

A Codec is created from proto file descriptors and produces standard SBE binary compatible with any SBE implementation.

Supported proto field types:

  • Scalar integers (int32, int64, uint32, uint64, sint32, sint64, fixed*, sfixed*)
  • bool, float, double, enum
  • string and bytes (require [(sbe.length)] annotation for fixed size)
  • Nested messages (encoded as SBE composites, inlined at fixed offsets)
  • repeated messages (encoded as SBE repeating groups)

Unsupported: map fields, oneof, repeated scalars.

Concurrency

A Codec is safe for concurrent use after construction. Its internal template tables are read-only once NewCodec returns.

A View returned by Codec.View is a value; copies are independent and safe to read from multiple goroutines, provided the underlying buffer is not mutated. View accessors read from that buffer directly — see the View documentation for the trust model.

Encoder and Decoder (the SOFH-framed stream wrappers) hold a reusable scratch buffer and are not safe for concurrent use; create one per goroutine, or guard with a mutex.

Index

Constants

View Source 
const DefaultMaxFrameSize = 16 << 20

DefaultMaxFrameSize caps the SBE body size at 16 MiB by default.

Variables

View Source 
var ErrFramingCorrupt = errors.New("sbe: stream framing corrupt")

ErrFramingCorrupt is returned when the SOFH framing is invalid (wrong encoding type, length less than the header, frame larger than MaxFrameSize, etc.). The byte stream is desynchronized after this error and the decoder is permanently unusable.

Functions

func ProtoToXML 

func ProtoToXML(files ...protoreflect.FileDescriptor) ([]byte, error)

ProtoToXML converts proto file descriptors with SBE annotations to an SBE XML schema.

func XMLToProto 

func XMLToProto(xmlData []byte) ([]byte, error)

XMLToProto converts an SBE XML schema to a .proto file with SBE annotations.

Types

type Codec 

type Codec struct {
	// contains filtered or unexported fields
}

Codec encodes and decodes proto.Message values using the SBE binary format. A Codec is safe for concurrent use after creation.

func NewCodec 

func NewCodec(files ...protoreflect.FileDescriptor) (*Codec, error)

NewCodec creates an SBE codec from proto file descriptors. Each file must have (sbe.schema_id) and (sbe.version) file options, and each message to be encoded must have an (sbe.template_id) message option.

func (*Codec) Marshal 

func (c *Codec) Marshal(msg proto.Message) ([]byte, error)

Marshal encodes msg to SBE binary format.

func (*Codec) NewDecoder 

func (c *Codec) NewDecoder(r io.Reader) *Decoder

func (*Codec) NewEncoder 

func (c *Codec) NewEncoder(w io.Writer) *Encoder

func (*Codec) Unmarshal 

func (c *Codec) Unmarshal(data []byte, msg proto.Message) error

Unmarshal decodes SBE binary into msg.

func (*Codec) UnmarshalDescriptor 

func (c *Codec) UnmarshalDescriptor(data []byte, desc protoreflect.MessageDescriptor) (*dynamicpb.Message, error)

UnmarshalDescriptor decodes SBE binary into a new dynamicpb.Message.

func (*Codec) View 

func (c *Codec) View(data []byte) (View, error)

View creates a zero-allocation reader over SBE-encoded data. The message template is identified from the header's template ID.

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder reads SOFH-framed SBE messages. The internal scratch buffer is reused across Decode calls; Codec.Unmarshal copies parsed values into the destination message, so the buffer is the decoder's to overwrite.

func (*Decoder) Decode 

func (d *Decoder) Decode(msg proto.Message) error

func (*Decoder) SetMaxFrameSize 

func (d *Decoder) SetMaxFrameSize(n int)

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder writes SBE messages framed with a Simple Open Framing Header.

func (*Encoder) Encode 

func (e *Encoder) Encode(msg proto.Message) error

type GroupView 

type GroupView struct {
	// contains filtered or unexported fields
}

GroupView provides access to entries in an SBE repeating group.

func (GroupView) Entry 

func (g GroupView) Entry(i int) View

Entry returns a View over the i-th group entry.

func (GroupView) Len 

func (g GroupView) Len() int

Len returns the number of entries in the group.

type View 

type View struct {
	// contains filtered or unexported fields
}

View provides zero-allocation read access to SBE-encoded data. Field values are read directly from the underlying byte buffer.

Strings returned by View.String are backed by the original buffer (via unsafe.String) and are only valid while that buffer is alive.

Trust model

View is designed for the hot path and trades safety for speed. Once the constructing Codec.View call returns successfully, accessor methods (View.Int, View.Uint, View.String, View.Composite, View.Group, etc.) panic on schema mismatch — unknown field name, wrong primitive type, or a field accessed as the wrong category. They are intended for callers that own both the schema and the buffer.

Do not call View accessors with arbitrary, attacker-controlled bytes. For untrusted input, use Codec.Unmarshal or Codec.UnmarshalDescriptor, which return an error on every malformed-input path. View's constructor validates the message header and rejects buffers shorter than the template's declared block length, but accessor-level checks are intentionally skipped.

func (View) Bool 

func (v View) Bool(name string) bool

Bool reads a boolean field (uint8 encoding: 0 = false).

func (View) Bytes 

func (v View) Bytes(name string) []byte

Bytes reads a fixed-length bytes field as a sub-slice of the underlying buffer.

func (View) Composite 

func (v View) Composite(name string) View

Composite returns a sub-View over a nested message (SBE composite) field.

func (View) Enum 

func (v View) Enum(name string) int

Enum reads an enum field as an integer.

func (View) Float 

func (v View) Float(name string) float64

Float reads a floating-point field. Works with float and double encodings.

func (View) Group 

func (v View) Group(name string) GroupView

Group returns a GroupView for a repeating group field. Groups are located by walking the data sequentially from the end of the root block.

func (View) Int 

func (v View) Int(name string) int64

Int reads a signed integer field. Works with int8, int16, int32, int64 encodings.

func (View) String 

func (v View) String(name string) string

String reads a fixed-length string field. The returned string points directly into the underlying buffer (zero-copy) and is only valid while that buffer is alive. Trailing null padding is trimmed.

func (View) Uint 

func (v View) Uint(name string) uint64

Uint reads an unsigned integer field. Works with uint8, uint16, uint32, uint64 encodings. Also works for bool and enum fields (returns raw numeric value).

type XMLComposite 

type XMLComposite struct {
	Name        string    `xml:"name,attr"`
	Description string    `xml:"description,attr,omitempty"`
	Types       []XMLType `xml:"type"`
	Refs        []XMLRef  `xml:"ref"`
}

XMLComposite represents an SBE composite type definition.

type XMLEnum 

type XMLEnum struct {
	Name         string          `xml:"name,attr"`
	EncodingType string          `xml:"encodingType,attr"`
	Description  string          `xml:"description,attr,omitempty"`
	ValidValues  []XMLValidValue `xml:"validValue"`
}

XMLEnum represents an SBE enum type definition.

type XMLField 

type XMLField struct {
	Name string `xml:"name,attr"`
	ID   uint32 `xml:"id,attr"`
	Type string `xml:"type,attr"`
}

XMLField represents a field within an SBE message or group.

type XMLGroup 

type XMLGroup struct {
	Name   string     `xml:"name,attr"`
	ID     uint32     `xml:"id,attr"`
	Fields []XMLField `xml:"field"`
}

XMLGroup represents an SBE repeating group.

type XMLMessage 

type XMLMessage struct {
	Name        string     `xml:"name,attr"`
	ID          uint32     `xml:"id,attr"`
	Description string     `xml:"description,attr,omitempty"`
	Fields      []XMLField `xml:"field"`
	Groups      []XMLGroup `xml:"group"`
}

XMLMessage represents an SBE message (template).

type XMLRef 

type XMLRef struct {
	Name string `xml:"name,attr"`
	Type string `xml:"type,attr"`
}

XMLRef represents a reference to another type within a composite.

type XMLSchema 

type XMLSchema struct {
	XMLName     xml.Name     `xml:"messageSchema"`
	Package     string       `xml:"package,attr"`
	ID          uint32       `xml:"id,attr"`
	Version     uint32       `xml:"version,attr"`
	ByteOrder   string       `xml:"byteOrder,attr"`
	Description string       `xml:"description,attr"`
	Types       XMLTypes     `xml:"types"`
	Messages    []XMLMessage `xml:"message"`
}

XMLSchema represents an SBE XML message schema.

func ParseXMLSchema 

func ParseXMLSchema(data []byte) (*XMLSchema, error)

ParseXMLSchema parses an SBE XML schema.

type XMLType 

type XMLType struct {
	Name          string `xml:"name,attr"`
	PrimitiveType string `xml:"primitiveType,attr"`
	Length        uint32 `xml:"length,attr,omitempty"`
	Description   string `xml:"description,attr,omitempty"`
}

XMLType represents an SBE simple type definition.

type XMLTypes 

type XMLTypes struct {
	Types      []XMLType      `xml:"type"`
	Composites []XMLComposite `xml:"composite"`
	Enums      []XMLEnum      `xml:"enum"`
}

XMLTypes holds the type definitions within an SBE schema.

type XMLValidValue 

type XMLValidValue struct {
	Name  string `xml:"name,attr"`
	Value string `xml:",chardata"`
}

XMLValidValue represents one value in an SBE enum.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.