README
¶
fastfix
A pure-Go implementation of FAST 1.1 (FIX Adapted for STreaming), the binary encoding used by most exchange market-data feeds.
No dependencies outside the standard library. Builds on Go 1.21+.
go get github.com/bandari-abhilash/fastfix
What it does
- Decode — scalars, groups and sequences under all six field operators
(
constant,copy,default,delta,increment,tail), acrossuInt32/64,int32/64,string,byteVectoranddecimal. - Load templates — FAST 1.1 template XML, both flattened files and
reference-composed ones that assemble messages from shared
<templateRef>blocks. - Encode — client-initiated session messages (Logon, MarketDataRequest, Logout, ...) for a TCP request or recovery channel.
Performance
Apple M5 (4 performance + 6 efficiency cores), Go 1.26, median of 6 runs.
The workload is one 174-byte market-data packet: an incremental refresh with
10 book entries using constant, increment, copy and default
operators, optional decimals and a sequence. Each packet is decoded after a
dictionary reset, as a UDP feed handler would, and every entry's price and
size is read back.
Compared with other implementations
Every implementation below decodes the identical bytes
(benchmarks/testdata/), resets its dictionary before each packet, and
reads every price and size. Each harness exits with an error unless its
checksum matches, so a fast wrong decode can't count. Single core, median of
6 runs:
time/packet heap/packet
mFAST (C++), generated code 0.27 µs 0
mFAST (C++), runtime XML 0.67 µs 0
fastlib 0.3.8 (Rust), callback * 1.87 µs 283 B
OpenFAST 1.1.1 (Java 17) 2.41 µs 5.7 KB
fastfix (Go) 2.55 µs 5.3 KB
goFAST (Go), hand-written Receiver 2.92 µs 1.5 KB
fastlib 0.3.8 (Rust), serde structs 4.81 µs 13 KB
goFAST (Go), reflection 5.22 µs 3.0 KB
* streams values to a callback without building a message, so it does less work than the others.
What this shows:
- fastfix is the fastest Go FAST decoder measured: 1.15× faster than goFAST's hand-written path and 2× faster than its reflection path, while loading templates at runtime and needing no per-template code.
- It is level with OpenFAST, the Java reference implementation (2.41 µs with ParallelGC and a fixed heap, 2.56 µs with JVM defaults).
- mFAST is 4–10× faster and allocates nothing per packet. It decodes into
storage it reuses, while fastfix builds a fresh
map[string]anytree per message. That gap is the target for future versions.
Fairness notes: OpenFAST ran on JDK 17 after a 5 s JIT warm-up. Rust was
built with LTO; fastlib needs an id on every field, so its harness adds
them in memory (ids never appear on the wire). mFAST was built at -O3.
About half of mFAST's runtime-XML time is its by-name field lookup.
goFAST returns decimals as float64; the others are exact.
Multiple cores
Decoders share nothing but the read-only Registry, so you can run one
Decoder per feed or channel on its own goroutine. Total packets per second:
cores 1 2 4 6 10
default GC 334k 559k 808k 645k 572k
GOGC=off 326k 583k 1.15M 1.44M 1.88M
With the default GC, throughput peaks at 4 cores (2.4×) and then falls. With
GC off it keeps rising to 5.7× at 10 cores. The ceiling is allocation and GC
pressure, not contention inside the decoder. Every message and sequence entry
is a map[string]any. For multi-feed deployments, raise GOGC or set
GOMEMLIMIT. Cutting allocations is the main planned optimization.
Small messages
decode Logon (35 B) 195 ns ~5M messages/s 12 allocs
encode Logon 131 ns ~7.6M messages/s 8 allocs
Reproduce
go test -run '^$' -bench . -benchmem # fastfix only
cd benchmarks && go test -bench . -benchmem # vs goFAST
cd benchmarks && go test -bench Parallel -cpu 1,2,4,6,10 # multi-core
benchmarks/java/run.sh # OpenFAST (needs a JDK)
benchmarks/rust/run.sh # fastlib (needs cargo)
benchmarks/cpp/run.sh # mFAST (needs cmake, boost)
The comparisons live under benchmarks/ in their own module and build
directories, so fastfix itself has no dependencies.
Quick start
reg, err := fastfix.LoadTemplates(templateXML)
dec := fastfix.NewDecoder(reg)
for pkt := range packets {
dec.Ctx.Reset() // see "Context resets" below
msgs, err := dec.DecodePacket(pkt)
for _, m := range msgs {
px, ok := m.Int("MDEntryPx") // ok=false if absent or fractional
...
}
}
Context resets — read this one
The Context holds the operator dictionary that delta, copy, increment
and tail build on. Whether that dictionary spans the whole stream or is
rebuilt per packet is a property of the feed, not of FAST.
UDP market-data feeds typically encode each datagram independently, so you
must call dec.Ctx.Reset() at every packet boundary. Skip it and the first
packet decodes correctly while every packet after it accumulates stale deltas
— values drift rather than failing loudly, which makes this expensive to
debug in production. TCP session channels usually do want the dictionary
carried across messages; there, don't reset. Check your feed's spec.
TestPerPacketContextReset pins this behaviour both ways.
Decimals
Decimal is a mantissa/exponent pair rendered the way OpenFAST's
DecimalValue does, so values match what the rest of the FAST ecosystem
produces: (10715,-2) and (1071500,-4) both render as 107.15, (5,2)
renders as 500.
GroupVal.Int returns ok=false for a fractional decimal rather than
truncating it — silently rounding a price is worse than refusing it.
Templates
- Templates with no numeric
idare reference-only building blocks: available for<templateRef>inlining, never registered as decodable messages. <typeRef>is a FIX type annotation with no wire meaning and is dropped.- Feeds commonly reference a reset template by id without declaring it in the
XML. Register it yourself:
reg.RegisterReset(120, "FastResetTemplate").
testdata/composed.xml is a synthetic template set exercising every
composition feature the loader supports; it is a useful starting point if you
are writing templates by hand.
Not implemented
These return an error rather than producing a wrong encoding:
- encoding of groups
- encoding of the
deltaoperator (decoding is supported) stringdelta (byteVectordelta is supported)
License
MIT. See LICENSE.
Documentation
¶
Overview ¶
Package fastfix implements the FAST (FIX Adapted for STreaming) 1.1 wire protocol in pure Go, with no dependencies outside the standard library.
FAST is the binary encoding used by most exchange market-data feeds: a presence map plus field operators (constant, copy, default, delta, increment, tail) let the encoder omit anything the decoder can reconstruct from its dictionary, so a typical update costs a handful of bytes.
Usage ¶
Load a template definition XML into a Registry, then decode packets:
f, err := os.Open("templates.xml")
reg, err := fastfix.LoadTemplates(f)
dec := fastfix.NewDecoder(reg)
for pkt := range packets {
dec.Ctx.Reset() // see "Context resets" below
msgs, err := dec.DecodePacket(pkt)
for _, m := range msgs {
price, ok := m.Int("MDEntryPx")
...
}
}
Context resets ¶
This is the single most common source of wrong values. A Decoder's Context holds the operator dictionary that delta, copy, increment and tail operators build on. Whether that dictionary spans the whole stream or is rebuilt per packet is a property of the feed, not of FAST.
UDP market-data feeds typically encode each datagram independently, so the caller must call Ctx.Reset() at every packet boundary. Skip it and the first packet decodes correctly while every later one accumulates stale deltas — values drift instead of failing loudly. TCP session channels usually do want the dictionary carried across messages; there, do not reset. Check the feed's specification.
Templates ¶
LoadTemplates accepts both flattened template files and reference-composed ones that assemble messages from shared <templateRef> blocks. Templates with no numeric id are treated as reference-only building blocks: available for inlining, never registered as decodable messages. <typeRef> is a FIX type annotation with no wire meaning and is ignored.
Feeds commonly reference a reset template by id without declaring it in the XML; register it with Registry.RegisterReset.
Encoding ¶
Encode covers client-initiated session messages (Logon, MarketDataRequest, Logout, ...). It is deliberately lazy: every operator-bearing field sets its presence-map bit and transmits explicitly rather than exploiting the dictionary. That is valid FAST 1.1 and keeps decoder state consistent, but it is not meant for encoding a high-rate feed.
Scope ¶
Decoding covers scalars, groups and sequences across all six operators. Not implemented: encoding of groups and of the delta operator, and string (as opposed to byteVector) delta. These return an error rather than producing a wrong encoding.
Example ¶
Encode a session message and decode it back.
package main
import (
"fmt"
"strings"
"github.com/bandari-abhilash/fastfix"
)
const templatesXML = `<?xml version="1.0" encoding="UTF-8"?>
<templates xmlns="http://www.fixprotocol.org/ns/fast/td/1.1">
<template name="Header">
<string name="MessageType"/>
<string name="SendingTime"><tail/></string>
</template>
<template name="Logon" id="1">
<templateRef name="Header"/>
<uInt32 name="HeartBtInt"/>
<string name="Username"/>
<string name="AppID" presence="optional"/>
</template>
</templates>`
// Encode a session message and decode it back.
func main() {
reg, err := fastfix.LoadTemplates(strings.NewReader(templatesXML))
if err != nil {
panic(err)
}
packet, err := fastfix.Encode(reg.Get(1), map[string]any{
"MessageType": "A",
"SendingTime": "20260716-09:15:00.000000",
"HeartBtInt": 30,
"Username": "USER01",
})
if err != nil {
panic(err)
}
fmt.Printf("%d bytes on the wire\n", len(packet))
dec := fastfix.NewDecoder(reg)
msgs, err := dec.DecodePacket(packet)
if err != nil {
panic(err)
}
m := msgs[0]
hb, _ := m.Int("HeartBtInt")
fmt.Printf("%s user=%s heartbeat=%ds appID-present=%v\n",
m.Template.Name, m.ValueString("Username"), hb, m.Has("AppID"))
}
Output: 35 bytes on the wire Logon user=USER01 heartbeat=30s appID-present=false
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrEOF = errors.New("fast: end of packet")
ErrEOF is returned when the packet has no further bytes.
Functions ¶
Types ¶
type Context ¶
type Context struct {
// contains filtered or unexported fields
}
Context carries decoder state across messages: the operator dictionary (global scope, keyed by field name, matching OpenFAST's default dictionary setup) and the last template id.
func NewContext ¶
func NewContext() *Context
func (*Context) Reset ¶
func (c *Context) Reset()
Reset clears the dictionaries and template state.
Many UDP market-data feeds encode every datagram against a fresh context rather than a stream-wide one. Against such a feed the caller MUST call Reset() at each packet boundary, otherwise delta and copy operators keep accumulating across packets and every value after the first is wrong. See TestPerPacketContextReset.
type Decimal ¶
Decimal is a FAST decimal value (mantissa * 10^exponent).
type Decoder ¶
type Decoder struct {
Reg *Registry
Ctx *Context
// ResetOnResetTemplate controls whether a reset template registered via
// Registry.RegisterReset clears the dictionaries MID-packet. It defaults
// to false: feeds commonly carry a reset template in the stream without
// intending a dictionary reset, and per-packet reset is the caller's job
// via Ctx.Reset().
ResetOnResetTemplate bool
}
Decoder decodes FAST messages against a template registry.
func NewDecoder ¶
func (*Decoder) DecodeNext ¶
DecodeNext decodes ONE message from the front of a (possibly incomplete) byte stream. Returns (msg, bytesConsumed, nil) on success, (nil, 0, nil) when the buffer holds only a partial message (feed more bytes and retry), or an error for malformed data. Context state is snapshotted and restored on a partial decode so a retry with more bytes decodes identically.
type Field ¶
type Field struct {
Name string
Kind Kind
Optional bool
Operator Op
HasInitial bool
InitInt int64
InitDec Decimal
InitStr string
Length *Field // sequences: the length field
Fields []*Field // groups/sequences: members
}
Field is one node of a template.
type GroupVal ¶
type GroupVal struct {
// contains filtered or unexported fields
}
GroupVal holds decoded field values by name.
func NewGroupValFrom ¶
NewGroupValFrom builds a GroupVal from decoded values; used by consumer tests to exercise parsing logic without a wire round-trip.
func (*GroupVal) Int ¶
Int mirrors OpenFAST's GetInt/GetLong: ok=false when the field is absent, and ok=false when the value is a Decimal with a fractional part (OpenFAST raises there rather than truncating). Callers that must not silently round should treat ok=false as "skip this entry".
func (*GroupVal) ValueString ¶
ValueString renders a field as text, returning "" for a null/absent field rather than reporting an error.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry holds templates by id.
func LoadTemplates ¶
LoadTemplates parses a FAST 1.1 template definition XML.
Two template-composition styles are supported:
- Flattened files, where every template inlines its own fields and sequences.
- Reference-composed files, which build messages from shared <templateRef> blocks (headers, trailers, sequence element templates) and carry <typeRef> annotations.
A <templateRef name="X"/> is expanded in place by inlining template X's fields (recursively); <typeRef> is a FIX type annotation with no bearing on the wire encoding and is ignored. Templates without a numeric id are reference-only building blocks: they are available for expansion but are not registered for decoding.
func (*Registry) RegisterReset ¶
RegisterReset registers a FAST reset template under the given id. Feeds typically reference a reset template by id without declaring it in the template XML, so the caller supplies it programmatically — e.g. reg.RegisterReset(120, "FastResetTemplate").
Source Files