protocol

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 25, 2026 License: MIT Imports: 5 Imported by: 0

Documentation

Overview

Package protocol defines the FrogoDB binary wire protocol. Shared between server and client.

Index

Constants

View Source 
const (
	ProtoVersionLegacy      uint8 = 0x01 // synchronous request-response mode
	ProtoVersionMultiplexed uint8 = 0x02 // multiplexed concurrent requests
)

Connection-level protocol version bytes sent as the first byte on a new connection for version negotiation (T17.8). The server reads this byte to select the connection handler. Old clients that do not send a version byte start with the Magic byte 'F' (0x46), which the server detects and falls back to legacy mode with RequestID-based auto-detection.

View Source 
const (
	OpPing         uint8 = 0x01
	OpPut          uint8 = 0x02
	OpGet          uint8 = 0x03
	OpDelete       uint8 = 0x04
	OpOperate      uint8 = 0x05
	OpBatch        uint8 = 0x06
	OpScan         uint8 = 0x07 // scan namespace/set records with limit + cursor
	OpCount        uint8 = 0x08 // return approximate record count for namespace/set
	OpGetMap       uint8 = 0x10
	OpBatchGet     uint8 = 0x20 // native batch GET with thresholded server-side shard execution (T16.6)
	OpBatchPut     uint8 = 0x21 // native batch PUT with server-side shard fan-out (T16.7)
	OpBatchOperate uint8 = 0x22 // native batch OPERATE with server-side shard fan-out (T16.7)
	OpBatchDelete  uint8 = 0x23 // native batch DELETE with server-side shard fan-out (T19.3)
	OpLSHDedup     uint8 = 0xA0 // LSH string deduplication (server-side)
	OpLSHVector    uint8 = 0xA1 // LSH vector behavioural ID (server-side)
)

OpCodes for client requests.

View Source 
const (
	OpPingResp         uint8 = 0x81
	OpPutResp          uint8 = 0x82
	OpGetResp          uint8 = 0x83
	OpDeleteResp       uint8 = 0x84
	OpOperateResp      uint8 = 0x85
	OpBatchResp        uint8 = 0x86
	OpScanResp         uint8 = 0x87
	OpCountResp        uint8 = 0x88
	OpGetMapResp       uint8 = 0x90
	OpBatchGetResp     uint8 = 0xA0 // 0x20 | 0x80
	OpBatchPutResp     uint8 = 0xA1 // 0x21 | 0x80
	OpBatchOperateResp uint8 = 0xA2 // 0x22 | 0x80
	OpBatchDeleteResp  uint8 = 0xA3 // 0x23 | 0x80
)

OpCodes for server responses (request OpCode | 0x80).

View Source 
const (
	FlagSendKey       uint16 = 1 << 0
	FlagCreateOnly    uint16 = 1 << 1
	FlagReplace       uint16 = 1 << 2
	FlagRespondAllOps uint16 = 1 << 3
)

Protocol flags (bitmask in Header.Flags).

View Source 
const (
	FlagReadMaster          uint16 = 0 << 4 // default: read from partition master
	FlagReadMasterOrReplica uint16 = 1 << 4 // read from master or replica (whichever is local)
	FlagReadReplicaOnly     uint16 = 2 << 4 // read from replica only
)

Read policy flags (2-bit field in bits 4-5 of Header.Flags). Use ReadPolicyMask to extract.

View Source 
const (
	FieldNamespace uint8 = 0x01
	FieldSet       uint8 = 0x02
	FieldKey       uint8 = 0x03
	FieldDigest    uint8 = 0x04
	FieldFilter    uint8 = 0x05 // serialized filter expression tree
	FieldGroup     uint8 = 0x06 // LSH group name
)

Field types in the payload.

View Source 
const (
	OpTypeRead    uint8 = 0x01
	OpTypeWrite   uint8 = 0x02
	OpTypeIncr    uint8 = 0x05
	OpTypeAppend  uint8 = 0x09
	OpTypePrepend uint8 = 0x0A
)

Operation types within OPERATE requests.

View Source 
const (
	RequestPayloadHeaderSize  = 8         // NumFields(2) + NumOps(2) + TTL(4)
	ResponsePayloadHeaderSize = 4         // ResultCode(1) + NumBins(2) + Reserved(1)
	FieldHeaderSize           = 5         // Type(1) + Len(4)
	OperationHeaderSize       = 8         // OpType(1) + BinNameLen(1) + ParticleType(1) + Flags(1) + DataLen(4)
	ResponseBinHeaderSize     = 6         // BinNameLen(1) + ParticleType(1) + DataLen(4)
	MaxPayloadSize            = 128 << 20 // 128 MB
)

Payload size constants.

View Source 
const (
	ResultOK                   uint8 = 0x00
	ResultKeyNotFound          uint8 = 0x02
	ResultKeyExists            uint8 = 0x05
	ResultGenerationError      uint8 = 0x06
	ResultOverloaded           uint8 = 0x18 // server backpressure: too many in-flight requests (T17.6)
	ResultFilteredOut          uint8 = 0x1A // record exists but excluded by filter expression
	ResultPartitionUnavailable uint8 = 0x14
	ResultInDoubt              uint8 = 0x28
	ResultServerError          uint8 = 0x50
)

Result codes returned in server responses.

View Source 
const FlagCommitMaster uint16 = 1 << 8

FlagCommitMaster is the commit policy flag (bit 8 of Header.Flags). Return after local write, replicate async.

View Source 
const FlagPreserveTTL uint16 = 1 << 11

FlagPreserveTTL requests that PUT and BatchPut preserve the existing record TTL when a record already exists. When absent, the TTL field keeps the legacy meaning: TTL > 0 sets a new TTL, TTL == 0 clears expiration.

View Source 
const FlagPrimaryOnly uint16 = 1 << 9

FlagPrimaryOnly limits local scan/count execution to partitions where the serving node is master. Only OpScan and OpCount honor it.

View Source 
const FlagWriteMerge uint16 = 1 << 10

FlagWriteMerge requests merge/update semantics for PUT and BatchPut. When absent, writes use the legacy full-record replace path.

View Source 
const HeaderSize = 20

HeaderSize is the fixed size of the wire protocol header in bytes.

View Source 
const (
	// OpFlagCreateOnly skips a Write operation if the bin already has a value.
	// Used for FirstSeen semantics: write only if the bin does not exist.
	OpFlagCreateOnly uint8 = 0x10
)

Per-operation flags (bitmask in Operation.Flags).

View Source 
const ReadPolicyMask uint16 = 0x30

ReadPolicyMask extracts the read policy from Header.Flags (bits 4-5).

View Source 
const VersionV1 uint8 = 0x01

VersionV1 is the current protocol version.

Variables

View Source 
var Magic = [2]byte{'F', 'D'}

Magic bytes identifying a FrogoDB protocol frame.

View Source 
var ResponseBufPool = sync.Pool{
	New: func() any {
		b := make([]byte, 0, 4096)
		return &b
	},
}

ResponseBufPool provides reusable byte buffers for response payload marshalling. Each buffer starts at 4KB capacity, sufficient for most single-record responses. Callers must return buffers via ReleaseResponseBuf after use.

Functions

func AcquireOperations 

func AcquireOperations() *[]Operation

AcquireOperations returns a pooled Operation slice for temporary use. Caller must call ReleaseOperations after use.

func AcquireResponseBuf 

func AcquireResponseBuf() *[]byte

AcquireResponseBuf returns a pooled byte buffer for response marshalling.

func EncodeScanResponse 

func EncodeScanResponse(nextCursor uint32, records []ScanRecord) []byte

EncodeScanResponse encodes scan records and a continuation cursor into a blob.

Wire format: [NextCursor:4B][NumRecords:2B][Records...] Record: [Generation:4B][NumBins:2B][Bins...] Bins use the standard ResponseBin wire format.

func ExpAnd 

func ExpAnd(exprs ...[]byte) []byte

ExpAnd creates a logical AND of all sub-expressions. Short-circuits on first false.

func ExpEqual 

func ExpEqual(left, right []byte) []byte

ExpEqual creates a "left == right" comparison expression.

func ExpFloatBin 

func ExpFloatBin(name string) []byte

ExpFloatBin creates a filter expression that reads a float bin value.

func ExpFloatVal 

func ExpFloatVal(v float64) []byte

ExpFloatVal creates a constant float value expression.

func ExpGreater 

func ExpGreater(left, right []byte) []byte

ExpGreater creates a "left > right" comparison expression.

func ExpGreaterOrEqual 

func ExpGreaterOrEqual(left, right []byte) []byte

ExpGreaterOrEqual creates a "left >= right" comparison expression.

func ExpIntBin 

func ExpIntBin(name string) []byte

ExpIntBin creates a filter expression that reads an integer bin value.

func ExpIntVal 

func ExpIntVal(v int64) []byte

ExpIntVal creates a constant integer value expression.

func ExpLess 

func ExpLess(left, right []byte) []byte

ExpLess creates a "left < right" comparison expression.

func ExpLessOrEqual 

func ExpLessOrEqual(left, right []byte) []byte

ExpLessOrEqual creates a "left <= right" comparison expression.

func ExpNot 

func ExpNot(expr []byte) []byte

ExpNot creates a logical NOT of the inner expression.

func ExpNotEqual 

func ExpNotEqual(left, right []byte) []byte

ExpNotEqual creates a "left != right" comparison expression.

func ExpOr 

func ExpOr(exprs ...[]byte) []byte

ExpOr creates a logical OR of all sub-expressions. Short-circuits on first true.

func ExpStringBin 

func ExpStringBin(name string) []byte

ExpStringBin creates a filter expression that reads a string bin value.

func ExpStringVal 

func ExpStringVal(v string) []byte

ExpStringVal creates a constant string value expression.

func HasPrimaryOnly 

func HasPrimaryOnly(opCode uint8, flags uint16) bool

HasPrimaryOnly returns true when Header.Flags requests primary-only execution for a scan/count request. Unrelated opcodes ignore the flag.

func MarshalBatchGetResults 

func MarshalBatchGetResults(results []BatchGetResult) []byte

MarshalBatchGetResults encodes batch GET results into a byte slice.

Wire format: [Count:4B][Result1][Result2]... Each result: [ResultCode:1B][Generation:4B][NumBins:2B][Bins...] Each bin uses the standard ResponseBin wire format.

func ReleaseOperations 

func ReleaseOperations(ops *[]Operation)

ReleaseOperations returns an Operation slice to the pool after clearing all Data references to prevent memory leaks.

func ReleaseRequest 

func ReleaseRequest(req *Request)

ReleaseRequest returns a Request to the pool after clearing all references. Operation.Data and BatchRecord slices are zeroed to prevent memory leaks.

func ReleaseResponse 

func ReleaseResponse(resp *Response)

ReleaseResponse returns a Response to the pool after clearing all fields. The Bins backing array is retained; individual entries are zeroed to release Data slice references for GC.

func ReleaseResponseBuf 

func ReleaseResponseBuf(b *[]byte)

ReleaseResponseBuf returns a buffer to the pool after resetting its length.

Types

type BatchGetResult 

type BatchGetResult struct {
	ResultCode uint8
	Generation uint32
	Bins       []ResponseBin
}

BatchGetResult holds the server-side result for one key in a batch GET.

func UnmarshalBatchGetResults 

func UnmarshalBatchGetResults(buf []byte) ([]BatchGetResult, error)

UnmarshalBatchGetResults decodes batch GET results from the wire format.

type BatchRecord 

type BatchRecord struct {
	Key        []byte
	TTL        uint32
	Generation uint32
	Flags      uint16 // per-record flags (CreateOnly, Replace, WriteMerge, PreserveTTL, etc.)
	Operations []Operation
}

BatchRecord holds per-key data for a batch PUT or OPERATE request (T16.7).

type Field 

type Field struct {
	Type uint8
	Data []byte
}

Field represents a typed field in the protocol payload.

Wire format: [FieldType:1B][FieldLen:4B][FieldData:var]

func UnmarshalField 

func UnmarshalField(buf []byte) (Field, int, error)

UnmarshalField reads a single Field from buf. Returns the field, bytes consumed, and any error.

func (*Field) MarshalSize 

func (f *Field) MarshalSize() int

MarshalSize returns the total wire size of the field.

func (*Field) MarshalTo 

func (f *Field) MarshalTo(buf []byte) int

MarshalTo writes the field into buf and returns the number of bytes written. 

type Header struct {
	Magic       [2]byte
	Version     uint8
	OpCode      uint8
	Flags       uint16
	PartitionID uint16
	Generation  uint32
	PayloadLen  uint32
	RequestID   uint32
}

Header is the 20-byte fixed header for every protocol frame.

Layout (BigEndian): 

[0:2]   Magic       "FD" (0x46, 0x44)
[2]     Version     0x01
[3]     OpCode      operation code
[4:6]   Flags       bit flags
[6:8]   PartitionID 0-4095
[8:12]  Generation  optimistic locking
[12:16] PayloadLen  payload size in bytes
[16:20] RequestID   multiplexing request ID (0 = legacy synchronous mode)

func (*Header) Marshal 

func (h *Header) Marshal(buf []byte)

Marshal writes the header into buf. buf must be at least HeaderSize bytes.

func (*Header) Unmarshal 

func (h *Header) Unmarshal(buf []byte) error

Unmarshal reads the header from buf. buf must be at least HeaderSize bytes.

type Operation 

type Operation struct {
	OpType       uint8
	BinName      string
	ParticleType uint8
	Flags        uint8
	Data         []byte
}

Operation represents a single operation within an OPERATE request.

Wire format: [OpType:1B][BinNameLen:1B][BinName:var][ParticleType:1B][Flags:1B][DataLen:4B][Data:var]

func UnmarshalOperation 

func UnmarshalOperation(buf []byte) (Operation, int, error)

UnmarshalOperation reads a single Operation from buf. Returns the operation, bytes consumed, and any error.

func (*Operation) MarshalSize 

func (o *Operation) MarshalSize() int

MarshalSize returns the total wire size of the operation.

func (*Operation) MarshalTo 

func (o *Operation) MarshalTo(buf []byte) int

MarshalTo writes the operation into buf and returns the number of bytes written.

type Request 

type Request struct {
	Header           Header
	Namespace        string
	Set              string
	Key              []byte
	Digest           [20]byte
	Group            string // LSH group name (FieldGroup)
	Operations       []Operation
	TTL              uint32
	FilterExpression []byte // serialized filter expression tree (FieldFilter)

	// Pre-resolved namespace/set IDs set by the transport layer's per-connection
	// cache (T10.5). When NsResolved is true, the engine skips the registry lock.
	NsID       uint16
	SetID      uint16
	NsResolved bool

	// BatchKeys holds multiple keys for native batch operations (T16.6).
	// Populated by unmarshalBatchGetPayload; slices reference the payload buffer.
	BatchKeys [][]byte

	// BatchRecords holds per-key record data for native batch PUT/OPERATE (T16.7).
	// Populated by unmarshalBatchWritePayload.
	BatchRecords []BatchRecord
}

Request represents a parsed client request.

func AcquireRequest 

func AcquireRequest() *Request

AcquireRequest returns a pooled Request with pre-allocated Operations slice. Caller must call ReleaseRequest after the request has been fully processed.

func (*Request) Marshal 

func (r *Request) Marshal() []byte

Marshal encodes the full request (header + payload). Sets Header.Magic, Header.Version, and Header.PayloadLen automatically.

func (*Request) MarshalInto 

func (r *Request) MarshalInto(buf []byte) int

MarshalInto encodes the full request into buf and returns the number of bytes written. buf must be at least MarshalSize() bytes. Sets Header.Magic, Header.Version, and Header.PayloadLen automatically.

func (*Request) MarshalPayload 

func (r *Request) MarshalPayload() []byte

MarshalPayload encodes the request payload (excluding the 16-byte header).

Payload format: [NumFields:2B][NumOps:2B][TTL:4B][Fields...][Operations...]

func (*Request) MarshalSize 

func (r *Request) MarshalSize() int

MarshalSize returns the total wire size of the request (header + payload) without allocating the output buffer. Use with MarshalInto for zero-alloc marshaling on hot paths.

func (*Request) Reset 

func (r *Request) Reset()

Reset clears all fields for reuse, retaining allocated backing arrays for Key, FilterExpression, and Operations slices. This enables per-connection Request reuse without re-allocating on every request.

func (*Request) Unmarshal 

func (r *Request) Unmarshal(buf []byte) error

Unmarshal decodes the full request (header + payload) from buf.

func (*Request) UnmarshalPayload 

func (r *Request) UnmarshalPayload(buf []byte) error

UnmarshalPayload decodes the request payload (excluding the 16-byte header). When called on a Reset() request, this method reuses existing backing arrays for Key, FilterExpression, and Operations to minimize allocations.

type Response 

type Response struct {
	Header     Header
	ResultCode uint8
	Generation uint32
	Bins       []ResponseBin
}

Response represents a server response to a client.

func AcquireResponse 

func AcquireResponse() *Response

AcquireResponse returns a pooled Response with pre-allocated Bins slice. Caller must call ReleaseResponse after the response has been fully written.

func (*Response) Marshal 

func (r *Response) Marshal() []byte

Marshal encodes the full response (header + payload). Sets Header.Magic, Header.Version, Header.Generation, and Header.PayloadLen automatically. Uses MarshalInto internally to produce the result in a single allocation.

func (*Response) MarshalInto 

func (r *Response) MarshalInto(buf []byte) int

MarshalInto encodes the full response into buf and returns the number of bytes written. buf must be at least MarshalSize() bytes. Sets Header.Magic, Header.Version, Header.Generation, and Header.PayloadLen automatically.

func (*Response) MarshalPayload 

func (r *Response) MarshalPayload() []byte

MarshalPayload encodes the response payload (excluding the 16-byte header).

Payload format: [ResultCode:1B][NumBins:2B][Reserved:1B][Bins...]

func (*Response) MarshalPayloadInto 

func (r *Response) MarshalPayloadInto(buf []byte) []byte

MarshalPayloadInto encodes the response payload into the provided buffer, growing it if necessary. Returns the buffer (which may be a new slice if growth was needed). Callers should update their pointer with the returned slice to ensure pool-returned buffers reflect any capacity growth.

func (*Response) MarshalSize 

func (r *Response) MarshalSize() int

MarshalSize returns the total wire size of the response (header + payload) without allocating. Use with MarshalInto for zero-alloc marshaling.

func (*Response) Reset 

func (r *Response) Reset()

Reset clears all fields for reuse, retaining the Bins backing array.

func (*Response) Unmarshal 

func (r *Response) Unmarshal(buf []byte) error

Unmarshal decodes the full response (header + payload) from buf.

func (*Response) UnmarshalPayload 

func (r *Response) UnmarshalPayload(buf []byte) error

UnmarshalPayload decodes the response payload (excluding the 16-byte header).

type ResponseBin 

type ResponseBin struct {
	Name         string
	ParticleType uint8
	Data         []byte
}

ResponseBin represents a single bin in a response payload.

Wire format: [BinNameLen:1B][BinName:var][ParticleType:1B][DataLen:4B][Data:var]

type ScanRecord 

type ScanRecord struct {
	Generation uint32
	Bins       []ResponseBin
}

ScanRecord represents one record in a scan response.

func DecodeScanResponse 

func DecodeScanResponse(data []byte) (uint32, []ScanRecord, error)

DecodeScanResponse decodes a scan blob into a continuation cursor and records.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.