zap

package module
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 2, 2026 License: BSD-3-Clause Imports: 18 Imported by: 2

README

ZAP

Zero-copy Application Protocol for Lux.

17x faster 11x less memory 29x fewer allocations 14x lower GC 

import "github.com/luxfi/zap"

ZAP is a high-performance binary protocol for AI agent communication and inter-process messaging. It provides 17x faster serialization and 11x less memory than MCP JSON-RPC, while maintaining full compatibility with existing MCP tools.

Features

  • Zero-copy reads - Access data directly from byte buffers (2.9ns parse time)
  • Zero allocation - No heap allocations during deserialization
  • MCP Bridge - Auto-discover and accelerate existing MCP servers
  • mDNS Discovery - Automatic peer discovery for distributed systems
  • Request/Response - Built-in correlation for async RPC calls
  • Environmental - 96% energy reduction vs JSON-RPC at scale

Quick Start

// Build a message
b := zap.NewBuilder(256)

ob := b.StartObject(24)
ob.SetUint32(0, 42)           // Field at offset 0
ob.SetUint64(8, 0xDEADBEEF)   // Field at offset 8
ob.SetBool(16, true)          // Field at offset 16
ob.FinishAsRoot()

data := b.Finish()  // Wire-ready bytes

// Read zero-copy
msg, _ := zap.Parse(data)
root := msg.Root()

fmt.Println(root.Uint32(0))   // 42
fmt.Println(root.Uint64(8))   // 0xDEADBEEF
fmt.Println(root.Bool(16))    // true

Wire Format

┌─────────────────────────────────────────────────┐
│ Header (16 bytes)                               │
│  ├─ Magic (4 bytes): "ZAP\x00"                  │
│  ├─ Version (2 bytes): 1                        │
│  ├─ Flags (2 bytes): compression, etc.          │
│  ├─ Root Offset (4 bytes): offset to root       │
│  └─ Size (4 bytes): total message size          │
├─────────────────────────────────────────────────┤
│ Data Segment (variable)                         │
│  └─ Structs, lists, text, bytes...             │
└─────────────────────────────────────────────────┘

Types

Type Size Description
Bool 1 Boolean (0 or 1)
Int8/Uint8 1 8-bit integer
Int16/Uint16 2 16-bit integer
Int32/Uint32 4 32-bit integer
Int64/Uint64 8 64-bit integer
Float32 4 IEEE 754 float
Float64 8 IEEE 754 double
Text 8 String (offset + length)
Bytes 8 Byte slice (offset + length)
List 8 Array (offset + length)
Struct 4 Nested object (offset)

Lists

// Build a list
lb := b.StartList(4)  // 4-byte elements
lb.AddUint32(100)
lb.AddUint32(200)
lb.AddUint32(300)
listOffset, listLen := lb.Finish()

// Reference from object
ob.SetList(fieldOffset, listOffset, listLen)

// Read
list := obj.List(fieldOffset)
for i := 0; i < list.Len(); i++ {
    fmt.Println(list.Uint32(i))
}

Nested Objects

// Build inner object
inner := b.StartObject(8)
inner.SetUint32(0, 111)
innerOffset := inner.Finish()

// Reference from outer
outer := b.StartObject(16)
outer.SetObject(4, innerOffset)
outer.FinishAsRoot()

// Read
innerObj := root.Object(4)
fmt.Println(innerObj.Uint32(0))  // 111

Schema Definition

schema := zap.NewSchema("myapp")

person := zap.NewStructBuilder("Person").
    Uint32("id").
    Text("name").
    Int32("age").
    Bool("active").
    List("tags", zap.TypeText).
    Build()

schema.AddStruct(person)

// Use generated offsets
const (
    PersonID     = 0
    PersonName   = 4
    PersonAge    = 12
    PersonActive = 16
    PersonTags   = 20
)

Performance

Core Operations
BenchmarkZAPParse-10          411944325     2.9 ns/op    0 B/op    0 allocs/op
BenchmarkZAPBuild-10           26375461    44.0 ns/op    0 B/op    0 allocs/op
BenchmarkConsensusRound-10    203780178     5.5 ns/op    0 B/op    0 allocs/op
ZAP vs MCP JSON-RPC (Tool Calling)
Metric MCP JSON-RPC ZAP Improvement
Serialization 5,579 ns/op 322 ns/op 17x faster
Memory/call 2,826 bytes 256 bytes 11x less
Allocations 58/op 2/op 29x fewer
Parse time ~1,000 ns 2.9 ns 345x faster
GC runs 41 per 50K ops 3 per 50K ops 14x less
20-Tool Orchestrator Benchmark
BenchmarkMCPOrchestrator20Tools    10000    109175 ns/op    54292 B/op    1060 allocs/op
BenchmarkZAPOrchestrator20Tools   150036      8195 ns/op     5282 B/op      60 allocs/op
Real Network Performance
MCP JSON-RPC (simulated):  ~140,000 calls/sec
ZAP Zero-copy (simulated): ~4,200,000 calls/sec  (30x faster)
ZAP Network (real TCP):    ~34,000 calls/sec     (20 tool servers)
Memory Profile (100K operations)
=== MCP JSON-RPC ===                 === ZAP Zero-Copy ===
Total Allocated: 304.01 MB           Total Allocated: 26.70 MB
Malloc Count:    6,001,513           Malloc Count:    200,003
Bytes/Op:        3,187               Bytes/Op:        280
Mallocs/Op:      60                  Mallocs/Op:      2

Memory Saved: 277.30 MB (91.2%)
Allocations Saved: 5,801,510 (96.7%)

MCP Bridge

Auto-discover MCP servers and accelerate them with ZAP:

import (
    "github.com/luxfi/zap"
    "github.com/luxfi/zap/mcp"
)

func main() {
    node := zap.NewNode(zap.NodeConfig{
        NodeID: "orchestrator",
        Port:   9000,
    })
    node.Start()

    // Create bridge - auto-discovers MCP server tools
    bridge := mcp.NewBridge(node)
    bridge.AddServer("filesystem", "npx", "-y", "@anthropic/mcp-filesystem")
    bridge.AddServer("github", "npx", "-y", "@anthropic/mcp-github")

    // List all discovered tools
    for _, tool := range bridge.GetTools() {
        fmt.Printf("[%d] %s - %s\n", tool.ID, tool.Name, tool.Description)
    }

    // Call tools (17x faster than native MCP)
    result, _ := bridge.CallToolByName(ctx, "read_file", map[string]interface{}{
        "path": "/etc/hosts",
    })
}

Environmental Impact

At scale (1 billion tool calls/day):

Metric MCP JSON-RPC ZAP Savings
Memory throughput 2.9 TB/day 0.3 TB/day 2.6 TB/day
Energy consumption 1.8 kWh/day 0.1 kWh/day 96% reduction
CO2 emissions ~0.7 kg/day ~0.03 kg/day ~0.67 kg/day
Yearly CO2 savings - - ~245 kg

Comparison

Feature ZAP Cap'n Proto FlatBuffers Protobuf
Zero-copy read ✓ ✓ ✓ ✗
Schema required ✗ ✓ ✓ ✓
Code generation Optional Required Required Required
Random access ✓ ✓ ✓ ✗
Mutable Build only ✓ ✗ ✓

EVM Types

Built-in support for Ethereum/EVM types:

// Read EVM types zero-copy
addr := obj.Address(0)      // 20-byte address
hash := obj.Hash(20)        // 32-byte hash
sig := obj.Signature(52)    // 65-byte signature

// Build with EVM types
ob.SetAddress(0, addr)
ob.SetHash(20, hash)

// Parse from hex
addr, _ := zap.AddressFromHex("0x742d35Cc6634C0532925a3b844Bc9e7595f...")
hash, _ := zap.HashFromHex("0xabc123...")

Predefined schemas: TransactionSchema, BlockHeaderSchema, LogSchema

Node Discovery

Auto-discover peers via mDNS and communicate with ZAP:

node := zap.NewNode(zap.NodeConfig{
    NodeID:      "node-1",
    ServiceType: "_luxd._tcp",
    Port:        zap.DefaultPort, // 9999 — canonical ZAP port
})

// Handle incoming messages
node.Handle(MsgTypePing, func(ctx context.Context, from string, msg *zap.Message) (*zap.Message, error) {
    // Process ping, return pong
    return buildPong(), nil
})

node.Start()
defer node.Stop()

// Send to peer
node.Send(ctx, "node-2", msg)

// Broadcast to all
node.Broadcast(ctx, msg)

// List peers
for _, peer := range node.Peers() {
    fmt.Println(peer)
}

Use Cases

AI Agent Orchestration
// Orchestrator managing 20 tool servers
orchestrator := NewOrchestrator("agent", 9000)
for i := 1; i <= 20; i++ {
    orchestrator.ConnectToolServer(fmt.Sprintf("127.0.0.1:%d", 9000+i))
}

// Call tools with automatic routing
result, _ := orchestrator.CallTool(ctx, "search_code", args)
GPU Cluster / FHE Kernel Communication
// GPU nodes discovering each other via mDNS
gpuNode := zap.NewNode(zap.NodeConfig{
    NodeID:      "gpu-node-1",
    ServiceType: "_lux-gpu._tcp",
    Port:        7000,
})

// Handle FHE ciphertext operations
gpuNode.Handle(MsgTypeFHEOp, func(ctx context.Context, from string, msg *zap.Message) (*zap.Message, error) {
    root := msg.Root()
    opType := root.Uint32(0)          // Operation type
    ctLen := root.Uint32(4)           // Ciphertext length

    // Zero-copy access to ciphertext data
    // Process with GPU kernel...

    return buildResult(resultCiphertext), nil
})

// Cross-VM kernel-to-kernel communication
resp, _ := gpuNode.Call(ctx, "vm-fhe-2", encryptedPayload)
VM-to-VM Communication (Lux Network)
// Cross-VM calls for crypto operations
vmNode := zap.NewNode(zap.NodeConfig{
    NodeID:      "vm-evm-1",
    ServiceType: "_lux-vm._tcp",
    Port:        8545,
})

// Route messages between VMs
vmNode.Handle(MsgTypeCrossVM, func(ctx context.Context, from string, msg *zap.Message) (*zap.Message, error) {
    // Zero-copy message routing
    return forwardToTargetVM(msg)
})
High-Performance Consensus
// 5-node consensus reaching agreement in ~450µs
nodes := make([]*ConsensusNode, 5)
for i := range nodes {
    nodes[i] = newConsensusNode(i, 19000+i)
    nodes[i].Start()
}

// Propose and reach consensus
nodes[0].propose(ctx, round, value)
// Consensus reached in ~0.45ms (local network)

Running the Examples

# Run the 20-tool MCP bridge example
cd examples/mcp-bridge
go run main.go

# Run benchmarks
go test -bench=. -benchmem

# Run memory profiling
go test -v -run TestMemoryUsage

License

Copyright (C) 2025, Lux Industries Inc. All rights reserved.

Documentation

Overview

Package zap implements the Zero-copy Application Protocol (ZAP) for Lux.

ZAP is a binary serialization format designed for high-performance inter-process and network communication. Like Cap'n Proto and FlatBuffers, ZAP enables zero-copy reads - data can be accessed directly from the underlying byte buffer without parsing or allocation.

Transport security: set NodeConfig.TLS to a *tls.Config to wrap all TCP connections with TLS. This supports PQ-TLS 1.3 when the Go runtime and configured cipher suites provide post-quantum key exchange (e.g. X25519Kyber768). When TLS is nil (the default), connections are plaintext.

Wire Format: 

┌─────────────────────────────────────────────────┐
│ Header (16 bytes)                               │
│  ├─ Magic (4 bytes): "ZAP\x00"                  │
│  ├─ Version (2 bytes): 1                        │
│  ├─ Flags (2 bytes): compression, etc.          │
│  ├─ Root Offset (4 bytes): offset to root       │
│  └─ Size (4 bytes): total message size          │
├─────────────────────────────────────────────────┤
│ Data Segment (variable)                         │
│  └─ Structs, lists, text, bytes...             │
└─────────────────────────────────────────────────┘

All multi-byte integers are little-endian. Offsets are relative to the position of the offset field itself.

Index

Constants

View Source 
const (
	// AddressSize is the size of an EVM address (20 bytes)
	AddressSize = 20

	// HashSize is the size of a keccak256 hash (32 bytes)
	HashSize = 32

	// SignatureSize is the size of an ECDSA signature (65 bytes: r[32] + s[32] + v[1])
	SignatureSize = 65

	// BloomSize is the size of a bloom filter (256 bytes)
	BloomSize = 256
)
View Source 
const (
	FieldReqID   = 0 // uint32 - request ID for correlation
	FieldReqFlag = 4 // uint32 - 1=request, 2=response
	ReqFlagReq   = 1
	ReqFlagResp  = 2
)

Reserved header fields for request/response correlation These are the first 8 bytes of every Call message

View Source 
const (
	// HeaderSize is the size of the ZAP message header
	HeaderSize = 16

	// Magic bytes identifying a ZAP message
	Magic = "ZAP\x00"

	// Version of the ZAP format
	Version = 1

	// DefaultPort is the canonical TCP port for ZAP transport across the
	// Lux ecosystem. Like 80 means HTTP and 443 means HTTPS, 9999 means
	// ZAP — every ZAP-hosting service binds this port; the DNS name (e.g.
	// zap.kms.svc, zap.mpc.svc) disambiguates which service is on the
	// other end.
	DefaultPort = 9999

	// Alignment for data segments
	Alignment = 8
)
View Source 
const (
	FlagNone       uint16 = 0
	FlagCompressed uint16 = 1 << 0
	FlagEncrypted  uint16 = 1 << 1
	FlagSigned     uint16 = 1 << 2
)

Flags for message header

View Source 
const MaxPQRecord = 64 * 1024

MaxPQRecord is the largest plaintext payload one Write turns into a single DATA frame. Writes larger than this are chunked. Sized well below the §5 16-MiB hard cap so the AEAD tag + frame envelope always fit.

Variables

View Source 
var (
	ErrInvalidMagic   = errors.New("zap: invalid magic bytes")
	ErrInvalidVersion = errors.New("zap: unsupported version")
	ErrBufferTooSmall = errors.New("zap: buffer too small")
	ErrOutOfBounds    = errors.New("zap: offset out of bounds")
	ErrInvalidOffset  = errors.New("zap: invalid offset")
)
View Source 
var BlockHeaderSchema = NewStructBuilder("BlockHeader").
	Hash("parentHash").
	Hash("uncleHash").
	Address("coinbase").
	Hash("stateRoot").
	Hash("transactionsRoot").
	Hash("receiptsRoot").
	Bytes("logsBloom").
	Uint64("difficulty").
	Uint64("number").
	Uint64("gasLimit").
	Uint64("gasUsed").
	Uint64("timestamp").
	Bytes("extraData").
	Hash("mixHash").
	Uint64("nonce").
	Build()

BlockHeaderSchema defines the schema for an EVM block header.

View Source 
var ErrNotPQConn = errors.New("zap: not a ZAP-PQ connection")

ErrNotPQConn is returned by AsPQConn when the caller passes a net.Conn that is not a *pqConn (e.g. a legacy TCP conn).

View Source 
var ErrRegistrationMalformed = errors.New("zap-pq: VMRegistration malformed")

ErrRegistrationMalformed is returned when fields are missing or the wrong length.

View Source 
var ErrTransportUnavailable = errors.New("zap: transport unavailable (did you import _ \"github.com/luxfi/zap/quic\"?)")

ErrTransportUnavailable is returned by NewNode when the requested Transport has no registered factory. For TransportQUIC, this means the quic subpackage was not imported.

View Source 
var LogSchema = NewStructBuilder("Log").
	Address("address").
	List("topics", TypeBytes).
	Bytes("data").
	Uint64("blockNumber").
	Hash("txHash").
	Uint32("txIndex").
	Hash("blockHash").
	Uint32("logIndex").
	Bool("removed").
	Build()

LogSchema defines the schema for an EVM log entry.

View Source 
var TransactionSchema = NewStructBuilder("Transaction").
	Hash("hash").
	Uint64("nonce").
	Address("from").
	Address("to").
	Bytes("value").
	Bytes("data").
	Uint64("gas").
	Bytes("gasPrice").
	Uint64("chainId").
	Signature("signature").
	Build()

TransactionSchema defines the schema for an EVM transaction.

Functions

func AsPQConn added in v0.6.1 

func AsPQConn(c net.Conn) (*pqConn, error)

AsPQConn type-asserts an interface{} into a ZAP-PQ wrapper so callers can fetch PeerID without import cycles.

func DecodeNodeIDHandshake added in v0.6.1 

func DecodeNodeIDHandshake(data []byte) (string, bool)

DecodeNodeIDHandshake reads a NodeID exchange message and returns the peer's nodeID. An empty string (with ok=false) indicates a malformed or out-of-range length field.

func EncodeNodeIDHandshake added in v0.6.1 

func EncodeNodeIDHandshake(nodeID string) []byte

EncodeNodeIDHandshake builds the NodeID exchange message. nodeIDs longer than maxNodeIDLen are truncated; the receiver validates length on Decode.

func RegisterTransport added in v0.6.1 

func RegisterTransport(t Transport, f TransportFactory)

RegisterTransport plugs a TransportFactory into the registry. The quic subpackage calls this in its init function.

Registration is idempotent — re-registering the same Transport overwrites — but in practice each Transport has exactly one factory linked into the binary.

func TLSCertFingerprintFromBytes added in v0.3.1 

func TLSCertFingerprintFromBytes(certDER []byte) [32]byte

TLSCertFingerprintFromBytes returns sha256(certBytes) sized for the AttestationContext field. Helper so callers don't need to reach into crypto/sha256 separately.

func TranscriptHash added in v0.3.1 

func TranscriptHash(ctx *AttestationContext) [48]byte

TranscriptHash returns the 48-byte SHAKE256-384 commitment a PQ Attestation signature MUST cover. Domain-separated with the "ZAP-PQ-V1" string so a signature produced for ZAP cannot be replayed on any other ML-DSA-signed transcript (warp envelopes, validator-set commitments, etc.).

SP 800-185 left_encode framing on each field so a malicious transcript field whose first bytes spell another field's payload cannot collide with a legitimate transcript.

func TypeSize 

func TypeSize(t Type) int

TypeSize returns the size of a type in bytes.

func UnwrapCorrelated added in v0.6.1 

func UnwrapCorrelated(data []byte) (reqID uint32, flag uint32, body []byte, ok bool)

UnwrapCorrelated reads the correlation header off `data` and returns (reqID, flag, body, ok). If `data` is shorter than the header or the flag isn't a recognised value, ok is false and the caller should treat the message as uncorrelated.

func VerifyRegistration added in v0.6.1 

func VerifyRegistration(
	reg *VMRegistration,
	chainAuthority *mldsa.PublicKey,
	mode RegistrationMode,
) (*mldsa.PublicKey, error)

VerifyRegistration checks AuthoritySig — and PrevVMSig under ModeRotation — against the chain authority public key. Returns the verified VM public key on success.

Signing context for both signatures is the §6.4 SignCtx ("lux-zap-pq-v1") so the same audited verifier handles them. Payload is `VMID ∥ VMPubKey` so a signature over one (VMID, pubkey) pair cannot be re-used for a different pair.

func WrapCorrelated added in v0.6.1 

func WrapCorrelated(reqID uint32, flag uint32, body []byte) []byte

WrapCorrelated prepends the Call/response correlation header to `body`. The result is what writeMessage emits onto the wire.

func WrapPQ added in v0.6.1 

func WrapPQ(conn net.Conn, sess *handshake.Session) net.Conn

WrapPQ wraps an established net.Conn with ZAP-PQ-v1 AEAD framing after a completed handshake. The returned net.Conn implements stream Read/Write on top of the record-oriented Session — one Write becomes one DATA frame (chunked at MaxRecord), reads buffer across frames so callers see the byte stream they expect.

Close closes the Session (which zeros the keys and closes the underlying TCP conn). LocalAddr / RemoteAddr / SetDeadline* delegate to the wrapped conn.

Drop-in usage: 

tcp, _ := net.Dial("tcp", "...")
sess, _ := (&handshake.Initiator{Local: id}).Run(tcp)
pq := zap.WrapPQ(tcp, sess)
// use pq as any net.Conn from here on

Types

type Address 

type Address [AddressSize]byte

Address is a 20-byte EVM address (zero-copy view). 

var ZeroAddress Address

ZeroAddress is the zero address.

func AddressFromHex 

func AddressFromHex(s string) (Address, error)

AddressFromHex parses an address from hex string.

func (Address) Hex 

func (a Address) Hex() string

Hex returns the hex-encoded address with 0x prefix.

func (Address) IsZero 

func (a Address) IsZero() bool

IsZero returns true if the address is zero.

func (Address) String 

func (a Address) String() string

String implements fmt.Stringer.

type Attestation added in v0.3.1 

type Attestation struct {
	// PubKey is the peer's strict-PQ public key. The verifier
	// confirms membership in the chain's validator set BEFORE
	// trusting the signature.
	PubKey []byte
	// Sig is the signature over TranscriptHash(...).
	Sig []byte
}

Attestation is the wire shape a ZAP peer presents after the TLS handshake completes. PubKey + Sig are opaque bytes from zap's perspective; the AttestationVerifier owns the format (FIPS 204 ML-DSA-65 pubkey 1952 bytes, signature 3293 bytes for Liquid; ML-DSA-87 with different byte counts for high-value Zoo chains).

func (*Attestation) HasPQEvidence added in v0.3.1 

func (a *Attestation) HasPQEvidence() bool

HasPQEvidence implements pq.PQEvidencer. A non-nil Attestation with a non-empty Sig counts as evidence — the gate then dispatches to the verifier, which actually checks the signature + validator-set membership.

type AttestationContext added in v0.3.1 

type AttestationContext struct {
	// TLSCertFingerprint is sha256 of the peer's TLS certificate
	// DER. Binding the attestation to this fingerprint means a
	// stolen-but-orthogonal TLS cert can't be paired with a
	// different PQ identity to MITM the channel.
	TLSCertFingerprint [32]byte
	// ChainID is the 32-byte chain identifier this connection is
	// scoped to. Different chains produce different transcripts,
	// so an attestation harvested on chain A is useless on chain B.
	ChainID [32]byte
	// PeerMLKEMPub is the peer's ML-KEM-768 public key (1184
	// bytes). Including it in the transcript binds the
	// attestation to the same KEM key the TLS handshake used
	// (when X25519MLKEM768 hybrid was negotiated). On classical
	// TLS this is empty and the field doesn't contribute.
	PeerMLKEMPub []byte
	// Timestamp (unix seconds) of the connection. Verifier checks
	// |now - timestamp| < some window (e.g. 60s) to refuse
	// replays of old captured attestations.
	Timestamp uint64
	// Nonce is per-session entropy from the verifier — 32 bytes.
	// Refuses an attacker who records one valid attestation from
	// replaying it on a future session.
	Nonce [32]byte
}

AttestationContext bundles the inputs a verifier needs to rebuild the transcript hash. Same inputs on both peers; the PQ signature anchors the binding.

type Bloom 

type Bloom [BloomSize]byte

Bloom is a 256-byte bloom filter.

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder constructs ZAP messages.

func NewBuilder 

func NewBuilder(capacity int) *Builder

NewBuilder creates a new builder with the given initial capacity.

func (*Builder) Finish 

func (b *Builder) Finish() []byte

Finish finalizes the message and returns the bytes.

func (*Builder) FinishWithFlags 

func (b *Builder) FinishWithFlags(flags uint16) []byte

FinishWithFlags finalizes with specific flags.

func (*Builder) Reset 

func (b *Builder) Reset()

Reset resets the builder for reuse.

func (*Builder) StartList 

func (b *Builder) StartList(elemSize int) *ListBuilder

StartList starts building a list.

func (*Builder) StartObject 

func (b *Builder) StartObject(dataSize int) *ObjectBuilder

StartObject starts building an object with the given data size.

func (*Builder) WriteBytes 

func (b *Builder) WriteBytes(data []byte) int

WriteBytes writes raw bytes and returns the offset.

func (*Builder) WriteText 

func (b *Builder) WriteText(s string) int

WriteText writes a string and returns the offset.

type Conn 

type Conn struct {
	NodeID string
	Addr   string
	// contains filtered or unexported fields
}

Conn is a ZAP connection to a peer.

func (*Conn) Recv 

func (c *Conn) Recv() (*Message, error)

Recv receives a message from the connection.

func (*Conn) Send 

func (c *Conn) Send(msg *Message) error

Send sends a message over the connection.

type Enum 

type Enum struct {
	Name   string
	Type   Type // Underlying type (Uint8, Uint16, etc.)
	Values map[string]uint64
}

Enum describes a ZAP enum.

type Field 

type Field struct {
	Name       string
	Type       Type
	Offset     int    // Byte offset within struct
	ListElem   Type   // Element type if Type == TypeList
	StructName string // Struct name if Type == TypeStruct
	Default    any    // Default value
}

Field describes a struct field.

type Handler 

type Handler func(ctx context.Context, from string, msg *Message) (*Message, error)

Handler handles incoming ZAP messages.

type Hash 

type Hash [HashSize]byte

Hash is a 32-byte hash (zero-copy view). 

var ZeroHash Hash

ZeroHash is the zero hash.

func HashFromHex 

func HashFromHex(s string) (Hash, error)

HashFromHex parses a hash from hex string.

func (Hash) Bytes32 

func (h Hash) Bytes32() [32]byte

Bytes32 returns the hash as a 32-byte array.

func (Hash) Hex 

func (h Hash) Hex() string

Hex returns the hex-encoded hash with 0x prefix.

func (Hash) IsZero 

func (h Hash) IsZero() bool

IsZero returns true if the hash is zero.

func (Hash) String 

func (h Hash) String() string

String implements fmt.Stringer.

type List 

type List struct {
	// contains filtered or unexported fields
}

List is a zero-copy view into a ZAP list.

func (List) Address 

func (l List) Address(i int) Address

Address returns an address from a list of addresses.

func (List) Bytes 

func (l List) Bytes() []byte

Bytes returns the raw bytes of the list (for byte lists).

func (List) Hash 

func (l List) Hash(i int) Hash

Hash returns a hash from a list of hashes.

func (List) IsNull 

func (l List) IsNull() bool

IsNull returns true if the list is null.

func (List) Len 

func (l List) Len() int

Len returns the number of elements.

func (List) Object 

func (l List) Object(i int, elemSize int) Object

Object returns an object list element.

func (List) Uint8 

func (l List) Uint8(i int) uint8

Uint8 returns a uint8 list element.

func (List) Uint32 

func (l List) Uint32(i int) uint32

Uint32 returns a uint32 list element.

func (List) Uint64 

func (l List) Uint64(i int) uint64

Uint64 returns a uint64 list element.

type ListBuilder 

type ListBuilder struct {
	// contains filtered or unexported fields
}

ListBuilder builds a ZAP list.

func (*ListBuilder) AddBytes 

func (lb *ListBuilder) AddBytes(data []byte)

AddBytes adds raw bytes (for byte lists).

func (*ListBuilder) AddUint8 

func (lb *ListBuilder) AddUint8(v uint8)

AddUint8 adds a uint8 element.

func (*ListBuilder) AddUint32 

func (lb *ListBuilder) AddUint32(v uint32)

AddUint32 adds a uint32 element.

func (*ListBuilder) AddUint64 

func (lb *ListBuilder) AddUint64(v uint64)

AddUint64 adds a uint64 element.

func (*ListBuilder) Finish 

func (lb *ListBuilder) Finish() (offset int, length int)

Finish returns the list offset and length.

type Message 

type Message struct {
	// contains filtered or unexported fields
}

Message is a ZAP message that can be read zero-copy.

func Parse 

func Parse(data []byte) (*Message, error)

Parse parses a ZAP message from bytes without copying.

func (*Message) Bytes 

func (m *Message) Bytes() []byte

Bytes returns the underlying byte slice.

func (*Message) Flags 

func (m *Message) Flags() uint16

Flags returns the message flags.

func (*Message) Root 

func (m *Message) Root() Object

Root returns the root object of the message.

func (*Message) Size 

func (m *Message) Size() int

Size returns the total message size.

type Node 

type Node struct {
	// contains filtered or unexported fields
}

Node is a ZAP node that combines mDNS discovery with zero-copy RPC.

func NewNode 

func NewNode(cfg NodeConfig) *Node

NewNode creates a new ZAP node.

func (*Node) Broadcast 

func (n *Node) Broadcast(ctx context.Context, msg *Message) map[string]error

Broadcast sends a message to all connected peers.

func (*Node) Call 

func (n *Node) Call(ctx context.Context, peerID string, msg *Message) (*Message, error)

Call sends a request and waits for a response.

func (*Node) ConnectDirect 

func (n *Node) ConnectDirect(addr string) error

ConnectDirect connects directly to a peer at the given address (bypasses mDNS).

func (*Node) Handle 

func (n *Node) Handle(msgType uint16, handler Handler)

Handle registers a handler for a message type.

func (*Node) NodeID 

func (n *Node) NodeID() string

NodeID returns this node's ID.

func (*Node) Peers 

func (n *Node) Peers() []string

Peers returns connected peer IDs.

func (*Node) Send 

func (n *Node) Send(ctx context.Context, peerID string, msg *Message) error

Send sends a ZAP message to a peer.

func (*Node) Start 

func (n *Node) Start() error

Start starts the node (discovery + listener).

func (*Node) Stop 

func (n *Node) Stop()

Stop stops the node.

type NodeConfig 

type NodeConfig struct {
	NodeID      string
	ServiceType string // e.g., "_luxd._tcp", "_fhed._tcp"
	Port        int
	Metadata    map[string]string
	Logger      *slog.Logger
	NoDiscovery bool        // Disable mDNS discovery (use ConnectDirect only)
	TLS         *tls.Config // optional PQ-TLS 1.3; nil = plaintext

	// Transport selects the network transport: TransportTCP (default,
	// preserves back-compat) or TransportQUIC. TransportQUIC requires
	// `import _ "github.com/luxfi/zap/quic"` somewhere in the binary.
	Transport Transport

	// QUICConfig, if non-nil, is passed to the QUIC transport as a
	// *quic.Config (github.com/quic-go/quic-go). Ignored for
	// TransportTCP. Typed as any here to avoid pulling quic-go into
	// the parent package's import graph.
	QUICConfig any
}

NodeConfig configures a ZAP node.

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object is a zero-copy view into a ZAP struct.

func (Object) Address 

func (o Object) Address(fieldOffset int) Address

Address reads an address at the given field offset (zero-copy).

func (Object) AddressSlice 

func (o Object) AddressSlice(fieldOffset int) []byte

AddressSlice returns a slice of the address bytes (zero-copy).

func (Object) Bool 

func (o Object) Bool(fieldOffset int) bool

Bool reads a bool at the given field offset.

func (Object) Bytes 

func (o Object) Bytes(fieldOffset int) []byte

Bytes reads a byte slice at the given field offset (zero-copy).

Wire-format rule: relOffset is an UNSIGNED forward pointer from the field position into the variable-section. Negative bit-patterns (high bit set) flow through uint32→int conversion as large positive values and are rejected by the absPos+length > len(data) bounds check. This closes the memo-pointer-escape malleability surface where a signed cast would let a crafted relOffset alias bytes back inside the fixed section.

func (Object) Float32 

func (o Object) Float32(fieldOffset int) float32

Float32 reads a float32 at the given field offset.

func (Object) Float64 

func (o Object) Float64(fieldOffset int) float64

Float64 reads a float64 at the given field offset.

func (Object) Hash 

func (o Object) Hash(fieldOffset int) Hash

Hash reads a hash at the given field offset (zero-copy).

func (Object) HashSlice 

func (o Object) HashSlice(fieldOffset int) []byte

HashSlice returns a slice of the hash bytes (zero-copy).

func (Object) Int8 

func (o Object) Int8(fieldOffset int) int8

Int8 reads an int8 at the given field offset.

func (Object) Int16 

func (o Object) Int16(fieldOffset int) int16

Int16 reads an int16 at the given field offset.

func (Object) Int32 

func (o Object) Int32(fieldOffset int) int32

Int32 reads an int32 at the given field offset.

func (Object) Int64 

func (o Object) Int64(fieldOffset int) int64

Int64 reads an int64 at the given field offset.

func (Object) IsNull 

func (o Object) IsNull() bool

IsNull returns true if the object is null.

func (Object) List 

func (o Object) List(fieldOffset int) List

List reads a list at the given field offset.

relOffset is signed: see Object() for the rationale.

func (Object) Object 

func (o Object) Object(fieldOffset int) Object

Object reads a nested object at the given field offset.

relOffset is signed: builders may emit a nested object that lives BEFORE the parent's fixed section (the parent is finalized last). The bounds check at the bottom rejects any absOffset outside the message; for the Bytes-malleability fix see Bytes().

func (Object) Signature 

func (o Object) Signature(fieldOffset int) Signature

Signature reads a signature at the given field offset.

func (Object) Text 

func (o Object) Text(fieldOffset int) string

Text reads a string at the given field offset (zero-copy).

func (Object) Uint8 

func (o Object) Uint8(fieldOffset int) uint8

Uint8 reads a uint8 at the given field offset.

func (Object) Uint16 

func (o Object) Uint16(fieldOffset int) uint16

Uint16 reads a uint16 at the given field offset.

func (Object) Uint32 

func (o Object) Uint32(fieldOffset int) uint32

Uint32 reads a uint32 at the given field offset.

func (Object) Uint64 

func (o Object) Uint64(fieldOffset int) uint64

Uint64 reads a uint64 at the given field offset.

type ObjectBuilder 

type ObjectBuilder struct {
	// contains filtered or unexported fields
}

ObjectBuilder builds a ZAP object (struct).

func (*ObjectBuilder) Finish 

func (ob *ObjectBuilder) Finish() int

Finish finalizes the object and returns its offset. Writes deferred text/bytes data after the object's fixed section and patches relative offsets.

func (*ObjectBuilder) FinishAsRoot 

func (ob *ObjectBuilder) FinishAsRoot() int

FinishAsRoot finalizes and sets as the message root.

func (*ObjectBuilder) SetAddress 

func (ob *ObjectBuilder) SetAddress(fieldOffset int, addr Address)

SetAddress sets an address field.

func (*ObjectBuilder) SetBool 

func (ob *ObjectBuilder) SetBool(fieldOffset int, v bool)

SetBool sets a bool field.

func (*ObjectBuilder) SetBytes 

func (ob *ObjectBuilder) SetBytes(fieldOffset int, v []byte)

SetBytes sets a bytes field. The data is written after the object's fixed section during Finish().

func (*ObjectBuilder) SetFloat32 

func (ob *ObjectBuilder) SetFloat32(fieldOffset int, v float32)

SetFloat32 sets a float32 field.

func (*ObjectBuilder) SetFloat64 

func (ob *ObjectBuilder) SetFloat64(fieldOffset int, v float64)

SetFloat64 sets a float64 field.

func (*ObjectBuilder) SetHash 

func (ob *ObjectBuilder) SetHash(fieldOffset int, h Hash)

SetHash sets a hash field.

func (*ObjectBuilder) SetInt8 

func (ob *ObjectBuilder) SetInt8(fieldOffset int, v int8)

SetInt8 sets an int8 field.

func (*ObjectBuilder) SetInt16 

func (ob *ObjectBuilder) SetInt16(fieldOffset int, v int16)

SetInt16 sets an int16 field.

func (*ObjectBuilder) SetInt32 

func (ob *ObjectBuilder) SetInt32(fieldOffset int, v int32)

SetInt32 sets an int32 field.

func (*ObjectBuilder) SetInt64 

func (ob *ObjectBuilder) SetInt64(fieldOffset int, v int64)

SetInt64 sets an int64 field.

func (*ObjectBuilder) SetList 

func (ob *ObjectBuilder) SetList(fieldOffset int, listOffset int, length int)

SetList sets a list field.

func (*ObjectBuilder) SetObject 

func (ob *ObjectBuilder) SetObject(fieldOffset int, objOffset int)

SetObject sets a nested object field (by offset).

func (*ObjectBuilder) SetSignature 

func (ob *ObjectBuilder) SetSignature(fieldOffset int, sig Signature)

SetSignature sets a signature field.

func (*ObjectBuilder) SetText 

func (ob *ObjectBuilder) SetText(fieldOffset int, v string)

SetText sets a text (string) field.

func (*ObjectBuilder) SetUint8 

func (ob *ObjectBuilder) SetUint8(fieldOffset int, v uint8)

SetUint8 sets a uint8 field.

func (*ObjectBuilder) SetUint16 

func (ob *ObjectBuilder) SetUint16(fieldOffset int, v uint16)

SetUint16 sets a uint16 field.

func (*ObjectBuilder) SetUint32 

func (ob *ObjectBuilder) SetUint32(fieldOffset int, v uint32)

SetUint32 sets a uint32 field.

func (*ObjectBuilder) SetUint64 

func (ob *ObjectBuilder) SetUint64(fieldOffset int, v uint64)

SetUint64 sets a uint64 field.

type RegistrationMode added in v0.6.1 

type RegistrationMode int

RegistrationMode selects which signatures VerifyRegistration requires. The chain-state loader decides which to pass based on whether the registry already holds an entry for this VMID.

Passing the wrong mode is a contract bug detectable at the call site: ModeInitial refuses any registration that carries a PrevVMSig; ModeRotation requires both PrevVMSig and PrevVMPubKey. A loader that always calls ModeInitial would let a compromised chain authority overwrite an existing VM key without the old key's consent — which §10.2 forbids. 

const (
	// ModeInitial: this VMID is being registered for the first time.
	// PrevVMSig MUST be empty; the chain-authority signature alone
	// is sufficient.
	ModeInitial RegistrationMode = iota
	// ModeRotation: this VMID already exists in the registry; the
	// caller is replacing its public key. Both the chain-authority
	// AND the previous VM key must sign the new (VMID, VMPubKey).
	ModeRotation
)

type Schema 

type Schema struct {
	Name    string
	Structs map[string]*Struct
	Enums   map[string]*Enum
}

Schema describes a complete ZAP schema.

func NewSchema 

func NewSchema(name string) *Schema

NewSchema creates a new empty schema.

func (*Schema) AddEnum 

func (s *Schema) AddEnum(e *Enum)

AddEnum adds an enum to the schema.

func (*Schema) AddStruct 

func (s *Schema) AddStruct(st *Struct)

AddStruct adds a struct to the schema.

type Signature 

type Signature [SignatureSize]byte

Signature is a 65-byte ECDSA signature.

type StaticVMRegistry added in v0.6.1 

type StaticVMRegistry struct {
	// contains filtered or unexported fields
}

StaticVMRegistry is an in-memory map implementation of VMRegistry suitable for genesis-loaded registrations. Production deployments will typically back this with an indexed chain-state cache, but the interface stays the same.

func NewStaticVMRegistry added in v0.6.1 

func NewStaticVMRegistry() *StaticVMRegistry

NewStaticVMRegistry returns an empty registry.

func (*StaticVMRegistry) Add added in v0.6.1 

func (r *StaticVMRegistry) Add(reg *VMRegistration)

Add inserts a registration. The caller is responsible for having verified the registration via VerifyRegistration first.

func (*StaticVMRegistry) Lookup added in v0.6.1 

func (r *StaticVMRegistry) Lookup(vmID [32]byte) (*VMRegistration, bool)

Lookup implements VMRegistry.

type Struct 

type Struct struct {
	Name   string
	Size   int // Total size in bytes
	Fields []Field
}

Struct describes a ZAP struct.

type StructBuilder 

type StructBuilder struct {
	// contains filtered or unexported fields
}

StructBuilder helps build struct definitions.

func NewStructBuilder 

func NewStructBuilder(name string) *StructBuilder

NewStructBuilder creates a struct builder.

func (*StructBuilder) Address 

func (sb *StructBuilder) Address(name string) *StructBuilder

Address adds an address field.

func (*StructBuilder) Bool 

func (sb *StructBuilder) Bool(name string) *StructBuilder

Bool adds a bool field.

func (*StructBuilder) Build 

func (sb *StructBuilder) Build() *Struct

Build finalizes and returns the struct.

func (*StructBuilder) Bytes 

func (sb *StructBuilder) Bytes(name string) *StructBuilder

Bytes adds a bytes field.

func (*StructBuilder) Float64 

func (sb *StructBuilder) Float64(name string) *StructBuilder

Float64 adds a float64 field.

func (*StructBuilder) Hash 

func (sb *StructBuilder) Hash(name string) *StructBuilder

Hash adds a hash field.

func (*StructBuilder) Int32 

func (sb *StructBuilder) Int32(name string) *StructBuilder

Int32 adds an int32 field.

func (*StructBuilder) Int64 

func (sb *StructBuilder) Int64(name string) *StructBuilder

Int64 adds an int64 field.

func (*StructBuilder) List 

func (sb *StructBuilder) List(name string, elemType Type) *StructBuilder

List adds a list field.

func (*StructBuilder) Signature 

func (sb *StructBuilder) Signature(name string) *StructBuilder

Signature adds a signature field.

func (*StructBuilder) Struct 

func (sb *StructBuilder) Struct(name string, structName string) *StructBuilder

Struct adds a nested struct field.

func (*StructBuilder) Text 

func (sb *StructBuilder) Text(name string) *StructBuilder

Text adds a text field.

func (*StructBuilder) Uint32 

func (sb *StructBuilder) Uint32(name string) *StructBuilder

Uint32 adds a uint32 field.

func (*StructBuilder) Uint64 

func (sb *StructBuilder) Uint64(name string) *StructBuilder

Uint64 adds a uint64 field.

type Transport added in v0.6.1 

type Transport int

Transport selects which network transport a Node uses.

The default zero value, TransportTCP, preserves the historical behavior of NewNode (TCP + optional TLS via NodeConfig.TLS) so every existing caller keeps working untouched.

TransportQUIC selects the QUIC transport defined in the quic subpackage. The quic subpackage must be imported anonymously by the process for TransportQUIC to be available; otherwise NewNode returns ErrTransportUnavailable. 

const (
	// TransportTCP is ZAP's original transport: framed Cap'n Proto
	// over TCP, with optional TLS 1.3 supplied via NodeConfig.TLS.
	// This is the default for backward compatibility.
	TransportTCP Transport = iota

	// TransportQUIC selects the QUIC transport from the quic
	// subpackage. Requires:
	//
	//   import _ "github.com/luxfi/zap/quic"
	//
	// in some package linked into the binary, so the QUIC factory
	// registers itself at init time.
	TransportQUIC
)

type TransportConn added in v0.6.1 

type TransportConn interface {
	// Send writes one ZAP frame to the peer.
	Send(frame []byte) error

	// Recv blocks until the next ZAP frame arrives, or returns
	// io.EOF when the peer cleanly closes.
	Recv() ([]byte, error)

	// Close performs a graceful close.
	Close() error

	// RemoteAddr returns the peer's network address (best effort —
	// for QUIC this is the address at handshake time, not necessarily
	// the latest after migration).
	RemoteAddr() string
}

TransportConn is the transport-level abstraction over a single peer connection. It mirrors the existing TCP *Conn semantics (Send/Recv/Close) and is used by the transport-aware Node code path in node.go.

type TransportFactory added in v0.6.1 

type TransportFactory interface {
	// Listen starts a listener on the address derived from cfg.
	// The listener calls onConn for each accepted, post-handshake
	// connection. Listen must not block; it returns the bound
	// address (so tests using port 0 can learn the kernel port) and
	// a Close func.
	Listen(ctx context.Context, cfg NodeConfig, onConn func(peerID string, conn TransportConn)) (string, func() error, error)

	// Dial opens a connection to addr. The returned TransportConn
	// is symmetric with the conns yielded by Listen.
	Dial(ctx context.Context, cfg NodeConfig, addr string) (peerID string, conn TransportConn, err error)
}

TransportFactory is the extension point the quic subpackage uses to register itself with this package at init time, avoiding an import cycle.

A TransportFactory wraps the existing Node so all the higher-level APIs (Handle, Send, Call, Broadcast) work identically regardless of transport. The factory is responsible for:

  • Binding a listener on n.port (cfg.Port).
  • Yielding accepted connections via the supplied dispatch hook.
  • Implementing outbound dial via the supplied dispatch hook.

Today only QUIC uses this hook; TCP is wired directly in node.go because the original Node embeds the TCP listener fields. This is pragmatic — once QUIC ships we can refactor TCP onto the same factory shape without a single user-visible change.

type Type 

type Type uint8

Type represents a ZAP type. 

const (
	TypeVoid Type = iota
	TypeBool
	TypeInt8
	TypeInt16
	TypeInt32
	TypeInt64
	TypeUint8
	TypeUint16
	TypeUint32
	TypeUint64
	TypeFloat32
	TypeFloat64
	TypeText
	TypeBytes
	TypeList
	TypeStruct
	TypeEnum
	TypeUnion
)

type VMRegistration added in v0.6.1 

type VMRegistration struct {
	VMID         [32]byte
	VMPubKey     []byte
	AuthoritySig []byte
	PrevVMSig    []byte

	// PrevVMPubKey is the prior VM key for rotation. Verifiers
	// supply this out of band (from chain state) when checking
	// PrevVMSig. Stored here so a single struct round-trips the
	// rotation evidence.
	PrevVMPubKey []byte
}

VMRegistration is the §10.2 chain-authority-signed record binding a VM plugin's ML-DSA-65 identity to its VMID.

  • VMID : SHA3-256(VMPubKey) — the canonical handle
  • VMPubKey : ML-DSA-65 public key bytes (MLDSA65PubLen)
  • AuthoritySig : chain-authority ML-DSA-65 signature over (VMID ∥ VMPubKey)
  • PrevVMSig : optional — prior VM key's signature over the same payload. Required for rotation; nil/empty for the initial registration.

type VMRegistry added in v0.6.1 

type VMRegistry interface {
	Lookup(vmID [32]byte) (*VMRegistration, bool)
}

VMRegistry is loaded from a signed chain config and consulted by the node side of every ZAP-PQ handshake to a VM plugin (§10.2).

A node holds one VMRegistry per chain it serves; the registry is populated from on-chain VMRegistration records each of which is signed by the chain authority.

Source Files

View all Source files

Directories

Path Synopsis
Real-world benchmark: ZAP vs gRPC/Protobuf
 Real-world benchmark: ZAP vs gRPC/Protobuf
examples
agents command
Multi-Agent LLM Consensus via ZAP Protocol
 Multi-Agent LLM Consensus via ZAP Protocol
mcp-bridge command
Example: MCP-to-ZAP Bridge with 20 Tool Servers
 Example: MCP-to-ZAP Bridge with 20 Tool Servers
Package handshake implements SPEC-ZAP-PQ-v1: the native post-quantum handshake and AEAD framing for ZAP.
 Package handshake implements SPEC-ZAP-PQ-v1: the native post-quantum handshake and AEAD framing for ZAP.
Package mcp provides a bridge between MCP (Model Context Protocol) servers and ZAP for high-performance tool calling.
 Package mcp provides a bridge between MCP (Model Context Protocol) servers and ZAP for high-performance tool calling.
Package quic implements the QUIC transport for the ZAP messaging substrate.
 Package quic implements the QUIC transport for the ZAP messaging substrate.
Package transport is the pluggable GPU-aware ZAP transport.
 Package transport is the pluggable GPU-aware ZAP transport.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.