raptorgo

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

README

raptorgo

Go Reference CI License

RaptorQ (RFC 6330) forward error correction in pure Go. No CGo, no dependencies.

Deliver an object over a lossy one-way channel without retransmission: the sender emits equal-sized packets, and the receiver rebuilds the object from roughly any K of them, no matter which were lost. Typical overhead is zero to two extra packets per block.

Compared to fixed-rate Reed-Solomon shards, RaptorQ generates fresh repair packets on demand (so redundancy adapts to unknown or varying loss), scales to 56,403 symbols per block and 255 blocks per object, and lets multicast receivers with different losses each complete independently. If you have a low-latency return channel, use TCP/QUIC instead; if your shard count is small and static, plain Reed-Solomon is simpler.

Install

go get github.com/fgn/raptorgo

Go 1.26 or newer. Pure Go on every platform.

Quickstart

Publish the 12-byte OTI (codec parameters) and a digest over an authenticated channel; send packets over the lossy one. The decoder delivers only after your verifier accepts the reconstructed bytes.

// Sender
limits := raptorgo.DefaultLimits()
oti, err := raptorgo.DeriveOTI(raptorgo.DerivationInput{
    TransferLength:             uint64(len(object)),
    DecoderBlockBytes:          1 << 20, // largest block decoded in memory
    MaxPayloadSize:             1280,    // symbol size; fit your path MTU
    SymbolAlignment:            4,
    MinSubSymbolAlignmentUnits: 8,
}, limits)
encoder, err := raptorgo.NewObjectEncoder(object, oti, limits)
otiWire, err := oti.MarshalBinary()
digest := sha256.Sum256(object)

// Receiver
receivedOTI, err := raptorgo.ParseOTI(otiWire, limits)
decoder, err := raptorgo.NewObjectDecoder(receivedOTI, limits, "transfer-42",
    func(request raptorgo.VerificationRequest) error {
        hash := sha256.New()
        if _, err := io.Copy(hash, request.Object); err != nil {
            return err
        }
        if !bytes.Equal(hash.Sum(nil), digest[:]) {
            return errors.New("digest mismatch")
        }
        return nil
    },
    func(verified raptorgo.VerifiedObject) error {
        result = verified.Bytes // owned copy, delivered exactly once
        return nil
    },
)

// Transfer: source symbols are ESI 0..K-1; repair symbols start at ESI K.
k, err := raptorgo.SourceBlockSymbols(receivedOTI, 0)
for esi := uint32(0); esi < k; esi++ {
    packet, err := encoder.Packet(0, esi, 1, true)
    // ... transmit; lost packets are simply never added ...
    _, err = decoder.AddPacket(packet)
}
for esi := k; decoder.State() == raptorgo.ObjectReceiving; esi++ {
    packet, err := encoder.Packet(0, esi, 1, true) // repair symbol
    _, err = decoder.AddPacket(packet)
}
// decoder.State() == raptorgo.ObjectDelivered

Run a complete transfer over a simulated lossy channel:

go run ./examples/roundtrip -size 1048576 -mtu 1280 -loss 10

Next steps: the runnable package examples, docs/transport.md for framing and repair strategies, and PacketInto for allocation-free packet emission on hot paths.

Performance

Single-threaded scalar Go decodes 12.8 MB with ten percent loss in about a second (AMD Ryzen 5 3600); systematic packets cost about 80 ns each with a reused buffer. The opt-in AVX2 build (GOEXPERIMENT=simd, amd64, runtime scalar fallback) is over 4x faster end to end: 117.5 ms encode and 112.2 ms decode for the same 12.8 MB workload on an i9-14900HX. Measured tables and reproduction commands are in docs/performance.md.

Safety and limits

FEC repairs erasures; it authenticates nothing. The decoder therefore requires a verifier and a deliverer and enforces decoded, then verified, then delivered: FEC success alone never releases bytes. All decoder memory is preflighted against explicit Limits, so a forged OTI is refused before it can consume resources. The API is in-memory (not streaming); one block spans at most 56,403 symbols; transfers cap at 942,574,504,275 bytes. Threat model: SECURITY.md.

Correctness

The full RFC surface is implemented and tested through K=56,403: parameter derivation and OTI handling, block and sub-block partitioning, grouped and shortened packets, systematic and repair encoding, and multi-block recovery under loss, reordering, and duplicates. Evidence comes from independent layers: exhaustive algebra and table tests against the pinned RFC, byte-identical Rust and C++ cross-decoding, dense algebraic oracles against the production solver, bounded TLA+ models with production trace replay, and pinned mutation-testing gates. This is strong layered evidence, not a machine-checked proof of the Go code, and no such claim is made. The exact claim-to-evidence map is in docs/invariants.md, recovery statistics in docs/recovery.md, and formal scope in the dev/formal documentation.

Development

task ci runs the local correctness suite; see the Taskfile for the full gate list. Rust, C++, and TLA+ are development-time oracles behind the nested dev module and never touch production builds or module downloads. Contribution and evidence expectations are in CONTRIBUTING.md; mutation policy in docs/mutation-testing.md. End-to-end validation lives in the companion repository raptorgo-e2e.

Attribution and license

RaptorQ is the work of Amin Shokrollahi, Michael Luby, and their collaborators, standardized by the IETF as RFC 6330. This repository is an independent implementation and claims no rights in the RaptorQ name or technology; any defects are this project's own.

Copyright 2026 FGN Research and Development B.V. Licensed under the Apache License 2.0; third-party material is listed in THIRD_PARTY_NOTICES.md.

Patent notice. IETF IPR disclosures exist for RFC 6330 (see the disclosure search). Apache-2.0 grants patent rights only from this project's contributors and cannot grant rights to third-party patents. Evaluate your own use; this is information, not legal advice.

Documentation

Overview

Package raptorgo implements the RaptorQ forward error correction scheme from RFC 6330 in pure Go.

ObjectEncoder produces systematic and repair encoding packets for an object described by an OTI. ObjectDecoder accepts single-symbol or grouped encoding packets incrementally and requires an application verifier before it invokes delivery. FEC recovery is not an authenticity check; callers remain responsible for packet and object trust.

All allocation-sensitive public operations take Limits or retain limits from construction. DefaultLimits is conservative operational policy; RFCWireLimits is intended for parsing and conformance across the complete RFC wire domain.

All errors returned by this package wrap one of the exported sentinel errors and can be matched with errors.Is.

RaptorQ is the work of its inventors and the RFC 6330 authors; this package is an independent implementation and claims no rights in the RaptorQ code or technology. See the repository README for attribution, the project's research status, and exactly which properties have been verified by which method.

Example 
package main

import (
	"bytes"
	"fmt"
	"io"

	"github.com/fgn/raptorgo"
)

func main() {
	object := []byte("hello, RaptorQ")
	limits := raptorgo.DefaultLimits()
	oti, err := raptorgo.DeriveOTI(raptorgo.DerivationInput{
		TransferLength: uint64(len(object)), DecoderBlockBytes: 1 << 20,
		MaxPayloadSize: 8, SymbolAlignment: 1, MinSubSymbolAlignmentUnits: 1,
	}, limits)
	if err != nil {
		panic(err)
	}
	encoder, err := raptorgo.NewObjectEncoder(object, oti, limits)
	if err != nil {
		panic(err)
	}

	decoder, err := raptorgo.NewObjectDecoder(
		oti, limits, "generation-7",
		func(request raptorgo.VerificationRequest) error {
			candidate, readErr := io.ReadAll(request.Object)
			if readErr != nil {
				return readErr
			}
			if request.Context != "generation-7" || !bytes.Equal(candidate, object) {
				return fmt.Errorf("object authentication failed")
			}
			return nil
		},
		func(verified raptorgo.VerifiedObject) error {
			fmt.Println(string(verified.Bytes))
			return nil
		},
	)
	if err != nil {
		panic(err)
	}
	const shortenFinal = true
	for sourceBlockIndex := 0; sourceBlockIndex < int(oti.SourceBlocks); sourceBlockIndex++ {
		sourceBlock := uint8(sourceBlockIndex)
		k, symbolsErr := raptorgo.SourceBlockSymbols(oti, sourceBlock)
		if symbolsErr != nil {
			panic(symbolsErr)
		}
		packet, packetErr := encoder.Packet(sourceBlock, 0, k, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
	}

}

Output:
hello, RaptorQ
Example (AuthenticatedTransfer)

Example_authenticatedTransfer is the recommended receiver pattern. The sender transmits the 12-byte OTI and a SHA-256 digest over an authenticated out-of-band channel, the network loses one source packet in five, repair packets (ESI >= K) stand in for the losses, and the receiver delivers the object only after the digest verifies. FEC repairs erasures; it never authenticates bytes, so the verifier compares against the out-of-band digest rather than trusting FEC success. 

package main

import (
	"bytes"
	"crypto/sha256"
	"errors"
	"fmt"
	"io"

	"github.com/fgn/raptorgo"
)

func main() {
	// Sender: derive an OTI, build the encoder, and publish the OTI and
	// object digest through an authenticated out-of-band channel.
	object := bytes.Repeat([]byte("raptorgo"), 512)
	limits := raptorgo.DefaultLimits()
	oti, err := raptorgo.DeriveOTI(raptorgo.DerivationInput{
		TransferLength:             uint64(len(object)),
		DecoderBlockBytes:          1 << 20,
		MaxPayloadSize:             64,
		SymbolAlignment:            4,
		MinSubSymbolAlignmentUnits: 16,
	}, limits)
	if err != nil {
		panic(err)
	}
	encoder, err := raptorgo.NewObjectEncoder(object, oti, limits)
	if err != nil {
		panic(err)
	}
	otiWire, err := oti.MarshalBinary()
	if err != nil {
		panic(err)
	}
	digest := sha256.Sum256(object)

	// Receiver: parse the announced OTI against local limits and refuse
	// delivery unless the reconstructed object matches the announced digest.
	receivedOTI, err := raptorgo.ParseOTI(otiWire, limits)
	if err != nil {
		panic(err)
	}
	decoder, err := raptorgo.NewObjectDecoder(
		receivedOTI, limits, "transfer-42",
		func(request raptorgo.VerificationRequest) error {
			hash := sha256.New()
			if _, copyErr := io.Copy(hash, request.Object); copyErr != nil {
				return copyErr
			}
			if !bytes.Equal(hash.Sum(nil), digest[:]) {
				return errors.New("SHA-256 digest mismatch")
			}
			return nil
		},
		func(verified raptorgo.VerifiedObject) error {
			fmt.Printf("delivered %d verified bytes\n", len(verified.Bytes))
			return nil
		},
	)
	if err != nil {
		panic(err)
	}

	// Network: every fifth source packet is lost in transit.
	k, err := raptorgo.SourceBlockSymbols(receivedOTI, 0)
	if err != nil {
		panic(err)
	}
	const shortenFinal = true
	lost := uint32(0)
	for esi := uint32(0); esi < k; esi++ {
		if esi%5 == 4 {
			lost++
			continue
		}
		packet, packetErr := encoder.Packet(0, esi, 1, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
	}
	// Repair symbols begin at ESI K; emit them until the decoder finishes.
	repairs := uint32(0)
	for esi := k; decoder.State() == raptorgo.ObjectReceiving; esi++ {
		packet, packetErr := encoder.Packet(0, esi, 1, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
		repairs++
	}
	fmt.Printf("state=%s lost=%d repairs=%d\n", decoder.State(), lost, repairs)

}

Output:
delivered 4096 verified bytes
state=delivered lost=12 repairs=12
Example (LossRecovery)

Example_lossRecovery loses one source packet in transit and recovers the object from a repair packet (ESI >= K) instead. 

package main

import (
	"bytes"
	"fmt"
	"io"

	"github.com/fgn/raptorgo"
)

func main() {
	object := []byte("recovered by RaptorQ")
	limits := raptorgo.DefaultLimits()
	oti, err := raptorgo.DeriveOTI(raptorgo.DerivationInput{
		TransferLength: uint64(len(object)), DecoderBlockBytes: 1 << 20,
		MaxPayloadSize: 8, SymbolAlignment: 1, MinSubSymbolAlignmentUnits: 1,
	}, limits)
	if err != nil {
		panic(err)
	}
	encoder, err := raptorgo.NewObjectEncoder(object, oti, limits)
	if err != nil {
		panic(err)
	}

	decoder, err := raptorgo.NewObjectDecoder(
		oti, limits, "generation-8",
		func(request raptorgo.VerificationRequest) error {
			candidate, readErr := io.ReadAll(request.Object)
			if readErr != nil {
				return readErr
			}
			if request.Context != "generation-8" || !bytes.Equal(candidate, object) {
				return fmt.Errorf("object authentication failed")
			}
			return nil
		},
		func(verified raptorgo.VerifiedObject) error {
			fmt.Println(string(verified.Bytes))
			return nil
		},
	)
	if err != nil {
		panic(err)
	}

	// This small object fits one source block with K source symbols.
	k, err := raptorgo.SourceBlockSymbols(oti, 0)
	if err != nil {
		panic(err)
	}
	const shortenFinal = true
	// Deliver all source packets except the last one, which is lost.
	for esi := uint32(0); esi+1 < k; esi++ {
		packet, packetErr := encoder.Packet(0, esi, 1, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
	}
	// Repair packets (ESI >= K) stand in for the lost source packet.
	for esi := k; decoder.State() == raptorgo.ObjectReceiving; esi++ {
		packet, packetErr := encoder.Packet(0, esi, 1, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
	}
	fmt.Println(decoder.State())

}

Output:
recovered by RaptorQ
delivered

Index

Examples

Constants

View Source 
const (
	// MaxTransferLength is the largest RFC 6330 transfer length F in bytes,
	// as corrected by verified RFC Editor erratum 5548.
	MaxTransferLength uint64 = 942_574_504_275
	// MaxSourceSymbolsPerBlock is K'_max, the largest number of source
	// symbols in one source block, from RFC 6330 Section 5.1.2.
	MaxSourceSymbolsPerBlock = 56_403
	// MaxIntermediateSymbols is the largest number of intermediate symbols
	// L that any source block can require, reached at K'_max.
	MaxIntermediateSymbols uint32 = 57_326
	// MaxEncodingSymbolID is the largest encoding symbol ID that fits the
	// 24-bit wire field from RFC 6330 Section 3.2.
	MaxEncodingSymbolID = 16_777_215
)
View Source 
const OTIEncodedSize = 12

OTIEncodedSize is the encoded size in bytes of the combined RFC 6330 Common and Scheme-Specific FEC Object Transmission Information produced by OTI.MarshalBinary and consumed by ParseOTI.

View Source 
const PayloadIDEncodedSize = 4

PayloadIDEncodedSize is the encoded size in bytes of the RFC 6330 FEC Payload ID that prefixes every encoding packet.

Variables

View Source 
var (
	// ErrInvalidLimits is returned when a Limits value has a zero,
	// out-of-range, or otherwise unsafe field.
	ErrInvalidLimits = errors.New("raptorgo: invalid limits")
	// ErrLimitExceeded is returned when an operation would exceed a
	// caller-supplied Limits field, the RFC 6330 wire domain, or the
	// address space of the platform.
	ErrLimitExceeded = errors.New("raptorgo: resource limit exceeded")
	// ErrInvalidOTI is returned when Object Transmission Information
	// violates an RFC 6330 constraint or has the wrong encoded length.
	ErrInvalidOTI = errors.New("raptorgo: invalid object transmission information")
	// ErrInvalidDerivation is returned when DeriveOTI input is inconsistent
	// or admits no valid RFC 6330 Section 4.3 parameter derivation.
	ErrInvalidDerivation = errors.New("raptorgo: invalid RFC parameter derivation input")
	// ErrInvalidPayloadID is returned when a FEC Payload ID has the wrong
	// encoded length or an encoding symbol ID that exceeds 24 bits.
	ErrInvalidPayloadID = errors.New("raptorgo: invalid FEC payload ID")
	// ErrInvalidPacket is returned when an encoding packet violates
	// RFC 6330 grouped-symbol or shortening semantics, targets an invalid
	// or empty source block, or requests an out-of-range symbol.
	ErrInvalidPacket = errors.New("raptorgo: invalid encoding packet")
	// ErrInvalidObject is returned when a source object does not match its
	// OTI, for example when its length differs from the transfer length.
	ErrInvalidObject = errors.New("raptorgo: invalid source object")
	// ErrObjectEncoder is returned when object encoding fails, including
	// nil-encoder use and internal source-block encoder failures.
	ErrObjectEncoder = errors.New("raptorgo: object encoder failed")
	// ErrObjectDecoder is returned when object decoding fails, including
	// nil-decoder use, construction preflight failures, and internal
	// source-block decoder failures.
	ErrObjectDecoder = errors.New("raptorgo: object decoder failed")
	// ErrObjectVerification is returned when the application verifier
	// rejects a reconstructed object.
	ErrObjectVerification = errors.New("raptorgo: decoded object verification failed")
	// ErrObjectDelivery is returned when the application deliverer rejects
	// a verified object.
	ErrObjectDelivery = errors.New("raptorgo: verified object delivery failed")
	// ErrObjectTerminal is returned when a packet is added to an
	// ObjectDecoder that has already left the ObjectReceiving state.
	ErrObjectTerminal = errors.New("raptorgo: object decoder is terminal")
)

Functions

func ObjectDecoderWorkspaceBackingBytes 

func ObjectDecoderWorkspaceBackingBytes(oti OTI, limits Limits) (uint64, error)

ObjectDecoderWorkspaceBackingBytes returns the conservative peak logical backing reserved by NewObjectDecoder and all later operations under limits. It includes persistent block decoders, the largest bounded decode attempt, reconstruction, and one owned delivery copy.

func ObjectEncoderWorkspaceBackingBytes 

func ObjectEncoderWorkspaceBackingBytes(oti OTI, limits Limits) (uint64, error)

ObjectEncoderWorkspaceBackingBytes returns the largest logical backing peak needed to own the object and construct any one repair encoder cache. Packet output backing is checked separately because it depends on each request.

func SourceBlockSymbols 

func SourceBlockSymbols(oti OTI, sourceBlockNumber uint8) (uint32, error)

SourceBlockSymbols returns K for a source block using RFC 6330 Section 4.4.1.2 partitioning.

Types

type DerivationInput 

type DerivationInput struct {
	// TransferLength is F, the object size in bytes, at most
	// MaxTransferLength.
	TransferLength uint64
	// DecoderBlockBytes is WS, the largest source-block size in bytes the
	// receiver is willing to hold in working memory while decoding.
	DecoderBlockBytes uint64
	// MaxPayloadSize is P', the target packet payload size in bytes; it
	// becomes the symbol size T and must be a nonzero multiple of
	// SymbolAlignment.
	MaxPayloadSize uint16
	// SymbolAlignment is Al in bytes, 1 to 255. Symbol and sub-symbol sizes
	// are multiples of it; 4 suits most transports.
	SymbolAlignment uint8
	// MinSubSymbolAlignmentUnits is SS: the smallest allowed sub-symbol is
	// SS*Al bytes, which must not exceed MaxPayloadSize.
	MinSubSymbolAlignmentUnits uint32
}

DerivationInput contains the inputs to the RFC 6330 Section 4.3 example parameter derivation.

type EncodingPacketView 

type EncodingPacketView struct {
	// contains filtered or unexported fields
}

EncodingPacketView is a validated, zero-copy view of a grouped RFC 6330 source or repair packet. The caller must not mutate the input byte slice while using the view.

func ParseEncodingPacket 

func ParseEncodingPacket(encoded []byte, oti OTI, limits Limits) (EncodingPacketView, error)

ParseEncodingPacket validates grouped-symbol semantics from RFC 6330 Section 4.4.2, including the one permitted shortened final source symbol.

func (EncodingPacketView) CopySymbol 

func (packet EncodingPacketView) CopySymbol(index uint32, destination []byte) error

CopySymbol copies a symbol into a canonical T-byte destination, adding the mandatory zero padding when the final source symbol was shortened on wire.

func (EncodingPacketView) IsSource 

func (packet EncodingPacketView) IsSource() bool

IsSource reports whether the packet carries source symbols rather than repair symbols.

func (EncodingPacketView) Payload 

func (packet EncodingPacketView) Payload() []byte

Payload returns the symbol payload that follows the FEC Payload ID. The returned bytes alias the parsed input and must not be mutated.

func (EncodingPacketView) PayloadID 

func (packet EncodingPacketView) PayloadID() PayloadID

PayloadID returns the packet's parsed FEC Payload ID.

func (EncodingPacketView) Symbol 

func (packet EncodingPacketView) Symbol(index uint32) ([]byte, error)

Symbol returns a read-only view of one encoding symbol within the payload. A legally shortened final source symbol is returned without its trailing zero padding, so the view may be shorter than the symbol size T; use CopySymbol to obtain the canonical T-byte form.

func (EncodingPacketView) SymbolCount 

func (packet EncodingPacketView) SymbolCount() uint32

SymbolCount returns the number of encoding symbols grouped in the packet.

type Limits 

type Limits struct {
	// MaxTransferLength bounds OTI.TransferLength (F) in bytes; it may not
	// exceed the RFC ceiling MaxTransferLength.
	MaxTransferLength uint64
	// MaxSymbolSize bounds OTI.SymbolSize (T) in bytes.
	MaxSymbolSize uint16
	// MaxSourceBlocks bounds OTI.SourceBlocks (Z).
	MaxSourceBlocks uint8
	// MaxSubBlocks bounds OTI.SubBlocks (N).
	MaxSubBlocks uint16
	// MaxSymbolAlignment bounds OTI.SymbolAlignment (Al) in bytes.
	MaxSymbolAlignment uint8
	// MaxSourceSymbolsPerBlock bounds K, the source symbols in any single
	// source block; it may not exceed K'_max (MaxSourceSymbolsPerBlock).
	MaxSourceSymbolsPerBlock uint32
	// MaxSourceBlockBytes bounds K*T, the size in bytes of any single
	// source block.
	MaxSourceBlockBytes uint64
	// MaxPacketBytes bounds the encoded size in bytes of one encoding
	// packet, including its FEC Payload ID.
	MaxPacketBytes uint64
	// MaxSymbolsPerPacket bounds G, the symbols grouped into one packet.
	MaxSymbolsPerPacket uint32
	// MaxRetainedSymbols bounds the source plus repair symbols one block
	// decoder retains at once. It must be at least K for every block the
	// OTI derives, or NewObjectDecoder fails.
	MaxRetainedSymbols uint32
	// MaxDecodeAttempts bounds bounded decode attempts per source block
	// before the decoder gives up.
	MaxDecodeAttempts uint32
	// MaxRepairSymbolsPerBlock bounds the repair symbols retained per source
	// block beyond its K source symbols.
	MaxRepairSymbolsPerBlock uint32
	// MaxDenseCompletionSymbols bounds the dense-completion stage of the
	// hybrid solver, at most MaxIntermediateSymbols. It must be at least P,
	// the block's PI symbol count, or NewObjectDecoder fails.
	MaxDenseCompletionSymbols uint32
	// MaxWorkingMemory bounds the peak logical backing bytes of any single
	// encoder or decoder operation, computed before allocation.
	MaxWorkingMemory uint64
}

Limits bounds all sizes controlled by OTI or packet data before allocation. Every allocation-sensitive operation validates externally controlled input against a Limits value taken as an argument or retained from construction. Every field must be nonzero; see Validate.

func DefaultLimits 

func DefaultLimits() Limits

DefaultLimits returns conservative production defaults. They intentionally reject some syntactically valid, extremely large RFC configurations. Applications should lower these defaults when their workload permits it.

func RFCWireLimits 

func RFCWireLimits() Limits

RFCWireLimits admits every wire value allowed by RFC 6330. It is useful for serialization and conformance tests, not as an allocation policy.

func (Limits) Validate 

func (limits Limits) Validate() error

Validate checks that every field is nonzero and within the RFC 6330 wire domain. It returns an error wrapping ErrInvalidLimits that names the first offending field, or nil when all fields are usable.

type OTI 

type OTI struct {
	// TransferLength is F, the object size in bytes, at most
	// MaxTransferLength. Zero describes a zero-length object.
	TransferLength uint64
	// SymbolSize is T, the encoding symbol size in bytes, 1 to 65535 and a
	// multiple of SymbolAlignment.
	SymbolSize uint16
	// SourceBlocks is Z, the number of source blocks, 1 to 255.
	SourceBlocks uint8
	// SubBlocks is N, the number of sub-blocks per source block, 1 to 65535.
	SubBlocks uint16
	// SymbolAlignment is Al in bytes, 1 to 255; SymbolSize and every
	// sub-symbol size are multiples of it.
	SymbolAlignment uint8
}

OTI is the RFC 6330 FEC Object Transmission Information: the parameter contract between encoder and decoder. It travels out of band and must be authenticated by the application before use.

func DeriveOTI 

func DeriveOTI(input DerivationInput, limits Limits) (OTI, error)

DeriveOTI applies the RFC 6330 Section 4.3 example algorithm. Applications may instead supply an OTI selected by their content-delivery protocol.

func ParseOTI 

func ParseOTI(encoded []byte, limits Limits) (OTI, error)

ParseOTI decodes an OTIEncodedSize-byte RFC 6330 encoded OTI and validates the result against both RFC constraints and the supplied limits before returning it.

func (OTI) MarshalBinary 

func (oti OTI) MarshalBinary() ([]byte, error)

MarshalBinary encodes the OTI into its OTIEncodedSize-byte RFC 6330 wire form. It validates against RFCWireLimits, not caller limits, so every wire-legal OTI can be serialized. ParseOTI is the limits-aware inverse; there is deliberately no UnmarshalBinary, so parsing untrusted OTI bytes always takes explicit Limits.

func (OTI) Validate 

func (oti OTI) Validate(limits Limits) error

Validate checks RFC constraints and application resource limits before any object, symbol, or matrix allocation.

type ObjectDecoder 

type ObjectDecoder struct {
	// contains filtered or unexported fields
}

ObjectDecoder incrementally decodes every nonempty source block and enforces decoded -> verified -> delivered ordering. FEC success alone never invokes delivery. It is safe for concurrent method calls. Callbacks run without the decoder mutex held, so they may inspect or call the decoder; packet additions made after finalization starts are rejected as terminal.

func NewObjectDecoder 

func NewObjectDecoder(
	oti OTI,
	limits Limits,
	context string,
	verify ObjectVerifier,
	deliver ObjectDeliverer,
) (*ObjectDecoder, error)

NewObjectDecoder preflights all persistent block decoders, their largest bounded attempt, reconstruction, verification, and one owned delivery copy before allocating retained packet storage.

A zero-length object (OTI.TransferLength 0) has no nonempty source blocks and therefore needs no packets: verify and deliver run synchronously on the empty object before NewObjectDecoder returns. In that fast path the decoder is returned non-nil even on failure, in the corresponding terminal state, together with an error wrapping ErrObjectVerification or ErrObjectDelivery; on success it is returned already in the ObjectDelivered state.

func (*ObjectDecoder) AddPacket 

func (decoder *ObjectDecoder) AddPacket(encoded []byte) (ObjectDecoderState, error)

AddPacket validates and copies every grouped symbol. Once all blocks decode, it synchronously verifies and invokes delivery exactly once. AddPacket on a nil decoder reports ObjectDeliveryFailed with an error wrapping ErrObjectDecoder.

func (*ObjectDecoder) CompletedBlocks 

func (decoder *ObjectDecoder) CompletedBlocks() uint32

CompletedBlocks reports how many nonempty source blocks have fully decoded so far. CompletedBlocks on a nil decoder reports 0.

func (*ObjectDecoder) PersistentBackingBytes 

func (decoder *ObjectDecoder) PersistentBackingBytes() uint64

PersistentBackingBytes reports the logical backing bytes reserved by the persistent per-block decoders for the lifetime of this decoder, excluding transient decode-attempt and reconstruction peaks. PersistentBackingBytes on a nil decoder reports 0.

func (*ObjectDecoder) State 

func (decoder *ObjectDecoder) State() ObjectDecoderState

State reports the decoder's current lifecycle state. State on a nil decoder reports ObjectDeliveryFailed.

type ObjectDecoderState 

type ObjectDecoderState uint8

ObjectDecoderState makes the authenticated-delivery boundary explicit. Once a decoder leaves ObjectReceiving it never accepts another packet; AddPacket then fails with ErrObjectTerminal. 

const (
	// ObjectReceiving is the initial state: the decoder accepts packets and
	// has not yet decoded every nonempty source block.
	ObjectReceiving ObjectDecoderState = iota
	// ObjectDecodeFailed is terminal: FEC decoding or reconstruction failed
	// permanently.
	ObjectDecodeFailed
	// ObjectDecoded is a transient finalization state: every nonempty source
	// block decoded and the reconstructed object awaits verification.
	ObjectDecoded
	// ObjectVerified is a transient finalization state: the verifier accepted
	// the reconstructed object and delivery is about to run.
	ObjectVerified
	// ObjectDelivered is terminal: the deliverer received the verified object
	// exactly once and accepted it.
	ObjectDelivered
	// ObjectVerificationFailed is terminal: the application verifier rejected
	// the reconstructed object.
	ObjectVerificationFailed
	// ObjectDeliveryFailed is terminal: the application deliverer rejected
	// the verified object.
	ObjectDeliveryFailed
)

func (ObjectDecoderState) String 

func (state ObjectDecoderState) String() string

String returns a short lower-case name for the state, such as "receiving", "delivered", or "verification-failed".

type ObjectDeliverer 

type ObjectDeliverer func(VerifiedObject) error

ObjectDeliverer receives the reconstructed object exactly once, after the verifier has accepted it. Returning a non-nil error moves the decoder to ObjectDeliveryFailed.

type ObjectEncoder 

type ObjectEncoder struct {
	// contains filtered or unexported fields
}

ObjectEncoder owns a source object and emits canonical grouped RFC 6330 source or repair packets. It retains at most one repair encoder cache and is safe for concurrent Packet calls.

func NewObjectEncoder 

func NewObjectEncoder(object []byte, oti OTI, limits Limits) (*ObjectEncoder, error)

NewObjectEncoder validates and owns object. Repair-plan construction is preflighted for every distinct block size, but is deferred until the first repair packet for a block is requested.

func (*ObjectEncoder) OTI 

func (encoder *ObjectEncoder) OTI() OTI

OTI returns the Object Transmission Information supplied at construction. OTI on a nil encoder returns the zero OTI.

func (*ObjectEncoder) Packet 

func (encoder *ObjectEncoder) Packet(
	sourceBlockNumber uint8,
	firstESI, symbolCount uint32,
	shortenFinal bool,
) ([]byte, error)

Packet returns one owned grouped packet beginning at firstESI. A group must lie wholly in the source range or wholly in the repair range. When shortenFinal is true, trailing FEC padding is omitted from the packet's last source symbol when RFC 6330 permits it. Packet on a nil encoder returns an error wrapping ErrObjectEncoder.

func (*ObjectEncoder) PacketInto 

func (encoder *ObjectEncoder) PacketInto(
	destination []byte,
	sourceBlockNumber uint8,
	firstESI, symbolCount uint32,
	shortenFinal bool,
) ([]byte, error)

PacketInto writes the same canonical packet as Packet into destination and returns its exact-length view. Destination must have room for the four-byte payload ID and the requested symbols; passing a full-sized buffer is valid when shortenFinal may produce a shorter result. The returned bytes alias destination, whose unused tail is left unchanged. A source-packet call does not allocate when destination is large enough. Concurrent calls are safe, but callers must use distinct destination buffers for concurrently live results and must not mutate a buffer until its call returns.

Example 
package main

import (
	"fmt"

	"github.com/fgn/raptorgo"
)

func main() {
	object := []byte("abcdefgh")
	oti := raptorgo.OTI{
		TransferLength: 8, SymbolSize: 4,
		SourceBlocks: 1, SubBlocks: 1, SymbolAlignment: 1,
	}
	encoder, err := raptorgo.NewObjectEncoder(object, oti, raptorgo.DefaultLimits())
	if err != nil {
		panic(err)
	}

	// Reuse this buffer for successive source packets. PacketInto returns the
	// exact packet-length prefix and performs no heap allocation on this path.
	buffer := make([]byte, raptorgo.PayloadIDEncodedSize+int(oti.SymbolSize))
	packet, err := encoder.PacketInto(buffer, 0, 1, 1, false)
	if err != nil {
		panic(err)
	}
	view, err := raptorgo.ParseEncodingPacket(packet, oti, raptorgo.DefaultLimits())
	if err != nil {
		panic(err)
	}
	fmt.Printf("payload=%q aliases-buffer=%v\n", view.Payload(), &packet[0] == &buffer[0])

}

Output:
payload="efgh" aliases-buffer=true

type ObjectVerifier 

type ObjectVerifier func(VerificationRequest) error

ObjectVerifier authenticates a reconstructed object before delivery. Returning a non-nil error rejects the object and moves the decoder to ObjectVerificationFailed; the deliverer is then never invoked.

type PayloadID 

type PayloadID struct {
	SourceBlockNumber uint8
	EncodingSymbolID  uint32
}

PayloadID is the four-octet FEC Payload ID from RFC 6330 Section 3.2.

func NewPayloadID 

func NewPayloadID(sourceBlockNumber uint8, encodingSymbolID uint32) (PayloadID, error)

NewPayloadID constructs a PayloadID, rejecting encoding symbol IDs that do not fit the 24-bit wire field.

func ParsePayloadID 

func ParsePayloadID(encoded []byte) (PayloadID, error)

ParsePayloadID decodes a four-octet RFC 6330 FEC Payload ID.

func (PayloadID) MarshalBinary 

func (id PayloadID) MarshalBinary() ([]byte, error)

MarshalBinary encodes the payload ID into its four-octet RFC 6330 wire form: the source block number followed by the 24-bit big-endian encoding symbol ID.

type SourceBlock 

type SourceBlock struct {
	// contains filtered or unexported fields
}

SourceBlock contains one RFC 6330 source block in symbol-major order. Its storage owns a copy of the corresponding source-object bytes and any zero padding required for the final source symbol.

func BuildSourceBlock 

func BuildSourceBlock(object []byte, oti OTI, sourceBlockNumber uint8, limits Limits) (SourceBlock, error)

BuildSourceBlock constructs one source block from a complete source object. For N > 1, it converts RFC sub-block-major input ordering into symbol-major storage. All externally controlled sizes are validated before allocation.

func (SourceBlock) Number 

func (block SourceBlock) Number() uint8

Number returns the block's source block number (SBN).

func (SourceBlock) Symbol 

func (block SourceBlock) Symbol(index uint32) ([]byte, error)

Symbol returns a view into the block-owned storage. Callers must not mutate the returned bytes.

func (SourceBlock) SymbolCount 

func (block SourceBlock) SymbolCount() uint32

SymbolCount returns K, the number of source symbols in the block.

func (SourceBlock) SymbolSize 

func (block SourceBlock) SymbolSize() uint16

SymbolSize returns the symbol size T in bytes.

type VerificationRequest 

type VerificationRequest struct {
	// OTI describes the reconstructed object being verified.
	OTI OTI
	// Context is the identifier supplied to NewObjectDecoder, so one shared
	// verifier can distinguish concurrent transfers.
	Context string
	// Length equals OTI.TransferLength and exists for convenience.
	Length uint64
	// Object streams the reconstructed bytes. It is valid only for the
	// duration of the verifier call and must not be retained.
	Object io.Reader
}

VerificationRequest binds reconstructed bytes to their OTI and an application-supplied generation/context identifier. Object is read-only.

type VerifiedObject 

type VerifiedObject struct {
	// OTI describes the delivered object.
	OTI OTI
	// Context is the identifier supplied to NewObjectDecoder.
	Context string
	// Bytes is an owned copy of the verified object, exactly
	// OTI.TransferLength long; the callback may retain and mutate it.
	Bytes []byte
}

VerifiedObject is passed exactly once to the delivery callback after its corresponding VerificationRequest succeeds. Bytes are owned by the callback.

Source Files

View all Source files

Directories

Path Synopsis
examples
roundtrip command
Command roundtrip demonstrates a complete raptorgo transfer over a simulated lossy channel.
 Command roundtrip demonstrates a complete raptorgo transfer over a simulated lossy channel.
internal
codec
Package codec implements RFC 6330 source-block constraint construction, intermediate-symbol generation, raw encoding, and decoding substrate.
 Package codec implements RFC 6330 source-block constraint construction, intermediate-symbol generation, raw encoding, and decoding substrate.
gf256
Package gf256 implements the octet arithmetic defined by RFC 6330 Section 5.7.
 Package gf256 implements the octet arithmetic defined by RFC 6330 Section 5.7.
matrix
Package matrix implements bounded sparse matrix and permutation primitives used by the RFC 6330 constraint builder and inactivation schedule.
 Package matrix implements bounded sparse matrix and permutation primitives used by the RFC 6330 constraint builder and inactivation schedule.
rfc6330
Package rfc6330 contains deterministic formulas and constants from RFC 6330.
 Package rfc6330 contains deterministic formulas and constants from RFC 6330.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.