raptorgo

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

README

raptorgo

Go Reference CI License

raptorgo is a bounded, pure-Go implementation of the RaptorQ forward error correction scheme in RFC 6330. The runtime package has no CGo or third-party dependencies.

RaptorQ solves one problem well: delivering an object over a lossy one-way channel without retransmission. The sender turns the object into a stream of equal-sized encoding packets; the receiver reconstructs the object from (almost exactly) any subset whose size matches the original — regardless of which packets were lost. Reception overhead is typically zero to two extra packets per block.

When to use it

Reach for RaptorQ instead of fixed-rate erasure codes such as Reed–Solomon shards when:

  • The loss rate is unknown or varies. Reed–Solomon fixes n and k up front; RaptorQ generates fresh repair packets on demand (a 24-bit symbol ID space per block), so the sender adapts redundancy live or cycles a data carousel indefinitely.
  • Blocks are large. GF(256) Reed–Solomon tops out at 256 shards; a RaptorQ source block spans up to 56,403 symbols, and an object up to 255 blocks.
  • Receivers join independently. Multicast or broadcast receivers each lose different packets; any of them completes once it has collected roughly K packets of each block.

Prefer plain retransmission (TCP/QUIC) when you have a low-latency return channel, and fixed Reed–Solomon shards when the shard count is small and static (for example, striping across disks).

Install

go get github.com/fgn/raptorgo

Requires Go 1.26 or newer. Pure Go on every platform; CI runs amd64, 386, race, and checkptr gates.

Quickstart

A complete transfer: the 12-byte OTI (the codec parameters) and a SHA-256 digest travel over an authenticated out-of-band channel, encoding packets travel over the lossy channel, and the receiver delivers the object only after the digest verifies.

// Sender: derive parameters, build the encoder, publish OTI + digest.
limits := raptorgo.DefaultLimits()
oti, err := raptorgo.DeriveOTI(raptorgo.DerivationInput{
    TransferLength:             uint64(len(object)),
    DecoderBlockBytes:          1 << 20, // largest block decoded in memory
    MaxPayloadSize:             1280,    // symbol size; fit your path MTU
    SymbolAlignment:            4,
    MinSubSymbolAlignmentUnits: 8,
}, limits)
encoder, err := raptorgo.NewObjectEncoder(object, oti, limits)
otiWire, err := oti.MarshalBinary()
digest := sha256.Sum256(object)

// Receiver: parse the OTI against local limits; verify before delivery.
receivedOTI, err := raptorgo.ParseOTI(otiWire, limits)
decoder, err := raptorgo.NewObjectDecoder(receivedOTI, limits, "transfer-42",
    func(request raptorgo.VerificationRequest) error {
        hash := sha256.New()
        if _, err := io.Copy(hash, request.Object); err != nil {
            return err
        }
        if !bytes.Equal(hash.Sum(nil), digest[:]) {
            return errors.New("digest mismatch")
        }
        return nil
    },
    func(verified raptorgo.VerifiedObject) error {
        result = verified.Bytes // owned copy, delivered exactly once
        return nil
    },
)

// Transfer: source symbols are ESI 0..K-1; repair symbols start at ESI K.
k, err := raptorgo.SourceBlockSymbols(receivedOTI, 0)
for esi := uint32(0); esi < k; esi++ {
    packet, err := encoder.Packet(0, esi, 1, true)
    // ... transmit; lost packets are simply never added ...
    _, err = decoder.AddPacket(packet)
}
for esi := k; decoder.State() == raptorgo.ObjectReceiving; esi++ {
    packet, err := encoder.Packet(0, esi, 1, true) // repair symbol
    _, err = decoder.AddPacket(packet)
}
// decoder.State() == raptorgo.ObjectDelivered

Allocation-sensitive senders can use PacketInto to write packets into a reusable caller buffer instead of receiving an owned allocation per call. Runnable versions of both patterns live in the package examples and in examples/roundtrip, a standalone program that simulates a lossy channel end to end:

go run ./examples/roundtrip -size 1048576 -mtu 1280 -loss 10

docs/transport.md covers integration with a real transport: framing, ESI ranges, repair emission strategies, and resource limits.

Callers may also supply an OTI chosen by their content-delivery protocol; DeriveOTI implements the RFC's recommended example algorithm rather than a mandatory transport policy. DefaultLimits is conservative application policy, while RFCWireLimits admits the full wire domain for parsing and conformance work.

Trust boundary

FEC repairs erasures; it does not authenticate hostile packets or reconstructed bytes. The public decoder deliberately requires both a verifier and a deliverer and enforces decoded -> verified -> delivered: FEC success alone never triggers delivery. Authenticate the OTI and the object digest out of band, protect packet integrity in transit when the network is hostile, and treat the verifier as the last line of defense. All decoder resources are bounded by explicit Limits checked before allocation, so a forged OTI is refused before it can consume memory. See SECURITY.md for the threat model.

Performance and practical limits

Systematic packets are emitted in well under a microsecond regardless of block size (about 80 ns with a reused PacketInto buffer), cached repair symbols cost a few microseconds each at T=1280, the one-time repair plan per block builds at 13–21 MB/s of object bytes, and end-to-end decoding with ten percent loss runs at 11–19 MB/s (AMD Ryzen 5 3600, go1.26.5, single-threaded, scalar GF(256)). Measured tables, the exact reproduction command, and block-size guidance are in docs/performance.md.

Practical limits: the API is in-memory — object-sized buffers on both ends, preflighted against Limits — not a streaming file API; one source block spans at most 56,403 symbols; transfers are capped at 942,574,504,275 bytes (RFC erratum 5548 applied).

Implemented behavior

The implementation covers:

  • RFC Section 4.3 parameter derivation and canonical 12-byte OTI handling;
  • source-block and sub-block partitioning for Z >= 1 and N >= 1;
  • grouped source and repair packets, including legal shortened source symbols;
  • the complete RFC parameter tables, Rand, Deg, Tuple, and ESI/ISI map;
  • systematic and arbitrary repair encoding through K=56,403;
  • bounded hybrid binary/HDPC inactivation decoding through K=56,403;
  • incremental multi-block object recovery with loss, reorder, and duplicates;
  • an explicit decoded -> verified -> delivered trust boundary; and
  • checked limits for packet, retained-symbol, attempt, dense-completion, and aggregate logical backing allocations.

Correctness evidence and methodology

The implementation is checked in independent layers:

  1. strict extraction and SHA-256 pinning of the RFC and its normative tables;
  2. exhaustive GF(256), mapping, tuple-domain, and structural invariants;
  3. dense algebraic oracles against the bounded production hybrid solver;
  4. byte-identical Rust parity through K=56,403 and selected T=255 cases;
  5. bidirectional Rust cross-decoding for grouped/shortened, multi-block, multi-sub-block, source-loss, and grouped-repair frames;
  6. bidirectional C++ cross-decoding in the K=10/12/48 consensus domain for repair-only, mixed loss/reorder/duplicate, and deficient sets;
  7. independent rank-sufficiency checks and a bounded preregistered Section 5.8 K'=10 fixed-pool recovery profile, explicitly not an all-row proof;
  8. bounded TLA+ models plus production-derived positive traces and forged or deleted negative traces; and
  9. a pinned mutation-testing gate over critical resource, wire, GF(256), and partition kernels, plus broader changed-line and scheduled audits.

The public object decoder additionally carries a data-driven trace pipeline: every committed harvested execution replays through the TLA+ trace specification with machine-generated mutants that must be rejected exactly, the closed action manifest must be fully covered, and model-derived schedules drive the production decoder so that every edge of two exact bounded ObjectDecoderModel graph dumps has a realizable witness. The precise claim is edge coverage of those finite dumps — not exploration of every model behavior or production execution. Current counts and reproduction commands are pinned in docs/invariants.md and the dev/formal documentation.

RFC Editor erratum 5548 is applied: the maximum transfer length is 942574504275.

That evidence has limits. The TLA+ models and trace replays establish bounded safety properties of abstractions and of recorded executions, with negative controls proving the checks are not vacuous; differential vectors establish byte-identical behavior on the domains actually compared; exhaustive tests cover exactly the finite domains they enumerate (GF(256) algebra, parameter tables, ESI/ISI maps); and the Section 5.8 recovery profile is a preregistered, bounded statistical measurement, explicitly not a proof over every RFC row and ESI. None of this amounts to a machine-checked proof of functional correctness of the Go implementation, and no such claim is made.

The lifecycle trace/model oracles are also not RFC 6330 byte-level oracles. RFC source pinning, table extraction, algebraic checks, rank tests, and the Rust/C++ comparisons own that boundary; trace validation owns ordering, terminal-state, callback-binding, and model-to-production witness claims.

The compact evidence index is in docs/invariants.md. Recovery statistics and their explicit limits are in docs/recovery.md. TLA scope is documented in the dev/formal repository documentation, and the reproducible ObjectDecoder measurements, hashes, and exact failure messages are in dev/formal/trace-validation-log.md.

Development

task test
task test:386
task test:race
task mutation:gate
task verify
task oracle:test
task test:release
task bench

task ci runs the ordinary local correctness suite. Rust and C++ are development-time oracles only and are never linked into production Go code. All formal, oracle, mutation, generator, and specification inputs live behind the nested dev module boundary, so normal imports and module downloads do not include them or invoke Cargo, C++, Java, TLA+, Task, or network fetches. See CONTRIBUTING.md for the change-evidence expectations and SECURITY.md for the threat model and reporting channel. Mutation scope and baseline policy are in docs/mutation-testing.md.

End-to-end validation is maintained in the companion repository raptorgo-e2e, which builds a file-transfer tool and a lossy-network test suite exclusively on this module's public API.

Attribution and research status

RaptorQ is not our invention. Raptor codes are the work of Amin Shokrollahi, building on Michael Luby's LT fountain codes; RaptorQ was developed at Digital Fountain and Qualcomm and standardized by the IETF as RFC 6330, authored by M. Luby, A. Shokrollahi, M. Watson, T. Stockhammer, and L. Minder. This repository is an independent implementation of that specification and claims no rights in the RaptorQ code, the RaptorQ name, or the underlying technology. Any defects in this implementation are this project's own.

The project is a research effort in applying formal-methods and differential-verification techniques to a protocol implementation. It should not be read as "formally verified." The TLA+ artifacts model-check bounded abstractions of selected subsystems, replay production-generated traces against those models, and drive model-derived schedules back through the public decoder; they check finite safety properties of models, recorded traces, and bounded witness schedules, not the compiled Go code, and establish no liveness of the real implementation. The exact claim-to-evidence map is in docs/invariants.md, the formal scope and its limits in the dev/formal repository documentation, and the bounded recovery statistics in docs/recovery.md.

License

Copyright 2026 FGN Research and Development B.V. Licensed under the Apache License 2.0. Third-party material distributed with this repository is listed in THIRD_PARTY_NOTICES.md.

Patent notice. RaptorQ is specified by RFC 6330, for which intellectual property rights disclosures have been filed with the IETF (see the IETF IPR disclosures for RFC 6330). The Apache-2.0 license grants patent rights only from this project's contributors; it cannot and does not grant rights to any third-party patents. Evaluate your own use accordingly; this notice is information, not legal advice.

Documentation

Overview

Package raptorgo implements the RaptorQ forward error correction scheme from RFC 6330 in pure Go.

ObjectEncoder produces systematic and repair encoding packets for an object described by an OTI. ObjectDecoder accepts single-symbol or grouped encoding packets incrementally and requires an application verifier before it invokes delivery. FEC recovery is not an authenticity check; callers remain responsible for packet and object trust.

All allocation-sensitive public operations take Limits or retain limits from construction. DefaultLimits is conservative operational policy; RFCWireLimits is intended for parsing and conformance across the complete RFC wire domain.

All errors returned by this package wrap one of the exported sentinel errors and can be matched with errors.Is.

RaptorQ is the work of its inventors and the RFC 6330 authors; this package is an independent implementation and claims no rights in the RaptorQ code or technology. See the repository README for attribution, the project's research status, and exactly which properties have been verified by which method.

Example 
package main

import (
	"bytes"
	"fmt"
	"io"

	"github.com/fgn/raptorgo"
)

func main() {
	object := []byte("hello, RaptorQ")
	limits := raptorgo.DefaultLimits()
	oti, err := raptorgo.DeriveOTI(raptorgo.DerivationInput{
		TransferLength: uint64(len(object)), DecoderBlockBytes: 1 << 20,
		MaxPayloadSize: 8, SymbolAlignment: 1, MinSubSymbolAlignmentUnits: 1,
	}, limits)
	if err != nil {
		panic(err)
	}
	encoder, err := raptorgo.NewObjectEncoder(object, oti, limits)
	if err != nil {
		panic(err)
	}

	decoder, err := raptorgo.NewObjectDecoder(
		oti, limits, "generation-7",
		func(request raptorgo.VerificationRequest) error {
			candidate, readErr := io.ReadAll(request.Object)
			if readErr != nil {
				return readErr
			}
			if request.Context != "generation-7" || !bytes.Equal(candidate, object) {
				return fmt.Errorf("object authentication failed")
			}
			return nil
		},
		func(verified raptorgo.VerifiedObject) error {
			fmt.Println(string(verified.Bytes))
			return nil
		},
	)
	if err != nil {
		panic(err)
	}
	const shortenFinal = true
	for sourceBlockIndex := 0; sourceBlockIndex < int(oti.SourceBlocks); sourceBlockIndex++ {
		sourceBlock := uint8(sourceBlockIndex)
		k, symbolsErr := raptorgo.SourceBlockSymbols(oti, sourceBlock)
		if symbolsErr != nil {
			panic(symbolsErr)
		}
		packet, packetErr := encoder.Packet(sourceBlock, 0, k, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
	}

}

Output:
hello, RaptorQ
Example (AuthenticatedTransfer)

Example_authenticatedTransfer is the recommended receiver pattern. The sender transmits the 12-byte OTI and a SHA-256 digest over an authenticated out-of-band channel, the network loses one source packet in five, repair packets (ESI >= K) stand in for the losses, and the receiver delivers the object only after the digest verifies. FEC repairs erasures; it never authenticates bytes, so the verifier compares against the out-of-band digest rather than trusting FEC success. 

package main

import (
	"bytes"
	"crypto/sha256"
	"errors"
	"fmt"
	"io"

	"github.com/fgn/raptorgo"
)

func main() {
	// Sender: derive an OTI, build the encoder, and publish the OTI and
	// object digest through an authenticated out-of-band channel.
	object := bytes.Repeat([]byte("raptorgo"), 512)
	limits := raptorgo.DefaultLimits()
	oti, err := raptorgo.DeriveOTI(raptorgo.DerivationInput{
		TransferLength:             uint64(len(object)),
		DecoderBlockBytes:          1 << 20,
		MaxPayloadSize:             64,
		SymbolAlignment:            4,
		MinSubSymbolAlignmentUnits: 16,
	}, limits)
	if err != nil {
		panic(err)
	}
	encoder, err := raptorgo.NewObjectEncoder(object, oti, limits)
	if err != nil {
		panic(err)
	}
	otiWire, err := oti.MarshalBinary()
	if err != nil {
		panic(err)
	}
	digest := sha256.Sum256(object)

	// Receiver: parse the announced OTI against local limits and refuse
	// delivery unless the reconstructed object matches the announced digest.
	receivedOTI, err := raptorgo.ParseOTI(otiWire, limits)
	if err != nil {
		panic(err)
	}
	decoder, err := raptorgo.NewObjectDecoder(
		receivedOTI, limits, "transfer-42",
		func(request raptorgo.VerificationRequest) error {
			hash := sha256.New()
			if _, copyErr := io.Copy(hash, request.Object); copyErr != nil {
				return copyErr
			}
			if !bytes.Equal(hash.Sum(nil), digest[:]) {
				return errors.New("SHA-256 digest mismatch")
			}
			return nil
		},
		func(verified raptorgo.VerifiedObject) error {
			fmt.Printf("delivered %d verified bytes\n", len(verified.Bytes))
			return nil
		},
	)
	if err != nil {
		panic(err)
	}

	// Network: every fifth source packet is lost in transit.
	k, err := raptorgo.SourceBlockSymbols(receivedOTI, 0)
	if err != nil {
		panic(err)
	}
	const shortenFinal = true
	lost := uint32(0)
	for esi := uint32(0); esi < k; esi++ {
		if esi%5 == 4 {
			lost++
			continue
		}
		packet, packetErr := encoder.Packet(0, esi, 1, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
	}
	// Repair symbols begin at ESI K; emit them until the decoder finishes.
	repairs := uint32(0)
	for esi := k; decoder.State() == raptorgo.ObjectReceiving; esi++ {
		packet, packetErr := encoder.Packet(0, esi, 1, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
		repairs++
	}
	fmt.Printf("state=%s lost=%d repairs=%d\n", decoder.State(), lost, repairs)

}

Output:
delivered 4096 verified bytes
state=delivered lost=12 repairs=12
Example (LossRecovery)

Example_lossRecovery loses one source packet in transit and recovers the object from a repair packet (ESI >= K) instead. 

package main

import (
	"bytes"
	"fmt"
	"io"

	"github.com/fgn/raptorgo"
)

func main() {
	object := []byte("recovered by RaptorQ")
	limits := raptorgo.DefaultLimits()
	oti, err := raptorgo.DeriveOTI(raptorgo.DerivationInput{
		TransferLength: uint64(len(object)), DecoderBlockBytes: 1 << 20,
		MaxPayloadSize: 8, SymbolAlignment: 1, MinSubSymbolAlignmentUnits: 1,
	}, limits)
	if err != nil {
		panic(err)
	}
	encoder, err := raptorgo.NewObjectEncoder(object, oti, limits)
	if err != nil {
		panic(err)
	}

	decoder, err := raptorgo.NewObjectDecoder(
		oti, limits, "generation-8",
		func(request raptorgo.VerificationRequest) error {
			candidate, readErr := io.ReadAll(request.Object)
			if readErr != nil {
				return readErr
			}
			if request.Context != "generation-8" || !bytes.Equal(candidate, object) {
				return fmt.Errorf("object authentication failed")
			}
			return nil
		},
		func(verified raptorgo.VerifiedObject) error {
			fmt.Println(string(verified.Bytes))
			return nil
		},
	)
	if err != nil {
		panic(err)
	}

	// This small object fits one source block with K source symbols.
	k, err := raptorgo.SourceBlockSymbols(oti, 0)
	if err != nil {
		panic(err)
	}
	const shortenFinal = true
	// Deliver all source packets except the last one, which is lost.
	for esi := uint32(0); esi+1 < k; esi++ {
		packet, packetErr := encoder.Packet(0, esi, 1, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
	}
	// Repair packets (ESI >= K) stand in for the lost source packet.
	for esi := k; decoder.State() == raptorgo.ObjectReceiving; esi++ {
		packet, packetErr := encoder.Packet(0, esi, 1, shortenFinal)
		if packetErr != nil {
			panic(packetErr)
		}
		if _, addErr := decoder.AddPacket(packet); addErr != nil {
			panic(addErr)
		}
	}
	fmt.Println(decoder.State())

}

Output:
recovered by RaptorQ
delivered

Index

Examples

Constants

View Source 
const (
	// MaxTransferLength is the largest RFC 6330 transfer length F in bytes,
	// as corrected by verified RFC Editor erratum 5548.
	MaxTransferLength uint64 = 942_574_504_275
	// MaxSourceSymbolsPerBlock is K'_max, the largest number of source
	// symbols in one source block, from RFC 6330 Section 5.1.2.
	MaxSourceSymbolsPerBlock = 56_403
	// MaxIntermediateSymbols is the largest number of intermediate symbols
	// L that any source block can require, reached at K'_max.
	MaxIntermediateSymbols uint32 = 57_326
	// MaxEncodingSymbolID is the largest encoding symbol ID that fits the
	// 24-bit wire field from RFC 6330 Section 3.2.
	MaxEncodingSymbolID = 16_777_215
)
View Source 
const OTIEncodedSize = 12

OTIEncodedSize is the encoded size in bytes of the combined RFC 6330 Common and Scheme-Specific FEC Object Transmission Information produced by OTI.MarshalBinary and consumed by ParseOTI.

View Source 
const PayloadIDEncodedSize = 4

PayloadIDEncodedSize is the encoded size in bytes of the RFC 6330 FEC Payload ID that prefixes every encoding packet.

Variables

View Source 
var (
	// ErrInvalidLimits is returned when a Limits value has a zero,
	// out-of-range, or otherwise unsafe field.
	ErrInvalidLimits = errors.New("raptorgo: invalid limits")
	// ErrLimitExceeded is returned when an operation would exceed a
	// caller-supplied Limits field, the RFC 6330 wire domain, or the
	// address space of the platform.
	ErrLimitExceeded = errors.New("raptorgo: resource limit exceeded")
	// ErrInvalidOTI is returned when Object Transmission Information
	// violates an RFC 6330 constraint or has the wrong encoded length.
	ErrInvalidOTI = errors.New("raptorgo: invalid object transmission information")
	// ErrInvalidDerivation is returned when DeriveOTI input is inconsistent
	// or admits no valid RFC 6330 Section 4.3 parameter derivation.
	ErrInvalidDerivation = errors.New("raptorgo: invalid RFC parameter derivation input")
	// ErrInvalidPayloadID is returned when a FEC Payload ID has the wrong
	// encoded length or an encoding symbol ID that exceeds 24 bits.
	ErrInvalidPayloadID = errors.New("raptorgo: invalid FEC payload ID")
	// ErrInvalidPacket is returned when an encoding packet violates
	// RFC 6330 grouped-symbol or shortening semantics, targets an invalid
	// or empty source block, or requests an out-of-range symbol.
	ErrInvalidPacket = errors.New("raptorgo: invalid encoding packet")
	// ErrInvalidObject is returned when a source object does not match its
	// OTI, for example when its length differs from the transfer length.
	ErrInvalidObject = errors.New("raptorgo: invalid source object")
	// ErrObjectEncoder is returned when object encoding fails, including
	// nil-encoder use and internal source-block encoder failures.
	ErrObjectEncoder = errors.New("raptorgo: object encoder failed")
	// ErrObjectDecoder is returned when object decoding fails, including
	// nil-decoder use, construction preflight failures, and internal
	// source-block decoder failures.
	ErrObjectDecoder = errors.New("raptorgo: object decoder failed")
	// ErrObjectVerification is returned when the application verifier
	// rejects a reconstructed object.
	ErrObjectVerification = errors.New("raptorgo: decoded object verification failed")
	// ErrObjectDelivery is returned when the application deliverer rejects
	// a verified object.
	ErrObjectDelivery = errors.New("raptorgo: verified object delivery failed")
	// ErrObjectTerminal is returned when a packet is added to an
	// ObjectDecoder that has already left the ObjectReceiving state.
	ErrObjectTerminal = errors.New("raptorgo: object decoder is terminal")
)

Functions

func ObjectDecoderWorkspaceBackingBytes 

func ObjectDecoderWorkspaceBackingBytes(oti OTI, limits Limits) (uint64, error)

ObjectDecoderWorkspaceBackingBytes returns the conservative peak logical backing reserved by NewObjectDecoder and all later operations under limits. It includes persistent block decoders, the largest bounded decode attempt, reconstruction, and one owned delivery copy.

func ObjectEncoderWorkspaceBackingBytes 

func ObjectEncoderWorkspaceBackingBytes(oti OTI, limits Limits) (uint64, error)

ObjectEncoderWorkspaceBackingBytes returns the largest logical backing peak needed to own the object and construct any one repair encoder cache. Packet output backing is checked separately because it depends on each request.

func SourceBlockSymbols 

func SourceBlockSymbols(oti OTI, sourceBlockNumber uint8) (uint32, error)

SourceBlockSymbols returns K for a source block using RFC 6330 Section 4.4.1.2 partitioning.

Types

type DerivationInput 

type DerivationInput struct {
	// TransferLength is F, the object size in bytes, at most
	// MaxTransferLength.
	TransferLength uint64
	// DecoderBlockBytes is WS, the largest source-block size in bytes the
	// receiver is willing to hold in working memory while decoding.
	DecoderBlockBytes uint64
	// MaxPayloadSize is P', the target packet payload size in bytes; it
	// becomes the symbol size T and must be a nonzero multiple of
	// SymbolAlignment.
	MaxPayloadSize uint16
	// SymbolAlignment is Al in bytes, 1 to 255. Symbol and sub-symbol sizes
	// are multiples of it; 4 suits most transports.
	SymbolAlignment uint8
	// MinSubSymbolAlignmentUnits is SS: the smallest allowed sub-symbol is
	// SS*Al bytes, which must not exceed MaxPayloadSize.
	MinSubSymbolAlignmentUnits uint32
}

DerivationInput contains the inputs to the RFC 6330 Section 4.3 example parameter derivation.

type EncodingPacketView 

type EncodingPacketView struct {
	// contains filtered or unexported fields
}

EncodingPacketView is a validated, zero-copy view of a grouped RFC 6330 source or repair packet. The caller must not mutate the input byte slice while using the view.

func ParseEncodingPacket 

func ParseEncodingPacket(encoded []byte, oti OTI, limits Limits) (EncodingPacketView, error)

ParseEncodingPacket validates grouped-symbol semantics from RFC 6330 Section 4.4.2, including the one permitted shortened final source symbol.

func (EncodingPacketView) CopySymbol 

func (packet EncodingPacketView) CopySymbol(index uint32, destination []byte) error

CopySymbol copies a symbol into a canonical T-byte destination, adding the mandatory zero padding when the final source symbol was shortened on wire.

func (EncodingPacketView) IsSource 

func (packet EncodingPacketView) IsSource() bool

IsSource reports whether the packet carries source symbols rather than repair symbols.

func (EncodingPacketView) Payload 

func (packet EncodingPacketView) Payload() []byte

Payload returns the symbol payload that follows the FEC Payload ID. The returned bytes alias the parsed input and must not be mutated.

func (EncodingPacketView) PayloadID 

func (packet EncodingPacketView) PayloadID() PayloadID

PayloadID returns the packet's parsed FEC Payload ID.

func (EncodingPacketView) Symbol 

func (packet EncodingPacketView) Symbol(index uint32) ([]byte, error)

Symbol returns a read-only view of one encoding symbol within the payload. A legally shortened final source symbol is returned without its trailing zero padding, so the view may be shorter than the symbol size T; use CopySymbol to obtain the canonical T-byte form.

func (EncodingPacketView) SymbolCount 

func (packet EncodingPacketView) SymbolCount() uint32

SymbolCount returns the number of encoding symbols grouped in the packet.

type Limits 

type Limits struct {
	// MaxTransferLength bounds OTI.TransferLength (F) in bytes; it may not
	// exceed the RFC ceiling MaxTransferLength.
	MaxTransferLength uint64
	// MaxSymbolSize bounds OTI.SymbolSize (T) in bytes.
	MaxSymbolSize uint16
	// MaxSourceBlocks bounds OTI.SourceBlocks (Z).
	MaxSourceBlocks uint8
	// MaxSubBlocks bounds OTI.SubBlocks (N).
	MaxSubBlocks uint16
	// MaxSymbolAlignment bounds OTI.SymbolAlignment (Al) in bytes.
	MaxSymbolAlignment uint8
	// MaxSourceSymbolsPerBlock bounds K, the source symbols in any single
	// source block; it may not exceed K'_max (MaxSourceSymbolsPerBlock).
	MaxSourceSymbolsPerBlock uint32
	// MaxSourceBlockBytes bounds K*T, the size in bytes of any single
	// source block.
	MaxSourceBlockBytes uint64
	// MaxPacketBytes bounds the encoded size in bytes of one encoding
	// packet, including its FEC Payload ID.
	MaxPacketBytes uint64
	// MaxSymbolsPerPacket bounds G, the symbols grouped into one packet.
	MaxSymbolsPerPacket uint32
	// MaxRetainedSymbols bounds the source plus repair symbols one block
	// decoder retains at once. It must be at least K for every block the
	// OTI derives, or NewObjectDecoder fails.
	MaxRetainedSymbols uint32
	// MaxDecodeAttempts bounds bounded decode attempts per source block
	// before the decoder gives up.
	MaxDecodeAttempts uint32
	// MaxRepairSymbolsPerBlock bounds the repair symbols retained per source
	// block beyond its K source symbols.
	MaxRepairSymbolsPerBlock uint32
	// MaxDenseCompletionSymbols bounds the dense-completion stage of the
	// hybrid solver, at most MaxIntermediateSymbols. It must be at least P,
	// the block's PI symbol count, or NewObjectDecoder fails.
	MaxDenseCompletionSymbols uint32
	// MaxWorkingMemory bounds the peak logical backing bytes of any single
	// encoder or decoder operation, computed before allocation.
	MaxWorkingMemory uint64
}

Limits bounds all sizes controlled by OTI or packet data before allocation. Every allocation-sensitive operation validates externally controlled input against a Limits value taken as an argument or retained from construction. Every field must be nonzero; see Validate.

func DefaultLimits 

func DefaultLimits() Limits

DefaultLimits returns conservative production defaults. They intentionally reject some syntactically valid, extremely large RFC configurations. Applications should lower these defaults when their workload permits it.

func RFCWireLimits 

func RFCWireLimits() Limits

RFCWireLimits admits every wire value allowed by RFC 6330. It is useful for serialization and conformance tests, not as an allocation policy.

func (Limits) Validate 

func (limits Limits) Validate() error

Validate checks that every field is nonzero and within the RFC 6330 wire domain. It returns an error wrapping ErrInvalidLimits that names the first offending field, or nil when all fields are usable.

type OTI 

type OTI struct {
	// TransferLength is F, the object size in bytes, at most
	// MaxTransferLength. Zero describes a zero-length object.
	TransferLength uint64
	// SymbolSize is T, the encoding symbol size in bytes, 1 to 65535 and a
	// multiple of SymbolAlignment.
	SymbolSize uint16
	// SourceBlocks is Z, the number of source blocks, 1 to 255.
	SourceBlocks uint8
	// SubBlocks is N, the number of sub-blocks per source block, 1 to 65535.
	SubBlocks uint16
	// SymbolAlignment is Al in bytes, 1 to 255; SymbolSize and every
	// sub-symbol size are multiples of it.
	SymbolAlignment uint8
}

OTI is the RFC 6330 FEC Object Transmission Information: the parameter contract between encoder and decoder. It travels out of band and must be authenticated by the application before use.

func DeriveOTI 

func DeriveOTI(input DerivationInput, limits Limits) (OTI, error)

DeriveOTI applies the RFC 6330 Section 4.3 example algorithm. Applications may instead supply an OTI selected by their content-delivery protocol.

func ParseOTI 

func ParseOTI(encoded []byte, limits Limits) (OTI, error)

ParseOTI decodes an OTIEncodedSize-byte RFC 6330 encoded OTI and validates the result against both RFC constraints and the supplied limits before returning it.

func (OTI) MarshalBinary 

func (oti OTI) MarshalBinary() ([]byte, error)

MarshalBinary encodes the OTI into its OTIEncodedSize-byte RFC 6330 wire form. It validates against RFCWireLimits, not caller limits, so every wire-legal OTI can be serialized. ParseOTI is the limits-aware inverse; there is deliberately no UnmarshalBinary, so parsing untrusted OTI bytes always takes explicit Limits.

func (OTI) Validate 

func (oti OTI) Validate(limits Limits) error

Validate checks RFC constraints and application resource limits before any object, symbol, or matrix allocation.

type ObjectDecoder 

type ObjectDecoder struct {
	// contains filtered or unexported fields
}

ObjectDecoder incrementally decodes every nonempty source block and enforces decoded -> verified -> delivered ordering. FEC success alone never invokes delivery. It is safe for concurrent method calls. Callbacks run without the decoder mutex held, so they may inspect or call the decoder; packet additions made after finalization starts are rejected as terminal.

func NewObjectDecoder 

func NewObjectDecoder(
	oti OTI,
	limits Limits,
	context string,
	verify ObjectVerifier,
	deliver ObjectDeliverer,
) (*ObjectDecoder, error)

NewObjectDecoder preflights all persistent block decoders, their largest bounded attempt, reconstruction, verification, and one owned delivery copy before allocating retained packet storage.

A zero-length object (OTI.TransferLength 0) has no nonempty source blocks and therefore needs no packets: verify and deliver run synchronously on the empty object before NewObjectDecoder returns. In that fast path the decoder is returned non-nil even on failure, in the corresponding terminal state, together with an error wrapping ErrObjectVerification or ErrObjectDelivery; on success it is returned already in the ObjectDelivered state.

func (*ObjectDecoder) AddPacket 

func (decoder *ObjectDecoder) AddPacket(encoded []byte) (ObjectDecoderState, error)

AddPacket validates and copies every grouped symbol. Once all blocks decode, it synchronously verifies and invokes delivery exactly once. AddPacket on a nil decoder reports ObjectDeliveryFailed with an error wrapping ErrObjectDecoder.

func (*ObjectDecoder) CompletedBlocks 

func (decoder *ObjectDecoder) CompletedBlocks() uint32

CompletedBlocks reports how many nonempty source blocks have fully decoded so far. CompletedBlocks on a nil decoder reports 0.

func (*ObjectDecoder) PersistentBackingBytes 

func (decoder *ObjectDecoder) PersistentBackingBytes() uint64

PersistentBackingBytes reports the logical backing bytes reserved by the persistent per-block decoders for the lifetime of this decoder, excluding transient decode-attempt and reconstruction peaks. PersistentBackingBytes on a nil decoder reports 0.

func (*ObjectDecoder) State 

func (decoder *ObjectDecoder) State() ObjectDecoderState

State reports the decoder's current lifecycle state. State on a nil decoder reports ObjectDeliveryFailed.

type ObjectDecoderState 

type ObjectDecoderState uint8

ObjectDecoderState makes the authenticated-delivery boundary explicit. Once a decoder leaves ObjectReceiving it never accepts another packet; AddPacket then fails with ErrObjectTerminal. 

const (
	// ObjectReceiving is the initial state: the decoder accepts packets and
	// has not yet decoded every nonempty source block.
	ObjectReceiving ObjectDecoderState = iota
	// ObjectDecodeFailed is terminal: FEC decoding or reconstruction failed
	// permanently.
	ObjectDecodeFailed
	// ObjectDecoded is a transient finalization state: every nonempty source
	// block decoded and the reconstructed object awaits verification.
	ObjectDecoded
	// ObjectVerified is a transient finalization state: the verifier accepted
	// the reconstructed object and delivery is about to run.
	ObjectVerified
	// ObjectDelivered is terminal: the deliverer received the verified object
	// exactly once and accepted it.
	ObjectDelivered
	// ObjectVerificationFailed is terminal: the application verifier rejected
	// the reconstructed object.
	ObjectVerificationFailed
	// ObjectDeliveryFailed is terminal: the application deliverer rejected
	// the verified object.
	ObjectDeliveryFailed
)

func (ObjectDecoderState) String 

func (state ObjectDecoderState) String() string

String returns a short lower-case name for the state, such as "receiving", "delivered", or "verification-failed".

type ObjectDeliverer 

type ObjectDeliverer func(VerifiedObject) error

ObjectDeliverer receives the reconstructed object exactly once, after the verifier has accepted it. Returning a non-nil error moves the decoder to ObjectDeliveryFailed.

type ObjectEncoder 

type ObjectEncoder struct {
	// contains filtered or unexported fields
}

ObjectEncoder owns a source object and emits canonical grouped RFC 6330 source or repair packets. It retains at most one repair encoder cache and is safe for concurrent Packet calls.

func NewObjectEncoder 

func NewObjectEncoder(object []byte, oti OTI, limits Limits) (*ObjectEncoder, error)

NewObjectEncoder validates and owns object. Repair-plan construction is preflighted for every distinct block size, but is deferred until the first repair packet for a block is requested.

func (*ObjectEncoder) OTI 

func (encoder *ObjectEncoder) OTI() OTI

OTI returns the Object Transmission Information supplied at construction. OTI on a nil encoder returns the zero OTI.

func (*ObjectEncoder) Packet 

func (encoder *ObjectEncoder) Packet(
	sourceBlockNumber uint8,
	firstESI, symbolCount uint32,
	shortenFinal bool,
) ([]byte, error)

Packet returns one owned grouped packet beginning at firstESI. A group must lie wholly in the source range or wholly in the repair range. When shortenFinal is true, trailing FEC padding is omitted from the packet's last source symbol when RFC 6330 permits it. Packet on a nil encoder returns an error wrapping ErrObjectEncoder.

func (*ObjectEncoder) PacketInto 

func (encoder *ObjectEncoder) PacketInto(
	destination []byte,
	sourceBlockNumber uint8,
	firstESI, symbolCount uint32,
	shortenFinal bool,
) ([]byte, error)

PacketInto writes the same canonical packet as Packet into destination and returns its exact-length view. Destination must have room for the four-byte payload ID and the requested symbols; passing a full-sized buffer is valid when shortenFinal may produce a shorter result. The returned bytes alias destination, whose unused tail is left unchanged. A source-packet call does not allocate when destination is large enough. Concurrent calls are safe, but callers must use distinct destination buffers for concurrently live results and must not mutate a buffer until its call returns.

Example 
package main

import (
	"fmt"

	"github.com/fgn/raptorgo"
)

func main() {
	object := []byte("abcdefgh")
	oti := raptorgo.OTI{
		TransferLength: 8, SymbolSize: 4,
		SourceBlocks: 1, SubBlocks: 1, SymbolAlignment: 1,
	}
	encoder, err := raptorgo.NewObjectEncoder(object, oti, raptorgo.DefaultLimits())
	if err != nil {
		panic(err)
	}

	// Reuse this buffer for successive source packets. PacketInto returns the
	// exact packet-length prefix and performs no heap allocation on this path.
	buffer := make([]byte, raptorgo.PayloadIDEncodedSize+int(oti.SymbolSize))
	packet, err := encoder.PacketInto(buffer, 0, 1, 1, false)
	if err != nil {
		panic(err)
	}
	view, err := raptorgo.ParseEncodingPacket(packet, oti, raptorgo.DefaultLimits())
	if err != nil {
		panic(err)
	}
	fmt.Printf("payload=%q aliases-buffer=%v\n", view.Payload(), &packet[0] == &buffer[0])

}

Output:
payload="efgh" aliases-buffer=true

type ObjectVerifier 

type ObjectVerifier func(VerificationRequest) error

ObjectVerifier authenticates a reconstructed object before delivery. Returning a non-nil error rejects the object and moves the decoder to ObjectVerificationFailed; the deliverer is then never invoked.

type PayloadID 

type PayloadID struct {
	SourceBlockNumber uint8
	EncodingSymbolID  uint32
}

PayloadID is the four-octet FEC Payload ID from RFC 6330 Section 3.2.

func NewPayloadID 

func NewPayloadID(sourceBlockNumber uint8, encodingSymbolID uint32) (PayloadID, error)

NewPayloadID constructs a PayloadID, rejecting encoding symbol IDs that do not fit the 24-bit wire field.

func ParsePayloadID 

func ParsePayloadID(encoded []byte) (PayloadID, error)

ParsePayloadID decodes a four-octet RFC 6330 FEC Payload ID.

func (PayloadID) MarshalBinary 

func (id PayloadID) MarshalBinary() ([]byte, error)

MarshalBinary encodes the payload ID into its four-octet RFC 6330 wire form: the source block number followed by the 24-bit big-endian encoding symbol ID.

type SourceBlock 

type SourceBlock struct {
	// contains filtered or unexported fields
}

SourceBlock contains one RFC 6330 source block in symbol-major order. Its storage owns a copy of the corresponding source-object bytes and any zero padding required for the final source symbol.

func BuildSourceBlock 

func BuildSourceBlock(object []byte, oti OTI, sourceBlockNumber uint8, limits Limits) (SourceBlock, error)

BuildSourceBlock constructs one source block from a complete source object. For N > 1, it converts RFC sub-block-major input ordering into symbol-major storage. All externally controlled sizes are validated before allocation.

func (SourceBlock) Number 

func (block SourceBlock) Number() uint8

Number returns the block's source block number (SBN).

func (SourceBlock) Symbol 

func (block SourceBlock) Symbol(index uint32) ([]byte, error)

Symbol returns a view into the block-owned storage. Callers must not mutate the returned bytes.

func (SourceBlock) SymbolCount 

func (block SourceBlock) SymbolCount() uint32

SymbolCount returns K, the number of source symbols in the block.

func (SourceBlock) SymbolSize 

func (block SourceBlock) SymbolSize() uint16

SymbolSize returns the symbol size T in bytes.

type VerificationRequest 

type VerificationRequest struct {
	// OTI describes the reconstructed object being verified.
	OTI OTI
	// Context is the identifier supplied to NewObjectDecoder, so one shared
	// verifier can distinguish concurrent transfers.
	Context string
	// Length equals OTI.TransferLength and exists for convenience.
	Length uint64
	// Object streams the reconstructed bytes. It is valid only for the
	// duration of the verifier call and must not be retained.
	Object io.Reader
}

VerificationRequest binds reconstructed bytes to their OTI and an application-supplied generation/context identifier. Object is read-only.

type VerifiedObject 

type VerifiedObject struct {
	// OTI describes the delivered object.
	OTI OTI
	// Context is the identifier supplied to NewObjectDecoder.
	Context string
	// Bytes is an owned copy of the verified object, exactly
	// OTI.TransferLength long; the callback may retain and mutate it.
	Bytes []byte
}

VerifiedObject is passed exactly once to the delivery callback after its corresponding VerificationRequest succeeds. Bytes are owned by the callback.

Source Files

View all Source files

Directories

Path Synopsis
examples
roundtrip command
Command roundtrip demonstrates a complete raptorgo transfer over a simulated lossy channel.
 Command roundtrip demonstrates a complete raptorgo transfer over a simulated lossy channel.
internal
codec
Package codec implements RFC 6330 source-block constraint construction, intermediate-symbol generation, raw encoding, and decoding substrate.
 Package codec implements RFC 6330 source-block constraint construction, intermediate-symbol generation, raw encoding, and decoding substrate.
gf256
Package gf256 implements the octet arithmetic defined by RFC 6330 Section 5.7.
 Package gf256 implements the octet arithmetic defined by RFC 6330 Section 5.7.
matrix
Package matrix implements bounded sparse matrix and permutation primitives used by the RFC 6330 constraint builder and inactivation schedule.
 Package matrix implements bounded sparse matrix and permutation primitives used by the RFC 6330 constraint builder and inactivation schedule.
rfc6330
Package rfc6330 contains deterministic formulas and constants from RFC 6330.
 Package rfc6330 contains deterministic formulas and constants from RFC 6330.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.