fec

Documentation

Overview

Package fec implements a forward error correction scheme using Reed Solomon Erasure Coding.

This codec has a limitation of 256 total shards and hard coded to allow 1kb sized shards for network transmission, and constructs segmented blocks based on 16x 1kb source shards with redundancy-adjusted extra shards to protect against data loss from signal noise or other corruption.

1kb chunks are used to ensure that regular disruptions of the signal have a better chance of not knocking out enough pieces to cause tx failure.

Index

• Constants
• func GetShardCodecParams(data []byte) (seg, segTot, num, tot, req, size int, err error)
• func Pieces(dLen, size int) (s int)
• func SegmentBytes(buf []byte, lim int) (out [][]byte)
• type PartialSegment
  • func (p PartialSegment) GetShardCount() (count int)
• type Partials
  • func NewPacket(firstShard []byte) (o *Partials, err error)
  • func (p *Partials) AddShard(newShard []byte) (err error)
  • func (p *Partials) Decode() (final []byte, err error)
  • func (p *Partials) GetRatio() (out float64)
  • func (p *Partials) HasAllDataShards() bool
  • func (p *Partials) HasMinimum() bool
• type Segments
• type ShardedSegments
  • func GetShards(buf []byte, redundancy int) (out ShardedSegments)

Constants

const (
	SegmentSize = 2 << 13
	ShardSize   = 2 << 9
)

Functions

func GetShardCodecParams

func GetShardCodecParams(data []byte) (
	seg, segTot, num, tot, req, size int, err error,
)

GetShardCodecParams reads the shard's prefix to provide the correct parameters for the RS codec the packet requires based on the prefix on a shard (presumably to create the codec when a new packet/group of shards arrives)

func Pieces

func Pieces(dLen, size int) (s int)

func SegmentBytes

func SegmentBytes(buf []byte, lim int) (out [][]byte)

Types

type PartialSegment

type PartialSegment struct {
	// contains filtered or unexported fields
}

PartialSegment is a max 16kb long segment with arbitrary redundancy parameters when all of the data segments are successfully received hasAll indicates the segment may be ready to reassemble

func (PartialSegment) GetShardCount

func (p PartialSegment) GetShardCount() (count int)

type Partials

type Partials struct {
	// contains filtered or unexported fields
}

Partials is a structure for storing a new inbound packet

func NewPacket

func NewPacket(firstShard []byte) (o *Partials, err error)

NewPacket creates a new structure to store a collection of incoming shards when the first of a new packet arrives

func (*Partials) AddShard

func (p *Partials) AddShard(newShard []byte) (err error)

AddShard adds a newly received shard to a Partials, ensuring that it has matching parameters (if the HMAC on the packet's wrapper passes it should be unless someone is playing silly buggers)

func (*Partials) Decode

func (p *Partials) Decode() (final []byte, err error)

func (*Partials) GetRatio

func (p *Partials) GetRatio() (out float64)

GetRatio is used after the receive delay period expires to determine how successful a packet was. If it was exactly enough with no surplus, return 0, if there is more than the minimum, return the proportion compared to the total redundancy of the packet, if there is less, return a negative proportion versus the amount, -1 means zero received, and a fraction of 1 indicates the proportion that was received compared to the minimum

func (*Partials) HasAllDataShards

func (p *Partials) HasAllDataShards() bool

HasAllDataShards returns true if all data shards are present in a Partials

func (*Partials) HasMinimum

func (p *Partials) HasMinimum() bool

HasMinimum returns true if there may be enough data to decode

type Segments

type Segments [][]byte

type ShardedSegments

type ShardedSegments []Segments

func GetShards

func GetShards(
	buf []byte, redundancy int,
) (out ShardedSegments)

GetShards returns a bundle of segments to be sent or stored in a 1kb segment size with redundancy shards added to each segment's shards that can reconstruct the original message by derivation via the available parity shards.