cobs

Documentation

Overview

Package cobs implements Consistent Overhead Byte Stuffing (COBS), a technique for reliable packet framing over serial byte streams.

COBS transforms any arbitrary payload such that the zero byte is guaranteed to never appear in the encoded output. A zero byte can then be appended as an unambiguous frame delimiter without colliding with the payload data.

The encoding has the following properties:

• Lossless and reversible. Decoding recovers the original payload exactly.

• Non-zero payload bytes are never modified. Only zero bytes are removed from the encoded form and replaced by length-prefix overhead bytes.

• The zero byte is guaranteed to never appear in encoded output, so a trailing null terminator byte can be used to delimit frames in a byte stream.

• Overhead is tightly bounded in the worst case, unlike PPP byte stuffing (up to 100% expansion) or HDLC bit stuffing (up to 20% expansion). For n > 0 payload bytes, encoded size is at most n + ⌈n/254⌉ bytes. An empty payload encodes to a single byte.

• Longer frames add at most one overhead byte per 254 bytes of data (about 0.4% for large frames, rounded up to whole bytes). This makes maximum frame size predictable, which matters for devices with fixed MTUs or hard transmission-time limits.

Framing convention: AppendEncode produces COBS-encoded payload only; callers typically append a null delimiter when writing to the wire. AppendDecode expects the input without the delimiter.

See https://www.stuartcheshire.org/papers/COBSforToN.pdf

Example

// Frames is a list of frames that may contain null bytes.
// Trivially joining the frames with a null byte would lead to
// ambiguity when parsing as null bytes within the frame itself
// cannot be distinguished from nulls used to mark frame boundaries.
frames := [][]byte{
	[]byte("Hello world!"),
	[]byte(""),
	[]byte("\x00\x00\x00"),
	[]byte("Fizz\x00Buzz"),
}

// COBS encoding ensures that each frame never contains null bytes.
for i, frame := range frames {
	frames[i] = AppendEncode(frame[:0], frame)
}

// Since the COBS-encoded frame lacks null bytes,
// we can trivially join the frames together using a null byte.
stream := bytes.Join(frames, []byte("\x00"))

// Print out the COBS-encoded stream.
fmt.Printf("COBS-encoded stream:\n\t%q\n\n", stream)

// When decoding, the frame boundaries can be trivially detected
// by splitting upon the null byte.
frames = bytes.Split(stream, []byte("\x00"))

// However, each individual frame is still COBS-encoded,
// so we need to decode each one back to the original frame payload.
for i, frame := range frames {
	frames[i] = must.Get(AppendDecode(frame[:0], frame))
}

// Print out each COBS-decoded frame to verify that it matches.
fmt.Println("COBS-decoded frames:")
for _, frame := range frames {
	fmt.Printf("\t%q\n", frame)
}

Output:
COBS-encoded stream:
	"\rHello world!\x00\x01\x00\x01\x01\x01\x01\x00\x05Fizz\x05Buzz"

COBS-decoded frames:
	"Hello world!"
	""
	"\x00\x00\x00"
	"Fizz\x00Buzz"

Index

• func AppendDecode(dst, src []byte) ([]byte, error)
• func AppendEncode(dst, src []byte) []byte
• func AppendNull(dst []byte) []byte
• func FrameLen(b []byte) int
• func MaxEncodedLen(n int) int
• func MinDecodedLen(n int) int
• func TrimNull(dst []byte) []byte

Examples

• Package

Functions

func AppendDecode

func AppendDecode(dst, src []byte) ([]byte, error)

AppendDecode appends the decoded bytes of src to the end of dst. The COBS-encoded src must not contain the trailing null terminator byte; use TrimNull to remove the trailing null terminator byte if needed. It reports an error if the src contains invalid COBS.

The src and dst buffers may exactly overlap. For example, it is valid to do:

b, _ = AppendDecode(b[:0], b)

func AppendEncode

func AppendEncode(dst, src []byte) []byte

AppendEncode appends the encoded bytes of src to the end of dst. The COBS-encoded output never contains null bytes and therefore also lacks a trailing null terminator byte; use AppendNull to append the trailing null terminator byte if needed.

The src and dst buffers may exactly overlap. For example, it is valid to do:

b = AppendEncode(b[:0], b)

func AppendNull

func AppendNull(dst []byte) []byte

AppendNull appends a trailing null terminator byte if it does not already exist.

func FrameLen

func FrameLen(b []byte) int

FrameLen reports the length of a COBS-encoded frame at the start of b by searching for the next trailing null terminator. The reported length includes the null terminator. If the null terminator could not be found, then it reports -1.

func MaxEncodedLen

func MaxEncodedLen(n int) int

MaxEncodedLen is the longest possible length for a COBS-encoded output for some payload of length n. The encoded length does not include the trailing null terminator byte.

Invariant: len(AppendEncode(nil, dec)) <= MaxEncodedLen(len(dec))

func MinDecodedLen

func MinDecodedLen(n int) int

MinDecodedLen is the shortest possible length for a decoded payload from a COBS-encoded input of a length of n, where n does not include the trailing null terminator byte.

Invariant: len(AppendDecode(nil, enc)) >= MinDecodedLen(len(enc))

func TrimNull

func TrimNull(dst []byte) []byte

TrimNull removes a trailing null terminator byte if it exists.