ztlv

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 27, 2026 License: MIT Imports: 7 Imported by: 0

README

ztlv

Go Reference Go Report Card

ztlv is a secure, low-allocation Tag-Length-Value (TLV) parser for Go.

It is designed for network protocols, critical infrastructure, and high-throughput systems where memory pressure and input validation are paramount. It operates directly on io.Reader and io.Writer interfaces, enabling streaming of datasets larger than available RAM.

Design Principles

  • Security First: Strict bounds checking prevents OOM (Out-Of-Memory) attacks. Integers are checked for overflows.
  • Low Allocation: Primitives (Tags, Lengths, Time) are encoded/decoded using stack-allocated scratchpads.
  • Optimized Strings: Uses unsafe.String (Go 1.20+) to convert byte buffers to strings without additional heap copying.
  • Robust Time: Timestamps are stored as 96-bit (64-bit Seconds + 32-bit Nanoseconds) to prevent Y2262 issues and maintain precision.
  • Big Endian: All numeric values conform to standard network byte order.

Installation

go get github.com/eraceo/ztlv

Usage

Encoder

The encoder wraps an io.Writer. It performs no heap allocations for primitive types.

package main

import (
	"bytes"
	"time"
	"github.com/eraceo/ztlv"
)

func main() {
	var buf bytes.Buffer
	enc := ztlv.NewEncoder(&buf)

	// Write Tag (0x01) + Length + String
	_ = enc.WriteTLVString(0x01, "admin")

	// Write Tag (0x02) + Length + Time (12 bytes)
	_ = enc.WriteTLVTime(0x02, time.Now())
}
Decoder

The decoder wraps an io.Reader and enforces configurable limits.

package main

import (
	"bytes"
	"log"
	"github.com/eraceo/ztlv"
)

func main() {
	// r is your io.Reader (e.g., net.Conn or bytes.Buffer)
	dec := ztlv.NewDecoder(r)

	// Decode loop
	tag, err := dec.ReadTag()
	if err != nil {
		log.Fatal(err)
	}

	switch tag {
	case 0x01:
		// Efficiently reads string (1 allocation only)
		val, _ := dec.ReadString()
		log.Printf("User: %s", val)
	case 0x02:
		// Efficiently reads Length + Time (0 allocation)
		// Uses ReadTimePrefixed because WriteTLVTime includes a length.
		ts, _ := dec.ReadTimePrefixed()
		log.Printf("Timestamp: %s", ts)
	default:
		// Efficiently skip unknown tags without allocation
		len, _ := dec.ReadLength()
		_ = dec.Skip(len)
	}
}
Advanced: Zero-Allocation Byte Reading

For hot paths where garbage collection pressure must be eliminated, use ReadBytesInto with a reused buffer. This method separates length reading from payload reading to ensure stream integrity.

// Pre-allocated buffer (or retrieved from sync.Pool)
buf := make([]byte, 4096)
// 1. Read the length first
length, err := dec.ReadLength()
if err != nil {
    handleError(err)
}

// 2. Check if buffer is large enough
if uint32(cap(buf)) < length {
    // Handle error, or grow buffer
    handleError(fmt.Errorf("buffer too small"))
}

// 3. Read payload directly into the slice (Zero Allocation)
err = dec.ReadBytesInto(length, buf)
if err != nil {
    handleError(err)
}

process(buf[:length])

Security Configuration

To prevent Denial of Service (DoS) attacks via malicious length prefixes, the decoder enforces default sanity limits. These can be adjusted per instance.

dec := ztlv.NewDecoder(r)

// Adjust limits based on your protocol requirements
dec.MaxStringSize = 64 * 1024       // Limit strings to 64KB
dec.MaxBytesSize  = 10 * 1024 * 1024 // Limit raw blobs to 10MB
dec.MaxListCount  = 5000            // Limit list items

Binary Format Specification

Component Size Type Description
Tag 1 byte uint8 Identifier (0-255).
Length 4 bytes uint32 Payload size in bytes (Big Endian).
Value N bytes Raw Payload data.

Time Format: Stored as 12 bytes.

  • Bytes 0-7: Seconds (int64, Big Endian)
  • Bytes 8-11: Nanoseconds (uint32, Big Endian)

Performance

  • Primitives: 0 allocs/op.
  • Strings: 1 alloc/op (buffer only, no string copy).
  • Bytes: 1 alloc/op (or 0 if using ReadBytesInto).
  • Skip: 0 allocs/op (uses io.Discard).

Documentation

Index

Constants

View Source 
const (
	// Default safety limits
	DefaultMaxStringSize = 1 << 20  // 1 MB
	DefaultMaxBytesSize  = 10 << 20 // 10 MB
	DefaultMaxListCount  = 1000
)

Variables

View Source 
var (
	ErrPayloadTooLarge = errors.New("ztlv: payload exceeds configured limit")
	ErrListTooLarge    = errors.New("ztlv: list count exceeds configured limit")
	ErrShortBuffer     = errors.New("ztlv: buffer too short")
	ErrInvalidTag      = errors.New("ztlv: invalid tag found")
	ErrInvalidLength   = errors.New("ztlv: invalid length for type")
)

Functions

This section is empty.

Types

type Decoder 

type Decoder struct {
	MaxStringSize uint32
	MaxBytesSize  uint32
	MaxListCount  uint32
	// contains filtered or unexported fields
}

func NewDecoder 

func NewDecoder(r io.Reader) *Decoder

func (*Decoder) ReadBool 

func (d *Decoder) ReadBool() (bool, error)

ReadBool reads a boolean (1 byte).

func (*Decoder) ReadBytes 

func (d *Decoder) ReadBytes() ([]byte, error)

ReadBytes is optimized for a single allocation. It is "Low Alloc" but safe (returns a clean copy).

func (*Decoder) ReadBytesInto 

func (d *Decoder) ReadBytesInto(length uint32, buf []byte) error

ReadBytesInto enables "Zero Alloc" scenarios if the caller reuses their buffer. Warning: This method reads strictly 'length' bytes. It does NOT read the length prefix itself. The caller must have read the length beforehand (e.g. via ReadLength()).

func (*Decoder) ReadFloat64 

func (d *Decoder) ReadFloat64() (float64, error)

ReadFloat64 reads a float64 (8 bytes, IEEE 754, Big Endian).

func (*Decoder) ReadInt64 

func (d *Decoder) ReadInt64() (int64, error)

ReadInt64 reads an int64 (8 bytes, Big Endian).

func (*Decoder) ReadLength 

func (d *Decoder) ReadLength() (uint32, error)

func (*Decoder) ReadNested 

func (d *Decoder) ReadNested(expected Tag, fn func(*Decoder) error) error

ReadNested handles a TLV container (a TLV that contains other TLVs). It reads the Tag and Length, creates a limited Decoder for the body, and executes the callback.

Security: The nested decoder is strictly limited to the container's length. It prevents reading past the container boundary.

Usage: 

err := dec.ReadNested(TagUser, func(d *zerotlv.Decoder) error {
    name, _ := d.ReadTLVString(TagName)
    age, _ := d.ReadTLVUint8(TagAge)
    return nil
})

func (*Decoder) ReadString 

func (d *Decoder) ReadString() (string, error)

ReadString uses "unsafe" optimizations (Go 1.20+) to reduce allocations.

func (*Decoder) ReadStrings 

func (d *Decoder) ReadStrings() ([]string, error)

func (*Decoder) ReadTLVBool 

func (d *Decoder) ReadTLVBool(expected Tag) (bool, error)

ReadTLVBool reads a Tag, verifies it, reads Length (must be 1), then reads Bool.

func (*Decoder) ReadTLVBytes 

func (d *Decoder) ReadTLVBytes(expected Tag) ([]byte, error)

ReadTLVBytes reads a Tag, verifies it, reads Length, then reads Bytes.

func (*Decoder) ReadTLVFloat64 

func (d *Decoder) ReadTLVFloat64(expected Tag) (float64, error)

ReadTLVFloat64 reads a Tag, verifies it, reads Length (must be 8), then reads Float64.

func (*Decoder) ReadTLVInt64 

func (d *Decoder) ReadTLVInt64(expected Tag) (int64, error)

ReadTLVInt64 reads a Tag, verifies it, reads Length (must be 8), then reads Int64.

func (*Decoder) ReadTLVString 

func (d *Decoder) ReadTLVString(expected Tag) (string, error)

ReadTLVString reads a Tag, verifies it, reads Length, then reads String.

func (*Decoder) ReadTLVTime 

func (d *Decoder) ReadTLVTime(expected Tag) (time.Time, error)

ReadTLVTime reads a Tag, verifies it, reads Length (must be 12), then reads Time.

func (*Decoder) ReadTLVUint8 

func (d *Decoder) ReadTLVUint8(expected Tag) (uint8, error)

ReadTLVUint8 reads a Tag, verifies it, reads Length (must be 1), then reads Uint8.

func (*Decoder) ReadTLVUint16 

func (d *Decoder) ReadTLVUint16(expected Tag) (uint16, error)

ReadTLVUint16 reads a Tag, verifies it, reads Length (must be 2), then reads Uint16.

func (*Decoder) ReadTLVUint32 

func (d *Decoder) ReadTLVUint32(expected Tag) (uint32, error)

ReadTLVUint32 reads a Tag, verifies it, reads Length (must be 4), then reads Uint32.

func (*Decoder) ReadTLVUint64 

func (d *Decoder) ReadTLVUint64(expected Tag) (uint64, error)

ReadTLVUint64 reads a Tag, verifies it, reads Length (must be 8), then reads Uint64.

func (*Decoder) ReadTag 

func (d *Decoder) ReadTag() (Tag, error)

func (*Decoder) ReadTime 

func (d *Decoder) ReadTime() (time.Time, error)

ReadTime decodes a robust timestamp (Seconds + Nanoseconds). Note: This reads strictly 12 bytes. It does NOT read a length prefix. Use ReadTimePrefixed if you are using standard TLV (WriteTLVTime).

func (*Decoder) ReadTimePrefixed 

func (d *Decoder) ReadTimePrefixed() (time.Time, error)

ReadTimePrefixed reads a Length prefix, verifies it is 12, and then reads the Time. This is the secure counterpart to WriteTLVTime.

func (*Decoder) ReadUint8 

func (d *Decoder) ReadUint8() (uint8, error)

ReadUint8 reads a uint8 (1 byte).

func (*Decoder) ReadUint16 

func (d *Decoder) ReadUint16() (uint16, error)

ReadUint16 reads a uint16 (2 bytes, Big Endian).

func (*Decoder) ReadUint32 

func (d *Decoder) ReadUint32() (uint32, error)

ReadUint32 reads a uint32 (4 bytes, Big Endian).

func (*Decoder) ReadUint64 

func (d *Decoder) ReadUint64() (uint64, error)

ReadUint64 reads a uint64 (8 bytes, Big Endian).

func (*Decoder) Skip 

func (d *Decoder) Skip(n uint32) error

func (*Decoder) VerifyTag 

func (d *Decoder) VerifyTag(expected Tag) error

VerifyTag reads a Tag and returns an error if it does not match the expected Tag.

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder remains lightweight. Allocations are primarily handled by the underlying Writer.

func NewEncoder 

func NewEncoder(w io.Writer) *Encoder

func (*Encoder) WriteBool 

func (e *Encoder) WriteBool(v bool) error

WriteBool writes a boolean (1 byte). 0x01 for true, 0x00 for false.

func (*Encoder) WriteBytes 

func (e *Encoder) WriteBytes(b []byte) error

WriteBytes writes a standard byte slice.

func (*Encoder) WriteFloat64 

func (e *Encoder) WriteFloat64(v float64) error

WriteFloat64 writes a float64 (8 bytes, IEEE 754, Big Endian).

func (*Encoder) WriteInt64 

func (e *Encoder) WriteInt64(v int64) error

WriteInt64 writes an int64 (8 bytes, Big Endian).

func (*Encoder) WriteLength 

func (e *Encoder) WriteLength(length uint32) error

func (*Encoder) WriteString 

func (e *Encoder) WriteString(s string) error

func (*Encoder) WriteStrings 

func (e *Encoder) WriteStrings(strs []string) error

func (*Encoder) WriteTLVBool 

func (e *Encoder) WriteTLVBool(tag Tag, v bool) error

WriteTLVBool writes a Tag, Length (1), then the Bool (1 byte).

func (*Encoder) WriteTLVBytes 

func (e *Encoder) WriteTLVBytes(tag Tag, b []byte) error

WriteTLVBytes writes a Tag, then the Length of the bytes, then the Bytes themselves.

func (*Encoder) WriteTLVFloat64 

func (e *Encoder) WriteTLVFloat64(tag Tag, v float64) error

WriteTLVFloat64 writes a Tag, Length (8), then the Float64 (8 bytes).

func (*Encoder) WriteTLVInt64 

func (e *Encoder) WriteTLVInt64(tag Tag, v int64) error

WriteTLVInt64 writes a Tag, Length (8), then the Int64 (8 bytes).

func (*Encoder) WriteTLVString 

func (e *Encoder) WriteTLVString(tag Tag, s string) error

WriteTLVString writes a Tag, then the Length of the string, then the String itself.

func (*Encoder) WriteTLVTime 

func (e *Encoder) WriteTLVTime(tag Tag, t time.Time) error

WriteTLVTime writes a Tag, then the Length of the time (12 bytes), then the Time itself.

func (*Encoder) WriteTLVUint8 

func (e *Encoder) WriteTLVUint8(tag Tag, v uint8) error

WriteTLVUint8 writes a Tag, Length (1), then the Uint8 (1 byte).

func (*Encoder) WriteTLVUint16 

func (e *Encoder) WriteTLVUint16(tag Tag, v uint16) error

WriteTLVUint16 writes a Tag, Length (2), then the Uint16 (2 bytes).

func (*Encoder) WriteTLVUint32 

func (e *Encoder) WriteTLVUint32(tag Tag, v uint32) error

WriteTLVUint32 writes a Tag, Length (4), then the Uint32 (4 bytes).

func (*Encoder) WriteTLVUint64 

func (e *Encoder) WriteTLVUint64(tag Tag, v uint64) error

WriteTLVUint64 writes a Tag, Length (8), then the Uint64 (8 bytes).

func (*Encoder) WriteTag 

func (e *Encoder) WriteTag(tag Tag) error

func (*Encoder) WriteTime 

func (e *Encoder) WriteTime(t time.Time) error

WriteTime encodes a timestamp robustly: compatible with all dates, avoids Year 2038 issues.

func (*Encoder) WriteUint8 

func (e *Encoder) WriteUint8(v uint8) error

WriteUint8 writes a uint8 (1 byte).

func (*Encoder) WriteUint16 

func (e *Encoder) WriteUint16(v uint16) error

WriteUint16 writes a uint16 (2 bytes, Big Endian).

func (*Encoder) WriteUint32 

func (e *Encoder) WriteUint32(v uint32) error

WriteUint32 writes a uint32 (4 bytes, Big Endian).

func (*Encoder) WriteUint64 

func (e *Encoder) WriteUint64(v uint64) error

WriteUint64 writes a uint64 (8 bytes, Big Endian).

type Tag 

type Tag byte

type UnexpectedTagError 

type UnexpectedTagError struct {
	Expected Tag
	Actual   Tag
}

UnexpectedTagError is returned when the read tag does not match the expected tag.

func (*UnexpectedTagError) Error 

func (e *UnexpectedTagError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.