record

Documentation

Overview

Package record implements CRC32C-checksummed framing for a single WAL record.

On-disk record layout (all integers little-endian):

+---------+---------+---------+------------------+
| CRC32C  | Length  |   LSN   |     Payload      |
| 4 bytes | 4 bytes | 8 bytes |  Length bytes    |
+---------+---------+---------+------------------+

CRC32C is computed over Length || LSN || Payload.

Index

• Constants
• Variables
• func Encode(dst []byte, lsn uint64, payload []byte) ([]byte, error)
• func EncodedSize(payloadLen int) int
• type Record
• type Scanner
  • func NewScanner(source io.Reader, maxRecordBytes int) *Scanner
  • func (s *Scanner) Err() error
  • func (s *Scanner) Next() bool
  • func (s *Scanner) Offset() int64
  • func (s *Scanner) Record() Record

Constants

const HeaderSize = 16

HeaderSize is the fixed per-record framing overhead: crc(4)+length(4)+lsn(8).

const MaxPayloadSize = math.MaxUint32

MaxPayloadSize is the largest payload the on-disk Length field (uint32) can represent. Callers enforce a smaller MaxRecordSize.

Variables

var ErrCorrupt = errors.New("wal/record: corrupt record (crc mismatch)")

ErrCorrupt is returned by Scanner.Err when a record's CRC32C checksum does not match the recorded header CRC, indicating on-disk data corruption.

var ErrPayloadTooLarge = errors.New("wal/record: payload exceeds MaxPayloadSize")

ErrPayloadTooLarge is returned by Encode when a payload cannot be represented by the uint32 Length field.

var ErrTooLarge = errors.New("wal/record: record length exceeds maximum")

ErrTooLarge is returned by Scanner.Err when the Length field in a record header exceeds the maximum configured for the Scanner.

var ErrTorn = errors.New("wal/record: torn record at tail")

ErrTorn is returned by Scanner.Err when the stream ends mid-record, indicating an incomplete (torn) write at the tail of a WAL segment. The segment may be safely truncated to Scanner.Offset().

Functions

func Encode

func Encode(dst []byte, lsn uint64, payload []byte) ([]byte, error)

Encode appends the framed record for (lsn, payload) to dst and returns the extended slice. dst may be nil. It returns ErrPayloadTooLarge if the payload cannot be represented by the uint32 Length field.

func EncodedSize

func EncodedSize(payloadLen int) int

EncodedSize returns the on-disk byte size of a record carrying a payload of payloadLen bytes.

Types

type Record

type Record struct {
	Payload []byte
	LSN     uint64
}

Record is a decoded log record. Fields are ordered to minimize struct padding and GC-scanned pointer bytes (slice first, then the scalar LSN).

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner reads framed WAL records sequentially from an io.Reader. The caller advances the stream with Next(); the current decoded record is available via Record(). Call Err() after Next() returns false to distinguish a clean end-of-stream from a torn write or data corruption.

Payload returned by Record() is valid only until the next call to Next(). Fields are ordered to minimize struct padding and GC-scanned pointer bytes: pointer-bearing fields (interfaces, slices, the Record's slice) are grouped first, followed by the scalar counters.

func NewScanner

func NewScanner(source io.Reader, maxRecordBytes int) *Scanner

NewScanner returns a Scanner that reads from source and rejects any record whose declared payload length exceeds maxRecordBytes.

func (*Scanner) Err

func (s *Scanner) Err() error

Err returns the first non-EOF error encountered, or nil after a clean EOF.

func (*Scanner) Next

func (s *Scanner) Next() bool

Next attempts to decode the next record from the stream. It returns true when a valid record is available via Record(). It returns false on clean EOF or on the first error (which is then accessible via Err()).

func (*Scanner) Offset

func (s *Scanner) Offset() int64

Offset returns the total number of bytes consumed by fully-valid records. On a torn or corrupt stream this is the safe truncation point.

func (*Scanner) Record

func (s *Scanner) Record() Record

Record returns the most recently decoded record. The Payload slice is only valid until the next call to Next().