format

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 23, 2026 License: BSD-3-Clause Imports: 3 Imported by: 0

Documentation

Overview

Package format is the byte-level on-disk layout of the .aki file (spec 2064 doc 02). It defines the magic, the file header on page 0, the double-buffered meta pages on pages 1 and 2, the common page header that prefixes every other page, and the page-type constants. A second engineer reading doc 02 plus this package can produce a byte-compatible reader and writer.

Everything multi-byte is little-endian, unconditionally, on every platform (doc 02 §6). Structures marshal into and unmarshal out of fixed byte ranges at the offsets the spec fixes; the checksum fields are CRC-32C over the bytes that precede them.

Index

Constants

View Source 
const (
	// FormatVersion is the current file format version, carried in the header.
	FormatVersion uint16 = 1

	// DefaultPageSize is the page size used when a file is created without an
	// explicit override.
	DefaultPageSize uint32 = 16384

	// MinPageSize and MaxPageSize bound the legal page sizes; a page size must
	// be a power of two in this inclusive range.
	MinPageSize uint32 = 4096
	MaxPageSize uint32 = 65536

	// HeaderSize is the size in bytes of the file header on page 0. The rest of
	// page 0 is zero padding.
	HeaderSize = 128

	// CommonHeaderSize is the size of the header prefixing every page except
	// page 0.
	CommonHeaderSize = 16

	// LeafHeaderSize is the effective header size of a B-tree leaf page, which
	// adds a 4-byte right-sibling pointer after the common header.
	LeafHeaderSize = 20

	// MetaHeaderSize is the size of the meta-page header (doc 02 §9).
	MetaHeaderSize = 124

	// DefaultDBCount is the default number of logical databases.
	DefaultDBCount uint32 = 16

	// MaxDBCount is the maximum number of logical databases.
	MaxDBCount uint32 = 256

	// MetaPageA and MetaPageB are the fixed page numbers of the two meta pages.
	MetaPageA uint32 = 1
	MetaPageB uint32 = 2
)

Format and size constants (doc 02 §1, §4, §6).

View Source 
const (
	PageTypeHeader    uint8 = 0x00
	PageTypeMeta      uint8 = 0x01
	PageTypeBTreeInt  uint8 = 0x02
	PageTypeBTreeLeaf uint8 = 0x03
	PageTypeOverflow  uint8 = 0x04
	PageTypeFLTrunk   uint8 = 0x05
	PageTypeFLLeaf    uint8 = 0x06
	PageTypeVExtent   uint8 = 0x07
	PageTypeWALIdx    uint8 = 0x08
	PageTypeStreamSeg uint8 = 0x09
	PageTypeHLLBlob   uint8 = 0x0A
	PageTypeCatalog   uint8 = 0x0B
	PageTypeFree      uint8 = 0xFF
)

Page-type values stored at byte 0 of the common header (doc 02 §7).

View Source 
const (
	CodecNone uint8 = 0x00
	CodecLZ4  uint8 = 0x01
	CodecZstd uint8 = 0x02

	EncryptionNone   uint8 = 0x00
	EncryptionAESGCM uint8 = 0x01
)

Codec and encryption identifiers (doc 02 §4).

View Source 
const (
	FileFlagInMemory          uint32 = 1 << 0
	FileFlagWALMode           uint32 = 1 << 1
	FileFlagEncrypted         uint32 = 1 << 2
	FileFlagCompressedDefault uint32 = 1 << 3
)

File-flag bits in the header's file_flags field (doc 02 §4).

View Source 
const NullPage uint32 = 0xFFFFFFFF

NullPage is the all-ones sentinel used in pointer fields to mean "no page" (doc 02 §6).

Variables

View Source 
var (
	// ErrBadMagic means the first 16 bytes are not the aki magic.
	ErrBadMagic = errors.New("aki/format: bad magic")
	// ErrBadChecksum means a stored CRC-32C does not match the bytes it covers.
	ErrBadChecksum = errors.New("aki/format: checksum mismatch")
	// ErrShortBuffer means a marshal/unmarshal target is too small.
	ErrShortBuffer = errors.New("aki/format: buffer too small")
	// ErrBadPageSize means a page size is out of range or not a power of two.
	ErrBadPageSize = errors.New("aki/format: invalid page size")
	// ErrUnknownPageType is returned when traversal hits an undefined page type.
	ErrUnknownPageType = errors.New("aki/format: unknown page type")
)

Errors returned by the format package.

View Source 
var Magic = [16]byte{
	't', 'a', 'm', 'n', 'd', 'a', 'k', 'i', ' ', 'f', 'm', 't', '0', '0', '1', '\n',
}

Magic is the first 16 bytes of every .aki file: the ASCII string "tamndaki fmt001" (15 bytes) followed by a 0x0A newline (doc 02 §3). The newline keeps file(1) and head -c 16 showing human-readable text.

Functions

func ValidPageSize 

func ValidPageSize(n uint32) bool

ValidPageSize reports whether n is a legal page size: a power of two in [MinPageSize, MaxPageSize].

Types

type FileHeader 

type FileHeader struct {
	// Magic is always format.Magic; ParseFileHeader rejects anything else.
	Magic [16]byte

	FormatVersion   uint16 // offset 16
	MinReadVersion  uint16 // offset 18
	MinWriteVersion uint16 // offset 20
	PageSize        uint32 // offset 22
	PageCount       uint32 // offset 26 (mutable)
	FreelistHead    uint32 // offset 30 (mutable)
	FreelistCount   uint32 // offset 34 (mutable)
	CatalogRoot     uint32 // offset 38 (mutable)
	DBCount         uint32 // offset 42
	ChangeCounter   uint64 // offset 46 (mutable)
	FileCreateTime  uint64 // offset 54 (microseconds)
	WALCheckpoint   uint64 // offset 62 (mutable)
	DefaultCodec    uint8  // offset 70
	EncryptionID    uint8  // offset 71
	KDFSalt         [16]byte
	KDFParams       uint32 // offset 88
	FileFlags       uint32 // offset 92
	MetaPageA       uint32 // offset 96
	MetaPageB       uint32 // offset 100
	SchemaVersion   uint32 // offset 104 (mutable)
	UserVersion     uint32 // offset 108 (mutable)
	AutovacuumMode  uint32 // offset 112
	// HeaderChecksum at offset 124 is CRC-32C of bytes 0..123; it is computed
	// by MarshalTo and verified by ParseFileHeader, not set by callers.
	HeaderChecksum uint32
}

FileHeader is the 128-byte header on page 0 of a .aki file (doc 02 §4). It is written atomically at create time; only the fields the spec marks mutable change afterward, and any change recomputes HeaderChecksum.

func NewFileHeader 

func NewFileHeader(pageSize, dbCount uint32, createTimeUS uint64) FileHeader

NewFileHeader returns a header for a freshly created file with the given page size, database count, and creation timestamp (in microseconds). Pointer fields start at their empty sentinels and the meta pages are fixed at 1 and 2.

func ParseFileHeader 

func ParseFileHeader(b []byte) (FileHeader, error)

ParseFileHeader reads and validates a header from the front of b. It checks the magic and the CRC-32C; a mismatch returns ErrBadMagic or ErrBadChecksum.

func (FileHeader) MarshalTo 

func (h FileHeader) MarshalTo(b []byte) error

MarshalTo writes the header into the first HeaderSize bytes of b and fills in the trailing CRC-32C over bytes 0..123. The remainder of the page (b past byte 128) is left untouched; callers zero-pad the page.

type MetaPage 

type MetaPage struct {
	Header PageHeader // page_type = PageTypeMeta

	MetaSeq       uint64    // offset 16
	TxnID         uint64    // offset 24
	ChangeCounter uint64    // offset 32
	PageCount     uint32    // offset 40
	FreelistHead  uint32    // offset 44
	FreelistCount uint32    // offset 48
	CatalogRoot   uint32    // offset 52
	WALCommitLSN  uint64    // offset 56
	DBCount       uint32    // offset 64
	SchemaVersion uint32    // offset 68
	DBRootPages   [8]uint32 // offset 72 (32 bytes)
	SystemRoot    uint32    // offset 104, root of the system table B-tree
	// MetaChecksum at offset 120 is CRC-32C of bytes 0..119; computed by
	// MarshalTo and verified by ParseMetaPage.
	MetaChecksum uint32
}

MetaPage is one of the two double-buffered meta pages on pages 1 and 2 (doc 02 §9). A commit writes the updated snapshot to the non-live page and fsyncs; the page with the higher valid MetaSeq is the live snapshot, giving atomic visibility of the root-pointer set without a separate journal.

func LiveMeta 

func LiveMeta(a, b MetaPage, aok, bok bool) (live MetaPage, ok bool)

LiveMeta returns whichever of a or b is the live snapshot: the valid page with the higher MetaSeq. The bools report whether each parsed cleanly. If neither is valid, ok is false.

func NewMetaPage 

func NewMetaPage(h FileHeader, metaSeq uint64) MetaPage

NewMetaPage returns the initial meta page for a freshly created file, derived from its header. Both meta pages start identical except MetaSeq: A gets seq 1 and B gets seq 0 so A is live.

func ParseMetaPage 

func ParseMetaPage(b []byte) (MetaPage, error)

ParseMetaPage reads and validates a meta page from b. A CRC-32C mismatch returns ErrBadChecksum; callers treat a bad meta page as not-live and fall back to the other one.

func (MetaPage) MarshalTo 

func (m MetaPage) MarshalTo(b []byte, pageSize uint32) error

MarshalTo writes the meta page into b, which must be at least one page. It fills the common header, the meta fields, and the trailing CRC-32C over bytes 0..119, then zero-pads the rest of the page. 

type PageHeader struct {
	Type      uint8
	Flags     uint8
	CellCount uint16
	FreeStart uint16
	FreeEnd   uint16
	PageLSN   uint64
}

PageHeader is the 16-byte common header at the front of every page except page 0 (doc 02 §8). RightSibling is only meaningful for B-tree leaf pages and lives just past this header; it is carried here for convenience but marshaled separately by the page-specific code.

func ParsePageHeader 

func ParsePageHeader(b []byte) (PageHeader, error)

ParsePageHeader reads a common header from the front of b.

func (PageHeader) FreeSpace 

func (h PageHeader) FreeSpace() int

FreeSpace returns the number of free bytes between FreeStart and FreeEnd.

func (PageHeader) MarshalTo 

func (h PageHeader) MarshalTo(b []byte) error

MarshalTo writes the common header into the first CommonHeaderSize bytes of b.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.