pack

Documentation

Overview

Package pack implements the immutable binary container that carries GoBeyond's pack-only runtime artifacts: render plans (.gbp) and packaged static entries (.gbs). See docs/architecture.md#render-plan-contract.

Both files share one container layout — a fixed header, a sorted index, and per-record zstd-compressed JSON — distinguished by the magic bytes. Opening a pack validates the header and index without decoding any record; records are read on demand through ReadAt so residency stays bounded.

Index

• Constants
• Variables
• func FallbackWeights(content ContentType, encodedLen uint64) (decodedWeight, peakWeight uint64)
• func PlanWeights(plan *renderplan.Plan, encodedLen int) (decodedWeight, peakWeight uint64)
• func StaticEntryKey(routeID string, params map[string]string) string
• func StaticWeights(props, metadata any, encodedLen int) (decodedWeight, peakWeight uint64)
• func WritePlans(path, buildID string, plans map[string][]byte) error
• func WriteStatic(path, buildID string, entries map[string][]byte) error
• type Builder
  • func NewBuilder(content ContentType, buildID string) (*Builder, error)
  • func (b *Builder) AddPlan(routeID string, encoded []byte) error
  • func (b *Builder) AddStatic(key string, encoded []byte) error
  • func (b *Builder) Len() int
  • func (b *Builder) WriteFile(path string) error
  • func (b *Builder) WriteTo(w io.Writer) (int64, error)
• type ContentType
  • func (c ContentType) String() string
• type Reader
  • func NewReader(src io.ReaderAt, size int64, content ContentType) (*Reader, error)
  • func Open(path string, content ContentType) (*Reader, error)
  • func (r *Reader) BuildID() string
  • func (r *Reader) Close() error
  • func (r *Reader) Content() ContentType
  • func (r *Reader) DecodeJSONRecord(key string) ([]byte, error)
  • func (r *Reader) Has(key string) bool
  • func (r *Reader) Len() int
  • func (r *Reader) ReadRecord(key string) ([]byte, error)
  • func (r *Reader) Record(key string) (Record, bool)
  • func (r *Reader) RecordCodec() string
  • func (r *Reader) Records() []Record
  • func (r *Reader) WeigherVersion() uint32
• type Record

Constants

const (
	ExtPlans  = ".gbp"
	ExtStatic = ".gbs"
)

Conventional file extensions for the two pack content types.

const (
	PlanPackCapability   = "gobeyond.plan-pack/v1"
	StaticPackCapability = "gobeyond.static-pack/v1"
)

Capability identifiers a build publishes in its deploy metadata (compatibility.json / artifacts.json) to declare the pack artifacts it emitted. Hosting and tooling key off these strings, not file extensions.

const (
	MaxBuildIDLen       = 256
	MaxKeyLen           = 1024
	MaxRecords          = 1 << 20
	MaxRecordStoredLen  = 1 << 30 // compressed record bytes on disk
	MaxRecordEncodedLen = 1 << 30 // JSON bytes after decompression
)

Size limits enforced when opening a pack, before any record is decoded.

const (
	// PlanPeakEncodedFactor: plan peak = decoded + encoded*3.
	PlanPeakEncodedFactor = 3
	// StaticPeakEncodedFactor: static peak = decoded + encoded*2.
	StaticPeakEncodedFactor = 2
	// PlanFallbackPeakFactor: unknown weigher, plan peak = encoded*8.
	PlanFallbackPeakFactor = 8
	// StaticFallbackPeakFactor: unknown weigher, static peak = encoded*5.
	StaticFallbackPeakFactor = 5
)

Stable pack-residency weight formulas. "Encoded" is the JSON record length in bytes before compression (Record.EncodedLen).

const FormatVersion = 1

FormatVersion is the container format implemented by this package.

const RecordCodecJSONZstd = "json+zstd"

RecordCodecJSONZstd is record codec v1: each record is one zstd-compressed JSON document. Whole-file compression is rejected by design because it breaks random access.

const WeigherVersion = 1

WeigherVersion identifies the deterministic pack-time weight model. Readers that see a different version must not trust stored weights and fall back to FallbackWeights.

Variables

var (
	// ErrNotFound reports a key absent from the pack index.
	ErrNotFound = errors.New("pack: record not found")
	// ErrDigestMismatch reports stored record bytes that do not match the
	// index digest. This is an immutable integrity failure.
	ErrDigestMismatch = errors.New("pack: record digest mismatch")
)

Functions

func FallbackWeights

func FallbackWeights(content ContentType, encodedLen uint64) (decodedWeight, peakWeight uint64)

FallbackWeights returns the conservative estimates readers must use when a pack was written by an unknown weigher version: peak = encoded*8 for plans and encoded*5 for static entries, with the decoded share derived from the standard peak formula.

func PlanWeights

func PlanWeights(plan *renderplan.Plan, encodedLen int) (decodedWeight, peakWeight uint64)

PlanWeights estimates the resident weight of a decoded render plan and its peak weight while decoding, for a plan whose JSON encoding is encodedLen bytes.

func StaticEntryKey

func StaticEntryKey(routeID string, params map[string]string) string

StaticEntryKey derives the canonical pack key for one static entry: routeID + "?" + sorted queryEscape(name)=queryEscape(value) pairs. Empty params yield routeID + "?". An optional catch-all matched with zero segments must be passed as a present, empty value.

func StaticWeights

func StaticWeights(props, metadata any, encodedLen int) (decodedWeight, peakWeight uint64)

StaticWeights estimates the resident weight of one decoded static entry (props plus metadata) and its peak weight while decoding, for an entry whose JSON encoding is encodedLen bytes.

func WritePlans

func WritePlans(path, buildID string, plans map[string][]byte) error

WritePlans writes the render-plan pack (.gbp) for one build from the route-ID-keyed plan JSON produced by the compiler.

func WriteStatic

func WriteStatic(path, buildID string, entries map[string][]byte) error

WriteStatic writes the packaged static-entry pack (.gbs) for one build from entry JSON keyed by StaticEntryKey.

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder assembles one pack at build time. Every added record is fully parsed during the CLI validation stage, weighed over the decoded value, and zstd-compressed; WriteTo then emits the sorted container.

func NewBuilder

func NewBuilder(content ContentType, buildID string) (*Builder, error)

func (*Builder) AddPlan

func (b *Builder) AddPlan(routeID string, encoded []byte) error

AddPlan validates one render plan and records it under its route ID. The encoded JSON must parse as a valid plan whose routeId matches routeID.

func (*Builder) AddStatic

func (b *Builder) AddStatic(key string, encoded []byte) error

AddStatic validates one static entry's JSON and records it under key, normally a StaticEntryKey value. The decoded value (props plus metadata) is weighed generically.

func (*Builder) Len

func (b *Builder) Len() int

func (*Builder) WriteFile

func (b *Builder) WriteFile(path string) error

WriteFile writes the pack to path via a temporary file and rename, so an interrupted build never leaves a partial pack behind.

func (*Builder) WriteTo

func (b *Builder) WriteTo(w io.Writer) (int64, error)

WriteTo emits the container: header, index sorted ascending by key, then record bytes in the same order.

type ContentType

type ContentType uint8

ContentType selects which artifact a pack carries.

const (
	// ContentPlans is a render-plan pack (.gbp); keys are route IDs.
	ContentPlans ContentType = iota + 1
	// ContentStatic is a packaged static-entry pack (.gbs); keys are
	// StaticEntryKey values.
	ContentStatic
)

func (ContentType) String

func (c ContentType) String() string

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader is an open pack. Open validates the header and index eagerly — magic, format version, size limits, bounds, overlaps, and duplicate keys — without decoding any record; record bytes are only read on demand through ReadAt. Methods are safe for concurrent use once NewReader returns.

func NewReader

func NewReader(src io.ReaderAt, size int64, content ContentType) (*Reader, error)

NewReader validates a pack held by any ReaderAt (a file, an mmap, or a byte slice via bytes.NewReader). The reader does not own src; Close is a no-op unless the reader came from Open.

func Open

func Open(path string, content ContentType) (*Reader, error)

Open opens the pack file at path and validates it as content.

func (*Reader) BuildID

func (r *Reader) BuildID() string

func (*Reader) Close

func (r *Reader) Close() error

Close releases the underlying file when the reader came from Open.

func (*Reader) Content

func (r *Reader) Content() ContentType

func (*Reader) DecodeJSONRecord

func (r *Reader) DecodeJSONRecord(key string) ([]byte, error)

DecodeJSONRecord reads, digest-verifies, and decompresses the record for key, returning its JSON bytes. Callers hand those to renderplan.Parse or the static entry decoder.

func (*Reader) Has

func (r *Reader) Has(key string) bool

func (*Reader) Len

func (r *Reader) Len() int

func (*Reader) ReadRecord

func (r *Reader) ReadRecord(key string) ([]byte, error)

ReadRecord returns the raw stored (zstd-compressed) bytes for key after verifying them against the index digest.

func (*Reader) Record

func (r *Reader) Record(key string) (Record, bool)

Record returns the index entry for key.

func (*Reader) RecordCodec

func (r *Reader) RecordCodec() string

func (*Reader) Records

func (r *Reader) Records() []Record

Records returns a copy of the index, sorted ascending by key.

func (*Reader) WeigherVersion

func (r *Reader) WeigherVersion() uint32

WeigherVersion is the weight model the pack was written with. When it is not this package's WeigherVersion, record weights have already been replaced with FallbackWeights.

type Record

type Record struct {
	Key           string
	Offset        uint64            // absolute file offset of the stored bytes
	Length        uint64            // stored (compressed) length in bytes
	Digest        [sha256.Size]byte // SHA-256 of the stored bytes
	EncodedLen    uint64            // JSON length in bytes before compression
	DecodedWeight uint64            // estimated resident weight once decoded
	PeakWeight    uint64            // estimated peak weight while decoding
}

Record describes one entry in a pack's sorted index.