jsonlledger

Documentation

Overview

Package jsonlledger holds the shared JSONL-ledger row helpers the report packages (cadencereport, milestonereport, programreport, …) each used to copy-paste: Parse scans a JSONL ledger into typed rows, and LatestBefore finds the newest prior row. Each caller keeps its own row type and delegates here so the duplicated bodies live in exactly one place.

Index

• Constants
• func AppendBounded(path string, line []byte, maxBytes int64) error
• func AppendValidated[T any](writer io.Writer, value T, validate func(T) error) error
• func LatestBefore[T any](row T, prior []T, date, tiebreak func(T) string) (T, bool)
• func Parse[T any](content string, keep func(T) bool) []T
• func ReadTail(path string, maxBytes int64) []byte
• type Checkpoint
  • func TailFold[T, S any](path string, ckpt Checkpoint[S], initial S, step func(S, T) S) (Checkpoint[S], error)

Constants

const DefaultActiveBytes int64 = 8 << 20

DefaultActiveBytes is the maximum active-ledger size used by unattended nightrun writers. One sealed generation may coexist beside it.

Functions

func AppendBounded

func AppendBounded(path string, line []byte, maxBytes int64) error

AppendBounded appends one complete JSONL line, rotating the prior active file to path+".1" before the append that would cross maxBytes. Rotation lives at the write site, so an unattended writer cannot bypass the disk bound.

func AppendValidated

func AppendValidated[T any](writer io.Writer, value T, validate func(T) error) error

AppendValidated marshals and appends one validated value as a JSONL row.

func LatestBefore

func LatestBefore[T any](row T, prior []T, date, tiebreak func(T) string) (T, bool)

LatestBefore returns the row in prior with the greatest (date, tiebreak) sort key, skipping any row whose non-empty tiebreak equals the reference row's (its own prior generation), or (zero, false) when none remain. date and tiebreak extract the primary sort key and the stable-sort tiebreaker from a row. It consolidates the identical "find the previous ledger row" scan the report packages each carried.

func Parse

func Parse[T any](content string, keep func(T) bool) []T

Parse scans content as JSONL, unmarshaling each non-blank line into a T and appending it when keep(row) reports true. Blank and malformed lines are skipped. A nil keep accepts every well-formed row. The 1 MiB line buffer matches the copies this consolidates, so long ledger lines still parse.

func ReadTail

func ReadTail(path string, maxBytes int64) []byte

ReadTail returns at most maxBytes from the end of path, dropping the first partial row when the window starts mid-line. Missing/unreadable files return nil, matching the ledgers' fail-open first-run contract.

Types

type Checkpoint

type Checkpoint[S any] struct {
	Path        string `json:"path"`   // file this checkpoint describes
	Offset      int64  `json:"offset"` // bytes folded: end of the last complete line
	Size        int64  `json:"size"`   // file size observed when the checkpoint was taken
	ModTimeNano int64  `json:"mtime"`  // file mtime observed, UnixNano
	Boundary    string `json:"fp"`     // fingerprint of the bytes ending at Offset
	State       S      `json:"state"`  // accumulated fold
}

Checkpoint records where a prior TailFold stopped so the next call can fold only the bytes appended since. Hold it in memory across reads or persist it — every field is JSON-encodable. The zero Checkpoint requests a full fold.

func TailFold

func TailFold[T, S any](path string, ckpt Checkpoint[S], initial S, step func(S, T) S) (Checkpoint[S], error)

TailFold folds only the JSONL rows appended to path since ckpt was taken. When ckpt still describes the file — same path, not shrunk below Offset, and the bytes ending at Offset unchanged — it seeks to ckpt.Offset and folds only the newly appended complete lines into ckpt.State via step. Otherwise (a zero ckpt, a different path, a shorter file, or a rewritten prefix) it re-folds the whole file from initial. Blank and malformed lines are skipped, matching Parse; a trailing line with no newline is left unconsumed until the writer completes it. The returned checkpoint carries the advanced offset and the new fold — feed it back on the next call.

Correctness rests on one assumption: when the file grows, the bytes before the prior Offset are unchanged (an append). A writer that rewrites the prefix in place is caught only when that rewrite alters the boundaryWindow bytes ending at Offset; a rewrite that preserves them exactly would be mis-resumed. Every current caller either appends or shifts contents (which changes those bytes), so the assumption holds — revisit it before pointing TailFold at a new writer.