Documentation
¶
Overview ¶
Package ledger reads and writes the three committed state files (docs/SPEC.md §15, §19): the ledger of every defined block, the reverse index of every reference, and the append-only ack log. All three are TSV with a one-line header that carries a format version, the repo name, and the commit they were produced at, so `publish` can ship them and `sync` can merge them.
Why TSV and not JSON: one row per id or per reference, sorted, means a move is a one-line diff in a pull request and a human can read the file. Why a format version on every file: a newer tool upgrades older files in place, an older tool refuses a newer file by name instead of misparsing it (§32).
Field escaping is the minimum that keeps a row on one line: tab, newline, carriage return, and backslash are escaped with a backslash. Nothing else is touched, so grep still works on ids, paths, and hashes.
Index ¶
- Constants
- Variables
- func Carry(prev, cur []RefRow, effective func(RefRow) string, current func(RefRow) string) []int
- func EscapeField(s string) string
- func ParseTime(s string) (time.Time, error)
- func UnescapeField(s string) string
- type Ack
- type AckKey
- type Acks
- type ActorKind
- type Baseline
- type Foreign
- type ForeignRow
- type Header
- type Ledger
- type RefRow
- type Refs
- type Row
Constants ¶
const ( KindLedger = "ledger" KindRefs = "refs" KindAcks = "acks" KindForeign = "foreign" )
File kinds, written into the header so a refs file can never be read as a ledger by mistake.
const ( LedgerFile = "ledger.tsv" RefsFile = "refs.tsv" AcksFile = "acks.tsv" ForeignFile = "foreign.tsv" )
Default file names under `.ds/`.
const AcksFormat = 3
AcksFormat is the ack log's own layout version. The ack log alone gained columns — 3 added rule and sentence — and it is never published, so it versions apart: bumping Format for it would stamp the published ledger and refs too, and a repository still on an older ds would refuse the whole workspace index for two columns it never reads.
const Format = 2
Format is the version of all three file layouts this package writes. Bump when a column is added or changes meaning; Decode refuses anything newer and reads every version down to MinFormat, so a ledger written by an older tool keeps working and only loses what its columns never carried.
2 added refs.seen_hash: the hash a block had when a citation was first recorded, which is the only baseline a citation that was never acked has. Without it a rescan silently adopted whatever the block said at that moment, and a change nobody had reviewed became invisible.
const MinFormat = 1
MinFormat is the oldest layout this tool still reads.
const RootShard = "_root"
RootShard names the shard for files at the repository root.
Variables ¶
var ( ErrBadHeader = errors.New("ledger: missing or malformed header line") ErrKind = errors.New("ledger: file kind does not match") ErrFormat = errors.New("ledger: format version not supported by this tool") ErrColumns = errors.New("ledger: column count mismatch") ErrField = errors.New("ledger: bad field value") // ErrConflict is a line git left behind when a merge conflicted. It is // reported as itself, not as the column-count mismatch it also is, so // the reader is told to finish the merge rather than to repair a row. ErrConflict = errors.New("ledger: unresolved merge conflict") )
Sentinel errors.
var ActorKindValues = []ActorKind{ActorHuman, ActorAgent}
ActorKindValues is the canonical order.
var ErrTimeRange = errors.New("ledger: timestamp outside the years RFC 3339 can write")
ErrTimeRange is a timestamp that parses but cannot be written back.
Functions ¶
func Carry ¶
Carry decides, for each current citation, which previous refs row it continues, so a citation keeps its baselines when it moves. It returns one index into prev per current row, or -1 for a citation that is new.
Why it exists: baselines used to be looked up by the exact doc and line. A citation that moved missed that lookup and was treated as new, and a new citation is baselined at the block's current hash — so adding one line above a citation, or renaming its doc, silently accepted every change to the cited block that nobody had reviewed. That is the one thing the tool exists to prevent, and it was reached by the most ordinary edit there is.
The rule it follows is that carrying an older baseline can only add a report, never remove one, while pairing two citations of the same block wrongly can remove one: if the acked citation's baseline lands on the unreviewed one, its drift disappears. So it pairs only where the pairing is certain, and everywhere else it is conservative.
- Same id and env with the same, non-empty sentence hash: the sentence itself moved. Same doc first, in line order, then across docs, which is a rename.
- What remains, per doc: one previous and one current citation of the id is a certain pairing. Anything else is ambiguous.
- What remains in docs with no previous citation of the id, against previous rows whose doc has none left: one and one is a rename whose sentence also changed. Anything else is ambiguous.
- An ambiguous citation gets the candidate whose baseline most needs reporting: the first, in doc and line order, whose effective baseline differs from the block's current hash. Over-reporting costs an ack; under-reporting is a wrong "up to date".
effective returns the baseline a previous row stood for (the latest ack at its position, else its acked hash, else its seen hash); current returns the hash now of the block a current citation resolves to — per citation, since its env and branch pick among defs sharing an id — or "" when it is not known. Both come from the caller because they are state this package does not hold.
Invariant: rows are grouped by id and env and never paired across groups, and for citations at distinct positions the result is independent of the order prev and cur arrive in (two citations of one id on one line are indistinguishable, and every consumer keys them by position anyway).
func EscapeField ¶
EscapeField escapes a value for one column of a docsync TSV file, so a tab, newline, carriage return or backslash in it cannot split a row. It is exported so every TSV docsync writes — tests.tsv as well as the ledger — uses one grammar; UnescapeField reverses it.
func ParseTime ¶
ParseTime reads an RFC 3339 timestamp the way every docsync file stores one, and rejects a value whose UTC form falls outside the years RFC 3339 can express.
Why it exists: every writer formats times in UTC. A value such as 0000-01-01T00:00:00+00:10 parses, but is year -1 in UTC, so re-saving the file writes a timestamp no reader accepts — the file loads once and never again. Rejecting it on read keeps every file that decodes re-savable.
Invariant: a time it returns formats, in UTC, to a string it accepts.
Types ¶
type Ack ¶
type Ack struct {
At time.Time
Actor string
ActorKind ActorKind
DelegatedBy string
ID string
Repo string
Doc string
Line int
Env string
BlockHash string
SentenceHash string
Note string
// Rule is the extraction rule (extract.Rule) the ack was recorded
// under, and Sentence the text of the sentence it approved. A sentence
// hash is only comparable with one bound by the same rule, so an ack
// from an older rule cannot say whether the wording changed; and a
// rewritten sentence is only reviewable when the old text is there to
// show beside the new. Both are empty in an ack log written before
// AcksFormat 3.
Rule int
Sentence string
}
Ack is one append-only approval event.
type Acks ¶
Acks is a header plus ack events, in the order they were recorded.
func (Acks) Encode ¶
Encode writes acks in recorded order; the log is append-only and never sorted, because order is part of the audit trail.
func (Acks) Latest ¶
Latest returns the newest ack per sentence: the one with the latest At, with file order breaking a tie.
Why not file order alone: one writer appends in time order, but a merge does not. Git's union driver, which keeps two branches' acks instead of conflicting, puts one branch's rows before the other's, so after a merge the last row for a sentence is whichever branch was merged second — not whichever ack was made last.
type Baseline ¶
type Baseline struct {
Acked string
Seen string
// Ack is the ack row Acked came from, when it came from one: the one
// at the citation's position or, for a moved citation, the one carried
// from its old position, or the one behind a hash refs.tsv carried
// (carriedAck). It carries the sentence the ack approved, which check
// compares with the sentence there now (SPEC §18). Nil when no ack row
// names the baseline.
Ack *Ack
}
Baseline is what one current citation is measured against: the hash it was last acked at, and the hash it was first seen at. Either may be empty; an empty Seen means the citation is new and the caller baselines it at the block's current hash.
func Baselines ¶
func Baselines(prev []RefRow, cur []RefRow, latest map[AckKey]Ack, scannedAt time.Time, current func(RefRow) string) []Baseline
Baselines resolves, for each current citation, the acked and seen hashes it is measured against, following it across moves (see Carry). It is the one place that decision is made, so `scan` writing refs.tsv and `check` reading it cannot disagree about which ack covers which sentence.
An ack recorded at a citation's current position is the newest statement about the sentence there, and wins — with one exception, which is the point of this function. When the citation moved, the ack at its new position may belong to a different citation that used to sit on that line; taking it would let an acked sentence's approval cover an unreviewed one. So for a moved citation that ack is trusted only when it names the same sentence (both sentence hashes non-empty and equal), or, for a citation with no sentence to compare, when it was recorded after scannedAt — the last scan, whose layout the move is measured from.
type Foreign ¶
type Foreign struct {
Header Header
Rows []ForeignRow
}
Foreign is the committed snapshot of the foreign blocks a repo cites (docs/SPEC.md §21). It is what makes `check` reproducible across repos: without it the answer for one commit depends on what upstream last published and on when the index was last synced, so CI can go red with no change in the repo under test and an old commit cannot be re-run.
It holds cited ids only, never the whole merged ledger, so an unrelated upstream edit does not churn every downstream repo's `.ds/`.
func DecodeForeign ¶
DecodeForeign reads a snapshot.
type ForeignRow ¶
type ForeignRow struct {
Row
// Commit is the publishing repo's commit for this row.
Commit string
}
ForeignRow is one block another repository published that this one cites: a published ledger row plus the commit that repo published it at, which a per-file header cannot carry because the rows come from several repos.
type Header ¶
type Header struct {
Kind string
Format int
// Extract is the version of the extraction rules the hashes in this
// file were computed under (extract.Rule). A hash is only comparable
// with another computed under the same rule, and nothing in a hash says
// which rule produced it — so the rule is recorded beside them. Absent,
// which is every file written before it existed, reads as 1.
Extract int
Repo string
Commit string
ScannedAt time.Time
}
Header is the metadata line at the top of every file.
type Ledger ¶
Ledger is a header plus rows.
func DecodeLedger ¶
DecodeLedger reads a ledger file. Rows come back in file order.
func Unshard ¶
Unshard merges shards back, sorted the way Encode sorts rows; the header is taken from the first shard by name.
func (Ledger) ByHash ¶
ByHash maps a body hash to every row carrying it, for `moved-unmarked` detection: a vanished id whose hash still exists somewhere.
func (Ledger) Bytes ¶
Bytes encodes to memory. Encode only fails on a writer error and a bytes.Buffer has none, so callers that need the bytes (stores, tests) get them without an error branch they could never exercise.
func (Ledger) Encode ¶
Encode writes the ledger sorted by id. Sorting here, not in the caller, is what makes the file diff-stable no matter how it was produced.
func (Ledger) Index ¶
Index maps id to row. The last row wins on duplicates, which cannot occur in a file this package wrote; merged workspace input is deduplicated by the caller with its own collision policy.
type RefRow ¶
type RefRow struct {
ID string
Repo string
Doc string
Line int
Verb string
Carrier block.Carrier
// AckedHash is the block hash a human or agent last accepted for this
// sentence. Empty until something acks it.
AckedHash string
// SeenHash is the block hash the first scan that recorded this citation
// saw, and it is written once and never revised: it is the fallback
// baseline for a citation nobody has acked, so that an unreviewed change
// is still reported. Revising it on every scan would recreate exactly
// the hole it exists to close. Empty when read from a format 1 file, and
// filled by the next scan.
SeenHash string
SentenceHash string
Env string
Args map[string]string
}
RefRow is one reference as stored. AckedHash, SeenHash and SentenceHash are what `check` compares to decide ok versus unacked.
func FromReference ¶
FromReference flattens a scanned reference. AckedHash and SeenHash start empty; acks fill the first and Snapshot fills the second.
func (RefRow) ToReference ¶
ToReference rebuilds a reference from a row for merged-workspace checks; Sentence is not stored, only its hash.
type Refs ¶
Refs is a header plus reference rows.
type Row ¶
type Row struct {
ID string
Repo string
Kind block.Kind
File string
Symbol string
Start int
End int
Hash string
Owner string
Stability block.Stability
Env string
Args map[string]string
}
Row is one defined block as stored. It is a flattened block.Block plus the repo that owns it. Args carries every directive key so nothing is lost; the named columns duplicate the ones reports sort and filter on.
Source Files