check

Documentation

Overview

Package check turns a scan, the previous ledger, and the ack log into findings: one row per reference or per problem, with a state from the spec's table, a severity, the change class and diff where relevant, and a remedy an agent can execute verbatim (docs/SPEC.md §16 passes 3 to 5, §17).

It is a pure function of its Input. Nothing here reads files, runs commands, or touches the network. Things that need any of those (resolvers, url checks, `run`, commit lookups) arrive as optional hooks in Options and are reported as `unverifiable` or `skipped` when absent, never silently passed.

Index

• Constants
• Variables
• func AboutHash(ids []string, defs ...[]block.Block) string
• func ClaimAbout(ref block.Reference) []string
• func CommandText(text, name string) string
• func ParseDuration(s string) (time.Duration, error)
• func ParseHTTPStatus(s string) (int, bool)
• func ParseRunTimeout(s string) (time.Duration, error)
• func ResolveDef(cands []block.Block, env, branch string) (block.Block, bool)
• func Tally(findings []Finding) (map[State]int, map[Severity]int, int)
• func WithPublishedBody(b block.Block, bodyAt func(hash string) (string, bool)) block.Block
• type Baseline
• type Finding
• type HashPair
• type Input
• type LocalResult
• type Options
• type Remedy
• type RemoteRef
• type Report
  • func ApplyRuns(rep Report, runs []RunResult) Report
  • func Run(in Input) Report
• type ResolveResult
• type Resolver
• type RunResult
• type Severity
  • func SeverityOf(st State) Severity
• type State
• type TestOutcome
• type URLResult
• type Undocumented
• type VerbHandler

Constants

const DateLayout = "2006-01-02"

DateLayout is the only date format accepted in directives. One format, no locale guessing.

const DefaultCommand = "ds"

DefaultCommand is the binary name the remedy texts are written with.

const DefaultMaxLines = 40

DefaultMaxLines is the rendered-block cap (§9.2).

Variables

var BaselineValues = []Baseline{BaselineAck, BaselineSeen}

BaselineValues is the canonical order.

var SeverityValues = []Severity{SeverityError, SeverityWarning, SeverityInfo, SeverityNone}

SeverityValues is the canonical order.

var StateValues = []State{StateOK, StateMoved, StateUnacked, StateBroken, StatePickFailed, StateTooLarge, StateRange, StateExpired, StateSunset, StateDeprecated, StateAssertFailed, StateTranslationStale, StateUnsourced, StateChainBroken, StateUnverifiable, StateSkipped, StateOrphan, StateUncovered, StateUnknown, StateProblem, StateDead, StateRetitled, StateURLMoved, StateUndocumented, StateResolveFailed, StateOutOfSync, StateStaleCopy, StateRotated, StateStale, StateTampered, StateUnscanned, StateRunFailed}

StateValues is the canonical order, used by the summary. dsself:def id=statevalues-vwvk2kq3 owner=@docsync stability=stable

Functions

func AboutHash

func AboutHash(ids []string, defs ...[]block.Block) string

AboutHash fingerprints what a claim is about: every current hash of every id in ids, across environments and repositories. The ack that renews a claim records it, and check then holds a changed about= block as reviewed while the fingerprint still matches. Without it, renewing a claim whose block had changed printed "renewed" and left the claim expired until the next `ds scan` recorded the new hash (bug 72). Empty when ids is empty.

func ClaimAbout

func ClaimAbout(ref block.Reference) []string

ClaimAbout lists the ids a claim's about= names, brackets trimmed.

func CommandText

func CommandText(text, name string) string

CommandText rewrites every command in text from DefaultCommand to name, so a binary built under another name (cli.WithName) prints remedies its users can run. An empty name, or DefaultCommand, returns text unchanged. It is exported because the CLI's own messages follow the same rule.

func ParseDuration

func ParseDuration(s string) (time.Duration, error)

ParseDuration reads the duration shape docsync writes. The grammar lives in internal/duration so configuration can validate the same shape without importing the checker.

func ParseHTTPStatus

func ParseHTTPStatus(s string) (int, bool)

ParseHTTPStatus reads an expect= value as an HTTP status code: three digits from 100 to 599, optionally quoted. ds:url's expect= takes only a status; ds:run's takes one as one of its modes, and the CLI uses this so both read the same grammar.

func ParseRunTimeout

func ParseRunTimeout(s string) (time.Duration, error)

ParseRunTimeout reads a ds:run timeout= value: a positive Go duration (`300ms`, `30s`, `2m`), the grammar run.timeout uses. The key was accepted and never applied (bug 86); a value that does not parse is now a `problem` at the directive rather than a silent fall back to run.timeout.

func ResolveDef

func ResolveDef(cands []block.Block, env, branch string) (block.Block, bool)

ResolveDef picks, from the defs sharing one id, the one a citation with the given env and branch resolves to: a def on that branch (or on none, for a citation naming no branch), then an exact env match; a citation with no env takes the first; one whose env no def names takes the def with no env.

Exported so scan's snapshot resolves a citation exactly as check does. Recording every citation of an id against whichever def came last gave a dev citation the prod value's hash, and a fresh repo with per-env defs reported its citations changed before anything had changed.

func Tally

func Tally(findings []Finding) (map[State]int, map[Severity]int, int)

Tally counts findings by state and by severity, and gives the exit code: 1 when any finding is an error. It is the one place a report is counted, so a report changed after Run counts exactly as Run would have.

func WithPublishedBody

func WithPublishedBody(b block.Block, bodyAt func(hash string) (string, bool)) block.Block

WithPublishedBody returns b with its content filled from the body store when b is another repository's block (block.KeyRepo set) and arrived without one, as every merged def does: the index carries ledger rows, and bodies separately by hash. A copy or a render of such a block was its title and a link only (bug 105). A secret or local block publishes no body, so it stays empty, and so does any miss.

Types

type Baseline

type Baseline string

Baseline names what a drifted finding was measured against (§26.2).

const (
	BaselineAck  Baseline = "ack"
	BaselineSeen Baseline = "seen"
)

The baselines: the hash of an explicit ack, or the hash recorded when the citation was first seen and nobody has acked it since.

type Finding

type Finding struct {
	State    State    `json:"state"`
	Severity Severity `json:"severity"`
	Message  string   `json:"message"`
	// Where the finding attaches: the doc line of the reference, or the
	// directive line of a def-level finding. DocRepo names the repository
	// the doc lives in when it is not this one (a merged workspace ref).
	Doc      string `json:"doc"`
	DocRepo  string `json:"doc_repo,omitempty"`
	Line     int    `json:"line"`
	Sentence string `json:"sentence,omitempty"`
	// What it is about.
	ID      string        `json:"id,omitempty"`
	Repo    string        `json:"repo,omitempty"`
	File    string        `json:"file,omitempty"`
	Lines   [2]int        `json:"lines,omitempty"`
	Verb    string        `json:"verb,omitempty"`
	Carrier block.Carrier `json:"carrier,omitempty"`
	Owner   string        `json:"owner,omitempty"`
	// Change detail for unacked and translation-stale.
	Classes []block.Class `json:"class,omitempty"`
	// Baseline says what the current hash was compared against: an explicit
	// ack, or the hash recorded when the citation was first seen. It lets a
	// consumer tell "a reviewed statement went stale" from "nobody has
	// reviewed this yet" without parsing the message.
	Baseline Baseline `json:"baseline,omitempty"`
	Hash     HashPair `json:"hash"`
	Diff     string   `json:"diff,omitempty"`
	// Remedy is the concrete command or edit that clears the finding.
	Remedy Remedy `json:"remedy"`
}

Finding is one row of `check` output. Fields are the JSON contract (§26.2); the JSON tags are the wire names and must not change within json_format 1.

type HashPair

type HashPair struct {
	Acked   string `json:"acked,omitempty"`
	Current string `json:"current,omitempty"`
}

HashPair is the block hash the sentence was last acked against and the hash it has now (§26.2 `hash`). Both empty means the finding is not about a block.

type Input

type Input struct {
	Repo string
	Now  time.Time
	// Prefix is the directive prefix, needed to render repo-mode copies for
	// comparison; empty means the default.
	Prefix string
	// Env is the default environment for references that do not say.
	Env string
	// Defs and Refs are the current scan; Merged are blocks from other repos
	// in the workspace (resolved through sync), referable but not matched.
	Defs         []block.Block
	Merged       []block.Block
	Refs         []block.Reference
	MergedRefs   []RemoteRef
	Problems     []scan.Problem
	Undocumented []Undocumented
	// Removed maps ids that were published by repositories since removed
	// from the workspace to that repo (§21): a reference to one is broken
	// with a message that says so rather than "deleted".
	Removed map[string]string
	// PrevForeign is the committed snapshot's rows, which is what a merged
	// block's previous position was at the last `ds sync`. `moved` is a
	// scan-to-scan state and needs a previous row; for a foreign block the
	// citing repo's own ledger never had one, so until the snapshot existed
	// a block that moved upstream could not be reported at all.
	PrevForeign []ledger.Row
	// SnapshotOnly says Merged came from the committed foreign snapshot
	// rather than the live workspace index (§21 `check --frozen`). It
	// changes what an unresolvable id means: not "nothing defines this"
	// but "nothing here defines this, and the snapshot does not record
	// it", which are different problems with different remedies.
	SnapshotOnly bool
	// Unreadable are files the scan skipped because of their form (see
	// scan.SkipReason.Unreadable). One that held citations or defs at the
	// last scan is reported: its citations are not being checked, and a
	// check that passed without saying so would be a wrong "up to date".
	Unreadable []scan.Skip
	Pages      map[string]extract.Page
	// Prev is the previous ledger for this repo; PrevRefs its reverse index;
	// Acks the append-only log.
	Prev     ledger.Ledger
	PrevRefs ledger.Refs
	Acks     ledger.Acks
	Opts     Options
}

Input is everything a run needs.

type LocalResult

type LocalResult struct {
	Present bool
	Err     error
}

LocalResult is what the Local hook reports for one `local=true` def. Present says the target file is readable here; Err, with Present, says the def's pick= found nothing in it. Neither carries the content: a local target is never hashed into the ledger, because the ledger is shared and the file is on one machine only.

type Options

type Options struct {
	MaxLines         int
	FuzzyThreshold   float64
	UnackedIsWarning bool
	Strict           bool
	// RepoMode is [include] mode = "repo": every block-position ds:block
	// is a copy `ds refresh` writes into the page, so a copy refresh cannot
	// write (a bad lines=) is an error, not a warning (bug 136). A citation
	// that already has a copy region is treated the same without it.
	RepoMode bool
	// Wording holds an ack to the sentence it approved (SPEC §18): a cited
	// sentence rewritten since its ack, or acked under an older sentence
	// rule, is unacked. False is [check] sentence = "position".
	Wording    bool
	OldContent func(row ledger.Row) (string, bool)
	// BodyAt returns a block's normalized body by its content hash, from the
	// body store (§20.1). It is what OldContent cannot be: the acked hash of
	// a citation may be many commits behind the previous ledger, and for a
	// merged def there is no local ledger row at all, so a by-id lookup
	// keyed to the previous scan cannot answer it. Nil, or a miss, degrades
	// a drift to block.ClassUnknown rather than a guess.
	BodyAt          func(hash string) (string, bool)
	CommentPrefixes func(file string) []string
	// TestResults maps a test block id to its last published outcome; nil
	// means no run was published and assert= references check as ordinary.
	TestResults map[string]TestOutcome
	// CommitExists answers `at=` pins; nil means unverifiable.
	CommitExists func(sha string) bool
	// URLCheck answers `ds:url`; nil means unverifiable.
	URLCheck func(href string) URLResult
	// Resolve answers secret addresses under --resolve; nil skips it.
	Resolve Resolver
	// StoredHashes are truth hashes kept from an earlier run when
	// resolve.store_hash is on, keyed by id; a differing hash is `rotated`.
	StoredHashes map[string]string
	// HasRecords says a record source is configured for `ds:table`.
	HasRecords bool
	// RunEnabled says `ds:run` directives may execute; the checker still
	// never executes anything, it only stops reporting them as skipped.
	RunEnabled bool
	// Local answers whether a `local=true` def's target is on this machine
	// and its pick= finds something (§9.1, §12). Nil, or Present false, is
	// `unverifiable`, the state everywhere the file is absent; before this
	// hook it was the state even beside the file (bug 85).
	Local func(b block.Block) LocalResult
	// ExtraVerbs are registered plugin verbs, so they are not `unknown`.
	ExtraVerbs map[string]bool
	// Handlers evaluate plugin verbs (§9.9, §37.3 Verb). A verb with no
	// handler is recorded as ok; one with a handler gets its findings.
	Handlers map[string]VerbHandler
	// KnownKeys extends the per-verb key tables for plugin verbs, so their
	// keys are not `unknown`; RequiredKeys makes a missing key a problem.
	KnownKeys    map[string][]string
	RequiredKeys map[string][]string
	// Classify replaces the change classifier (§37.3 Classifier).
	Classify match.Classifier
	// KnownEnvs is [env] known. When it lists any, an env= on this repo's
	// own defs and citations that it does not list is reported `unknown`:
	// a misspelled environment otherwise reads as a missing definition, or
	// as a def nobody can ever select. Empty disables the test.
	KnownEnvs []string
	// Command is the name of the binary a finding's message and remedy
	// tell the reader to run ("pds ack …"). Empty means DefaultCommand. A
	// custom build named itself in help and still told its users to run a
	// `ds` they did not have.
	Command string
}

Options tunes a run. Every hook is optional; absence degrades to an honest `unverifiable` or `skipped`.

type Remedy

type Remedy struct {
	IfStillTrue string `json:"if_still_true,omitempty"`
	IfNot       string `json:"if_not,omitempty"`
	Fix         string `json:"fix,omitempty"`
}

Remedy is what clears a finding. `unacked` offers the two branches the spec names (§26.2): ack if the sentence is still true, edit if not. Every other state has one Fix. An agent executes these verbatim; the texts are commands and file positions, never scanned content.

type RemoteRef

type RemoteRef struct {
	Repo      string
	Ref       block.Reference
	AckedHash string
	// SeenHash is that repo's recorded first-seen baseline for the citation,
	// used when it has never been acked.
	SeenHash string
}

RemoteRef is a reference published by another repository in the workspace that points at this repository's blocks (§21). Its acked hash comes from that repo's published refs, since its ack log is not here.

type Report

type Report struct {
	Findings   []Finding
	Changes    []match.Change
	Summary    map[State]int
	BySeverity map[Severity]int
	ExitCode   int
	// TruthHashes are the hashes resolved for truth=true defs this run, for
	// the caller to store when resolve.store_hash is on.
	TruthHashes map[string]string
}

Report is the result of Run.

func ApplyRuns

func ApplyRuns(rep Report, runs []RunResult) Report

ApplyRuns turns the outcomes of executed `ds:run` directives into their findings and recounts the report. Before it, check said "run directive will execute where enabled" for every run and the caller only printed what happened, so a failed command made `check --run` exit 1 with no finding naming it and a summary of only passing findings (bug 29). A run that was not executed becomes `skipped` with the reason. Findings with no result are left alone.

func Run

func Run(in Input) Report

Run performs the evaluation.

type ResolveResult

type ResolveResult struct {
	Checked bool
	Exists  bool
	Hash    string
	Err     error
}

ResolveResult is what a secret resolver reports for one address (§12). A resolver returns existence and, where the provider can be read, a hash of the value; never the value. Checked is false when the provider could not be reached, which is reported as unverifiable.

type Resolver

type Resolver func(provider, addr string) ResolveResult

Resolver answers for one address at one provider.

type RunResult

type RunResult struct {
	// Doc and Line locate the directive.
	Doc  string
	Line int
	// Command is what ran, as the directive gave it.
	Command string
	// Failed says the command exited non-zero, timed out, or did not
	// print what expect= asked for.
	Failed bool
	// Skipped, when not empty, says why the command was not run: an id
	// that is not runnable=true, a cmd= outside run.allow.
	Skipped string
}

RunResult is what executing one `ds:run` directive came to. The checker never executes anything, so the caller that did reports back through ApplyRuns, and the outcome becomes the directive's finding.

type Severity

type Severity string

Severity decides the exit code.

const (
	SeverityError   Severity = "error"
	SeverityWarning Severity = "warning"
	SeverityInfo    Severity = "info"
	SeverityNone    Severity = "none"
)

The severities, most to least severe. Any error makes `check` exit 1; the rest are reported and do not fail it.

func SeverityOf

func SeverityOf(st State) Severity

SeverityOf returns the table severity for a state; unknown states are errors so a typo in a plugin can never pass silently.

type State

type State string

State is one row of the findings table (docs/SPEC.md §17).

const (
	StateOK               State = "ok"
	StateMoved            State = "moved"
	StateUnacked          State = "unacked"
	StateBroken           State = "broken"
	StatePickFailed       State = "pick failed"
	StateTooLarge         State = "too-large"
	StateRange            State = "range"
	StateExpired          State = "expired"
	StateSunset           State = "sunset"
	StateDeprecated       State = "deprecated"
	StateAssertFailed     State = "assert failed"
	StateTranslationStale State = "translation stale"
	StateUnsourced        State = "unsourced"
	StateChainBroken      State = "chain broken"
	StateUnverifiable     State = "unverifiable"
	StateSkipped          State = "skipped"
	StateOrphan           State = "orphan"
	StateUncovered        State = "uncovered"
	StateUnknown          State = "unknown"
	StateProblem          State = "problem"
	StateDead             State = "dead"
	StateRetitled         State = "retitled"
	StateURLMoved         State = "url moved"
	StateUndocumented     State = "undocumented export"
	StateResolveFailed    State = "resolve failed"
	StateOutOfSync        State = "out of sync"
	StateStaleCopy        State = "stale copy"
	StateRotated          State = "rotated"
	StateStale            State = "stale"
	StateTampered         State = "tampered"
	StateUnscanned        State = "unscanned"
	StateRunFailed        State = "run failed"
)

The finding states. Each is one row of the docs/SPEC.md §17 table, which says what it means, its severity (severityOf), and how it is cleared; a new state needs all three plus a conformance fixture.

type TestOutcome

type TestOutcome string

TestOutcome is what a published CI run recorded for a test id (§21).

const (
	TestPassed  TestOutcome = "passed"
	TestFailed  TestOutcome = "failed"
	TestSkipped TestOutcome = "skipped"
)

The outcomes a CI run can publish for a test id: read from its JUnit report and stored in the workspace index's tests file, which refuses any other value.

type URLResult

type URLResult struct {
	Status  int
	Final   string // final URL after redirects
	Title   string
	Err     error
	Checked bool // false when the hook could not check (offline)
}

URLResult is what a url checker hook reports. The two error shapes mean different things and are reported differently:

• Checked false, Err set: no HTTP answer came back (offline, DNS, refused, timed out). Reported `unverifiable`; a hook must not cache it.
• Checked true, Err set: the request could not even be formed from href. Reported `problem` at the directive.

type Undocumented

type Undocumented struct {
	File   string
	Line   int
	Symbol string
}

Undocumented is an exported declaration under a policy.require_doc path that has no def (§23). The root computes these from the scan; check only turns them into findings.

type VerbHandler

type VerbHandler func(ref block.Reference) []Finding

VerbHandler is a plugin verb's check. Findings it returns get the doc, line, verb, carrier, and id of the reference filled in when empty, and the table severity applied, so a handler only says what it found.