check

Documentation

Overview

Package check turns a scan, the previous ledger, and the ack log into findings: one row per reference or per problem, with a state from the spec's table, a severity, the change class and diff where relevant, and a remedy an agent can execute verbatim (docs/SPEC.md §16 passes 3 to 5, §17).

It is a pure function of its Input. Nothing here reads files, runs commands, or touches the network. Things that need any of those (resolvers, url checks, `run`, commit lookups) arrive as optional hooks in Options and are reported as `unverifiable` or `skipped` when absent, never silently passed.

Index

• Constants
• Variables
• func ParseDuration(s string) (time.Duration, error)
• func ResolveDef(cands []block.Block, env, branch string) (block.Block, bool)
• type Baseline
• type Finding
• type HashPair
• type Input
• type Options
• type Remedy
• type RemoteRef
• type Report
  • func Run(in Input) Report
• type ResolveResult
• type Resolver
• type Severity
  • func SeverityOf(st State) Severity
• type State
• type TestOutcome
• type URLResult
• type Undocumented
• type VerbHandler

Constants

const DateLayout = "2006-01-02"

DateLayout is the only date format accepted in directives. One format, no locale guessing.

const DefaultMaxLines = 40

DefaultMaxLines is the rendered-block cap (§9.2).

Variables

var BaselineValues = []Baseline{BaselineAck, BaselineSeen}

BaselineValues is the canonical order.

var SeverityValues = []Severity{SeverityError, SeverityWarning, SeverityInfo, SeverityNone}

SeverityValues is the canonical order.

var StateValues = []State{StateOK, StateMoved, StateUnacked, StateBroken, StatePickFailed, StateTooLarge, StateRange, StateExpired, StateSunset, StateDeprecated, StateAssertFailed, StateTranslationStale, StateUnsourced, StateChainBroken, StateUnverifiable, StateSkipped, StateOrphan, StateUncovered, StateUnknown, StateProblem, StateDead, StateRetitled, StateURLMoved, StateUndocumented, StateResolveFailed, StateOutOfSync, StateRotated, StateStale, StateTampered, StateUnscanned}

StateValues is the canonical order, used by the summary. dsself:def id=statevalues-vwvk2kq3 owner=@docsync stability=stable

Functions

func ParseDuration

func ParseDuration(s string) (time.Duration, error)

ParseDuration reads the duration shape docsync writes. The grammar lives in internal/duration so configuration can validate the same shape without importing the checker.

func ResolveDef

func ResolveDef(cands []block.Block, env, branch string) (block.Block, bool)

ResolveDef picks, from the defs sharing one id, the one a citation with the given env and branch resolves to: a def on that branch (or on none, for a citation naming no branch), then an exact env match; a citation with no env takes the first; one whose env no def names takes the def with no env.

Exported so scan's snapshot resolves a citation exactly as check does. Recording every citation of an id against whichever def came last gave a dev citation the prod value's hash, and a fresh repo with per-env defs reported its citations changed before anything had changed.

Types

type Baseline

type Baseline string

Baseline names what a drifted finding was measured against (§26.2).

const (
	BaselineAck  Baseline = "ack"
	BaselineSeen Baseline = "seen"
)

The baselines: the hash of an explicit ack, or the hash recorded when the citation was first seen and nobody has acked it since.

type Finding

type Finding struct {
	State    State    `json:"state"`
	Severity Severity `json:"severity"`
	Message  string   `json:"message"`
	// Where the finding attaches: the doc line of the reference, or the
	// directive line of a def-level finding. DocRepo names the repository
	// the doc lives in when it is not this one (a merged workspace ref).
	Doc      string `json:"doc"`
	DocRepo  string `json:"doc_repo,omitempty"`
	Line     int    `json:"line"`
	Sentence string `json:"sentence,omitempty"`
	// What it is about.
	ID      string        `json:"id,omitempty"`
	Repo    string        `json:"repo,omitempty"`
	File    string        `json:"file,omitempty"`
	Lines   [2]int        `json:"lines,omitempty"`
	Verb    string        `json:"verb,omitempty"`
	Carrier block.Carrier `json:"carrier,omitempty"`
	Owner   string        `json:"owner,omitempty"`
	// Change detail for unacked and translation-stale.
	Classes []block.Class `json:"class,omitempty"`
	// Baseline says what the current hash was compared against: an explicit
	// ack, or the hash recorded when the citation was first seen. It lets a
	// consumer tell "a reviewed statement went stale" from "nobody has
	// reviewed this yet" without parsing the message.
	Baseline Baseline `json:"baseline,omitempty"`
	Hash     HashPair `json:"hash"`
	Diff     string   `json:"diff,omitempty"`
	// Remedy is the concrete command or edit that clears the finding.
	Remedy Remedy `json:"remedy"`
}

Finding is one row of `check` output. Fields are the JSON contract (§26.2); the JSON tags are the wire names and must not change within json_format 1.

type HashPair

type HashPair struct {
	Acked   string `json:"acked,omitempty"`
	Current string `json:"current,omitempty"`
}

HashPair is the block hash the sentence was last acked against and the hash it has now (§26.2 `hash`). Both empty means the finding is not about a block.

type Input

type Input struct {
	Repo string
	Now  time.Time
	// Prefix is the directive prefix, needed to render repo-mode copies for
	// comparison; empty means the default.
	Prefix string
	// Env is the default environment for references that do not say.
	Env string
	// Defs and Refs are the current scan; Merged are blocks from other repos
	// in the workspace (resolved through sync), referable but not matched.
	Defs         []block.Block
	Merged       []block.Block
	Refs         []block.Reference
	MergedRefs   []RemoteRef
	Problems     []scan.Problem
	Undocumented []Undocumented
	// Removed maps ids that were published by repositories since removed
	// from the workspace to that repo (§21): a reference to one is broken
	// with a message that says so rather than "deleted".
	Removed map[string]string
	// PrevForeign is the committed snapshot's rows, which is what a merged
	// block's previous position was at the last `ds sync`. `moved` is a
	// scan-to-scan state and needs a previous row; for a foreign block the
	// citing repo's own ledger never had one, so until the snapshot existed
	// a block that moved upstream could not be reported at all.
	PrevForeign []ledger.Row
	// SnapshotOnly says Merged came from the committed foreign snapshot
	// rather than the live workspace index (§21 `check --frozen`). It
	// changes what an unresolvable id means: not "nothing defines this"
	// but "nothing here defines this, and the snapshot does not record
	// it", which are different problems with different remedies.
	SnapshotOnly bool
	// Unreadable are files the scan skipped because of their form (see
	// scan.SkipReason.Unreadable). One that held citations or defs at the
	// last scan is reported: its citations are not being checked, and a
	// check that passed without saying so would be a wrong "up to date".
	Unreadable []scan.Skip
	Pages      map[string]extract.Page
	// Prev is the previous ledger for this repo; PrevRefs its reverse index;
	// Acks the append-only log.
	Prev     ledger.Ledger
	PrevRefs ledger.Refs
	Acks     ledger.Acks
	Opts     Options
}

Input is everything a run needs.

type Options

type Options struct {
	MaxLines         int
	FuzzyThreshold   float64
	UnackedIsWarning bool
	Strict           bool
	// Wording holds an ack to the sentence it approved (SPEC §18): a cited
	// sentence rewritten since its ack, or acked under an older sentence
	// rule, is unacked. False is [check] sentence = "position".
	Wording    bool
	OldContent func(row ledger.Row) (string, bool)
	// BodyAt returns a block's normalized body by its content hash, from the
	// body store (§20.1). It is what OldContent cannot be: the acked hash of
	// a citation may be many commits behind the previous ledger, and for a
	// merged def there is no local ledger row at all, so a by-id lookup
	// keyed to the previous scan cannot answer it. Nil, or a miss, degrades
	// a drift to block.ClassUnknown rather than a guess.
	BodyAt          func(hash string) (string, bool)
	CommentPrefixes func(file string) []string
	// TestResults maps a test block id to its last published outcome; nil
	// means no run was published and assert= references check as ordinary.
	TestResults map[string]TestOutcome
	// CommitExists answers `at=` pins; nil means unverifiable.
	CommitExists func(sha string) bool
	// URLCheck answers `ds:url`; nil means unverifiable.
	URLCheck func(href string) URLResult
	// Resolve answers secret addresses under --resolve; nil skips it.
	Resolve Resolver
	// StoredHashes are truth hashes kept from an earlier run when
	// resolve.store_hash is on, keyed by id; a differing hash is `rotated`.
	StoredHashes map[string]string
	// HasRecords says a record source is configured for `ds:table`.
	HasRecords bool
	// RunEnabled says `ds:run` directives may execute; the checker still
	// never executes anything, it only stops reporting them as skipped.
	RunEnabled bool
	// ExtraVerbs are registered plugin verbs, so they are not `unknown`.
	ExtraVerbs map[string]bool
	// Handlers evaluate plugin verbs (§9.9, §37.3 Verb). A verb with no
	// handler is recorded as ok; one with a handler gets its findings.
	Handlers map[string]VerbHandler
	// KnownKeys extends the per-verb key tables for plugin verbs, so their
	// keys are not `unknown`; RequiredKeys makes a missing key a problem.
	KnownKeys    map[string][]string
	RequiredKeys map[string][]string
	// Classify replaces the change classifier (§37.3 Classifier).
	Classify match.Classifier
}

Options tunes a run. Every hook is optional; absence degrades to an honest `unverifiable` or `skipped`.

type Remedy

type Remedy struct {
	IfStillTrue string `json:"if_still_true,omitempty"`
	IfNot       string `json:"if_not,omitempty"`
	Fix         string `json:"fix,omitempty"`
}

Remedy is what clears a finding. `unacked` offers the two branches the spec names (§26.2): ack if the sentence is still true, edit if not. Every other state has one Fix. An agent executes these verbatim; the texts are commands and file positions, never scanned content.

type RemoteRef

type RemoteRef struct {
	Repo      string
	Ref       block.Reference
	AckedHash string
	// SeenHash is that repo's recorded first-seen baseline for the citation,
	// used when it has never been acked.
	SeenHash string
}

RemoteRef is a reference published by another repository in the workspace that points at this repository's blocks (§21). Its acked hash comes from that repo's published refs, since its ack log is not here.

type Report

type Report struct {
	Findings   []Finding
	Changes    []match.Change
	Summary    map[State]int
	BySeverity map[Severity]int
	ExitCode   int
	// TruthHashes are the hashes resolved for truth=true defs this run, for
	// the caller to store when resolve.store_hash is on.
	TruthHashes map[string]string
}

Report is the result of Run.

func Run

func Run(in Input) Report

Run performs the evaluation.

type ResolveResult

type ResolveResult struct {
	Checked bool
	Exists  bool
	Hash    string
	Err     error
}

ResolveResult is what a secret resolver reports for one address (§12). A resolver returns existence and, where the provider can be read, a hash of the value; never the value. Checked is false when the provider could not be reached, which is reported as unverifiable.

type Resolver

type Resolver func(provider, addr string) ResolveResult

Resolver answers for one address at one provider.

type Severity

type Severity string

Severity decides the exit code.

const (
	SeverityError   Severity = "error"
	SeverityWarning Severity = "warning"
	SeverityInfo    Severity = "info"
	SeverityNone    Severity = "none"
)

The severities, most to least severe. Any error makes `check` exit 1; the rest are reported and do not fail it.

func SeverityOf

func SeverityOf(st State) Severity

SeverityOf returns the table severity for a state; unknown states are errors so a typo in a plugin can never pass silently.

type State

type State string

State is one row of the findings table (docs/SPEC.md §17).

const (
	StateOK               State = "ok"
	StateMoved            State = "moved"
	StateUnacked          State = "unacked"
	StateBroken           State = "broken"
	StatePickFailed       State = "pick failed"
	StateTooLarge         State = "too-large"
	StateRange            State = "range"
	StateExpired          State = "expired"
	StateSunset           State = "sunset"
	StateDeprecated       State = "deprecated"
	StateAssertFailed     State = "assert failed"
	StateTranslationStale State = "translation stale"
	StateUnsourced        State = "unsourced"
	StateChainBroken      State = "chain broken"
	StateUnverifiable     State = "unverifiable"
	StateSkipped          State = "skipped"
	StateOrphan           State = "orphan"
	StateUncovered        State = "uncovered"
	StateUnknown          State = "unknown"
	StateProblem          State = "problem"
	StateDead             State = "dead"
	StateRetitled         State = "retitled"
	StateURLMoved         State = "url moved"
	StateUndocumented     State = "undocumented export"
	StateResolveFailed    State = "resolve failed"
	StateOutOfSync        State = "out of sync"
	StateRotated          State = "rotated"
	StateStale            State = "stale"
	StateTampered         State = "tampered"
	StateUnscanned        State = "unscanned"
)

The finding states. Each is one row of the docs/SPEC.md §17 table, which says what it means, its severity (severityOf), and how it is cleared; a new state needs all three plus a conformance fixture.

type TestOutcome

type TestOutcome string

TestOutcome is what a published CI run recorded for a test id (§21).

const (
	TestPassed  TestOutcome = "passed"
	TestFailed  TestOutcome = "failed"
	TestSkipped TestOutcome = "skipped"
)

The outcomes a CI run can publish for a test id: read from its JUnit report and stored in the workspace index's tests file, which refuses any other value.

type URLResult

type URLResult struct {
	Status  int
	Final   string // final URL after redirects
	Title   string
	Err     error
	Checked bool // false when the hook could not check (offline)
}

URLResult is what a url checker hook reports.

type Undocumented

type Undocumented struct {
	File   string
	Line   int
	Symbol string
}

Undocumented is an exported declaration under a policy.require_doc path that has no def (§23). The root computes these from the scan; check only turns them into findings.

type VerbHandler

type VerbHandler func(ref block.Reference) []Finding

VerbHandler is a plugin verb's check. Findings it returns get the doc, line, verb, carrier, and id of the reference filled in when empty, and the table severity applied, so a handler only says what it found.