verifier

Documentation

Overview

Package verifier turns sandbox stage runs into signed verification receipts (goal.md §7.7, §16) and implements the cross-verification client that claims jobs from the server, re-verifies artifacts locally, and posts receipts back.

Index

• Constants
• func ResolvedEdges(ctx context.Context, dir string, m domain.SampleManifest, ...) ([]scanner.Edge, bool)
• func ResolvedLeaves(ctx context.Context, dir string, m domain.SampleManifest, ...) []domain.PURL
• func Run(ctx context.Context, r sandbox.Runner, cap domain.SandboxCapability, ...) (domain.VerificationReceipt, error)
• func RunLogged(ctx context.Context, r sandbox.Runner, cap domain.SandboxCapability, ...) (domain.VerificationReceipt, map[string]string, error)
• func TreeBatches(edges []scanner.Edge, declaredLeaves []domain.PURL, resolved []string, ...) []domain.ObservationBatch
• type ArtifactSource
• type AuditedReceipt
  • func ReadReceiptDump(r io.Reader) ([]AuditedReceipt, error)
• type CrossVerifier
  • func (cv *CrossVerifier) DownloadArtifact(ctx context.Context, sampleID string) ([]byte, error)
  • func (cv *CrossVerifier) FetchJob(ctx context.Context) (*Job, error)
  • func (cv *CrossVerifier) IsIdle() bool
  • func (cv *CrossVerifier) RunBudget(ctx context.Context, budget string, once bool) error
  • func (cv *CrossVerifier) RunOne(ctx context.Context) (worked bool, err error)
  • func (cv *CrossVerifier) SetStageLogSink(fn func(path string))
  • func (cv *CrossVerifier) UnsupportedWork() []string
  • func (cv *CrossVerifier) VerifyAndReport(ctx context.Context, job *Job) (domain.VerificationReceipt, error)
• type Job
• type ReceiptAudit
  • func AuditReceipts(rows []AuditedReceipt) ReceiptAudit
• type StageLogStore
  • func (s *StageLogStore) Keep(sampleID string, logs, stages map[string]string) (string, error)

Constants

const (
	CrossJobReason  = "cross"
	MatrixJobReason = "matrix"
)

Functions

func ResolvedEdges

func ResolvedEdges(ctx context.Context, dir string, m domain.SampleManifest, all []scanner.Adapter) ([]scanner.Edge, bool)

ResolvedEdges is the dependency tree this sample's resolver actually wrote into the workspace: (parent, child) pairs with both ends at concrete versions, exactly as the lockfile records them.

These are DIRECT edges, which is what makes them reportable at all. A receipt's resolved-package list is a flat closure, and turning that into edges would claim transitive dependencies as direct — an assertion nothing measured. The lockfile says who pulled whom; that is the fact, and it is the half of a version conflict a person can actually act on.

func ResolvedLeaves

func ResolvedLeaves(ctx context.Context, dir string, m domain.SampleManifest, all []scanner.Adapter) []domain.PURL

ResolvedLeaves reads explicit empty declarations from this verification's resolver. Other ecosystems and unreadable files contribute no absence facts.

func Run

func Run(
	ctx context.Context,
	r sandbox.Runner,
	cap domain.SandboxCapability,
	sampleDir string,
	m domain.SampleManifest,
	ident *identity.Identity,
	env domain.EnvironmentFingerprint,
) (domain.VerificationReceipt, error)

Run executes the verification pipeline over an unpacked sample dir and returns a signed receipt. Stage semantics:

• resolve → compile → contract run in order; a FAIL short-circuits every later stage to SKIPPED (never guessed results).
• load is derived, never guessed: PASS only when the contract actually ran and passed (the module necessarily loaded); SKIPPED otherwise.

Full logs stay local; only their SHA-256 digest enters the receipt. The sample ID is recomputed from the directory via the canonical artifact builder, so a receipt can never mis-attribute results.

func RunLogged

func RunLogged(
	ctx context.Context,
	r sandbox.Runner,
	cap domain.SandboxCapability,
	sampleDir string,
	m domain.SampleManifest,
	ident *identity.Identity,
	env domain.EnvironmentFingerprint,
) (domain.VerificationReceipt, map[string]string, error)

RunLogged is Run, and also returns each stage's output.

The receipt stores only a DIGEST of the logs, deliberately: raw output carries paths, usernames and tokens and must never leave the machine (goal.md §8.5). But the logs were then discarded entirely, so an author whose contract failed got a table of PASS/FAIL and nothing else -- no way to see what their own assertion did, on their own machine, in a container they cannot enter. The only way to find out was to rebuild the container by hand and guess at the flags.

That is the loop the whole contribution side depends on. These logs stay local: they go to the author's terminal and nowhere near a receipt.

func TreeBatches

func TreeBatches(edges []scanner.Edge, declaredLeaves []domain.PURL, resolved []string, m domain.SampleManifest, r domain.VerificationReceipt, epoch string) []domain.ObservationBatch

TreeBatches is the dependency tree a verification resolved, as the wire units that carry edges.

The batches are deliberately identical in identity to the ones the server already derives from this receipt — same peer, same sample-derived bucket, same day. A verification whose edges landed in a different bucket from its own observations would be counted as two projects, and a count of projects is the entire meaning of the number those edges feed.

Nothing here is anybody's private code. A cross-verification workspace is a fresh temp directory holding a published sample and whatever the public registry resolved into it; the machine's own projects are never in it. That is why this path needs no publicness pass, and why it must never be pointed at a directory a person works in.

Types

type ArtifactSource

type ArtifactSource interface {
	// Fetch returns the artifact bytes and the source they came from
	// ("local" | "peer" | "server").
	Fetch(ctx context.Context, sampleID string) ([]byte, string, error)
}

ArtifactSource is the peer node's fetch chain, kept as an interface so the verifier does not depend on the peer package (the daemon wires them).

type AuditedReceipt

type AuditedReceipt struct {
	ReceiptID string
	PeerID    string
	SampleID  string
	EnvHash   string
	CreatedAt string
	Receipt   domain.VerificationReceipt
	// RawReceipt is the receipt exactly as stored, before this package's
	// schema saw it. Without it the round-trip check below has nothing to
	// compare against and cannot fail.
	RawReceipt []byte
}

AuditedReceipt is one stored receipt as the store holds it: the signed document plus the columns it was indexed by. The columns are carried separately on purpose — an audit that reads the id out of the document it is checking cannot detect a document that was replaced after storage.

func ReadReceiptDump

func ReadReceiptDump(r io.Reader) ([]AuditedReceipt, error)

ReadReceiptDump decodes a production receipt dump: one base64-encoded JSON row per line, blank lines ignored.

Base64 because `COPY (...) TO STDOUT` uses Postgres' text format, which escapes backslashes — every \" inside a stored manifest comes back broken, and the corruption looks like a malformed receipt rather than a bad export. The producing query lives in docs/operations.md.

type CrossVerifier

type CrossVerifier struct {
	HTTP      *http.Client
	ServerURL string
	Ident     *identity.Identity
	Cap       domain.SandboxCapability

	// Runner overrides the capability-derived sandbox runner (tests, or a
	// daemon that already detected capability). Nil picks Docker for
	// CONTAINER_RUN and the native COMPILE_ONLY runner otherwise.
	Runner sandbox.Runner
	// ContainerOS is the kind of container this machine's Docker daemon
	// serves ("linux" or "windows"; empty means linux). It decides both
	// which jobs are claimable and which OS the receipt reports, so a
	// Windows worker contributes Windows evidence instead of quietly
	// producing Linux results on a Windows machine.
	ContainerOS string
	// Env is this peer's environment fingerprint stamped into receipts.
	Env domain.EnvironmentFingerprint
	// WorkDir hosts per-job temp workspaces; empty means the OS temp dir.
	WorkDir string
	// LastActivityFile is touched by csx run; its mtime drives the
	// "idle" budget (only verify when the user is not actively working).
	LastActivityFile string
	// StageLogs keeps the output of a verification that failed, on this
	// machine only. Nil disables it entirely, which is what tests and any
	// caller without a home directory want.
	StageLogs *StageLogStore
	// OnStageLogs is called with the path of a kept log so the worker can
	// print it. Without it a failed job left no trace an operator could
	// follow: stdout carried a "job completed" counter and nothing else.
	OnStageLogs func(path string)
	// OnVerified is called after each job whose receipt was accepted.
	//
	// Nothing counted them. The dashboard reads a crossVerifications
	// counter that no code path ever wrote, so a peer could spend hours
	// verifying other people's samples and its own stats would report zero
	// forever -- the one number that shows a peer giving back rather than
	// taking. Optional: nil simply counts nothing.
	OnVerified func()
	// Source resolves artifacts from the cheapest place first (local CAS,
	// then announced peers, then the server) — goal.md §15.1. *peer.Node
	// implements it. Nil downloads straight from the server.
	Source ArtifactSource
	// contains filtered or unexported fields
}

CrossVerifier claims verification jobs from the central server, re-verifies the artifacts in the local sandbox, and posts signed receipts back. It uses the persistent peer identity (goal.md §8.6) — verification is explicit-identity work, unlike anonymous evidence.

func (*CrossVerifier) DownloadArtifact

func (cv *CrossVerifier) DownloadArtifact(ctx context.Context, sampleID string) ([]byte, error)

DownloadArtifact returns the sample artifact, verifying that its SHA-256 equals the content-addressed sampleID BEFORE anything unpacks it. With a Source set it walks local CAS → peers → server first; any failure there falls back to a direct server download, so a broken peer path can never stall verification.

func (*CrossVerifier) FetchJob

func (cv *CrossVerifier) FetchJob(ctx context.Context) (*Job, error)

FetchJob asks the server for one open job matching this peer's capability and claims it via POST /v1/verification/jobs/{id}/claim. (nil, nil) means no work is available.

func (*CrossVerifier) IsIdle

func (cv *CrossVerifier) IsIdle() bool

IsIdle reports whether the last recorded csx run activity is older than the idle window. A missing activity file counts as idle.

func (*CrossVerifier) RunBudget

func (cv *CrossVerifier) RunBudget(ctx context.Context, budget string, once bool) error

RunBudget runs the claim→verify→report loop under the user's idle verification budget (goal.md §9.6): "off", "5m", "15m", "idle", or "unlimited". "idle" only verifies when no csx run happened within the last 10 minutes (mtime of LastActivityFile). With once=true at most one job is processed. It returns when the budget is spent, the queue is empty, or ctx is done.

func (*CrossVerifier) RunOne

func (cv *CrossVerifier) RunOne(ctx context.Context) (worked bool, err error)

RunOne claims and processes at most one cross-verification job. worked is true once a job was claimed, including when its verification or receipt submission failed; callers use that distinction to report job failures separately from queue/network errors.

func (*CrossVerifier) SetStageLogSink

func (cv *CrossVerifier) SetStageLogSink(fn func(path string))

SetStageLogSink installs the callback that announces a kept log. The worker uses it to print the path; without one, a failed job left no trace anyone could follow.

func (*CrossVerifier) UnsupportedWork

func (cv *CrossVerifier) UnsupportedWork() []string

UnsupportedWork reports the coordinates the last queue scan offered that no verifier image in this build can run, most useful when nothing was claimed. An empty result means the queue itself had nothing for this peer, which is a different fact and deserves a different sentence.

func (*CrossVerifier) VerifyAndReport

func (cv *CrossVerifier) VerifyAndReport(ctx context.Context, job *Job) (domain.VerificationReceipt, error)

VerifyAndReport downloads and hash-verifies the job's artifact, unpacks it into a fresh temp workspace, runs the verification engine, and posts the signed receipt to POST /v1/verifications.

type Job

type Job struct {
	ID       int64           `json:"id"`
	SampleID string          `json:"sampleId"`
	Reason   string          `json:"reason"` // "cross" | "matrix"
	WantEnv  json.RawMessage `json:"wantEnv,omitempty"`
}

Job is one open cross/matrix verification job (plan C5).

type ReceiptAudit

type ReceiptAudit struct {
	Receipts  int
	WithImage int

	// SignatureValid: the stored bytes are the ones the peer's key signed.
	SignatureValid int
	// PeerIDBinding: peerId == "ed25519:" + hex(sha256(peerPubkey))[:16], so
	// the key that signed and the identity claimed are the same peer.
	PeerIDBinding int
	// ReceiptIDMatches: the stored id is the content hash of the stored
	// document. This is what makes the image identity and the verdict parts
	// of ONE hashed document rather than two tables someone joined.
	ReceiptIDMatches int
	// CanonicalRoundTrip: re-canonicalising the document reproduces it, so
	// nothing in the stored JSON is outside the schema these checks read.
	CanonicalRoundTrip int
	// RowBinding: the store's own columns agree with the signed document.
	RowBinding int

	// DigestPinned: reference is <alias>@sha256:<64 hex> — not a mutable tag.
	DigestPinned int
	// ReferenceDigestAgree: the standalone digest field is the reference's.
	ReferenceDigestAgree int
	// PublishedImage: the executed reference is an entry this build's
	// registry publishes, digest included.
	PublishedImage int

	ByOS        map[string]int
	ByPeer      map[string]int
	ByReference map[string]int

	// Problems names every receipt that failed a check, so a non-zero
	// shortfall can be chased instead of merely noticed.
	Problems []string
}

ReceiptAudit counts, per check, how many receipts passed. Counts rather than a bool: "all 2,279 verified" and "the one receipt we looked at verified" are different evidence, and a total that does not match Receipts is itself the finding.

func AuditReceipts

func AuditReceipts(rows []AuditedReceipt) ReceiptAudit

AuditReceipts re-verifies stored receipts offline, using the same code the verifier signs with, and reports how many passed each check.

type StageLogStore

type StageLogStore struct {
	// Home is CSX_HOME. Logs live in a subdirectory of it.
	Home string
	// MaxFiles bounds the directory. Zero uses defaultStageLogFiles.
	MaxFiles int
	// Now overrides the clock for tests.
	Now func() time.Time
}

StageLogStore keeps the stage output of a verification that failed.

The receipt carries only a digest of these logs, deliberately and permanently: raw output holds paths, usernames and tokens, and none of it may leave the machine (goal.md §8.5). But the cross-verification worker then threw the logs away entirely, and its workspace is disposable, so a reproducible failure left nothing at all behind. Two peers hit the same Django 6.1 resolve failure a day apart; the sample was fine, the pinned hashes matched PyPI, and running the identical command in the identical image by hand exited 0 — and there was no way to see what differed, because the one artifact that would have said was discarded on the line between RunLogged and Run.

Nothing here is uploaded. These files exist so an operator can read them on the machine that produced them.

func (*StageLogStore) Keep

func (s *StageLogStore) Keep(sampleID string, logs, stages map[string]string) (string, error)

Keep writes the stage logs for one verification if any stage failed, and returns the path it wrote. A run where nothing failed returns "" and writes nothing: there is nothing to diagnose and a worker fills a disk otherwise.

An error here is never fatal to the caller. Logs are a diagnostic aid, not a precondition of verification, and a machine whose log directory is unwritable must still produce the evidence it was asked for.