README
¶
Probity Vectors
Test an agent-evidence verifier against saved records with known outcomes. The corpora cover valid records, altered bytes, missing evidence and open specification interpretations.
The recording is demo/four-arms.sh from a fresh clone; vhs demo/four-arms.tape re-records it.
Try it
From a reviewed source checkout, build the Go verifier and a wheel, then run the installed harness outside the checkout. You need Go 1.26+, Python 3.11+ and uv. Record the checkout's full commit ID with your report.
git rev-parse HEAD
run_dir="$(mktemp -d)"
GOWORK=off go build -o "$run_dir/aee-verify" ./cmd/aee-verify
uv build --wheel --out-dir "$run_dir"
wheel="$(find "$run_dir" -maxdepth 1 -name 'agent_evidence_vectors-*.whl')"
cd "$run_dir"
uv run --no-project --with "$wheel" agent-evidence-vectors --verifier "$run_dir/aee-verify --json"
Open conformance-report.json for the per-vector results. A complete external run has rail: external, verifier.vectorsExecuted equal to totals.vectors, and zero totals.suiteRefusals. The report separates conformance from reason-code agreement.
See installation routes and release identity for package availability and signature checks.
Select v0.18.1 on GitHub and PyPI. Install the matching release with Go's executable directory on your PATH:
go install github.com/probityai/agent-evidence-vectors/cmd/aee-verify@v0.18.1
uvx agent-evidence-vectors==0.18.1 --verifier "aee-verify --json"
Test your verifier
The harness calls your executable with a vector-file path. Return a verdict through the exit status and a JSON object on the last line of stdout. The verifier contract covers output fields, key policy and comparison rules.
Use the GitHub Action to run the same check in your build and retain its report. The CI guide gives a workflow step and the available inputs.
Verifiers that pin this corpus
agmi, Rul1an/aee-checker and MoltyCel/moltrust-api run a corpus from this repository in their own CI at a pinned release or commit, and this repository's CI runs agmi's anchored-chain verifier back against vectors-anchored-chain. The verifier table gives each one's corpus, pin and CI run.
Choose your next step
| I want to | Read |
|---|---|
| Pick a format and corpus | Corpus guide |
| Implement the verifier interface | Verifier contract |
| Understand what the fixtures test | Corpus measurements |
| Check release bytes and signatures | Release verification |
| Report a run or disagreement | Run reporting |
| See who runs and uses the corpora | Independent runs, Adopters |
Each corpus manifest records its source revision, digest and expected outcomes. DISTRIBUTION.md lists the shipped corpora and installation routes.
For fixture coverage, see Corpus measurements. To check signed release bytes, use Release verification.
Source maps: repository layout, corpus readers and witness attestor.
Contribute
Open an issue with the corpus revision, verifier build and report. For patches, start with CONTRIBUTING.md. For citations, use the citation guide.
Need to check a supplied claim against saved evidence? See Probity Verify. Browse the Probity projects for records, vocabulary and admission tools.
Apache-2.0. LICENSE.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package aee implements the verification core for the in-toto Adversarial Execution Evidence predicate, version 0.7 (predicateType https://in-toto.io/attestation/adversarial-execution-evidence/v0.7).
|
Package aee implements the verification core for the in-toto Adversarial Execution Evidence predicate, version 0.7 (predicateType https://in-toto.io/attestation/adversarial-execution-evidence/v0.7). |
|
Package aeetest builds deterministic, fully synthetic AEE v0.7 statements for tests and demos.
|
Package aeetest builds deterministic, fully synthetic AEE v0.7 statements for tests and demos. |
|
cmd
|
|
|
aee-verify
command
Command aee-verify is the consumer MVP for AEE v0.7 statements: it runs GATE 0 (well-formedness), GATE 1 (coverage validity), the result recompute, derives the per-row evidence tier against a consumer policy, and evaluates the consumer-policy step (expected corpus and substrate anchors; the Admitted conjunction).
|
Command aee-verify is the consumer MVP for AEE v0.7 statements: it runs GATE 0 (well-formedness), GATE 1 (coverage validity), the result recompute, derives the per-row evidence tier against a consumer policy, and evaluates the consumer-policy step (expected corpus and substrate anchors; the Admitted conjunction). |
|
ijson-gate-shim
command
Command ijson-gate-shim exposes the reference I-JSON ingress gate (aee.CheckIJSON) over a line-delimited JSON protocol on stdin/stdout, so a cross-implementation differential harness can drive the REAL reference gate as one rail alongside the other verifier implementations -- never a reimplementation.
|
Command ijson-gate-shim exposes the reference I-JSON ingress gate (aee.CheckIJSON) over a line-delimited JSON protocol on stdin/stdout, so a cross-implementation differential harness can drive the REAL reference gate as one rail alongside the other verifier implementations -- never a reimplementation. |
|
mutgen
command
Command mutgen enumerates and applies single-site WEAKENING mutations to the Go verification rail, so that what the conformance corpus FORCES can be measured instead of asserted.
|
Command mutgen enumerates and applies single-site WEAKENING mutations to the Go verification rail, so that what the conformance corpus FORCES can be measured instead of asserted. |
|
mutrun
command
Command mutrun replays the whole conformance corpus through the in-process aee rail and writes one JSON object per vector.
|
Command mutrun replays the whole conformance corpus through the in-process aee rail and writes one JSON object per vector. |
|
Package corpora judges every conformance corpus in this repository with one binary.
|
Package corpora judges every conformance corpus in this repository with one binary. |
|
Package observedeffect verifies statements carrying the Observed Effect predicate, whose normative definition is spec/predicates/observed-effect.md.
|
Package observedeffect verifies statements carrying the Observed Effect predicate, whose normative definition is spec/predicates/observed-effect.md. |
|
witnessattestor
module
|