README
¶
Agent Evidence Vectors
Verified by Probity (github.com/probityai/probity): latest passing run
Test an agent-evidence verifier against saved records with known outcomes. The corpora cover valid records, altered bytes, missing evidence and open specification interpretations.
The recording is demo/four-arms.sh from a fresh clone; vhs demo/four-arms.tape re-records it.
Add it to a project with uv add agent-evidence-vectors==0.25.0, then pip install agent-evidence-vectors==0.25.0. The probityai command line is planned, not yet released: uv add probityai, then pip install probityai.
Try it
Run every vector in the default corpus through the bundled reference verifier. You need Python 3.11+ and uv; there is nothing to clone or build, and a cold run takes about six seconds:
uvx agent-evidence-vectors==0.25.0
At v0.25.0 the default corpus is suiteRevision 32, and the run prints one row per vector and ends with:
totals: 288 vectors, 288 pass, 0 fail; report written to conformance-report.json
Now run the same corpus against a verifier other than the reference. The Go reference verifier installs with Go 1.26+ and Go's executable directory on your PATH; put your own command in its place:
go install github.com/probityai/agent-evidence-vectors/cmd/aee-verify@v0.25.0
uvx agent-evidence-vectors==0.25.0 --verifier "aee-verify -validity-only -json"
Open conformance-report.json for the per-vector results. A complete external run has rail: external, verifier.vectorsExecuted equal to totals.vectors, and zero totals.suiteRefusals. The report separates conformance from reason-code agreement. --list-corpora names the other shipped corpora and --corpus <name> selects one.
The release is v0.25.0 on GitHub and PyPI. To run from a reviewed source checkout instead, follow the source build. Installation routes and release identity covers package availability and signature checks.
Test your verifier
The harness calls your executable with a vector-file path. Return a verdict through the exit status and a JSON object on the last line of stdout. The verifier contract covers output fields, key policy and comparison rules.
Use the GitHub Action to run the same check in your build and retain its report. The CI guide gives a workflow step and the available inputs.
- uses: probityai/agent-evidence-vectors@v0.22.0
with:
verifier: VERIFIER_COMMAND --json
Replace VERIFIER_COMMAND with the command that runs your verifier.
Verifiers that pin this corpus
agmi, Rul1an/aee-checker and MoltyCel/moltrust-api run a corpus from this repository in their own CI at a pinned release or commit, and this repository's CI runs agmi's anchored-chain verifier back against vectors-anchored-chain. The verifier table gives each one's corpus, pin and CI run.
Choose your next step
| I want to | Read |
|---|---|
| Start a project with this Action already in CI | evidence-starter template |
| Pick a format and corpus | Corpus guide |
| Implement the verifier interface | Verifier contract |
| Understand what the fixtures test | Corpus measurements |
| Check release bytes and signatures | Release verification |
| Report a run or disagreement | Run reporting |
| Check a W3C intake record | W3C intake checker |
| See who runs and uses the corpora | Independent runs, Adopters |
Each corpus manifest records its source revision, digest and expected outcomes. DISTRIBUTION.md lists the shipped corpora and installation routes.
For fixture coverage, see Corpus measurements. To check signed release bytes, use Release verification.
Source maps: repository layout, corpus readers and witness attestor.
Contribute
Open an issue with the corpus revision, verifier build and report. For patches, start with CONTRIBUTING.md. For citations, use the citation guide.
Need to check a supplied claim against saved evidence? See Probity Verify. Browse the Probity projects for records, vocabulary and admission tools.
Apache-2.0. LICENSE.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package aee implements the verification core for the in-toto Adversarial Execution Evidence predicate, version 0.7 (predicateType https://in-toto.io/attestation/adversarial-execution-evidence/v0.7).
|
Package aee implements the verification core for the in-toto Adversarial Execution Evidence predicate, version 0.7 (predicateType https://in-toto.io/attestation/adversarial-execution-evidence/v0.7). |
|
Package aeetest builds deterministic, fully synthetic AEE v0.7 statements for tests and demos.
|
Package aeetest builds deterministic, fully synthetic AEE v0.7 statements for tests and demos. |
|
cmd
|
|
|
aee-verify
command
Command aee-verify is the reference verifier for AEE v0.7 statements: it runs GATE 0 (well-formedness), GATE 1 (coverage validity), the result recompute, derives the per-row evidence tier against a consumer policy, and decides ADMISSION: whether the consumer may act on the statement, under the pins it supplied.
|
Command aee-verify is the reference verifier for AEE v0.7 statements: it runs GATE 0 (well-formedness), GATE 1 (coverage validity), the result recompute, derives the per-row evidence tier against a consumer policy, and decides ADMISSION: whether the consumer may act on the statement, under the pins it supplied. |
|
ijson-gate-shim
command
Command ijson-gate-shim exposes the reference I-JSON ingress gate (aee.CheckIJSON) over a line-delimited JSON protocol on stdin/stdout, so a cross-implementation differential harness can drive the REAL reference gate as one rail alongside the other verifier implementations -- never a reimplementation.
|
Command ijson-gate-shim exposes the reference I-JSON ingress gate (aee.CheckIJSON) over a line-delimited JSON protocol on stdin/stdout, so a cross-implementation differential harness can drive the REAL reference gate as one rail alongside the other verifier implementations -- never a reimplementation. |
|
mutgen
command
Command mutgen enumerates and applies single-site WEAKENING mutations to the Go verification rail, so that what the conformance corpus FORCES can be measured instead of asserted.
|
Command mutgen enumerates and applies single-site WEAKENING mutations to the Go verification rail, so that what the conformance corpus FORCES can be measured instead of asserted. |
|
mutrun
command
Command mutrun replays the whole conformance corpus through the in-process aee rail and writes one JSON object per vector.
|
Command mutrun replays the whole conformance corpus through the in-process aee rail and writes one JSON object per vector. |
|
Package corpora judges every conformance corpus in this repository with one binary.
|
Package corpora judges every conformance corpus in this repository with one binary. |
|
Package observedeffect verifies statements carrying the Observed Effect predicate, whose normative definition is spec/predicates/observed-effect.md.
|
Package observedeffect verifies statements carrying the Observed Effect predicate, whose normative definition is spec/predicates/observed-effect.md. |
|
witnessattestor
module
|