Documentation
¶
Overview ¶
Package diff answers "which step caused it".
Two runs of the same agent, side by side, is a spot-the-difference puzzle: a 47-step trace against a 47-step trace, most of which differ in ways nobody cares about. This package does the three things that turn that into an answer — it ALIGNS the runs step by step (align.go), COMPARES each aligned pair through an explicit noise filter (compare.go), and names the FIRST divergence plus the one later step whose arguments carry that divergence forward, suppressing everything else (causality.go).
Three disciplines govern it, inherited from the sibling `behalf why`:
- Nothing is written back. Alignment, classification and the causal reading are computed from the stored bytes on every run. A comparison bug can never freeze into evidence (Q11, schema §1).
- Nothing is invented. Every value on screen comes out of a stored payload span, and every name comes from the local alias map, which is an asserted label rather than a cryptographic claim (Q16, Q40). Where the causal link cannot be shown by value equality the output says "later difference", not "consequence".
- The suppression rule is a heuristic and says so, on screen, with `--all` as the escape hatch. A diff that hides a difference without admitting it is worse than one that shows all 47.
What is compared, and what is not ¶
The unit of comparison is the ACTION: the operation (name, target, arguments) and what came back (result, outcome).
"Arguments" means the semantically meaningful cut — the tool's own arguments — and NOT the params blob as the proxy forwarded it. The forwarded blob carries `_meta`: the delegation chain, and W3C baggage holding `behalf-run-id`. The run id is by definition different between the two runs being compared, so anything covering `_meta` differs at every single step, and a diff built on it reports 47 of 47 steps as changed while explaining none of them. `_meta` is therefore filtered by construction (NoisyPathSegments), and the digest that covers the whole forwarded blob is not compared at all (NotCompared).
NotCompared lists the receipt fields deliberately outside the scope. Three of them are worth naming here because they are not noise:
- authority.* and attribution.* — a run whose delegation chain is verified and a run whose leaf hop is merely asserted differ on every single receipt, at every step, including step 0. Counting that as a step difference would make step 0 the "first divergence" of every such pair and bury the step that actually caused the outcome. It is a real finding and it is `behalf why`'s finding; diff surfaces it as the attribution warning on the step it features, and hands off.
- step_key — the alignment key (Q85), not a difference. It embeds the causal ordinal, so it changes whenever a step moves; that is alignment's input, and reporting it would report every insertion twice.
- emitter.counter — a custody primitive, not a step number. Two runs recorded through one proxy share one monotonic counter, so run A is 0..46 and run B is 47..93. A diff that indexed steps by it would work on hand-built fixtures and silently misalign every recorded pair. The step number here is always Step.Ordinal: the receipt's position in the run view, which is log-index order filtered to the run (Q82), exactly the coordinate `behalf why <run>:<step>` takes.
Step identity (Q85) ¶
The primary key is the stored step_key — hash of tool name, normalized argument schema and causal ordinal — with sequence alignment as the documented fallback for runs whose keys do not line up (agent-version changes, inserted or removed steps). See align.go.
Index ¶
Constants ¶
const ( // AlignerStepKey is Q85's primary: the stored step_key, a hash of tool // name, normalized argument schema and causal ordinal. AlignerStepKey = "step_key" // AlignerSequence is the documented fallback: global sequence alignment // over a similarity score. AlignerSequence = "sequence" // AlignerBlocked is the fallback under a run long enough that the full // dynamic-programming matrix is refused; alignment is then decomposed on // unique step_key anchors and run within each block. Named separately so // output can be honest that a cheaper method was used. AlignerBlocked = "sequence (anchored)" )
The two alignment tiers, named in Result.Aligner.
Variables ¶
var NoisyFields = []string{
"receipt_id", "log_index", "leaf_hash", "run_id",
"captured_at", "created_at", "updated_at", "started_at",
"finished_at", "completed_at", "expires_at", "timestamp", "ts",
"duration_ms", "latency_ms", "elapsed_ms", "took_ms",
"request_id", "trace_id", "span_id", "session_id", "correlation_id",
"nonce", "etag", "cursor",
}
NoisyFields are field names ignored wherever they appear inside a compared value, because two runs differ in them by construction. This is the fine half of the noise filter — the coarse half, whole receipt fields that never enter the compared view at all, is NotCompared in diff.go.
The list is deliberately short and deliberately boring. Every entry here is a field that a correct system re-mints per run; a field that merely often changes (an amount, an order id, a status) is not on it, because hiding one of those is exactly the failure that would make the answer wrong rather than merely noisy. Volatile values under names not on this list are caught by shape instead — see volatileValue.
var NoisyPathSegments = []string{"_meta"}
NoisyPathSegments are path components that make everything BENEATH them noise, not just a leaf of that name.
`_meta` is the MCP envelope the proxy forwards alongside a tool call's real arguments: it carries the delegation chain and the W3C baggage header, and that baggage carries `behalf-run-id`. The run id is the one thing guaranteed to differ between any two runs, so every field under `_meta` differs at every step by construction. Comparing them would report a whole run as changed and explain nothing — the precise failure this filter exists to prevent.
This is why the comparison reaches for `params.arguments` and the tool name and target rather than the params blob as forwarded. See also the input-slot digest entry in NotCompared, which is the same problem one level up: a digest over the whole forwarded blob covers `_meta` too, and is therefore not a digest of the action.
var NotCompared = map[string]string{
"schema_version": "the writer's schema version, not the action",
"otel_conventions_version": "the conventions version, not the action",
"receipt_id": "a client-minted ULID, unique per receipt by construction (Q46)",
"captured_at": "wall-clock capture time; two runs never share one",
"run_id": "the thing being compared; equal by definition of the query",
"run_id_provenance": "how run_id was derived (Q7), not what the agent did",
"correlation": "run-scoped keys — session, trace, txn, acti, conversation (Q7)",
"emitter": "the capture surface and its per-emitter counter, not the action",
"step_key": "the alignment key (Q85), not a difference — see align.go",
"risk_class": "capture-time policy assignment (Q6), not the action. It is READ — to " +
"choose which linked downstream step is featured as the consequence (causality.go) — " +
"but never compared: two runs of one script under one policy agree on it, and a pair " +
"that did not would be reporting a policy change as an agent difference",
"risk_policy_digest": "which policy was in force, not what the agent did",
"authority": "the delegation chain — `behalf why`'s question; surfaced as the attribution warning",
"attribution": "the stored verification rollup (Q12) — same, and see the package doc",
"provenance": "native vs imported (Q93), a property of the record",
"log_index": "a log coordinate, assigned at append",
"leaf_hash": "a log coordinate, derived from the stored bytes",
"payload[input].digest": "a digest of the params object AS FORWARDED, which carries `_meta` " +
"(delegation chain, and W3C baggage holding behalf-run-id) — so it differs at every step " +
"of any two runs and is not a digest of the action. The per-field manifest is the comparable cut",
}
NotCompared documents the receipt fields that never enter the compared view, with the reason each is out of scope. It exists as data so a test can assert the list rather than trusting a comment, and so the reason is written down next to the field it excuses.
This is the coarse half of the noise filter: these fields differ by construction between any two runs, or belong to a different question. The fine half — field names and value shapes inside a compared value — lives in compare.go as NoisyFields and the volatile-value shapes.
Functions ¶
Types ¶
type Change ¶
type Change struct {
Class Class
Path string
Kind ChangeKind
A, B json.RawMessage
// Count is the element count for a reordered array.
Count int
}
Change is one field-level finding inside an aligned pair. A and B are the stored bytes at that path — never re-serialized, only sliced.
type ChangeKind ¶
type ChangeKind string
ChangeKind is what happened to one field.
const ( KindChanged ChangeKind = "changed" KindOnlyInA ChangeKind = "only-in-A" KindOnlyInB ChangeKind = "only-in-B" KindReordered ChangeKind = "reordered" )
type Class ¶
type Class string
Class is what kind of difference one aligned pair carries. A pair may carry several at once (the demo's step 31 differs in both `arguments` and `result`), which is why Difference holds a set.
const ( // ClassArguments — the call the agent made differs: its target, its // idempotency key, or the digest evidence for its (customer-held) input. ClassArguments Class = "arguments" // ClassResult — what came back differs. ClassResult Class = "result" // ClassOutcome — the operation's status or error differs. An `ok` in one // run and an `error` in the other is the loudest thing a diff can find. ClassOutcome Class = "outcome" // ClassOnlyInA — the step exists in the first run and has no counterpart // in the second: a deletion. ClassOnlyInA Class = "only-in-A" // ClassOnlyInB — the step exists only in the second run: an insertion. // A retry storm shows up as a run of these. ClassOnlyInB Class = "only-in-B" // ClassOrder — the same values came back in a different sequence. This // is a deliberately separate class from ClassResult because it is the // divergence that reads as "nothing changed" to every other tool: same // data, same count, same everything, and the agent picks a different // element. ClassOrder Class = "order" )
type Difference ¶
type Difference struct {
// Index is the pair's position in Result.Pairs — the aligned order the
// causality rule reads.
Index int
Pair Pair
Classes []Class
Changes []Change
// NoiseFiltered names the paths the noise filter dropped, so a reader
// can audit the filter instead of trusting it. `--all` prints them.
NoiseFiltered []string
// Opaque marks a pair whose ONLY evidence is a payload-slot digest: the
// receipt records that customer-held content diverged, but not what
// changed in it (Q34–Q38 keep the content out of the record).
//
// These are reported and never named as a cause, because a hash cannot
// answer "which step caused it" — and because a digest is the signal
// most likely to fire for reasons that have nothing to do with the
// action. A response blob routinely carries a session id, a cursor or a
// timestamp the receipt's outcome never records, so counting these as
// result differences turns a perfect alignment into a wall of findings
// that cannot be explained — the exact failure this feature exists to
// avoid. Only the OUTPUT slot can land here; the input slot's digest is
// not compared at all (NotCompared).
Opaque bool
// Truncated is set when a pair carried more field-level changes than
// maxChangesPerPair; the pair is still counted once, and the render says
// how many were kept.
Truncated bool
// Suppressed is the downstream heuristic's verdict (causality.go).
Suppressed bool
}
Difference is one aligned pair that is not identical.
func Compare ¶
func Compare(pairs []Pair) []Difference
Compare runs the structural comparison over every aligned pair and returns the pairs that are not identical, in aligned order.
func (*Difference) Has ¶
func (d *Difference) Has(c Class) bool
Has reports whether the difference carries class c.
type Link ¶
Link is the value-equality evidence behind a consequence claim: a value that appears in the first divergence's differing result on one run's side and again in the featured step's differing arguments on that same side, differing between the runs. Index is the element position when the link came from a reordered array (the demo's results[0]), and -1 otherwise.
type Options ¶
type Options struct {
Color bool
// All turns suppression off and lists every difference, grouped.
All bool
Aliases why.Aliases
}
Options controls rendering. Colour is opt-in and off by default so piped and captured output is plain text.
type Pair ¶
Pair is one aligned position: a step from each run, or a step from one run and nothing from the other. Insertions and deletions are first-class here, not an afterthought, because the single most common reason two runs of the same agent stop lining up is that one of them retried.
func Align ¶
Align pairs up the steps of two runs and reports which tier produced the result.
Tier 1 — step_key (Q85's primary) ¶
If every step on both sides carries a step_key, the keys are unique within each run, and the two key sets are equal, the pairing is by key: exact, O(n), and it needs no scoring at all. The bijection requirement is the whole guard. step_key hashes the causal ordinal, so a single inserted or deleted step shifts every key after it and the bijection fails — which is correct, because in that case a key pairing would report ~100% differences and the fallback reports one. A single step whose argument shape changed also breaks the bijection, and again the fallback is what gets it right: it can pair those two steps on tool name and position, which key equality by definition cannot.
So tier 1 is used only when it explains the entire pair of runs, and anything less hands over to tier 2.
Tier 2 — sequence alignment (the documented fallback) ¶
Global Needleman–Wunsch over a similarity score of (tool name, argument shape, ordinal proximity), with step_key equality as the score's top tier so that wherever keys do line up the fallback reproduces tier 1's answer. This is deliberately not zip-by-position: a run that inserts one step at position 3 differs from its sibling in one step, and a positional comparison would call that 44 differences. Gaps in either direction are what make an insertion an insertion.
type Result ¶
type Result struct {
RunA, RunB string
CountA, CountB int
// StartA and StartB are each run's first captured_at, verbatim — the
// origin the "t+412ms" coordinate is measured from.
StartA, StartB string
// WeakestA and WeakestB are each run's weakest stored attribution
// rollup (Q12). They are not compared as a step difference — see the
// package doc — but the render states it when they disagree, because a
// diff that says "2 differ" while one run cannot prove its authority at
// all has answered the smaller question.
WeakestA, WeakestB string
// Aligner names which tier produced Pairs, for the record and for tests:
// AlignerStepKey or AlignerSequence.
Aligner string
Pairs []Pair
// Differences are the aligned pairs that differ in a way the receipt can
// explain, in aligned order. This is the number the headline counts.
Differences []Difference
// Opaque are the aligned pairs whose only difference is a payload-slot
// digest — see Difference.Opaque. They are reported as their own line
// and listed by --all, and they are never named as a cause.
Opaque []Difference
// First is the first divergence in aligned order — the step this feature
// exists to name. Nil when nothing differs.
First *Difference
// Featured is the step shown under the first divergence. When
// FeaturedIsConsequence it is the highest-risk differing step whose
// differing argument values are traceable to First's differing result
// values by value equality (ties to the latest), and Link carries that
// evidence; otherwise it is simply the last differing step, and the
// render labels it "later difference" rather than claiming a causal link
// it cannot show.
Featured *Difference
FeaturedIsConsequence bool
Link *Link
// SuppressedCount is how many differences the downstream heuristic hid.
SuppressedCount int
}
Result is everything one `behalf diff` render needs.
func Analyze ¶
Analyze is the whole engine over two already-projected runs: align, compare, then read causality. It is pure, which is what lets each of the three pieces be tested on its own.
type Step ¶
type Step struct {
RunID string
// Ordinal is the receipt's position in the RUN VIEW — log-index order
// filtered to the run (Q82) — counting from 0, which is the coordinate
// `behalf why <run>:<step>` takes. It is deliberately never
// emitter.counter: that counter is per-emitter and monotonic across
// runs, so two runs recorded through one proxy would number 0..46 and
// 47..93 and every recorded pair would misalign.
Ordinal int
LogIndex uint64
LeafHash string
Payload []byte
StepKey string
Operation string
Target string
CapturedAt string
ActorJKT string
// RiskClass is the capture-time tool policy's assignment (Q6), read and
// never recomputed. It is NOT compared — see NotCompared — and is used
// for one thing: choosing which of several linked downstream steps the
// render features as the consequence (causality.go).
RiskClass string
// Attribution is the stored receipt-level rollup (Q12); LeafHopStatus is
// the stored verification state of the acting hop; RootJKT is the
// confirmation-key thumbprint of the depth-0 hop — the human the run was
// carried out on behalf of. All three are read, never recomputed.
Attribution string
LeafHopStatus string
RootJKT string
HopCount int
// contains filtered or unexported fields
}
Step is one receipt as the diff reads it: the log coordinates, the alignment key, and the projected argument / result / outcome views the comparison runs over. Payload is the exact stored payload span, kept verbatim — nothing here is ever re-serialized back into the log.
Source Files