t324

README

T32.4 — Search-topology and cost spike

Retained engineering record. This directory selects the initial physical search topology for the multi-service program from source-free T32.2 evidence and T32.3's public oracle. It is not current user guidance, a production catalog contract, a target SLO, an accuracy result, or multi-service release authorization. Current sequencing remains in docs/ROADMAP.md.

Completed: 2026-08-04 · Status: COMPLETE · Production changes: none

Decision

The initial v2 topology is one direct whole-repository zoekt generation per exact repository revision set. All code reads that generation directly. A service query adds the complete catalog-derived filename predicate to the zoekt query alongside its exact revision before ranking and top-K truncation; it never filters a whole-repository result afterward.

Alternative	Decision	Reason
Direct whole-repository shards	GO to T32.5	T32.2 completed its prospectively frozen direct envelope, and the neutral oracle is exact with in-query service predicates.
Bounded prefix-aware/hybrid cohorts	NO-GO; trigger not met	The preregistered direct baseline did not fail. Building cohorts now would add shared-file duplication and cross-cohort ranking/publication complexity without a measured need.
P6 fleet	NO-GO; trigger not met	No single-node capacity or correctness failure requires fleet escalation. P6 remains demand-driven.

NO-GO here does not claim cohorts or P6 can never work. Either alternative requires a new named direct-topology failure, a prospectively frozen experiment, and the same all-code/service equality bar.

Bound inputs

results.json binds these exact retained inputs:

• T32.2 source-free baseline: sha256:d1ec7b658eef84d2974c50c66d6dca00160a412fd49154c1ad4e232baae695ad;
• T32.3 artifact receipt: sha256:ce94187fd3b9c1ad42b64f131c9234399a5df918a07c5f452b94393873ab8611;
• T32.3 Git bundle: sha256:05a1b845a2eaee1c6a2b0beda972aa0ea6ffe9cc636d886014887202728e2194.

The T32.3 generator now exposes the exact frozen load-profile bytes to later spikes. Tests verify every materialized path, byte count, and digest against the already-retained profile inventory; the profile and corpus artifacts did not change.

Correctness result

The harness rejects a child whose Go build metadata does not match this module's exact zoekt version and checksum. That admitted zoekt-git-index child indexed all five neutral revisions in one direct shard set. The retained run checked 40 independent oracle queries: 33 exact/stale searches executed and seven restricted, removed, or unavailable queries stopped at authority admission with the oracle's empty result.

All checks passed:

• exact All code and service result sets at every revision;
• catalog-derived service filename atoms present inside the zoekt query;
• a raw-byte-derived broad package oracle at every revision;
• stable ordering for a branch-bound all-document adversarial ranking case and equality between its full first three results and zoekt's top-K=3 result;
• exact branch binding and per-revision catalog/oracle equality across all five snapshots, plus repeatable prior-revision and static A → B → A branch selection within one immutable shard set;
• service top-K=1 execution over the 1,000/5,000-service profiles, proving unrelated higher-ranked documents never enter a post-filtering budget.

The correctness test authors a fresh public repository, runs the real child, and replays the oracle. It compares semantic booleans and the path-result digest; timings are observations, not reproducibility pins.

Cost result

Measured on the source-free environment recorded in results.json (Darwin arm64, Go 1.26.5, 10 logical CPUs). The neutral builds disable ctags to isolate physical text-search topology; T32.2 separately measured the target path under its frozen production configuration.

Input	Files	Memberships	Initial build	One-file revision build	Peak child RSS	Shard bytes	Shards	Cold / warm query battery
Five-revision correctness corpus	14–15/revision	oracle-defined	54 ms	n/a	97.0 MiB	89,333	1	1 / 1 ms
1,000 services	3,151	5,000	345 ms	288 ms	109.2 / 97.3 MiB	803,142 / 803,361	1	1,059 / 884 µs
5,000 services	15,751	25,000	1,827 ms	1,300 ms	135.7 / 167.9 MiB	3,876,442 / 3,876,661	1	4,540 / 4,788 µs

The 1,000- and 5,000-service predicate passes compile all five placements for every service only to expose the frozen aggregate cost: 10.7/52.8 ms, 369,000/1,845,000 canonical query-tree bytes, and 369 bytes maximum for one service. A production request compiles one authorized service predicate, not all services, and indexing remains one repository build rather than a loop over services.

Each visible shard maps once under the pinned zoekt Unix reader contract. The run therefore recorded one shard mmap in each case. Numeric process descriptors were measured with lsof: opening each one-shard directory reader added five descriptors. Peak child RSS comes from wait4's ru_maxrss, with Linux KiB normalized to bytes and Darwin's byte value retained. “Cold” means a newly opened reader; the experiment did not evict the host page cache. A measured one-file update uses the same clean staging/full-build posture as today's -incremental=false production path.

The profile no-op check loads desired HEAD from Git and the active branch version from published shard metadata before timing one identity comparison. The timed comparison records zero child runs, file scans, shard reads, or shard-byte change. T32.2's separate real restart measurement supplies the target-side confirmation: already current, zero index children, zero shard-byte delta, and zero shard-mtime changes. The neutral replay does not interrupt publication or transition physical generations; those recovery semantics remain owned by T34.3.

Boundaries and steady-state cost

These tiny synthetic source bytes measure service-count metadata and query predicate behavior; they do not reproduce target source volume and cannot set a service-count, latency, memory, disk, or shard SLO. Warm being slower than cold in one 5,000-service micro-run is retained honestly rather than promoted to a cache claim.

This package is development-only. Production packages do not import it, and no request, query, startup/restart, sync tick, retry/no-op, publication transition, store, cache, filesystem, Git, child-process, memory, descriptor, or mmap work changed because of T32.4. Only explicit measurement or the targeted replay test creates Git children and a zoekt child; both use temporary directories and retain only the closed source-free receipt.

Reproduction

go run ./spike/t324/cmd/measure \
  -root . \
  -zoekt bin/zoekt-git-index \
  -out spike/t324/results.json
go test ./spike/t323 ./spike/t324/... -count=1

Re-authoring timings or environment identity changes the receipt and requires review. Semantic result digests and input bindings must remain unchanged unless their owning retained artifact advances explicitly.

Documentation

Overview

Package t324 measures the T32.4 direct-search topology against T32.2's source-free trigger and T32.3's independent public oracle. Production packages must not import spike packages.

Index

• Constants
• func DecodeStrict[T any](data []byte) (T, error)
• func MarshalCanonical(value any) ([]byte, error)
• func SHA256(data []byte) string
• func ValidateReceipt(receipt Receipt) error
• type BuildMeasurement
• type Claims
• type Correctness
• type Environment
• type Inputs
• type NeutralCost
• type NoOpMeasurement
• type PredicateMeasurement
• type ProfileMeasurement
• type QueryMeasurement
• type ReaderMeasurement
• type Receipt
  • func Measure(ctx context.Context, root, zoektBinary string) (Receipt, error)
• type TopologyDecision
• type TriggerDecision

Constants

const ReceiptSchema = "t324-search-topology-cost-receipt-v1"

Functions

func DecodeStrict

func DecodeStrict[T any](data []byte) (T, error)

func MarshalCanonical

func MarshalCanonical(value any) ([]byte, error)

func SHA256

func SHA256(data []byte) string

func ValidateReceipt

func ValidateReceipt(receipt Receipt) error

Types

type BuildMeasurement

type BuildMeasurement struct {
	WallMS       int64 `json:"wall_ms"`
	PeakRSSBytes int64 `json:"peak_rss_bytes"`
	ShardBytes   int64 `json:"shard_bytes"`
	ShardCount   int   `json:"shard_count"`
}

type Claims

type Claims struct {
	SourceFree                    bool `json:"source_free"`
	SelectsInitialTopology        bool `json:"selects_initial_topology"`
	EstablishesSLO                bool `json:"establishes_slo"`
	EstablishesAccuracy           bool `json:"establishes_accuracy"`
	AuthorizesMultiServiceRelease bool `json:"authorizes_multi_service_release"`
	ChangesProductionBehavior     bool `json:"changes_production_behavior"`
}

type Correctness

type Correctness struct {
	Revisions                 int  `json:"revisions"`
	OracleQueries             int  `json:"oracle_queries"`
	ExecutedQueries           int  `json:"executed_queries"`
	AuthorityEmptyQueries     int  `json:"authority_empty_queries"`
	AuthorityAdmissionChecked bool `json:"authority_admission_checked"`
	AllCodeEqual              bool `json:"all_code_equal"`
	ServiceEqual              bool `json:"service_equal"`
	PredicatesInsideZoekt     bool `json:"predicates_inside_zoekt"`
	BroadQueriesEqual         bool `json:"broad_queries_equal"`
	AdversarialRankingStable  bool `json:"adversarial_ranking_stable"`
	TopKEqual                 bool `json:"top_k_equal"`
	RevisionBound             bool `json:"revision_bound"`
	PerRevisionCatalogEqual   bool `json:"per_revision_catalog_equal"`
	PriorRevisionRepeatable   bool `json:"prior_revision_repeatable"`
	BranchABASelectionEqual   bool `json:"branch_a_b_a_selection_equal"`
}

type Environment

type Environment struct {
	GOOS          string `json:"goos"`
	GOARCH        string `json:"goarch"`
	GoVersion     string `json:"go_version"`
	LogicalCPUs   int    `json:"logical_cpus"`
	ZoektSHA256   string `json:"zoekt_binary_sha256"`
	CtagsDisabled bool   `json:"ctags_disabled"`
}

type Inputs

type Inputs struct {
	T322ResultsSHA256 string `json:"t322_results_sha256"`
	T323ReceiptSHA256 string `json:"t323_receipt_sha256"`
	T323BundleSHA256  string `json:"t323_bundle_sha256"`
}

type NeutralCost

type NeutralCost struct {
	Build        BuildMeasurement  `json:"build"`
	Reader       ReaderMeasurement `json:"reader"`
	ColdQueryMS  int64             `json:"cold_query_ms"`
	WarmQueryMS  int64             `json:"warm_query_ms"`
	QueryCount   int               `json:"query_count"`
	ResultDigest string            `json:"result_digest"`
}

type NoOpMeasurement

type NoOpMeasurement struct {
	WallUS           int64 `json:"wall_us"`
	StateComparisons int   `json:"state_comparisons"`
	ChildRuns        int   `json:"child_runs"`
	FileScans        int   `json:"file_scans"`
	ShardReads       int   `json:"shard_reads"`
	ShardBytesDelta  int64 `json:"shard_bytes_delta"`
}

type PredicateMeasurement

type PredicateMeasurement struct {
	CompiledServices int   `json:"compiled_services"`
	Placements       int   `json:"placements"`
	WallUS           int64 `json:"wall_us"`
	CanonicalBytes   int64 `json:"canonical_bytes"`
	MaxBytes         int   `json:"max_bytes"`
}

type ProfileMeasurement

type ProfileMeasurement struct {
	Name           string               `json:"name"`
	Services       int                  `json:"services"`
	Files          int                  `json:"files"`
	Memberships    int                  `json:"memberships"`
	Build          BuildMeasurement     `json:"build"`
	SingleFileEdit BuildMeasurement     `json:"single_file_edit"`
	Reader         ReaderMeasurement    `json:"reader"`
	Queries        QueryMeasurement     `json:"queries"`
	Predicates     PredicateMeasurement `json:"predicates"`
	NoOp           NoOpMeasurement      `json:"no_op"`
}

type QueryMeasurement

type QueryMeasurement struct {
	QueryCount    int    `json:"query_count"`
	ColdTotalUS   int64  `json:"cold_total_us"`
	WarmTotalUS   int64  `json:"warm_total_us"`
	ColdWarmEqual bool   `json:"cold_warm_equal"`
	BroadTopK     int    `json:"broad_top_k"`
	ServiceTopK   int    `json:"service_top_k"`
	ResultDigest  string `json:"result_digest"`
}

type ReaderMeasurement

type ReaderMeasurement struct {
	MMapCount        int    `json:"mmap_count"`
	MMapMethod       string `json:"mmap_method"`
	DescriptorBase   int    `json:"descriptor_base"`
	DescriptorOpen   int    `json:"descriptor_open"`
	DescriptorDelta  int    `json:"descriptor_delta"`
	DescriptorMethod string `json:"descriptor_method"`
}

type Receipt

type Receipt struct {
	Schema      string               `json:"schema"`
	MeasuredOn  string               `json:"measured_on"`
	Inputs      Inputs               `json:"inputs"`
	Environment Environment          `json:"environment"`
	Trigger     TriggerDecision      `json:"cohort_trigger"`
	Correctness Correctness          `json:"correctness"`
	NeutralCost NeutralCost          `json:"neutral_cost"`
	Profiles    []ProfileMeasurement `json:"profiles"`
	Decisions   []TopologyDecision   `json:"decisions"`
	Claims      Claims               `json:"claims"`
}

func Measure

func Measure(ctx context.Context, root, zoektBinary string) (Receipt, error)

type TopologyDecision

type TopologyDecision struct {
	Topology string `json:"topology"`
	Decision string `json:"decision"`
	Reason   string `json:"reason"`
	NextGate string `json:"next_gate"`
}

type TriggerDecision

type TriggerDecision struct {
	BaselineOutcome string `json:"baseline_outcome"`
	IndexOutcome    string `json:"index_outcome"`
	SearchOutcome   string `json:"search_outcome"`
	Triggered       bool   `json:"triggered"`
	Reason          string `json:"reason"`
}