httpapi

Documentation

Overview

Package httpapi implements the csx-server /v1 HTTP API (plan contract C5). All responses are JSON; errors are {"error": string}; handlers never panic outward. Registry and web reads only ever touch materialized snapshots — raw evidence is aggregated by internal/compatibility, never in-request (goal.md §14.5). The server never stores raw error text, paths, or user agents (§2.2).

Index

• func NewMux(d Deps) *http.ServeMux
• func ObservationsFromReceipt(r serverstore.ReceiptRow, symbols ...string) []domain.ObservationBatch
• func RecomputeStatus(rows []serverstore.ReceiptRow, now time.Time) string
• func ReconcileCrossJobLanes(ctx context.Context, store serverstore.Store, limit int) (repaired, unsupported int, err error)
• func ReconcileDependencyAtlas(ctx context.Context, store serverstore.Store, limit int) (int, error)
• func ReconcileStrandedDrafts(ctx context.Context, store serverstore.Store, limit int) (int, error)
• func ReconcileUncheckedPublicness(ctx context.Context, store serverstore.Store, checker PublicnessChecker, ...) (int, error)
• type CachedPublicnessChecker
• type Deps
• type MavenJarProber
• type PublicnessChecker
• type WantedSnapshot
  • func LoadWantedSnapshot(ctx context.Context, store serverstore.Store) (*WantedSnapshot, error)

Functions

func NewMux

func NewMux(d Deps) *http.ServeMux

NewMux builds the /v1 API mux with every C5 route registered.

func ObservationsFromReceipt

func ObservationsFromReceipt(r serverstore.ReceiptRow, symbols ...string) []domain.ObservationBatch

ObservationsFromReceipt turns one verification receipt into the usage observations it already proves.

Exported because the receipts already stored need the same conversion, and backfilling them through a second implementation would be a second set of rules about what a receipt proves.

A receipt that does not name the packages it resolved is attributed to no coordinate at all: inventing one would be worse than the dash it leaves. When sample manifest symbols are provided, passing contract executions are also attributed to those declared symbols at StageProjectTest with SymbolExact confidence so verified samples satisfy Observation >= 1.

func RecomputeStatus

func RecomputeStatus(rows []serverstore.ReceiptRow, now time.Time) string

RecomputeStatus re-derives a sample's status from its receipts under the CURRENT rules, starting from PUBLISHED rather than from whatever the row already says.

The live path only ever upgrades, which is right while the rules hold still. When a rule is corrected — as the environment-diversity gate was, after receipts that misstated where they ran had already granted MATRIX_PASS — a status earned under the old rule is simply wrong, and leaving it in place would advertise verification the evidence no longer supports. Downgrading is therefore an honesty operation, deliberately kept out of the request path and run by an operator.

func ReconcileCrossJobLanes

func ReconcileCrossJobLanes(ctx context.Context, store serverstore.Store, limit int) (repaired, unsupported int, err error)

ReconcileCrossJobLanes re-derives what every reachable cross job may ask for and reports how many it repaired and how many no lane can run.

crossRequirementsAFleetCanRun guards the jobs created from now on. It cannot reach the ones already in the table, and nothing else will: once a job is open, no request path ever reads it again — the same absence ReconcileStrandedDrafts exists for. Three such jobs held the production cross queue open and unclaimable for three days.

It runs both ways on purpose. A job asking for a line no image serves is relaxed to one the fleet can run; a job whose coordinates gained a lane comes back to the open queue by itself, so "unsupported" stays a statement about the images this build pins rather than a verdict on the sample. Live claims are left alone: the receipt a worker is about to sign is bound to the requirements it was handed.

func ReconcileDependencyAtlas

func ReconcileDependencyAtlas(ctx context.Context, store serverstore.Store, limit int) (int, error)

ReconcileDependencyAtlas reconciles dependency edges and leaf states from verified samples. It scans passing receipts of verified samples, producing DependsOn and DependsOnNone batches, which in turn populate dependency_resolution and dependency_edge so the atlas is restored and the scheduler has fresh work.

func ReconcileStrandedDrafts

func ReconcileStrandedDrafts(ctx context.Context, store serverstore.Store, limit int) (int, error)

ReconcileStrandedDrafts offers every stranded authoring draft for cross verification again, and reports how many it woke.

The retry on the receipt path only fires when an attempt CLOSES a job, so the drafts stranded before that code existed have no future event to reach them: their job is done, no receipt passed, and nothing will ever look at them again. This runs once at boot, which is often enough — deploys are frequent and the backlog is finite — and stays off every request path.

It is idempotent by construction: queueCrossVerificationOn declines a sample that already has work queued, and StrandedDrafts excludes anything that has spent its attempts.

func ReconcileUncheckedPublicness

func ReconcileUncheckedPublicness(ctx context.Context, store serverstore.Store, checker PublicnessChecker, limit int) (int, error)

ReconcileUncheckedPublicness resolves publicness for packages that have never been verified against a registry (checked_at IS NULL, publicness UNKNOWN).

Background: in production, packages seeded or ingested past the per-request lookup budget were stored with checked_at IS NULL and refused on every subsequent evidence upload (#176). Running this at server boot reconciles that historical backlog without charging active ingest requests.

Types

type CachedPublicnessChecker

type CachedPublicnessChecker interface {
	PublicnessChecker
	CachedPublicness(ctx context.Context, p domain.PURL) (string, bool)
}

CachedPublicnessChecker is a checker that can answer from what it already knows, without reaching a registry.

The per-request lookup cap exists to bound how many NEW names one anonymous request can push at npmjs.org or pypi.org. Without this, the cap was charged for every answer including the ones that never left the process, so a request whose batches cluster on packages the server already knows — the ordinary case, since evidence follows what people actually run — spent the whole budget on cache hits and returned UNKNOWN for everything after the twentieth package.

A checker that cannot answer this way is not required to: the fallback is the old behaviour, which is safe and merely thriftier than it needs to be.

type Deps

type Deps struct {
	Store serverstore.Store
	Blobs blob.Store
	Cfg   serverstore.ServerConfig
	// Build is the identity of this server process, served by /version. It
	// comes from stamps the deployment put on the artifact, so a rollback
	// changes it back without anyone editing anything.
	Build buildinfo.Info
	// PublicnessChecker is an interface so the outbound-lookup behaviour is
	// testable: this dependency is the one that reaches a third-party
	// registry with a name the caller chose, and a test has to be able to
	// count that.
	Checker PublicnessChecker

	// GitHub device-flow endpoints; empty fields use the github.com
	// defaults. Tests point these at httptest servers.
	GitHubDeviceURL string
	GitHubTokenURL  string
	GitHubUserURL   string

	// HTTPClient performs outbound GitHub calls; nil means a 10s-timeout
	// client.
	HTTPClient *http.Client

	// PeerProbe dials an announcing peer back to confirm other peers could
	// actually reach it. nil uses a real HTTP ping; tests supply a stub so
	// no test ever opens a socket to the outside.
	PeerProbe func(ctx context.Context, addr string, port int) bool

	// Limits holds the per-class rate limiters; nil builds the defaults.
	// Tests that hammer an endpoint on purpose install their own.
	Limits *limiters

	// Now is a test seam; nil means time.Now.
	Now func() time.Time

	// WantedSnapshot is loaded before the aggregation builder starts. A
	// process must never make its first public wanted request compete with the
	// restart builder for PostgreSQL; NewMux takes its own copy and serves it
	// while one bounded background refresh keeps it current.
	WantedSnapshot *WantedSnapshot
	// contains filtered or unexported fields
}

Deps carries everything the API handlers need. Checker is nil in trust mode (CSX_PUBLIC_CHECK=trust): every syntactically valid public-ecosystem batch is accepted without a registry probe (dev/e2e only).

type MavenJarProber

type MavenJarProber interface {
	MavenHasJar(ctx context.Context, p domain.PURL) (hasJar, known bool)
}

MavenJarProber answers whether a maven coordinate publishes a compiled artifact. *registry.Checker satisfies it.

The second return says whether the answer is known. A registry that will not answer has not said no, and treating those the same way would starve the authoring queue on a bad afternoon — silently, which is how the last outage looked.

type PublicnessChecker

type PublicnessChecker interface {
	Check(ctx context.Context, p domain.PURL) string
}

PublicnessChecker answers whether a package exists on its public registry. *registry.Checker satisfies it.

type WantedSnapshot

type WantedSnapshot struct {
	GeneratedAt time.Time
	Rows        []serverstore.WantedRow
}

WantedSnapshot is the exact public wanted feed loaded before background aggregation starts. Rows may be empty: an empty successful read is still a valid snapshot and must be distinguished from a process that was never primed.

func LoadWantedSnapshot

func LoadWantedSnapshot(ctx context.Context, store serverstore.Store) (*WantedSnapshot, error)

LoadWantedSnapshot performs the one database read readiness depends on. Calling it before StartBuilder gives the public endpoint a last-good value that is independent of builder CPU, I/O and connection pressure.