artifact

package module
v0.87.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: BSD-3-Clause Imports: 56 Imported by: 0

README

artifact

pkg.go.dev Go License

The pure-Go artifact client — the shared backend of the zpkgx family. It resolves a package's dependency closure, downloads the artifacts from the configured ZPKG_DIST (default: oci://ghcr.io/go-pkgx/packages, the signed registry the published artifacts are actually in — see DistBase in zpkg.go), completes the implicit libc/gcc closure a FROM scratch image needs, and execs through the glibc artifact's own loader. CGO_ENABLED=0, no runtime dependencies of its own.

Consumed by:

so both tools share one source of truth for the artifact protocol.

Highlights

  • Reads package.hcl + versions.txt from the pantry — falling back to package.yml, which is converted to HCL on the way in (zpkg.go:276) and is being retired with the rest of the pkgx compatibility — then BFS the runtime dependency closure and streams gzip/xz + tar into ZPKG_DIR. ⛔ "The pantry" is two trees: our own recipes merged over pkgxdev/pantry, which PantryBase still reads at resolve time (zpkg.go:68) and which is a dependency being wound down — the sovereign pantry is written, not converted, because that corpus carries no licence to convert (go-zpkg/packages#431, #440).
  • Soname-exact FROM-scratch completion: walks each artifact's ELF DT_NEEDED and pulls the provider version that ships the exact soname an ABI needs (not merely the latest), so drifted sonames resolve.
  • Embedded Mozilla CA bundle (net/http with no system trust store), pure-Go DNS.
  • Nothing outside the store. A blob is staged inside $ZPKG_DIR, not in the system temp directory: FROM scratch has no /tmp, and until this was measured by building that image and running it, zpkgs catalog update failed on the one image this package exists to serve.

What Extract refuses

Extract is the one tar extractor in this ecosystem, and it is pointed at archives nobody here produced: artifacts from a registry, and — through zbk/fetch — source tarballs from 233 distinct upstream hosts, unpacked as root inside a chroot in the sovereign build. So what it refuses is part of its API, not an implementation detail.

it rejects with ErrInsecurePath why
an absolute entry name /etc/passwd is not inside anything
a name that escapes dest after component-stripping the classic ../..
a hard-link source that escapes dest vetted exactly like a name
an entry whose path crosses a symlink an earlier entry created a name check is lexical and a write is not: dest/a/b is inside dest even when a is a link out of it

and it drops setuid, setgid and sticky — an entry gets its permission bits and nothing else.

A leaf that an earlier entry made a symlink is unlinked rather than refused, because the archive does say what belongs at that name. A directory component that is already a symlink is refused, because there the archive contradicts itself.

What it does not promise. A symlink entry may point anywhere, including out of dest — that is what tar(1) does, and artifacts need relative links. Nothing Extract writes afterwards follows such a link, but a caller that walks the result and resolves symlinks itself can still leave dest, and has to say whether it means to.

Each of these rules has a test that fails without it, and a positive control that an ordinary nested archive still extracts: a check tightened too far refuses everything and reports green.

What bounds an .xz artifact

Decompressor decodes .tar.xz with go-compressions/xz, not ulikunitz/xz. The latter allocates the LZMA2 dictionary whole from the block header (DictCap is a minimum, not a ceiling): a 44-byte .xz declaring a 1.5 GiB dictionary allocated 1,610,663,976 bytes, and an artifact or a source tarball is input nobody here produced. Here the window grows with the decoded data: a stream declaring 1.5 GiB over a few bytes allocates about 22 KB (decompress_bomb_test.go, run in CI as a compiled binary under ulimit -v and GOMEMLIMIT). The dictionary a stream may make the decoder keep is capped at 256 MiB (maxXzDictionary, above the 64 MiB of xz -9); data that needs more fails with an error that satisfies errors.Is(err, xz.ErrTooLarge). ulikunitz/xz remains in go.mod only for the tests that build .xz fixtures (encode side).

What gets verified

With ZPKG_VERIFY on — the default — every pull that brings bytes onto the machine checks the artifact's cosign signatures against the configured trust set and fails closed. Not "every install": every pull.

That distinction is the whole of it, and it has been got wrong twice. The first time, the check lived in DownloadArtifact, which only the mirror tooling calls, so zpkgx and zpkgm installed unsigned artifacts while advertising otherwise. The second time was the catalogue — the list of project names zpkgs ls and <TAB> read — which was fetched past the check because it is "only a list of names".

A list of names is what a person then types. Forging it does not get an unsigned artifact installed, because the install path still verifies; it gets a near-miss name offered at the prompt, which is the whole of a typosquat. And since that catalogue is now fetched once and read from disk for days, it is verified less often than an artifact, not more — so the one check it gets has to happen.

The rule, for anything added later: if it pulls bytes, it goes through verifyPulled, whatever it means to do with them. The store cache is no exception: a cache hit is re-checked against the signatures the registry advertises, because a cache is an optimisation and never an authority.

Who may vouch, and how many of them

Verification is against a trust set, not a single key: TUF's shape (a set of keys and a threshold of them) with Nix's spelling for the list.

variable what it does
ZPKG_TRUSTED_KEYS replaces the trust set — name:KEY, whitespace separated
ZPKG_EXTRA_TRUSTED_KEYS adds to it, which is usually what you want
ZPKG_TRUST_THRESHOLD how many distinct trusted keys must have signed (default 1)
ZPKG_EXTRA_TRUSTED_KEYS='alice:RWQ…' ZPKG_TRUST_THRESHOLD=2 zpkgx +fftw.org

Two people building the same inputs get the same hash — deliberately, and that collision is the evidence. What tells them apart is the attestation: each attaches their own signature referrer to the one artifact, and the client counts how many distinct trusted keys it can verify. A threshold of 2 means "I install nothing that two builders did not independently reproduce".

Three rules this hangs on:

  • Each key counts once (TUF §5.4.2). Otherwise a threshold of two is met by pushing one signature twice, which anyone who can push can do.
  • A signature from outside the trust set is not an error. Anybody may sign a public artefact; if a stranger's signature invalidated an artifact, invalidating any artifact would cost one push. It is counted and ignored.
  • A threshold nobody could meet is refused when the configuration is read, not at install time, because that mistake is in the configuration and a message about the artifact would send you looking in the wrong place.

The signer name you see comes from your trust set and never from the registry: a name on the artefact is written by whoever pushed it. The referrer carries only a key id, which is checkable, and which is used to pick which key to try first — never to decide which may succeed.

Rotating a key, and revoking one

Three states, and the difference between the last two is the whole reason there are two lists:

signs counts towards the threshold reported as
SigningPublicKey — current yes yes its name
RetiredSigningPublicKeys — retired no yes its name
RevokedSigningKeys — revoked no no named, with the reason

Retiring is changing the key you sign with. It says nothing about what is already signed, so a retired key stays in the trust set and nothing needs re-signing. Rotation is an edit to two variables: the new key goes up, the old one moves down.

This has happened once, on 2026-11-06:

key id state what it signed
df35c3a46931d3e6 current everything from 2026-11-06
3eae61fe7d7cb689 retired, as go-zpkg-until-2026-11 every artifact before it

⛔ Retired and not revoked, which was a judgement and not a finding. The old key's private half could not be located — two scans of the maintainer's workstation covered 2,167,120 files, and what sat in the CI secret turned out to be its public half. Lost is not compromised: nothing suggests anybody else holds it, so everything it signed stays trusted and nothing is rebuilt. If evidence of compromise ever appears it moves to RevokedSigningKeys, and then it does.

rotation_test.go pins both key ids as literals, so neither variable can change without a test saying so.

And a retired key is not discounted: an artifact signed by the current key and a retired one counts as two. That is exactly right during an overlap, where the publisher deliberately signs with both — a scheme that ranked retired keys lower would make a threshold of 2 unmeetable precisely while rotating.

Revoking is saying somebody else may hold the key. Nothing it signed can be told from a forgery, so its signatures count for nothing — including on artifacts it signed legitimately before the compromise.

⛔ And they cannot be told apart by date, because there is no date. Every published artifact carries the same pinned timestamp so that a re-publish is byte-identical, so "was this signed before the key leaked?" has no answer on the artefact. Honouring signatures made "before" a revocation would be a check that cannot fail — and it would restore trust in exactly the artefacts an attacker would plant. Whether to buy that distinction with a third-party timestamp is artifact#215.

A revoked signature is reported as itself, not folded into "signed by an untrusted key". A stranger's signature is noise anybody can make; your own revoked key on an artifact is a fact about your supply chain. Every entry carries a Because, because a revocation without a reason is a rumour.

What a project NAME may be

A name is a name. It is not a path and it is not a URL fragment — and it reached one anyway, because every lookup builds a URL by concatenation:

fmt.Sprintf("%s/%s/%s", base, project, "package.yml")

Go sends a request path verbatim — it does not clean .. — and raw.githubusercontent.com answers a traversing path with a 307 to the normalised one, which Go's client follows. Measured, not reasoned about:

httpGet(".../pkgxdev/pantry/main/projects/../../../../go-zpkg/pantry-overlay/main/README.md")
→ 1270 bytes of ANOTHER repository's file

A recipe is not inert. It declares dependencies, a build script and the runtime environment a package exports, so fetching one from a repository an attacker chose is the whole game.

The check is where the name is used — the HTTP pantry lookup, the OCI reference, ParseLock, UnmarshalCatalog — because hardly any of these names were typed. They arrive in a lock fetched from a repository, in a catalogue pulled from a registry and read by every <TAB>, and in the dependency lists inside both. One forged name refuses the whole lock or catalogue: a hostile member means the list was written to be one.

An allowlist, not a list of dangerous spellings — .. is obvious, ..%2f is not, and the probe showed it reaching a different path. Segments of [A-Za-z0-9._+-], at most 8, at most 200 characters. All 1908 published projects fit; the deepest is four segments (github.com/GoogleContainerTools/container-structure-test), and that list is the test's positive control, because a rule tightened too far refuses everything and reports green.

A version can carry a rebuild counter

1.8.2_1 is the same upstream software as 1.8.2, built again — for a compiler security release, say, where nothing about the recipe or the version changes and the binary does.

Absent means 0. That is Debian's rule for its own revision ("the absence of a debian_revision is equivalent to a debian_revision of 0") and, here, the literal state of the registry: measured 2026-10-10, 0 of 1371 published version strings carry one, and the only non-alphanumeric rune among them is ..

⛔ Before this, a rebuild was invisible and an exact pin lied about it. ParseVer stops each component at the first non-digit, so 1.8.2_1 and 1.8.2 both yielded Nums [1 8 2] and compared equal:

1.8.2   satisfies "=1.8.2_1" ? true
1.8.2_1 satisfies "=1.8.2"   ? true

A lock pinning the rebuilt artifact would have installed the un-rebuilt one — the vulnerable one, since a rebuild is what a toolchain security fix produces — and said nothing. satisfies routes every operator through one comparator, so ordering and exactness were the same fix.

It orders numerically, and a real version bump still dominates any number of rebuilds below it:

1.8.2 < 1.8.2_1 < 1.8.2_2 < 1.8.2_10 < 1.8.3
⛔ And it has no shorter spelling

tagSpellings offers 1.2.0 as another way of writing 1.2, because a trailing zero really is the same release. It built those alternatives from the numeric components alone — and the rebuild counter is not one of them, so 3.3.11_1 used to offer 3.3.11, which is not another spelling of it but the build it supersedes.

PublishedTagFor walks these and asks the registry about each, so a lock pinning the rebuilt artifact was answered "published" on finding the un-rebuilt one, and a catalogue would advertise a rebuild that does not exist. That is the same defect as the comparator's, by a second route, and the same one the non-zero rule already guards against for neighbouring releases — 26.8.1 must not suggest 26.8.0.

A rebuild now has exactly one spelling: its own.

⛔ And it does not delete the build it supersedes

The same cause, by a third route. numericKey gives a version its canonical identity so that 2.44 and 2.44.0 — one release, two tags, and our registry carries both — collapse into one candidate instead of letting the registry's listing order decide. It too was built from the numeric components alone, so 3.3.11 and 3.3.11_1 shared the key 3.3.11, and selectVersions kept one and dropped the other from the listing entirely.

Measured against ghcr.io/go-zpkg/packages/fftw.org, which publishes both tags:

VersionsFor(fftw.org) = 3.3.11_1          # 3.3.11 is published, and absent
=3.3.11  darwin/aarch64 -> (nothing)      # the artifact is there, and pullable

So rebuilding an artifact silently broke every lock that pinned the version it rebuilt — the exact mirror of the comparator defect, where a pin on the rebuilt artifact was satisfied by the un-rebuilt, vulnerable one. With three builds of one version the listing kept only the last: 1.8.2, 1.8.2_1 and 1.8.2_2 all answered 1.8.2_2.

Superseding is a question of order, which cmpVer answers, not a licence to delete the loser. The counter now joins the key, so the newest is still the rebuild and every published build stays pinnable. Two spellings of one build still collapse — 3.3.11_1 and v3.3.11_1 are one artifact — which is what the key is for.

⛔ pkgx cannot read a _N tag, and that is a deliberate trade

_N follows Homebrew's rebuild and Debian's +bN, and pkgx/libpkgx cannot parse it at all. Its SemVer constructor (src/utils/semver.ts:26-41, read 2026-11-02) splits on . and accepts a part only if it matches /^(\d+)([a-z])$/ — last part only — or /^\d+$/, and otherwise throws invalid version. For 3.3.11_1 the parts are 3, 3, 11_1; the third matches neither.

So pointing a pkgx client at this registry is not a supported configuration once a project carries a rebuild. It throws rather than resolving the wrong artifact, which is the better of the two failures, but it is a failure.

The alternatives were weighed and rejected: a fourth numeric component (3.3.11.1) parses in pkgx but cannot be told apart from a genuine four-component release such as ghc 5.64.3.2; a letter suffix (3.3.11a) parses in pkgx but caps at 26 and occupies the slot real versions use, as openssl's 1.1.1w does. Build metadata (+neon) is not an option at all — the OCI distribution spec requires a tag to match [a-zA-Z0-9_][a-zA-Z0-9._-]{0,127}, which excludes +.

An underscore is not always a rebuild

_ is admitted by ValidateVersionString because it is ordinary in versions elsewhere, so 2026_09_25 is a shape this parser can meet — and reading its _25 as a rebuild would turn one release into the 25th build of another.

The rule is that the part before the underscore must be dotted. Every version this registry carries is; a date-like one is not. _beta is not a rebuild either: what follows must be all digits. And it splits at the last underscore, so in 1.0_2_3 the version is 1.0_2 and the rebuild is 3 — zbk appends _N to a version it already resolved.

An end-of-life line is ranked last, never removed

ZPKG_DEPRECATED may name a JSON file of end-of-life release lines per project, e.g. {"openssl.org": ["3.6", "3.4"]}. Among the versions that satisfy a constraint, one on a listed line is chosen only when nothing else satisfies, and the fall back is announced on stderr (PickVersionForAllWarn returns the warning as data). So ^3 resolves to 3.5.8 while ^3.6.4, =3.6.4 and a lock still resolve to 3.6.4: it is a rank, not a filter, and it never reaches across a soname boundary (^3 and ^4 share no satisfier). The list is read locally; the resolver never needs the network to choose.

A binary may also carry a default, DefaultDeprecated, set before the first resolution — because an opt-in variable fixes nothing: with the variable as the only source, this feature changed no answer for anybody, and openssl's 3.6 line went end-of-life with ^3 still resolving to 3.6.4 everywhere. zpkgx sets it from an embedded, dated list.

The precedence is deliberate, and the surprising direction is the one to avoid:

ZPKG_DEPRECATED list used
set, readable that file, and only that file — never a union with the default
set, missing or unparsable empty, with a warning — explicitly not the default
unset DefaultDeprecated
unset, and no default no list, i.e. the behaviour without this feature

An operator who names a file has said what they want; quietly substituting the default for an unreadable one would hide their mistake behind behaviour that looks like it worked.

The lock format lives here

zbk lock wrote locks and nothing could read one back, because the reader was in zbk's package main — importable by nothing. A format with exactly one program able to read it has not left the program it was written for, and the whole point of a lock is that something else acts on it later.

So the format is here — Lock, LockPin, RenderLock, ParseLock, ReadLock, LockAge — and the resolution stays in zbk, which is the real seam: deciding what a set means today needs a pantry, an overrides set and a version resolver; reading a file that records the answer needs none of them. Writing moved with reading, because a format whose two ends live in different repositories is a round trip waiting to break on a field one side forgot.

The version rule is Spack's: new readers read old locks, old readers refuse new ones.

A recipe can declare a service

A daemon's recipe says how to run it as a service, as data, in one service "<name>" { … } block in its package.hcl. ParseService reads it and FetchService fetches it (overlay first, then the base pantry); the installer renders the unit from it, so the hardening every service shares is written once and a recipe states only what differs:

service "authn-bridge" {
  description   = "authn-bridge, an OpenID Connect provider in front of a SAML federation"
  command       = "authn-bridge"                  # a program in bin/, never a path
  args          = ["--config", "/etc/authn-bridge"]
  stop-timeout  = "20s"
  state-directory { mode = "0700" }               # /var/lib/authn-bridge
  runtime-directory { mode = "0700" }             # /run/authn-bridge
  configuration-directory { mode = "0750" }       # /etc/authn-bridge
}

Also user (default: the name), capabilities (default: none), restart (on-failure | always), reload (none | hup), proc-subset (pid | all) and documentation. The fields are the ones that varied between three real daemons (go-authn's authnd, authn-bridge and authn-revokd), nothing more.

Readers before this version are not broken by it, which was measured rather than assumed: the recipe reader is a generic one, so the block is one more key every consumer ignores — FetchMeta returns the same dependencies and provides with it as without it, and zbk lint passes. That reader drops the block's label, though, so two service blocks are a duplicate key to it and the whole recipe becomes unreadable for every older client. Hence one service per recipe, refused here as well.

The block itself is strict: an unknown attribute is refused rather than ignored, because a misspelt capabilites would otherwise render a unit without the capability. Arguments are a closed alphabet (letters, digits, _ . / : = , @ + -), because they are written into a unit that root loads, where a newline is a new directive and %, $, quotes and ; all mean something.

"It is not here" is two states, and both are types

A caller that wants to offer a next step — ask for this to be published — cannot act on prose, so the two ways a project can be unavailable each have a type:

*NotPublishedError the registry does not carry the project at all. ghcr spells this as 403: denied at the token endpoint for a repository that does not exist, which reads as a credentials fault and is not one
*NoArtifactHereError the project is published, just not for your platform. It carries the versions that exist and the ones with no artifact, so the message stays the sentence the field reports settled on

ProjectToRequest(err) answers both with the project's name, and "" for everything else. One call, because two predicates at the call site is how one of them comes to be forgotten — which is what happened: *NotPublishedError alone covers the rare half. Measured against the published darwin/aarch64 catalogue on 2026-10-23, 567 of 1940 projects had a recipe and no artifact, and for the length of one release zpkgx's "you can ask for this" hint could not see any of them.

⛔ The prose predicate that preceded them is still here as IsNotPublished's last resort, and it is too wide once it leaves this package: it matches not found anywhere in a message, and its first use outside fired on command not found in the requested packages: true, where the package was published and the command was missing.

A closure can be completed for a platform you are not on

CompleteClosure resolves a package's declared closure, installs it, then reads the artifacts' own DT_NEEDED (or Mach-O load commands) and pulls whatever provides each unsatisfied soname — the implicit glibc/gcc libraries and the undeclared ones that package.yml graphs omit — until the closure is complete enough to run on a FROM scratch image.

CompleteClosureFor does that for an explicit os/arch, so one machine can audit all of them:

closure, err := artifact.CompleteClosureFor(roots, dir, "linux", "x86-64")

⛔ Why it had to exist. zbk unresolved reports every reference an installed closure cannot answer; wired into CI it found 967 unresolvable references over 259 of 1407 packaged projects on linux/x86-64. The other five platforms had never been asked, because this function resolved and installed for the host and nothing else could be said honestly — the same shape as a CVE lane that scanned two platforms of six for two months.

⛔ The target's OS picks the binary format, and that is the only line of it that is a behaviour change rather than a pass-through to ResolveClosureFor and InstallFor. Reading a linux closure from a mac has to take the ELF path; taking the host's would complete nothing, find every soname satisfied because it never looked, and report a clean closure — a wrong answer that looks like good news. Two tests pin it in both directions, and CompleteClosure is now exactly CompleteClosureFor(…, HostSlug()), so the host path is identical by construction rather than by a second reading of the same logic.

⛔ A load command is input from a stranger. The @rpath references this reads come out of artifacts that were downloaded, and both halves of one go somewhere: the reference becomes the path that is stat'd to decide a reference is already satisfied, and the project it names becomes a registry repository and a store path written to (<dir>/<project>/v<version>). Since v0.58.0 a reference that does not stay inside the store (filepath.IsLocal) and a project that is not a usable name (ValidateProjectName) are both refused — and both refusals are said, because a silent skip is the failure this code already paid for once at 1266 references. Without the first, an artifact naming @rpath/../../../../etc/passwd had that reference passed over as "satisfied" on the strength of a file that is neither a library nor in the store.

FuzzMachoRef holds the invariant over arbitrary input; its committed corpus includes @rpath/..0, which refuted the first version of the test's own assertion in 0.16s — ..0 is an ordinary filename, and a hand-rolled re-derivation of IsLocal was wrong where IsLocal was right.

⛔ And since v0.59.0 InstallFor checks the name it writes under. It builds <pkgxDir>/<project>/v<version>, and was safe only by order: the fetch below it goes through OCIClient.repository, which validates. But the os.Stat deciding "already present" runs first, so a project of ../x naming a directory that exists returned (false, nil) — this function's way of saying already installed. The caller was told the work was done, nothing was installed, and the evidence was a directory outside the store. Resolved is an exported struct with exported fields, so a caller can fill it without passing through a resolver or a catalogue; the order of two unrelated checks inside one function is not a boundary. Measured before adding it: the pantry's 1929 recipes carry 6381 dependency keys and every one passes, so it rejects nothing that exists.

Where it is proven to work

This package reads formats it did not write — ELF and Mach-O headers, OCI manifests, tar and xz streams — every one of them little-endian on the wire. So any place that reads an integer through the host's byte order rather than through encoding/binary is wrong on a big-endian machine and nowhere else, and a compiler cannot see it. CI therefore does not stop at cross-compiling: it runs the suite on seven runtimes that are not the host.

lane how
test (arm64|riscv64|ppc64le|s390x|loong64, qemu) go test -exec qemu-<arch>-static, -count=1 so a cached PASS from the host architecture cannot stand in for a run that never happened
test (js/wasm, node) node 24. Go's js/wasm port takes net/http through the host's fetch() and the filesystem through the host's fs — two runtimes no other lane exercises. A browser that installs a signed artifact into its own store runs this code
test (wasip1/wasm, wasmtime) a different wasm port with a different syscall surface: no network at all, and a filesystem only where one is granted. A package that installs artifacts has to know which of the two it is on

An artifact for the wasip1/wasm platform is published with the OCI platform os=wasip1, architecture=wasm (the Docker+Wasm / CNCF wasm-oci-artifact spelling; only x86-64 and aarch64 are renamed), and a recipe may scope a block to wasip1:, windows: or js: without it being read as a plain value on another host.

s390x is in the list because it is big-endian and nothing else here is. The build matrix adds linux/darwin on amd64 and arm64 and both wasm targets, so everything that runs also compiles with CGO_ENABLED=0.

BSD-3-Clause.

Documentation

Overview

Package artifact is the reusable package-artifact backend: it resolves a project's runtime dependency closure from the pantry, downloads the artifacts from the configured ZPKG_DIST and installs them — with no runtime dependencies of its own (pure Go, CGO_ENABLED=0, runnable on a `FROM scratch` image). It is imported by zpkgm, zpkgx and zpkb so there is one source of truth.

⛔ THE DEFAULT REGISTRY IS STILL oci://ghcr.io/go-pkgx/packages, and this sentence has been wrong before. DefaultDistBase (zpkg.go:64) is the authority; readmedefault_test.go asserts the README agrees with it, and asserts specifically that it does NOT say go-zpkg/packages — because a rename pass once rewrote the prose here while every artifact stayed where it was. The move to go-zpkg is decided and is a republication, not an edit: the factory run lands first, this constant flips after.

Index

Constants

View Source
const (
	// GlibcVersionAnnotation / GlibcMinKernelAnnotation describe a
	// glibc-flavored build.
	GlibcVersionAnnotation   = "io.github.go-zpkg.glibc.version"
	GlibcMinKernelAnnotation = "io.github.go-zpkg.glibc.min-kernel"
	// ABIProvidesAnnotation lists the sonames an artifact's shared libraries
	// provide.
	ABIProvidesAnnotation = "io.github.go-zpkg.abi.provides"
	// SpecAnnotation carries an artifact's concrete spec.
	SpecAnnotation = "io.github.go-zpkg.spec"
	// AnnotationSourceURI records where a stored source archive came from.
	AnnotationSourceURI = "io.github.go-zpkg.source.uri"
)

The annotation keys this project mints, and the ones it still has to read.

⛔⛔ A REVERSE-DNS KEY IS A CLAIM ON A DOMAIN, AND WE HAD MADE THREE WE COULD NOT BACK. The published artifacts carry `org.go-pkgx.*`; the rename turned that into `org.go-zpkg.*`. Neither is ours: `go-pkgx.org` and `go-zpkg.org` are domains nobody here has ever registered. It is the same error as `go-pkgx.dev/catalog` and as `application/vnd.pkgx.*` — a name that looks like ours because we are the only ones using it.

`io.github.go-zpkg` is the reverse-DNS of a host we demonstrably control: go-zpkg.github.io answers, and the Pages repository exists. It is the same convention OCI itself uses (`org.opencontainers.image.*`) applied honestly.

⛔ WRITE ONE, READ THREE. Nothing rewrites a published manifest, and 42 756 artifacts carry `org.go-pkgx.*`. A reader that accepted only the new key would not fail loudly — it would find the annotation ABSENT, and every one of these keys means something different when absent: no glibc flavour, no declared sonames, no recorded spec. Silence is the worst possible failure for exactly these fields, so the legacy keys stay readable until nothing in the registry carries them.

View Source
const (
	// CatalogFileName is the one file a catalogue layer holds.
	CatalogFileName = "catalog.json"
	// CatalogMaxBytes bounds what a reader will take from that file. The
	// whole pkgx pantry is ~1900 projects; 32 MiB is two orders of magnitude
	// of headroom and still refuses an artifact-sized surprise.
	CatalogMaxBytes = 32 << 20
)
View Source
const (
	PinsFromRecipes = "recipes"
	PinsPublished   = "published"
)

The two questions a lock's versions can answer. See Lock.Pinned.

View Source
const (
	MediaLayerGz = "application/vnd.go-zpkg.artifact.layer.v1.tar+gzip"
	MediaLayerXz = "application/vnd.go-zpkg.artifact.layer.v1.tar+xz"
	// +zstd is a media type the OCI image-spec itself defines (unlike, say,
	// brotli, which has none and would make every generic tool — skopeo, oras,
	// registry UIs — treat the layer as opaque).
	MediaLayerZst = "application/vnd.go-zpkg.artifact.layer.v1.tar+zstd"
	// ArtifactType marks an artifact image manifest (its manifest.config
	// carries this via ORAS PackManifest v1.1's artifactType).
	ArtifactType = "application/vnd.go-zpkg.artifact"
)

OCI media types.

⛔⛔ `vnd.pkgx.*` WAS A VENDOR NAMESPACE WE DO NOT OWN, FOR A FORMAT ITS OWNER DOES NOT DEFINE. Checked against upstream's own corpus (docs.pkgx.sh/llms-full.txt, read 2026-11-04) rather than assumed: pkgx distributes **plain HTTPS tarballs** from `dist.pkgx.dev/<pkg>/<os>/<arch>/` with `.sha256sum` and `.asc` beside them. It mentions no OCI registry, no artifact type and no media type at all. The OCI packaging is entirely ours — `bk` invented it — and we minted its names inside somebody else's vendor space.

That is the same mistake as `go-pkgx.dev/catalog`: a name that looks like ours because we are the only ones using it, in a namespace whose owner never agreed. Once we stop promising compatibility with legacy pkgx, there is not even an argument of convenience left.

⛔ WRITE THE NEW ONE, READ BOTH. 42 756 artifacts are published carrying the old types and nothing rewrites a published manifest; a reader that accepted only the new type would make every one of them unreadable the day it shipped. A rename that breaks every install is not a rename.

⛔ AND THE COMPATIBILITY IS STRUCTURAL, WHICH IS WHY THERE ARE NO `legacy…` CONSTANTS HERE. `extForLayer` matches the codec SUFFIX (`tar+zstd`), never the whole string, so it reads both vendor spellings without being told about either; nothing in the read path compares against ArtifactType at all. Declaring unused legacy constants would have looked like a compatibility mechanism while being dead code.

What pins it is a test holding the old strings as LITERALS (mediacompat_test.go). A literal cannot be changed in sympathy with the new constant during a tidy-up; a constant can.

View Source
const (
	// MediaSourceLayer marks the stored archive. Deliberately codec-agnostic:
	// what is stored is the upstream bytes exactly as they arrived, whatever
	// they were compressed with, because changing a byte would change the digest
	// that is the whole point.
	//
	// ⛔ `vnd.go-zpkg.*`, not `vnd.pkgx.*`. Upstream pkgx defines no OCI
	// artifacts at all — it serves plain tarballs from dist.pkgx.dev — so
	// these names were minted by us inside a vendor namespace belonging to
	// somebody who never agreed to them. See oci.go for the measurement and
	// the read-compatibility rule.
	MediaSourceLayer = "application/vnd.go-zpkg.source.layer.v1"
	// ArtifactTypeSource marks a source manifest, so a registry listing can tell
	// these from artifacts.
	ArtifactTypeSource = "application/vnd.go-zpkg.source"
)

A source mirror: the bytes an artifact was BUILT FROM, stored in a registry so a rebuild does not depend on an upstream host answering.

This matters more than "upstream might go away". 57% of the pkgx pantry is built from GitHub `/archive/refs/tags/…` tarballs, which are GENERATED on request rather than stored: of 165 such URLs probed, not one advertises a Content-Length, because the object does not exist until someone asks. The compression pipeline behind them has changed before and every checksum in the ecosystem changed with it. A mirror of those is not a copy of anything — it is the first and only stored artefact those sources have ever had.

Addressed by CONTENT, not by URL. `SourceTag` turns the sha256 the fetcher already computed into the tag, so the digest recorded in an artifact's SBOM IS the address of the bytes, and nothing has to be named twice or kept in step.

View Source
const (
	// The artifact tarball extensions; ext also selects the OCI layer media type
	// on push.
	//
	// Measured on real artifact payloads (25 MiB coreutils, 68 MiB llvm/lib/clang),
	// same machine, pure-Go codecs:
	//
	//	codec       ratio        compress   decompress
	//	gzip -6     3.50x/4.45x  1.4s       0.2s
	//	xz          6.85x/7.97x  3.5s       0.9s
	//	zstd -19    7.40x/8.89x  1.1s       ~0.0s
	//
	// zstd beats xz on all three axes, and gzip is the worst of both worlds: the
	// poorest ratio AND a slower decompression than zstd. Every install pays the
	// decompression; the factory pays the compression once.
	ExtTarGz  = ".tar.gz"
	ExtTarXz  = ".tar.xz"
	ExtTarZst = ".tar.zst"

	// GlibcProject is the pkgx project that provides the C library + loader; it
	// is the implicit from-scratch root on linux and the artifact whose manifest
	// carries the glibc min-kernel annotation.
	GlibcProject = "gnu.org/glibc"
)

Shared string symbols for the pkgx artifact ecosystem — named once here so artifact, bk and pkgm reference constants instead of scattering magic literals. (The OCI media types live near their use in oci.go as MediaLayer*, and the manifest annotation keys in verify.go as *Annotation; those are already named constants — this file gathers the remaining recurring literals.)

View Source
const (
	// TrustedKeysEnv REPLACES the trust set. An operator who sets it has
	// said which keys they trust, and silently keeping ours in the list
	// would be deciding that for them.
	TrustedKeysEnv = "ZPKG_TRUSTED_KEYS"
	// ExtraTrustedKeysEnv ADDS to it, which is what almost everyone wants:
	// trust the project plus a colleague. Nix's `extra-` convention, for
	// the reason Nix has it — replacing a default list by hand is how you
	// drop the entry that made everything work.
	ExtraTrustedKeysEnv = "ZPKG_EXTRA_TRUSTED_KEYS"
	// TrustThresholdEnv is how many DISTINCT trusted keys must have signed.
	TrustThresholdEnv = "ZPKG_TRUST_THRESHOLD"
)
  • ⛔⛔ TWO PEOPLE BUILDING THE SAME THING PRODUCE THE SAME HASH, ON PURPOSE
  • ⛔ WHICH IS WHY ONE PINNED KEY WAS NOT ENOUGH
  • THE MODEL, AND WHERE IT COMES FROM

Who may vouch for a build, and how many of them it takes.

⛔⛔ TWO PEOPLE BUILDING THE SAME THING PRODUCE THE SAME HASH, ON PURPOSE

The question this answers is "how do I tell a build made by two different people apart?", and the first half of the answer is that you do not, and must not want to. A build's identity is a function of its inputs; if my name went into it, my rebuild of your artifact would get a different address and could never be compared with yours. Identity has to collide for two independent builds of the same inputs — that collision IS the evidence.

What differs between two builders is therefore not the artefact but the ATTESTATION: who put their key on it. So the registry holds one artifact and, potentially, several signatures over it, and the client counts them.

⛔ WHICH IS WHY ONE PINNED KEY WAS NOT ENOUGH

Until now artifact carried a single SigningPublicKey and oci.go verified the FIRST signature referrer it found. With one signer that is correct and with two it is not merely incomplete, it is wrong twice over:

  • it cannot count — "signed by both builders" and "signed by one" are the same answer;
  • it is order-dependent — if the registry lists the other builder's signature first, verification FAILS on an artifact that our own key signed, and the message reads as tampering.

THE MODEL, AND WHERE IT COMES FROM

A trust SET plus a THRESHOLD, which is TUF's shape (§4.3: a threshold is "a positive integer number of keys (>=1) of that role whose signatures are required"), with TUF's counting rule (§5.4.2: a client "MUST NOT count more than one verified SIGNATURE from that KEYID towards the THRESHOLD").

The spelling of the key list is Nix's `trusted-public-keys` — whitespace separated, `name:key` — because an operator who has configured a Nix binary cache already knows it. Nix requires one signature from one trusted key; Guix likewise checks a single authorised signature on a narinfo, and compares servers out of band with `guix challenge`. The threshold is what lets that comparison be a precondition of installing rather than an audit someone runs afterwards.

View Source
const (
	MaxTrustedKeys = 64
	MaxSignatures  = 32
)

Bounds. A trust set comes from the operator's own configuration, so its cap is there to catch a mistake; the signature list comes from a REGISTRY, so its cap is a resource bound against a subject that can be made to carry thousands of referrers — each of which would otherwise cost a fetch and a signature verification per trusted key.

⛔ BOTH REFUSE RATHER THAN TRUNCATE. Verifying the first 32 of 5000 signatures and reporting success would be a filter pretending to be a check: the answer "this is signed by alice" would depend on where the list happened to be cut.

View Source
const (
	// ArtifactTypeSignature marks a cosign signature referrer manifest.
	ArtifactTypeSignature = "application/vnd.dev.cosign.artifact.sig.v1+json"
	// ArtifactTypeProvenance marks the in-toto SLSA provenance referrer: the
	// statement that says WITH WHAT an artifact was built.
	//
	// ⛔ IT IS THE ONLY PLACE THE BUILDER IS RECORDED. The build's identity
	// (Concrete) deliberately excludes the builder — a zbk release must not
	// re-address every artifact in the registry, which is Nix's and Spack's
	// position too. So "which factory made this?" is answerable ONLY from
	// this attestation, and anything that wants to ask has to read it.
	ArtifactTypeProvenance = "application/vnd.in-toto+json"
	// MediaSimpleSigning is the media type of the simple-signing payload blob.
	MediaSimpleSigning = "application/vnd.dev.cosign.simplesigning.v1+json"
	// CosignSignatureAnnotation carries the base64 signature on the referrer
	// manifest (cosign's convention).
	CosignSignatureAnnotation = "dev.cosignproject.cosign/signature"
	// SignerKeyIDAnnotation names, in hex, the minisign key id that made the
	// signature on this referrer.
	//
	// ⛔ IT IS A HINT AND MUST NEVER BE TRUSTED. A cosign blob signature is a
	// bare Ed25519 signature: it does not say which key made it, so with two
	// signers a verifier would have to try every trusted key against every
	// signature. This annotation turns that product back into a walk — and
	// nothing more. Verify tries the hinted key FIRST and then the others, so
	// an annotation that lies costs time and changes no verdict.
	//
	// There is deliberately no annotation for the signer's NAME. A name on the
	// artefact is written by whoever pushed it; the name a user reads comes
	// from their own trust set, which is the only place it means anything.
	SignerKeyIDAnnotation = "org.go-zpkg.signer.keyid"
)

Signature referrer conventions.

View Source
const CatalogProjectName = "go-pkgx.dev/catalog"

CatalogProjectName is the reserved project the catalogue is published under. A project name rather than a special path, because that is what makes it an ordinary artifact.

⛔⛔ THIS NAME ADDRESSES A PUBLISHED ARTEFACT, AND THE RENAME MOVED THE NAME WITHOUT MOVING THE ARTEFACT. The go-pkgx → go-zpkg pass (#192) rewrote it from `go-pkgx.dev/catalog` to `go-zpkg.github.io/catalog`. Nothing publishes that, so every `zpkgs catalog update` on defaults has failed since, and a FROM-scratch image built around the result has no catalogue in it at all:

zpkgs: catalog update: go-zpkg.github.io/catalog is not published in
ghcr.io/go-pkgx/packages/go-zpkg.github.io/catalog … 403: denied

Measured 2026-11-04 with ghcrprobe, anonymously, against a positive control so that "absent" could not be the probe failing:

ghcr.io/go-pkgx/packages/gnu.org/bash                 PRESENT (control)
ghcr.io/go-pkgx/packages/go-pkgx.dev/catalog          PRESENT
ghcr.io/go-pkgx/packages/go-zpkg.github.io/catalog    ABSENT
ghcr.io/go-pkgx/packages/go-pkgx.dev/catalog (as .github.io) ABSENT

It is the SECOND constant this happened to — see DistBase in zpkg.go, where the same pass pointed the whole fleet at an empty registry. The rule both of them earned: a constant that addresses something published follows the artefact, and moves only after a copy has run and been verified.

View Source
const CatalogVersion = "1"

CatalogVersion is the version it is published as. One moving tag rather than a dated series: a catalogue is a cache of the registry, every reader wants the newest, and a hundred dated versions of it would be a hundred things to garbage-collect.

View Source
const DefaultDistBase = "oci://ghcr.io/go-pkgx/packages"

Base URLs for the pkgx distribution + pantry; overridable in tests, and at runtime via $ZPKG_DIST / $ZPKG_PANTRY so a consumer can point pkgm/pkgx at a local mirror produced by the `mirror` tool.

DistBase defaults to the **signed** OCI registry so the default install path is verifiable end-to-end (pairs with VerifyRequired defaulting on). That registry is a growing subset of the pantry; for the full upstream catalogue set ZPKG_DIST=https://dist.pkgx.dev (unsigned → also set ZPKG_VERIFY=0).

⛔⛔ THE ORG IN THIS URL IS A DEPLOYMENT FACT, NOT OUR NAME, AND THE RENAME GOT IT WRONG. The go-pkgx → go-zpkg pass rewrote it to `ghcr.io/go-zpkg/packages`, which every release from v0.64.0 to v0.68.0 carries. Measured 2026-11-04 with ghcrprobe, anonymously, against a control:

ghcr.io/go-pkgx/packages/gnu.org/bash   PRESENT  (tag sha256-64caff4d…)
ghcr.io/go-pkgx/packages/zlib.net       PRESENT  (tag sha256-21c9e221…)
ghcr.io/go-zpkg/packages/gnu.org/bash   ABSENT
ghcr.io/go-zpkg/packages/zlib.net       ABSENT

GitHub's container packages are org-scoped and did NOT follow the repository transfer: the repositories all moved, every artifact stayed. So those five releases point the whole fleet at an empty registry, and `zpkgx` cannot fetch anything with its defaults — which is what the `FROM scratch, offline` check was reporting in go-zpkg/zpkgx#141 while I read it as a path problem.

Renaming a URL did not move what it addresses. The artifacts move by COPY, and this line follows them — not the other way round. Until that copy has run and been verified, this is where they are. DefaultDistBase is the same value as DistBase's initial one, as a CONSTANT.

⛔ DistBase IS A VAR THAT TESTS REASSIGN, so it cannot be used to check what the documentation claims the default is: the first version of readmedefault_test.go compared the README against DistBase, passed under `-run`, and failed in the full suite with

the README does not name the real default "http://127.0.0.1:55863"

— another test had pointed it at an httptest server first. A gate that reads mutable global state tests the order the suite happened to run in.

View Source
const DefaultSignerName = "go-zpkg"

DefaultSignerName names the project's own key in messages.

View Source
const DefaultThreshold = 1

DefaultThreshold is 1.

⛔ AND THAT IS A DELIBERATE CHOICE, NOT A PLACEHOLDER. The mechanism is worth having before a second builder exists, because it has to be in place BEFORE the registry is rebuilt — an artifact signed under the old scheme would have to be republished to gain a second signature. But shipping a default of 2 today would make every published artifact unverifiable by its own client the moment this is released, which is not a security improvement, it is an outage. Two is one environment variable away, and it becomes the right default the day a second builder publishes.

View Source
const DeprecatedEnv = "ZPKG_DEPRECATED"

DeprecatedEnv names a JSON file that lists, per project, release lines that are end-of-life:

{"openssl.org": ["3.6", "3.4", "1.1.1"]}

⛔ THIS IS A PREFERENCE, NOT A CONSTRAINT. A deprecated version stays resolvable: it is RANKED LAST among the versions that satisfy a constraint, never removed from the list. The two shapes behave the same on every request except one, and that one is the whole correctness question:

^3.6.4  admits ONLY 3.6.4, which is deprecated.

A RANK resolves it to 3.6.4 (and warns). A FILTER would leave nothing and fail the request. This is RANK, on purpose: reproducibility and an explicit request for a dead line outrank the preference, and a lock that stopped resolving would be a worse defect than the one this fixes.

It follows that the preference only changes an answer where a supported candidate ALSO satisfies the constraint. `=3.6.4` and `^3.6.4` have none, so they get 3.6.4; `^3` and `^3.5` have 3.5.x, so they get 3.5.8; `^4` and `^3` share no satisfier, so the preference can never reach across a soname boundary. No special case for exact pins is needed or present.

The file is read locally, once. The resolver never touches the network to choose a version (FROM scratch and air-gapped use must keep working). Where the list comes from — `bk eol` computing it from endoflife.date — is a separate matter. An unset variable, a missing file or an unreadable one all mean an empty list, i.e. exactly the behaviour without this feature; an unparsable file is reported once on stderr and also means an empty list.

View Source
const KnownDeprecatedAsOf = "2026-11-01"

KnownDeprecatedAsOf is the day KnownDeprecated was measured.

A reader has to be able to tell a current judgement from one left behind by a release, and a list with no date reads as fresh forever.

View Source
const LockfileVersion = 1

LockfileVersion is this format's number. Bumped when a reader of the previous one would MISREAD a file rather than merely miss a field.

The compatibility rule is Spack's, and worth copying: new readers read old locks, old readers refuse new ones.

View Source
const MaxSignerNameLen = 64

MaxSignerNameLen bounds a signer's name. A name is read aloud in a verdict, not stored; anything longer is a paragraph someone is trying to put on the screen.

View Source
const SpecVersion = 1

specJSON is the wire form. Field names are short because an OCI annotation value travels in every manifest fetch, and explicit because a Go field rename must not silently change what published artifacts mean. SpecVersion is the payload's format version.

⛔⛔ A NUMBER, NOT A SCHEMA, AND IT BUYS ONE THING: A LEGIBLE FAILURE. Without it, a payload written by a NEWER release is not rejected — it is MISREAD. encoding/json drops fields it does not know, silently, so a v2 spec carrying a new identity-bearing field parses cleanly here as a v1 spec that happens to lack it.

The artifact is not installed in that case, because VerifySpec recomputes the hash and the dropped field moves it. So the outcome is already fail-closed — but the message is "the artifact does not match its own description", which reads as tampering and sends its reader to look for an attacker. The truth is "this artifact was written by a release newer than yours", and only a version field can say so.

A JSON Schema would say more and cost more: this value travels in every manifest fetch, and a schema is a URL to fetch and a document to keep in step for a payload of five fields. The question a reader actually has is "can I understand this?", and an integer answers it.

View Source
const StorePlatformFile = ".zpkg-platform"

StorePlatformFile is the name of the marker a store carries saying which platform its artifacts were built for. ⛔ `.zpkg-platform`: this file is OURS, written by our installer into our store, and upstream pkgx writes nothing of the kind. Named after upstream it would claim a convention that is not theirs — and a store shared with them is exactly what moving to ~/.zpkg stops.

View Source
const WhyNoDate = "there is no build date: a published artifact's timestamp is pinned " +
	"(SOURCE_DATE_EPOCH) so that re-publishing it is byte-identical, which means every " +
	"artifact carries the same instant. --sort=installed orders by when THIS machine got them."

WhyNoDate is the one line a command prints when a reader asks for an order by build date, so the absence is explained rather than looking like an oversight.

View Source
const WithdrawnEnv = "ZPKG_WITHDRAWN"

WithdrawnEnv names a JSON file listing, per project, the BUILDS that should no longer be chosen — the ones a CVE has caught:

{"fftw.org": [
  {"version": "3.3.11",
   "hash": "27o3ysxva6swayci",
   "because": "CVE-2026-1234",
   "superseded_by": "rnp44isdgdtzxcit"}
]}

WHY A BUILD AND NOT A VERSION

ZPKG_DEPRECATED retires a release LINE, which is the right grain for "3.6 is end-of-life". It is the wrong grain for the case this exists for: the package is fine, the source is fine, and what is wrong is the TOOLCHAIN it was compiled with. A Go patch release fixes a standard-library defect; nothing about the recipe or the upstream version changes, only `go.dev` in the closure — so the fixed artefact is another build of the SAME version, and the broken one has to be nameable without condemning the version.

That is only expressible because a build has an identity. It is the first thing the hashed layout buys that a version-addressed store could not do at all.

⛔⛔ IT RANKS, IT DOES NOT FILTER — THE SAME RULE AS DEPRECATION

A withdrawn build stays resolvable. The reasoning is deprecated.go's and has not changed: a LOCK pins a build, and a lock that stops resolving is a worse defect than the one being fixed. Reproducibility and an explicit request outrank the preference.

So:

  • two builds, one withdrawn → the other, and the withdrawal is announced;
  • one build, withdrawn → that one, loudly, because there is nothing else and refusing would strand the caller;
  • several left after the withdrawn ones are set aside → still AMBIGUOUS. Setting aside a build somebody recorded a reason for is not the same as inventing an order among the rest.

⛔ AND IT CARRIES THE REASON AND THE REPLACEMENT

A warning that does not say why is noise, and one that does not say where to go next leaves the reader to find the fixed build themselves — which means reading hashes, which is the thing nobody can do. `because` is the advisory identifier; `superseded_by` is the hash that fixes it, when one exists. Empty is honest: a withdrawal with no replacement yet is a real state and must be sayable.

Read locally, once, like the deprecation list: the resolver never touches the network to choose, because FROM scratch and air-gapped use must keep working. Where the file comes from — `bk advisories` and the VEX documents attached to published artifacts — is a separate matter, the same separation as `bk eol` and ZPKG_DEPRECATED.

Variables

View Source
var (
	DistBase   = DefaultDistBase
	PantryBase = "https://raw.githubusercontent.com/pkgxdev/pantry/main/projects"

	// PantryOverlay is MERGED OVER PantryBase for a project's package.yml: a
	// key the overlay states replaces that key, a key it omits is inherited
	// from upstream, and a list replaces rather than merges. A small curated
	// overlay can then carry corrected recipes (e.g. a stale
	// `openssl.org: ^1.1` bumped to a constraint that matches the published
	// artifacts) without forking the whole pantry.
	//
	// This said "consulted BEFORE PantryBase, falling back when the overlay
	// has no recipe" until #103, and that description outlived the code by
	// long enough to matter. Preferring the overlay WHOLE was right while every
	// entry was a full copy of an upstream recipe; go-zpkg/pantry-overlay#50
	// then reduced all 183 to the keys they change, so an entry read whole is a
	// FRAGMENT — under the old rule curl.se would resolve to `dependencies` and
	// nothing else, with no distributable and no build. bk read one that way
	// and its closure collapsed from 54 projects to 8 (go-zpkg/bk#232).
	//
	// A comment that describes a superseded rule is worse than none, because it
	// is believed: this is the second one found today.
	//
	// It defaults to OUR overlay because DistBase defaults to OUR registry, and
	// the two have to agree. They did not, and the disagreement was silent:
	//
	//	$ pkgm install rust-lang.org/cargo      # defaults, darwin/aarch64
	//	$ cargo --version
	//	$ echo $?
	//	137
	//
	// The published cargo links openssl 3, and says so — `@rpath/openssl.org/
	// v3.6.4/lib/libssl.3.dylib`. Upstream's recipe does not mention openssl at
	// all, so the resolver installed 1.1.1w (pulled in by libssh2) and the
	// binary died on a library nothing had installed. The corrected recipe was
	// in the overlay the whole time, and nothing consulted it.
	//
	// ZPKG_PANTRY_OVERLAY overrides it; "none" turns it off and resolves
	// everything upstream.
	PantryOverlay = "https://raw.githubusercontent.com/go-zpkg/pantry-overlay/main/projects"

	// UpstreamDist is a SECOND distribution consulted for the versions of a
	// project our own DistBase does not carry. Empty means there is none, and
	// that is the default.
	//
	// ⛔⛔ THE MECHANISM IS KEPT, THE VALUE IS NOT. This was
	// `https://dist.pkgx.dev` — upstream pkgx's distribution — compiled in and
	// always on, so every lookup our registry could not answer silently became
	// a question asked of another project. That was right while this was a
	// mirror with a gap-filler. It is wrong for a distribution: a project we
	// do not carry should SAY SO, not resolve to somebody else's artifact.
	//
	// Measured 2026-11-04 on a one-in-forty sample of the 1907 recipes: of the
	// artifacts published and read, 0% were built here and 100% were mirrored,
	// on darwin/aarch64 and linux/x86-64 alike, toolchain included. So the
	// fallback is not a corner case today — which is exactly why it has to
	// become a stated choice rather than a built-in assumption.
	//
	// Set ZPKG_UPSTREAM_DIST to point at one; "none" is the explicit off, as
	// with ZPKG_PANTRY_OVERLAY. Our own factory sets it while the rebuild is
	// in flight, which puts the dependency in a workflow where it can be read
	// instead of in a constant where it cannot.
	UpstreamDist = ""

	// ImplicitRootsOff drops the implicit system-library roots from closure
	// completion. See implicitRoots; set from ZPKG_IMPLICIT_ROOTS=none.
	ImplicitRootsOff = false

	// CacheBase is an OCI registry consulted for the ARTIFACT ITSELF before
	// DistBase — a pull-through cache next to the builders, so a job does not
	// fetch llvm.org's ~1.7 GiB across the network again.
	//
	// It deliberately takes NO part in resolution. A pull-through cache cannot
	// list the tags of a repository it has not synced yet, so listing against it
	// falls back to the upstream dist, which yields a VERSION where our registry
	// wants a TAG — gnu.org/bash is published as `5.3`, not `5.3.0` — and the
	// pull then 404s on a tag that never existed. Measured; it is why this is a
	// separate variable and not simply ZPKG_DIST pointed elsewhere.
	//
	// What the cache serves is still verified against the signature DistBase
	// advertises, so a cache that returns the wrong bytes is rejected rather
	// than trusted.
	CacheBase = ""
)

BuildOrders lists the keys, for a flag's help and for validating one.

View Source
var DefaultDeprecated = KnownDeprecated

DefaultDeprecated is the list used when DeprecatedEnv is unset.

⛔ IT EXISTS BECAUSE AN OPT-IN ENVIRONMENT VARIABLE FIXES NOTHING. Shipped with the variable as the only source, this whole feature changed no answer for anybody: openssl's 3.6 line went end-of-life on 2026-11-01 and `^3` went on resolving to 3.6.4 across the fleet, because nothing set the variable. A capability nobody switches on is a capability the fleet does not have.

⛔⛔ AND LETTING EACH BINARY CARRY ITS OWN DEFAULT FIXED IT ONLY WHERE SOMEONE REMEMBERED. That was the second shape of the same defect: the activation lived in `pkgx/internal/app`, unimportable from any other module, so `pkgm` — the INSTALLER — went on resolving `openssl.org@^3` to the end-of-life 3.6.4 while `pkgx` gave 3.5.8. Measured 2026-11-03, A/B on one pkgm binary. See KnownDeprecated.

So this now HOLDS the library's list rather than starting empty: every consumer inherits the ranking, and a binary that wants a different list assigns this variable as before. The precedence is unchanged, and the surprising direction is still the one to avoid:

  • DeprecatedEnv set and readable → that file, and ONLY that file;
  • DeprecatedEnv set and broken → empty, with a warning. It does NOT fall back to this default: an operator who named a file has said what they want, and quietly substituting something else would hide their mistake behind working behaviour;
  • DeprecatedEnv unset → this default.

A caller that overrides it must do so before the first resolution: the list is latched once, so assigning it afterwards does nothing.

View Source
var DefaultWithdrawn = map[string][]Withdrawal{}

DefaultWithdrawn is the list used when WithdrawnEnv is unset.

⛔ EMPTY, AND THAT IS NOT THE SAME DECISION AS DefaultDeprecated's. The deprecation list ships populated because end-of-life dates are a public fact anybody can compute and being wrong about one costs a warning. A withdrawal says "do not use this build", which is a claim about OUR artefacts that only our own advisory process can make — and shipping a guess would either condemn a build nobody has examined or, worse, imply that an empty list means every build has been cleared.

View Source
var ErrBadProjectName = errors.New("artifact: not a project name")

ErrBadProjectName is returned for a name that cannot be used as one.

View Source
var ErrBadService = errors.New("service block")

ErrBadService is wrapped by every refusal of a service block's contents.

View Source
var ErrBadVersionString = errors.New("not a version")

ErrBadVersionString is returned for a version a lock has no business carrying.

View Source
var ErrInsecurePath = errors.New("artifact: insecure path in archive")

ErrInsecurePath is returned by Extract when an archive entry (or a hard-link source) names an absolute path, or one that after component-stripping would escape the destination directory. It is wrapped with the offending name so callers can test for it with errors.Is.

View Source
var ErrSameNamespace = errors.New("artifact: source and destination are the same namespace")

ErrSameNamespace means a copy was asked for between one namespace and itself. It is not a no-op worth tolerating: a copy that reports success without moving anything is exactly how an empty destination passes for a full one, which is the failure this whole operation exists to avoid.

View Source
var ErrSourceAbsent = errors.New("artifact: source not in the mirror")

ErrSourceAbsent is returned by PullSource when the store has no such digest.

View Source
var Exec func(argv0 string, argv []string, env []string) error

Exec runs (or, on UNIX, replaces the current process image with) the target binary; overridable in tests. Its concrete value is platform-specific:

  • UNIX (exec_unix.go): syscall.Exec — replaces the process, never returns on success.
  • Windows (exec_windows.go): spawns the child with inherited stdio, waits, and exits the parent with the child's exit code (Windows has no execve).

The signature is identical on every platform, so callers are unchanged.

View Source
var HTTPClient = NewHTTPClient()

HTTPClient is an HTTP client that trusts the embedded CA bundle (falling back to the system pool if, somehow, the embed is empty). It is overridable.

View Source
var KnownDeprecated = map[string][]string{
	"openssl.org": {"1.1.1", "3.0", "3.1", "3.2", "3.3", "3.4", "3.6"},
}

KnownDeprecated is, per project, the release lines upstream has abandoned.

Measured on KnownDeprecatedAsOf against the registry and endoflife.date:

1.1.1  ended 2023-09-11 (LTS)     3.4  ended 2026-10-22
3.0    ended 2026-09-07 (LTS)     3.6  ended 2026-11-01
3.1    ended 2025-03-14           3.5  supported to 2030-04-08 (LTS)
3.2    ended 2025-11-23           4.0  supported to 2027-05-14
3.3    ended 2026-04-09

Only openssl so far, and deliberately: this list changes what a constraint resolves to across the whole fleet, so each project earns its place by a measurement rather than by a sweep. go-zpkg/artifact#179 moves the judgement into the signed catalogue, where it is computed rather than committed.

View Source
var KnownSupported = map[string][]string{
	"openssl.org": {"3.5", "4.0"},
}

KnownSupported is the other half of the same measurement: the lines still supported on KnownDeprecatedAsOf, and which must therefore NOT appear in KnownDeprecated.

⛔ IT EXISTS TO BE CONTRADICTED. The expensive mistake here is listing a line that is still supported — that is how `^3` would be pushed off the 3.5 LTS onto something older, quietly, with no error anywhere. Recording both halves of the verdict makes that edit fail a test instead of shipping.

View Source
var RetiredSigningPublicKeys = []string{
	"go-zpkg-until-2026-11:RWQ+rmH+fXy2iYr+gReQAOQtYWtH0A7UlxcAa2hpr+txNBwGqtpFsR6L",
}

RetiredSigningPublicKeys are keys this project HAS signed with and still trusts, newest first, as `name:key` entries.

⛔⛔ RETIRING A KEY MUST NOT INVALIDATE WHAT IT SIGNED. A signature is a statement about bytes, made at a moment; replacing the key we sign WITH says nothing about the artifacts already signed. A trust set that dropped the old key would refuse the whole published registry overnight — and since the fix for a lost or compromised key is to rotate, a design where rotation breaks everything is a design where nobody rotates.

⛔ AND THEY ARE NOT EXPIRED BY DATE, BECAUSE THERE IS NO DATE TO EXPIRE THEM BY. Every published artifact carries the same pinned timestamp so that re-publishing is byte-identical, so "was this signed before the key was retired?" is a question nothing on the artefact can answer. Pretending otherwise would be a check that cannot fail. A key leaves this list only when a human decides its signatures are no longer to be believed — which is a withdrawal of trust, a different and louder act than rotation.

⛔ `go-zpkg-until-2026-11` IS RETIRED, NOT REVOKED, AND THE DISTINCTION IS A JUDGEMENT WE MADE RATHER THAN A FACT WE FOUND. Its private half could not be located: two scans of the maintainer's workstation covered 2,167,120 files and found no key, and what sat in the CI secret turned out to be its PUBLIC half. Lost is not compromised — nothing suggests anybody else holds it — so everything it signed stays trusted and nothing needs rebuilding. If evidence of compromise ever appears, it moves to RevokedSigningKeys and that changes.

It signed every artifact in the registry up to 2026-11-06.

View Source
var RevokedSigningKeys []RevokedKey

RevokedSigningKeys are keys that were OURS and are no longer trusted — compromised, exfiltrated, or signed something we did not authorise.

⛔⛔ REVOKING IS NOT RETIRING, AND THE DIFFERENCE IS THE WHOLE POINT OF HAVING BOTH. A retired key is one we stopped signing WITH; everything it signed is still ours and still true. A revoked key is one somebody else may have held, so NOTHING it signed can be distinguished from a forgery — including artifacts it signed perfectly legitimately before the compromise.

⛔ AND WE CANNOT TELL THE TWO APART, BECAUSE THERE IS NO DATE. Every published artifact carries the same pinned timestamp, so "was this signed before the key leaked?" has no answer on the artefact. A scheme that honoured signatures made "before" the revocation would be a check that cannot fail, which is worse than no check: it would restore trust in exactly the artefacts an attacker would plant.

So a revoked key's signature counts for NOTHING — and is reported, loudly and separately from a stranger's. A stranger signing our artifact is noise anybody can make; our own revoked key on an artifact is a fact about our supply chain that the reader has to act on.

The remedy for a revoked key is to rebuild and re-sign, not to argue. That is expensive, which is why it is a different list from the retired one and why entries carry a reason.

View Source
var SigningPublicKey = "RWTfNcOkaTHT5lsudyEtExOI45lQQEHCu+fckxlKljWtzfuFTQtgjLZT"

SigningPublicKey is the go-zpkg artifact signing key (minisign format). A signed artifact carries a cosign-style signature referrer whose signature verifies against this key; verification is fail-closed. It is a var (not const) only so tests can substitute a throwaway key.

⛔ IT IS THE DEFAULT TRUST SET, NOT THE WHOLE OF IT. What a client will accept is a TrustSet — this key unless the operator says otherwise, plus whatever ZPKG_EXTRA_TRUSTED_KEYS adds, and a threshold over the lot. See trust.go; everything that verifies an artifact goes through LoadTrustSet.

View Source
var Warn func(msg string)

Warn receives diagnostics a caller should SEE but that are not fatal: a closure the resolver could not complete, a soname nothing provides. It is nil by default — a library must not write to a process's stderr uninvited — so a command-line tool sets it once at startup:

artifact.Warn = func(msg string) { fmt.Fprintln(os.Stderr, "pkgx: "+msg) }

Silence here is expensive: an unprovided soname does not fail the install, it fails LATER, when the binary starts and reports "cannot open shared object file" with nothing pointing back at the resolution that gave up.

Functions

func Annotation

func Annotation(ann map[string]string, key string) string

Annotation reads key from ann, falling back to the spellings the same field has carried before.

⛔ THE CURRENT KEY WINS EVEN WHEN BOTH ARE PRESENT. A manifest holding two is one somebody wrote with two tools, and the newer spelling is the one this build agrees with; preferring the legacy one would make an upgrade read older data than the version before it.

An empty value is NOT a hit. `ann[k]` cannot tell "absent" from "present and empty", and for every key here an empty string means the same as absent — so falling through on it is what makes an artifact written with an empty new key still find its legacy value.

func AnnotationPresent

func AnnotationPresent(ann map[string]string, key string) (string, bool)

AnnotationPresent is Annotation for callers that must tell an EMPTY value from a MISSING one — the spec annotation does, because an empty spec is a malformed artifact and a missing one is an old artifact.

func BinNames

func BinNames(project string, provides []string) []string

BinNames returns the base names of the binaries a package provides, falling back to the project's leaf name when it declares no `provides:`.

func BinNamesFor

func BinNamesFor(project string, provides []string, version string) []string

BinNamesFor is BinNames with the package's version in hand, so a recipe that names its binary AFTER that version is reachable.

apache.org/apr-util provides exactly one program:

provides:
  - bin/apu-{{ version.major }}-config

and taken verbatim that is a file called `apu-{{ version.major }}-config`, which exists nowhere. `pkgx apache.org/apr-util` answered

command not found in the requested packages: apu-{{ version.major }}-config

Six pantry recipes put a moustache in a bin/ name; five of them also list a plain one and were saved by it. This is the sixth.

An empty version means "do not expand": the callers that have no version are asking what the recipe SAYS, and a placeholder silently becoming empty there would turn one unusable name into a different unusable name.

func CanonicalLoaderExists

func CanonicalLoaderExists() bool

CanonicalLoaderExists reports whether /lib{,64}/ld-linux-* is present.

func CatalogTarball

func CatalogTarball(c Catalog) ([]byte, error)

CatalogTarball packs a catalogue into the shape an artifact layer has: a gzipped tar holding one file, `catalog.json`.

A real tar, not gzipped JSON wearing a tar media type. The manifest says `…layer.tar+gzip` and a reader that believes it — `tar tzf`, or any tool that unpacks an artifact — must not be lied to for the sake of fifteen lines.

func CertPool

func CertPool() *x509.CertPool

CertPool is the trust store the go-zpkg tools use: the host's, plus the embedded Mozilla bundle. On a FROM-scratch image there is no system store at all, so the embedded bundle is the only thing standing between a tool and "x509: certificate signed by unknown authority" — which is exactly what bk hit when it tried to reach github from inside a scratch builder.

func CheckStorePlatform

func CheckStorePlatform(dir, osn, arch string) error

CheckStorePlatform says whether a store holds artifacts for the platform about to use it.

⛔ THE FAILURE IT REPLACES IS `Exec format error`, WITH NO EXPLANATION. A prefix is `$ZPKG_DIR/<project>/v<version>/…` with no platform in it, so a store shared over NFS between linux/x86-64 and linux/aarch64 holds whichever architecture was installed last — `lib/libssl.so.3` is the same file name on both — and the other architecture gets a kernel error naming nothing anybody can act on.

⛔ A STORE THAT DOES NOT SAY IS NOT A MISMATCH. Every store written before the marker existed is silent, and treating silence as disagreement would refuse to run on the installed base. Silence is reported as agreement here and the question is closed the first time somebody installs into it.

func CompanionsFor

func CompanionsFor(project, osn, arch string) (map[string]string, error)

CompanionsFor returns a recipe's `companions:` for an explicit os/arch slug.

A companion is not a dependency: the package works without it, and nothing links against it. It is what the recipe says belongs in the same ENVIRONMENT, and some packages are unusable without theirs — `rust-lang.org` ships bin/cargo-clippy and bin/cargo-fmt and no `cargo` at all, because cargo is the separate `rust-lang.org/cargo` project it names here.

Reading it was the whole of issue #65: a recipe that declares `rust-lang.org` and then runs `cargo` is correct against upstream pkgx and died against ours with `"cargo": executable file not found in $PATH` — 74 pantry recipes declare the key, and 21 of those are named as a dependency by some other recipe.

func ConfigError

func ConfigError() error

ConfigError returns the error, if any, from loading ~/.zpkg/config.hcl2. It is nil when the file parsed cleanly or is simply absent, and non-nil only on a real path or parse failure (in which case reads fall back to the environment).

func CopyPublished

func CopyPublished(ctx context.Context, src, dst *OCIClient, project, tag string) (ocispec.Descriptor, error)

CopyPublished copies one published tag of one project from src to dst, preserving digests, together with everything that hangs off it: the cosign signature manifests and the SBOM / provenance / VEX attestations.

⛔⛔ IT IS NOT HOW A REGISTRY NAMESPACE IS MOVED, AND IT WAS WRITTEN BELIEVING IT WAS. The premise holds as far as it goes: GitHub's container packages are org-scoped and do not follow a repository transfer — when every repository moved from go-pkgx to go-zpkg, every artifact stayed, DistBase was rewritten to match, and v0.64.0 through v0.68.0 shipped a fleet pointing at an empty registry (TestTheDefaultRegistryHoldsArtifacts is the gate that came out of it). So the new namespace must be POPULATED.

It is populated by BUILDING CORRECT RECIPES INTO IT, not by moving these artifacts. They were built from pkgxdev/pantry's recipes, which go-zpkg/packages#431 establishes can be neither read nor copied — 2453 upstream files, no LICENSE and no COPYING — and whose item 1 already concluded "the sovereign pantry is WRITTEN, not converted". The artifacts also carry no build identity (packages/.github/workflows/wasip1.yml:201: "Nothing in go-pkgx carries a build identity — it predates them"), so the sentence this function was justified with, "a rebuild changes artifacts that are already attested", was false about the thing it protected. go-zpkg/packages#440 carries the corrected order.

What is left is the case where republishing verbatim is the POINT: a mirror (go-zpkg/mirror; `zbk mirroraudit` opens with "A mirrored artifact is republished verbatim"). This has no caller in the migration.

⛔ EXTENDED copy, not Copy, and the difference is the whole point. oras.Copy walks a manifest and its blobs — it would move the artifact and leave its signature behind, because a signature is not a child of the manifest but a REFERRER to it. An artifact that arrives unverifiable is worse than one that does not arrive: `zpkgs install` verifies by default, so the first would be a silent trust failure at install time and the second is a 404 at copy time.

Digests survive by construction. ExtendedCopy moves content by descriptor, so the manifest digest on the far side is the same string, and the signature payload commits to exactly that (payloadCommitsTo, verify.go:185 — it checks critical.image.docker-manifest-digest and nothing about WHERE the artifact lives). That is why this is a copy and not a re-sign.

It streams: ORAS fetches each blob and pushes it, so a 1 GB artifact does not become 1 GB of resident memory. Nothing here holds a tarball.

The tag is carried over unchanged, deliberately. A copy that renamed a version would make the two namespaces disagree about what is published, and the catalogue is built by listing tags.

func Decompressor

func Decompressor(ext string, body io.Reader) (io.Reader, func(), error)

Decompressor wraps an artifact body in the decoder its extension names, and returns a close function for the decoders that hold resources.

An artifact's compression is not a detail the installer may guess: a reader that assumes gzip and meets zstd fails with "invalid header", which reads like a corrupt download rather than a format it was never taught. Every extension we have ever published is handled here, and an unknown one says so by name.

All three decoders stream, which matters now that the pull stages the tarball on disk: nothing here holds an artifact in memory.

func Describe

func Describe(c Concrete) string

Describe is Distinguish for a single spec with nothing to compare against: the whole of what was recorded, which is what `zpkgs builds` falls back to when a version has exactly one build.

⛔ A SUMMARY, NOT THE CLOSURE. Printing 54 dependencies answers a question nobody asked with a page nobody reads; the count is the honest short form, and the full list has its own command.

func Dir

func Dir() string

Dir resolves the artifact store (ZPKG_DIR, default ~/.zpkg), expanding the platform placeholders described by ExpandStorePath.

⛔⛔ `~/.zpkg`, AND IT IS UPSTREAM'S DIRECTORY WE ARE GETTING OUT OF. pkgx documents its own store as `~/.pkgx/<project>/v<version>/` — upstream's own corpus, read 2026-11-04 — and upstream's `pkgx` can now sit on the same PATH as ours, which is the entire reason the binaries were renamed. Two implementations writing one directory, under two different layouts (ours carries a variant hash; see storepath.go), is the collision the rename existed to remove, left in the one place it does the most damage.

It is also what every command PRINTS. The landing page's transcripts said

1965 project(s) → ~/.pkgx/catalog/linux-x86-64.json

after the rename, and the page was right — that is what the code returned. A product that answers with somebody else's directory has not been renamed, it has been relabelled.

⛔ NOTHING IS MIGRATED, ON PURPOSE. An existing `~/.pkgx` is pkgx's as much as ours and we cannot tell which artifacts in it are whose; moving it would be taking somebody else's files. A user who wants the old store keeps it with ZPKG_DIR=~/.pkgx, which is one line and says what it means.

func Distinguish

func Distinguish(specs []Concrete) []string

Distinguish says what each build IS, in the only terms that help: the axes on which it differs from the others it is listed beside.

⛔⛔ A LIST OF HASHES IS UNUSABLE AND THAT IS NOT A PRESENTATION DETAIL. A hash is opaque by construction; shown two of them, a reader has no way to pick, and the command that printed them has handed over its own problem. Nix pays exactly this price — a store path says WHICH build and never WHAT build — and the answer everywhere else is to keep the spec queryable: `spack find libdwarf +debug`, conda's info/hash_input.json. We publish the spec on the manifest for this reason; this is the function that uses it.

⛔ IT DESCRIBES THE DIFFERENCE, NOT THE BUILD. A full spec is 54 dependencies, identical in 53 of them, and printing it per row buries the one edge that matters in a screen of agreement. What a reader asked is "how do these differ", so that is what is answered — the same reason `diff` exists rather than printing both files.

The result is parallel to the input: one line per spec, "" for a spec that records nothing. A caller must say so rather than printing an empty cell — see the note zpkgs prints for an artifact published before identities existed.

func DocDiff

func DocDiff(a, b any) string

DocDiff is docDiff, exported: a caller comparing two recipes needs to say WHICH key parted, and a second implementation of this walk would be a second opinion about what a recipe means.

func DownloadArtifact

func DownloadArtifact(project, ver, osn, arch string) ([]byte, string, error)

DownloadArtifact fetches the raw compressed artifact tarball for a specific project/version/os/arch, trying .tar.gz then .tar.xz. It returns the bytes and the extension that succeeded (".tar.gz" or ".tar.xz") — for mirror tooling that copies artifacts verbatim without extracting them.

func ELFInterp

func ELFInterp(p string) (string, error)

ELFInterp is the program interpreter an ELF names in its PT_INTERP header — the absolute path the kernel execs to load it.

It is the first of the three things Nix requires of a bootstrapped toolchain: the final environment must not REFERENCE the seed it came from. An artifact whose PT_INTERP is /lib64/ld-linux-x86-64.so.2 runs because the MACHINE has that file, and says nothing about whether the store does. Our own builder README states the same property in prose and measured it once, by hand, on one recipe:

PT_INTERP  /pkgx/gnu.org/glibc/v2.44.0/lib/glibc-2.44/ld-linux-aarch64.so.1

An invariant nothing checks is a hope.

A statically linked file has no PT_INTERP, and that is not an error: it returns "" with a nil error. Only a file that cannot be read as an ELF at all fails.

func ELFNeeded

func ELFNeeded(p string) ([]string, error)

ELFNeeded is what an ELF file loads: its DT_NEEDED entries.

It is the linux counterpart of MachoNeeded, and the two answer differently shaped questions. A Mach-O names a PATH, so "is it satisfied" is a stat. An ELF names a SONAME and the loader searches for it, so the question is whether ANYTHING in the closure provides that name — which is a comparison against the whole store, not against one path.

Both readers live here because an audit that runs on one platform and not the other measures half a fleet. This one is exported for `bk unresolved`; CompleteClosure has used the same reading to repair a closure for a while.

func ELFRunpath

func ELFRunpath(p string) (runpath, rpath []string, err error)

ELFRunpath is where an ELF tells the loader to look: DT_RUNPATH, and the older DT_RPATH, in that order.

Both are returned, and NOT merged, because they do not mean the same thing to the loader: DT_RPATH is consulted before LD_LIBRARY_PATH and DT_RUNPATH after it, and a file carrying both is answering two different questions. A caller auditing for paths that escape the store cares about every entry; one reasoning about resolution order needs to know which list an entry came from, so the two are returned separately rather than concatenated.

Absent entries are not an error: a file with neither returns two nil slices.

func EmitHCL

func EmitHCL(doc map[string]any) string

Emit renders a recipe document as package.hcl text.

It is the inverse of ToMap, and the only thing that makes it trustworthy is that the two are checked against each other: Convert parses what this produces and refuses to hand it back unless the result is the document it started from. A converter nobody checks is a rewrite.

Shape follows this package's own convention. A map whose keys are all HCL identifiers becomes a BLOCK, because `build { script = … }` is what a reader expects; a map with a key like "openssl.org" cannot — an attribute name must be an identifier — so it becomes an object attribute with quoted keys.

func Env

func Env(name string) string

Env resolves a variable for the go-zpkg tools. A real process environment variable that is set AND non-empty wins; otherwise the value from ~/.zpkg/config.hcl2 is returned; otherwise the empty string.

func ExpandStorePath

func ExpandStorePath(p string) string

ExpandStorePath replaces the platform placeholders in a store path: `{platform}` becomes `linux-x86-64`, `{os}` becomes `linux` and `{arch}` becomes `x86-64`, all for the RUNNING machine.

⛔ ONE STORE CANNOT HOLD TWO ARCHITECTURES, and nothing said so. A prefix is `$ZPKG_DIR/<project>/v<version>/…` with no platform anywhere in it. Measured 2026-10-29 on one machine, the same project and version staged twice:

…/openssl.org/v3.6.4/lib/libssl.so.3     ELF 64-bit x86-64
…/openssl.org/v3.6.4/lib/libssl.3.dylib  Mach-O arm64

Those two were kept apart only by being under different ZPKG_DIRs. Here the FILE NAMES differ because the operating systems do; between linux/x86-64 and linux/aarch64 they are identical — `lib/libssl.so.3` both times — so a store shared over NFS holds whichever architecture was installed last, and the other one gets `Exec format error` from the kernel with no explanation.

⛔ WHY THIS MATTERS ON A CLUSTER, specifically. srun's `--export` defaults to ALL — Slurm's own documentation: "All of the user's environment will be loaded from the caller's environment" — so the PATH composed on the head node arrives unchanged on every compute node. A head of one architecture and nodes of another is then two failures at once: the PATH names the head's binaries, and the store they point into cannot contain the node's.

Giving the platform a place IN the path is what Spack does (its prefixes carry platform, os and target) and it costs nothing here: a path with no placeholder is returned unchanged, so every existing store keeps working.

ZPKG_DIR=/shared/pkgx/{platform}     one tree per architecture, one NFS export
ZPKG_DIR=/shared/pkgx/{os}/{arch}    the same, split

⛔ BRACES, NOT `$`: the shell expands `$PLATFORM` before pkgx ever sees it, and a value the shell has already eaten cannot be resolved per node. `{…}` survives quoting and reaches the program, which is the point — the SAME exported string has to mean a different directory on each node.

func Extract

func Extract(tr *tar.Reader, dest string, strip int) error

Extract unpacks every entry of tr into dest, stripping strip leading path components from each entry name (strip == 0 keeps the archive layout). It is the shared archive extractor for this ecosystem — both this package's installer and zpkb's source fetcher route through it, so a single implementation carries the invariants both rely on:

  • Regular files keep the modification time the archive recorded (tar(1) semantics). Recipes DEPEND on this: a release tarball ships generated files NEWER than the sources they derive from so make leaves them alone; stamping every file "now" re-orders them by archive position and make then tries to regenerate — libexpat 2.8.3 dies that way (doc/xmlwf.1 is archived before, hence would be stamped older than, doc/xmlwf.xml → make runs the docbook rule with no docbook2x-man installed → Error 1). A zero recorded time is left as written.
  • Absolute names, and names that escape dest after stripping, are rejected with ErrInsecurePath; a hard-link source is vetted the same way.
  • An entry whose path crosses a symlink an EARLIER entry created is rejected with ErrInsecurePath. A name check is lexical and a write is not: `dest/a/b` is inside dest even when `a` is a link out of it.
  • setuid, setgid and sticky are dropped: an entry gets its permission bits and nothing else.
  • Directories, regular files, symlinks and hard links are reproduced; unsupported entry types (fifos, devices, ...) are skipped.

What it does NOT promise: a symlink ENTRY may point anywhere, including out of dest, which is what tar(1) does and what an artifact's relative links need. Nothing Extract writes afterwards follows such a link — that is the ErrInsecurePath rule above — but a caller that walks the result and resolves symlinks itself can still leave dest, and must say whether it means to.

tar.ErrInsecurePath from tr.Next is tolerated because Extract performs its own stricter vetting on every entry name.

func FetchMeta

func FetchMeta(project string) (deps map[string]string, provides []string, err error)

FetchMeta returns the host-relevant runtime dependencies (project -> constraint) and the list of provided paths for a project's package.yml.

func FetchMetaFor

func FetchMetaFor(project, osn, arch string) (deps map[string]string, provides []string, err error)

FetchMetaFor is FetchMeta for an EXPLICIT pkgx os/arch slug. A recipe's dependencies are platform-keyed (`linux: {...}`, `darwin/aarch64: {...}`), so reading them through the host's slug while staging a rootfs for another platform silently produces the WRONG closure — a linux image missing the deps only linux declares, and carrying the ones only darwin needs.

func FetchRuntimeEnv

func FetchRuntimeEnv(project, prefix, version string) (map[string]string, error)

FetchRuntimeEnv returns a project's runtime env declarations with the recipe placeholders resolved against the version actually installed. A project that declares none yields an empty map, not an error.

func FetchRuntimeEnvIn

func FetchRuntimeEnvIn(project string, closure []Resolved, dir string) (map[string]string, error)

FetchRuntimeEnvIn is FetchRuntimeEnv with the resolved closure in hand, so `deps.<project>.prefix` and the dep version tokens resolve.

libpkgx's useMoustaches tokenises, for every installed dependency:

map.push({ from: `deps.${dep.pkg.project}.prefix`, to: dep.path.string })
map.push(...tokenize.version(dep.pkg.version, `deps.${dep.pkg.project}.version`))

Without them rust-lang.org/cargo declares

CARGO_HTTP_CAINFO: ${{deps.curl.se/ca-certs.prefix}}/ssl/cert.pem

and cargo builds with no CA bundle at all — `[77] Problem with the SSL CA cert … error setting certificate file: /ssl/cert.pem` — which is where the whole crates.io family stopped.

func FindClosureBin

func FindClosureBin(closure []Resolved, dir, project, name string) string

FindClosureBin returns the path to <project>/…/bin/<name> if that project is in the installed closure, else "".

func FindLoader

func FindLoader(dir string) string

FindLoader locates the pkgx glibc dynamic loader in an installed closure.

func FindLoaderFor

func FindLoaderFor(dir, arch string) string

FindLoaderFor is FindLoader for an EXPLICIT architecture: the loader of the rootfs being STAGED, whose ELF name (ld-linux-aarch64.so.1 vs ld-linux-x86-64.so.2) is the target's, not the staging machine's.

func GOARCH

func GOARCH() string

GOARCH returns the pkgx architecture slug (e.g. "x86-64", "aarch64") for the running machine.

func GOOS

func GOOS() string

GOOS returns the pkgx OS slug ("linux", "darwin", or "windows") for the running machine.

func GlibcConstraint

func GlibcConstraint() string

GlibcConstraint is the constraint gnu.org/glibc is resolved with when a closure needs the C library: an exact "=<version>" when a glibc has been pinned (ZPKG_GLIBC) or chosen for this host's kernel, else "*" (newest).

func GlibcFlavor

func GlibcFlavor() string

GlibcFlavor is the glibc build variant this host asks for: the ZPKG_GLIBC pin, or "" for ordinary builds. A pinned host wants BOTH that exact glibc artifact AND, wherever the factory published one, the tool built against it — the `<version>-glibc<ver>` tag.

func GlibcMinKernel

func GlibcMinKernel(path string) (string, error)

GlibcMinKernel reads the minimum supported Linux kernel version baked into a glibc's libc.so.6 (or any glibc-linked ELF) via the GNU `.note.ABI-tag` note, returned as "major.minor.subminor" (e.g. "3.2.0").

glibc records its `--enable-kernel` floor here: the loader+libc refuse to run on an older kernel. So this is exactly the datum a glibc-by-kernel selector needs — pick the newest glibc whose min-kernel <= the host's `uname -r`. Pure Go (debug/elf); no external readelf.

The note layout (ELF spec + GNU ABI tag): namesz(u32) descsz(u32) type(u32), then the name ("GNU\0", padded to 4 bytes), then desc = 4 u32 words [OS, major, minor, subminor] with OS==0 for Linux. type is NT_GNU_ABI_TAG(1).

func HCLToMap

func HCLToMap(src []byte, filename string) (map[string]any, error)

HCLToMap parses package.hcl into the generic document shape a package.yml decodes to.

func HCLToYAML

func HCLToYAML(src []byte, filename string) ([]byte, error)

HCLToYAML turns a package.hcl into the package.yml bytes the rest of this package already knows how to read.

It lives here, and not in bk where the HCL front-end was written, because the CLIENT has to read these files: the pantry overlay is fetched at install time, and a recipe written in HCL is unreadable to a pkgx that cannot parse it. bk imports artifact, so the parser cannot live in bk without a cycle — and two copies of it would drift.

Converting to YAML rather than decoding straight into a recipe keeps ONE parse path: whatever validation and quirk-handling the YAML side has is applied to both formats, and a recipe written either way cannot behave differently.

Cost, measured before it was added: pkgx goes from 8.81 MB to 9.45 MB.

func HostKernel

func HostKernel() (string, error)

HostKernel returns the running Linux kernel release (e.g. "6.8.0-45-generic"), or "" on a non-Linux host.

func HostSlug

func HostSlug() (string, string)

HostSlug returns the pkgx (os, arch) slug for the running machine.

func Install

func Install(r Resolved, pkgxDir string) (bool, error)

Install downloads and extracts one artifact into pkgxDir (skips if the versioned prefix already exists), then writes major/minor convenience links. Extraction is atomic — the artifact is unpacked into a temp dir and the versioned prefix is renamed into place — so concurrent installs sharing one ZPKG_DIR never observe a half-extracted prefix. Install materialises an artifact for the HOST platform. Use InstallFor to stage a rootfs for another one -- building a linux image from a mac, say.

func InstallFor

func InstallFor(r Resolved, pkgxDir, osn, arch string) (bool, error)

InstallFor materialises an artifact for an EXPLICIT platform. The extracted tree is identical whatever the host: only the artifact fetched differs, so a builder can stage a linux/aarch64 rootfs from a darwin/arm64 machine.

func InstalledVersions

func InstalledVersions(project, dir string) []string

InstalledVersions lists the versions of a project present in a store, newest first.

The other direction from PrefixOf: that one COMPOSES a path from a version somebody already resolved, this one ENUMERATES what is on disk. Both now go through storepath.go, which is where the layout lives.

Newest first by VERSION order, not by name: a directory listing is alphabetical, and alphabetically "v1.9" sorts above "v1.10", so the first entry would be the wrong one to put beside a project's name.

An unreadable or absent directory is no versions, not an error. "Nothing installed" is the normal state of almost every project in a catalogue of two thousand, and a browser cannot stop to report it.

func IsELF

func IsELF(p string) bool

IsELF reports whether the file at p starts with the ELF magic.

func IsNotPublished

func IsNotPublished(err error) bool

IsNotPublished reports whether err means this registry simply does not carry the project, as opposed to a transient or authentication failure.

⛔ IT IS ABOUT AN ERROR FROM A REGISTRY CALL. The typed form above is exact; the fallback is repoAbsent, whose last resort reads the error's TEXT, and text written for another purpose can contain "not found". Pass it what came back from the registry, not whatever reached your error printer — or match *NotPublishedError yourself and be sure.

func IsOCI

func IsOCI(base string) bool

IsOCI reports whether a dist base selects the OCI transport.

func IsPlatformTag

func IsPlatformTag(tag string) bool

IsPlatformTag reports whether a tag names a platform manifest rather than a version. Consumers walking a repository's tags need to skip these.

func IsVersionTag

func IsVersionTag(tag string) bool

IsVersionTag reports whether a registry tag names a package VERSION rather than registry bookkeeping.

This matters because ghcr implements no referrers API, so ORAS keeps each artifact's attestations under a fallback tag named `sha256-<digest>`: those tags sit in the same listing as the versions. Parsed as versions they became phantom "0" entries — inflating every count, and (worse) making a listing look non-empty so the upstream-dist fallback never engaged for a project we had published only some versions of.

⛔ EXPORTED BECAUSE EVERY CALLER OF ListTags NEEDS IT AND ONE COULD NOT REACH IT. `pkgs request` (go-zpkg/pkgx) filtered the listing with IsPlatformTag alone, and its first live run reported a project with no releases as "2 version(s) tagged" — the two being `sha256-…` referrers, one of which it then asked for platforms and got "the index has 3 entry(ies) and none of them names a platform". The rule was already written and tested here; what was missing was the ability to use it, and a second copy in the caller is how two answers to one question begin.

func LibDirs

func LibDirs(closure []Resolved, dir string) []string

LibDirs returns every library directory in an installed closure (including glibc's versioned sub-libdir on linux). It is the list form of LibPath, used where an OS-native path separator is required (Windows joins with ";" and resolves DLLs from these dirs via PATH, so a ":"-joined string is wrong).

func LibPath

func LibPath(closure []Resolved, dir string) string

LibPath returns a ":"-joined library path covering every lib dir in an installed closure (including glibc's versioned sub-libdir). It is the value of $LD_LIBRARY_PATH used on linux; on Windows use LibDirs with the native separator instead.

func LinkView

func LinkView(viewDir, storeDir, project string, v Ver, hash string) error

linkStoreAlias makes `<store>/<project>/v<version>` resolve to one build.

⛔⛔ THIS LINK IS PART OF THE PUBLISHED ABI, WHICH IS WHY IT LIVES IN THE STORE AND NOT IN A VIEW. Measured on a real published artifact — `otool -l` on zlib.net 1.3.2 for darwin/aarch64 gives `name @rpath/zlib.net/v1.3.2/lib/libz.1.dylib` — so every binary linking zlib names that exact path. A store without it breaks them at load time.

⛔ RELATIVE TARGET. An artifact tree is relocatable; an absolute link would be the one thing in it that is not, and the whole store is routinely mounted somewhere else (an NFS export, a container rootfs, a staged builder).

⛔ IT DOES NOT OVERWRITE A REAL DIRECTORY. One there is a store written by a release that did not hash paths, and replacing an installed tree with a symlink is not something to do silently — the install is refused instead, naming the path. LinkView materialises the `v<version>` alias for ONE build inside a VIEW — a user's profile, an environment, or a container rootfs being assembled.

⛔⛔ A DEFAULT IS NOT A PROPERTY OF THE STORE, AND AN HPC SITE IS WHERE THAT STOPS BEING A NICETY. The store is a shared NFS export: one copy of each build, read by everyone, writable by few. An alias written INTO it would make one user's choice of default everyone's, and would need write access to a tree that is often mounted read-only. The first version of this wrote the alias into the store, and that is the bug.

So the store holds hashed directories and nothing else — no mutable state, no selection, safe to share and safe to freeze. "Which build do I get when I do not say" is answered by a VIEW, and a view is built for a context:

a user's default      their own state directory
an environment        resolved to exact hashes when it is loaded, so it
                      never consults a default at all
a container/microVM   the links are COMPUTED INTO THE IMAGE at creation
                      time from the shared store, which is why they must
                      not be in the store to begin with

That is Nix's profiles and Spack's views, and the reason both keep the store immutable.

⛔ THE TARGET IS ABSOLUTE HERE, UNLIKE AN IN-STORE LINK. A view normally sits on a different filesystem from the store — a home directory beside an NFS export, an image being staged under a build root — so a relative target would resolve against the view and find nothing. A view is a projection of one particular store; it is not itself relocatable, and pretending otherwise would produce links that are silently dangling.

func LoaderName

func LoaderName() string

LoaderName is the dynamic-loader soname for the current architecture.

func LoaderNameFor

func LoaderNameFor(arch string) string

LoaderNameFor answers LoaderName for an explicit architecture, so a builder can name the loader of the image it is assembling.

An architecture that is not here returns "", and the caller must say so rather than report a missing glibc: the two look identical from the outside and are nothing alike. `bk builder` on linux/s390x said

builder: no s390x loader in the staged glibc — is gnu.org/glibc in the
toolchain?

with gnu.org/glibc plainly in the toolchain and all 42 packages installed.

s390x's loader is NOT named after its architecture, and that is the one thing about this table that cannot be guessed. The two entries above both read `ld-linux-<arch>.so.N`; extending that pattern gives `ld-linux-s390x.so.1`, which does not exist. The real name comes from OUR OWN artefacts, twice over and independently:

  • 362 occurrences of `ld64.so.1 is NEEDED` across the s390x lane's logs, and not one `ld-linux-*.so` (recorded in go-zpkg/packages' overrides/gnu.org-glibc.hcl, which needed the same fact for $LDSO);
  • `readelf -l` on the published s390x libc.so.6 (go-zpkg/bk#263): `/…/gnu.org/glibc/v2.44+brewing/lib/glibc-2.44/ld64.so.1`.

ppc64le, riscv64 and loong64 are deliberately ABSENT. bk is TESTED on them and no artifact is BUILT for them, so there is no artefact here to read a name out of — and a name deduced from a pattern is exactly what this comment exists to warn against. Add one when an artifact for that architecture can be read.

func LockAge

func LockAge(generated string, now time.Time) string

LockAge says how old a lock is, in the words every other age in this ecosystem is said in.

func LockPinned

func LockPinned(d Lock) string

LockPinned is d.Pinned with the default filled in, so every reader resolves "absent" the same way instead of each one deciding.

func MachoNeeded

func MachoNeeded(p string) ([]string, error)

MachoNeeded is what a Mach-O file loads. It is the darwin counterpart of ELF's DT_NEEDED, and it lives here rather than in a build tool because the closure needs it too: bk reads it to rewrite a reference, artifact reads it to find out whether that reference resolves.

A file that is not a Mach-O is not an error worth propagating — a store is mostly text — so the caller gets an empty list and the walk continues. ⛔ A UNIVERSAL ("fat") file is read too, through every one of its slices.

`macho.Open` handles only a thin file, and a universal one made the closure check say it could not read a library at all. Reported from the field on 2026-10-12, installing python-openstackclient on darwin/aarch64:

charset_normalizer/*.so looks like a Mach-O (universal) but cannot be read

which is the warning five lines down in unresolvedRefs — so the file was correctly identified as a Mach-O, correctly flagged as unreadable, and its references were simply unknown. An unknown reference is how a closure gets declared complete while it is not, which is the one failure this file exists to prevent.

The UNION of the slices, not the slice matching this host, and the choice is deliberate — the report suggested selecting the arm64 slice. Two reasons against it:

  • On macOS an x86-64 slice still runs, under Rosetta. "No slice for this architecture" therefore does not mean "will not run here", so picking by host arch can under-report for a binary that executes perfectly well.
  • Under-reporting is the dangerous direction. A universal binary's slices name the same dylib PATHS in practice, since the libraries are universal too, so the union is almost always identical to any single slice — and where it is not, the extra reference is a real thing something loads.

Deduplicated and sorted, because a union across slices has no natural order. The thin path keeps its load order untouched: there it is the file's own.

func MarkStorePlatform

func MarkStorePlatform(dir, osn, arch string)

MarkStorePlatform records, once, which platform a store holds.

⛔ IT NEVER FAILS A CALLER. A read-only $ZPKG_DIR is somebody's deliberate arrangement — a materialised squashfs image on a compute node is exactly that — and refusing to run because a note could not be left would break the case this was written to help. An unwritable marker means the store simply cannot say what it holds, which is where everything was before.

func MarshalCatalog

func MarshalCatalog(c Catalog) ([]byte, error)

MarshalCatalog renders a catalogue deterministically: projects sorted, versions and platforms sorted within each.

Deterministic because the artefact is signed and attested like an artifact, and a catalogue whose bytes moved for no reason would make every rebuild look like a change — the same argument the lock makes for sorting its pins.

func MarshalSpec

func MarshalSpec(c Concrete) string

MarshalSpec renders a concrete spec for the SpecAnnotation.

No error: specJSON is strings, string maps and slices of those, and encoding/json cannot fail on any of them. Returning one anyway would add a branch no caller can reach and no test can exercise — the shape the coverage measurement keeps finding and a reader later mistakes for a real case. TestMarshalSurvivesAwkwardBytes pins the property instead.

func NewHTTPClient

func NewHTTPClient() *http.Client

NewHTTPClient builds an *http.Client that trusts CertPool.

func ParseBuildRef

func ParseBuildRef(spec string) (rest, hash string)

ParseBuildRef splits a spec that may name a build by hash.

fftw.org                  → "fftw.org", ""
fftw.org@3.3.11           → "fftw.org@3.3.11", ""
fftw.org@3.3.11/7k2mq4xv  → "fftw.org@3.3.11", "7k2mq4xv"

⛔ `/<hash>` IS SPACK'S SPELLING AND IT IS BORROWED ON PURPOSE. Its manual: "In place of a spec in any command, substitute /<hash> where <hash> is any amount from the beginning of a spec hash." It is the one form that works without knowing a package's variant vocabulary, which matters most exactly when a user has just been handed two hashes by a refusal and has to pick.

⛔⛔ THE `@version` IS REQUIRED BEFORE THE HASH, AND THAT IS THE WHOLE RULE. A project name contains slashes — a third of the pantry is named that way — so a trailing component cannot be told from a hash by LOOKING at it.

The first attempt tried: split on the last slash when the tail "looks like a hash", justified by the claim that a pantry component would contain a letter outside base32's alphabet or be longer than a hash. That claim is FALSE, and a test refuted it immediately: base32 is a–z plus 2–7, which covers very nearly every lowercase word. `x.org/pciaccess`, `gnome.org/libxml2`, `wpewebkit.org/libwpe` and `github.com/gabime/spdlog` all parsed as hash references.

So the syntax decides, not the shape: a hash may only follow an `@version`, and without one a trailing component is part of the project name. Always, with no exception to get wrong.

func PinTag

func PinTag(uri string) string

PinTag is the registry tag recording WHICH digest a URL served.

SourceTag addresses bytes; this addresses a *source*. The mirror is content-addressed, which is right for storage and useless for the question that matters on a rebuild: "is this the same tarball the last build got?" Nothing in the pkgx recipe format answers it — exactly one recipe of 904 carries a `sha:` at all, and that one points at a `.sha256` served by the same host, so it detects corruption and not substitution.

A URL is hashed rather than escaped because an OCI tag may hold only [A-Za-z0-9_.-], at most 128 characters, and a distributable URL has slashes, colons and percent-escapes and is routinely longer than that.

func PreferNotWithdrawn

func PreferNotWithdrawn(project string, builds []Build) (keep []Build, setAside []Build)

PreferNotWithdrawn sets aside the builds that have been withdrawn, and reports which it set aside.

⛔⛔ IT RETURNS EVERYTHING WHEN EVERYTHING IS WITHDRAWN. That is the case the rank-not-filter rule exists for: a caller asking for the only build there is must get it, with the reason said out loud, rather than an empty list and a refusal it cannot act on.

func PrefixOf

func PrefixOf(project string, closure []Resolved, dir string) string

PrefixOf returns the installed prefix directory for the Resolved whose Project equals project in the closure, or "" if the project is not part of the closure. StorePrefix decides the layout; see storepath.go.

func PrimaryBin

func PrimaryBin(project string, provides []string) string

PrimaryBin picks the binary to run for a project: the one whose name matches the project's path leaf (gnu.org/wget -> wget) or its domain's second-level label (perl.org -> perl, not the first-listed corelist), else the first provided binary.

func PrimaryBinFor

func PrimaryBinFor(project string, provides []string, version string) string

PrimaryBinFor is PrimaryBin with the version in hand — see BinNamesFor.

func ProjectToRequest

func ProjectToRequest(err error) string

ProjectToRequest names the project a caller could ask to have published, for the errors that mean "there is nothing here you can install", and "" for every other error.

⛔ ONE QUESTION, ONE ANSWER. The two states above differ in their data — one has a registry and an HTTP refusal, the other has versions and a platform — but a caller deciding whether to offer `pkgs request` is asking the same thing of both. Two predicates at that call site is how one of them comes to be forgotten, which is exactly what happened to the second state for the length of one release.

func PublishCatalog

func PublishCatalog(c *OCIClient, cat Catalog, osn, arch string) error

PublishCatalog pushes a catalogue for ONE platform, through the path a artifact takes.

Per platform, and that is not an accident of the signature. What is available differs by architecture — the s390x lane has a fraction of what linux/x86-64 has — so a single catalogue would tell most readers that packages are available which, for them, are not. `bk weather` exists because that asymmetry is real.

func PublishedTagFor

func PublishedTagFor(project string, v Ver, osn, arch string) (string, bool, error)

PublishedTagFor reports whether THIS registry carries a version for a platform, and under which tag.

The second half of the catalogue's question, and the reason it is this rather than a bare OCIClient.HasPlatform: one version has several spellings and they do not always hold the same platforms — gnu.org/tar has its linux artifacts under `1.35` and its darwin ones under `1.35.0`. Asking about one spelling answers about one spelling. This asks about the version.

It also goes through the cached client, which matters when the caller is a catalogue build asking about two thousand projects in a row.

func RenderLock

func RenderLock(d Lock) string

RenderLock writes a lock. HCL, because every other file this ecosystem reads by hand is HCL and HCLToMap reads it straight back.

func ResolveBinPath

func ResolveBinPath(bin string) string

ResolveBinPath maps a logical bin path (…/bin/foo) to the file that actually exists on disk. On Windows it prefers "foo.exe" (the PE image the toolchain produces), on a wasm host (wasip1, js) "foo.wasm" (the module a wasip1/wasm artifact ships under bin/), falling back to the bare name; on UNIX it is the identity. This keeps BinNames/PrimaryBin returning logical, extension-free names while the run/stub sites resolve to the real executable.

func ResolveBuild

func ResolveBuild(dir, project string, v Ver) (string, bool)

ResolveBuild finds the installed build a (project, version) names.

⛔⛔ THIS IS WHAT REPLACES COMPUTING `<dir>/<project>/v<version>`. Since every real directory carries a hash, that path no longer exists in the store, and a reader cannot derive it — it has to ASK. The order is:

the version says which    a Ver carrying a variant hash is exact, and
                          nothing else is consulted
exactly one is installed  the ordinary case, and unambiguous
several are installed     AMBIGUOUS. "" with ok=false, and the caller
                          says so rather than picking one

⛔ SEVERAL IS NOT "TAKE THE NEWEST". Two builds of one version are siblings — that is the whole reason they both exist — and there is no order between them to take the first of. A reader that silently picked would give a different answer tomorrow for the same request, which is exactly the defect the hashed layout was introduced to remove.

func SetupScratchRootfs

func SetupScratchRootfs(loader, shellPath string)

SetupScratchRootfs makes the pkgx loader and a shell available at their canonical absolute paths (best-effort). On a FROM-scratch image this lets every artifact ELF — the one we exec AND any child processes it spawns — resolve its PT_INTERP=/lib/ld-linux natively, and (when shellPath is set) lets "#!/bin/sh" wrapper scripts resolve to the pkgx bash we installed. On a normal system these paths already exist, so os.Symlink fails and is ignored (no clobbering).

func SetupScratchRootfsAt

func SetupScratchRootfsAt(root, loaderName, loaderTarget, shellTarget string) error

SetupScratchRootfsAt is SetupScratchRootfs against an arbitrary root, for a rootfs being STAGED rather than the one being run on. A builder assembling a FROM-scratch image on a host cannot write /lib and /bin — those belong to the host — so it hands the staging directory here instead.

loaderName is the loader's ELF name for the TARGET architecture, which is not necessarily the host's: LoaderName() answers for the host, LoaderNameFor() for whatever is being built.

func ShortBuilder

func ShortBuilder(id string) string

ShortBuilder renders a SLSA builder id for a column:

https://github.com/go-zpkg/zbk@v0.64.0  →  zbk@v0.64.0
https://github.com/go-pkgx/bk@v0.26.0   →  bk@v0.26.0

⛔ THE ORGANISATION IS DROPPED AND THE TOOL AND VERSION ARE NOT. A reader scanning the column is asking "which factory, how old"; the host and owner are the same for every row we publish and spend a third of the line. An id that does not look like a URL is returned whole — a builder we do not recognise is exactly the one worth reading in full.

func ShortHash

func ShortHash(identity string) string

ShortHash renders any identity string in the one shape a store path and an OCI tag both use.

⛔ IT LIVES HERE BECAUSE BOTH WRITERS DO. The factory computes a build's identity (go-zpkg/zbk's spec package) and has to put it in a tag; the installer has to find the matching directory. If each shortened in its own way the two would silently address different things, and the symptom would be "the artifact is not published" for an artifact that is.

func SonameComesFromOutside

func SonameComesFromOutside(soname string) bool

SonameComesFromOutside reports whether a soname is one the CLOSURE is not expected to carry.

Two groups, and they are different kinds of outside. The implicit system libraries — glibc, the libstdc++/libgcc pair, the gcc runtime — are supplied by the installer from their own project groups rather than by the soname map, so a closure that has not run that step yet looks short of them without being wrong. The host-provided ones are a boundary rather than a gap: a GPU driver's userspace half is versioned in lockstep with a kernel module and belongs to whoever installed the driver.

An audit that counted either as a defect would report a hole in every closure on the fleet.

func SonameProviders

func SonameProviders() []string

SonameProviders is every project the soname maps can pull into a closure: the bounded set of dependencies a RECIPE never mentions.

A dependency discovered this way exists only in the compiled artefact. perl declares nothing about libcrypt, glibc 2.38+ stopped providing it, and the map above supplies github.com/besser82/libxcrypt when a perl binary turns out to need it. Anything planning a build order from recipes alone is blind to that channel, and finds out one failed build at a time.

Sorted and deduplicated, so a caller can diff it against an order it computed and see what that order cannot have known.

func SonamesUnder

func SonamesUnder(prefixes []string) map[string]bool

SonamesUnder is every shared-library file name the given installed prefixes offer. On linux a DT_NEEDED entry is satisfied by any of them, wherever it sits, because the loader searches a path rather than following one.

func SourceDigest

func SourceDigest(data []byte) string

SourceDigest is the sha256 of data in lowercase hex, the form SourceTag takes and the form an artifact's SBOM records.

func SourceTag

func SourceTag(sha256hex string) string

SourceTag is the registry tag for a source digest. OCI tags cannot contain a colon, so `sha256:abcd…` becomes `sha256-abcd…` — the same substitution the referrers fallback uses.

func SplitFlavor

func SplitFlavor(tag string) (version, glibc string)

SplitFlavor splits a registry tag into the software version and the glibc flavor it was built against; the flavor is "" for an ordinary tag.

"8.20.0"             → "8.20.0", ""
"8.20.0-glibc2.27.0" → "8.20.0", "2.27.0"

func SplitVariantHash

func SplitVariantHash(tag string) (version, hash string)

SplitVariantHash cuts a trailing variant hash off a registry tag.

"3.3.11"                   → "3.3.11", ""
"3.3.11-habcdefghij012345" → "3.3.11", "abcdefghij012345"

A suffix that is not exactly hashLen characters of the hash alphabet is NOT a hash: a version like "1.0-h2" would otherwise read as a truncated one, and that silently changes which artifact a pin means.

func StoreBuildDirName

func StoreBuildDirName(v Ver, hash string) string

StoreBuildDirName is the REAL directory a build occupies: always hashed.

⛔⛔ EVERY INSTALLED TREE CARRIES ITS HASH, WITH NO EXCEPTION. A layout with two shapes — `v3.3.11` here and `v3.3.11-h…` there — is one a reader has to branch on, and the branch is where two builds of one version silently become one. The alias `v3.3.11` still exists, as a SYMLINK, so nothing that computes a path from (project, version) has to change; what it points at names which build it is.

The hash comes from the strongest witness available, in this order:

the tag's variant hash   the concrete spec: platform, declared variants,
                         resolved closure, recipe plan. What `bk` published.
the content digest       the artifact's own manifest digest. Says "these
                         exact bytes" and nothing about why.
the source               neither is available: an unverified HTTP dist,
                         which by construction offers no digest to trust.

⛔ THE THIRD IS NAMED AND NOT HIDDEN. It hashes where the bytes were asked for, not what came back, so it does NOT distinguish two different builds served from one URL. That is the honest limit of an unsigned dist — ZPKG_VERIFY already has to be off to reach it — and giving it the same shape as the others keeps the layout uniform without pretending the guarantee is the same.

func StoreBuildPrefix

func StoreBuildPrefix(dir, project string, v Ver, hash string) string

StoreBuildPrefix is StorePrefix for the real, hashed directory.

func StoreDirName

func StoreDirName(v Ver) string

StoreDirName is the single directory component under the project — the ALIAS when there is no recorded variant, because that is what a reader computes from (project, version) alone.

⛔ THIS IS NOT ALWAYS THE REAL DIRECTORY. Since every install is written to a hashed directory (see StoreBuildDirName), a bare `v<version>` here names a SYMLINK to one. Readers follow it; the installer never writes it as a directory.

func StoreHash

func StoreHash(v Ver, contentDigest, source string) string

StoreHash picks the hash a build's directory carries. See StoreBuildDirName for the order and why.

func StorePlatform

func StorePlatform(dir string) (osn, arch string, ok bool)

StorePlatform reads a store's marker. It reports ok=false when the store does not say, which every store written before this existed does not.

func StorePrefix

func StorePrefix(dir, project string, v Ver) string

StorePrefix returns the directory an artifact occupies in a store.

It is the ONLY place that composes one. Nine inline spellings is nine chances for one of them to forget the variant hash, and a path that forgets it silently addresses somebody else's build. ⛔⛔ IT RESOLVES FIRST, AND THAT IS WHAT KEEPS EVERY CALLER CORRECT. Since every installed directory carries a hash, `<dir>/<project>/v<version>` is no longer a path that exists in a store; six read sites compute it and would all have to learn to ask. They ask through here instead.

When nothing is installed — or when several builds are and the request does not say which — it falls back to the COMPOSED path, which is the same string those callers got before anything was installed. Their "not installed" branch is a failing os.Stat either way, so none of them changes behaviour; what changes is that an installed build is now found under its hash.

func StoreTotals

func StoreTotals(entries []StoreEntry) (bytes int64, versions, projects int, olderBytes int64)

StoreTotals sums a scan: bytes, versions, distinct projects, and the bytes held by versions that are NOT the newest of their project present.

That last number is the one worth printing and the easiest to misread, so its name says what it is: older, not garbage. Whether an older version can go depends on roots — a lock, an environment — that this package cannot see.

func StubBins

func StubBins(closure []Resolved, dir, prefix string) (int, error)

StubBins writes a small env-setting shell stub into <prefix>/bin for every binary in the closure, mirroring the reference pkgm: the stub exports the closure's LD_LIBRARY_PATH and exec's the real artifact binary.

func StubBinsStaged

func StubBinsStaged(closure []Resolved, s Stage) (int, error)

StubBinsStaged is StubBins for a rootfs being assembled elsewhere.

A stub is a shell script holding ABSOLUTE paths — the closure's LD_LIBRARY_PATH and the artifact binary to exec. Written with the staging machine's paths, every one of them is wrong the moment the rootfs boots: they point at /tmp/whatever-the-builder-used, a directory the guest does not have. So the paths baked in come from GuestDir while the existence checks use Dir.

func UnknownStorePlaceholders

func UnknownStorePlaceholders(p string) []string

UnknownStorePlaceholders lists the `{…}` names left in a store path that ExpandStorePath does not know.

⛔ A MISSPELT PLACEHOLDER IS WORSE THAN NO PLACEHOLDER. `{platfrom}` expands to nothing, so the store becomes a directory literally called `{platfrom}` — which works, silently, as a single-architecture store, and the cluster it was meant to fix fails the same way it did before. Dir cannot return an error without changing every caller, so the question is asked separately and the commands report it.

func ValidatePlatformSlug

func ValidatePlatformSlug(s string) error

ValidatePlatformSlug refuses anything that is not an `os/arch` slug.

A platform is written by the factory, not by a recipe, so a bad one means the file was forged rather than that somebody upstream was careless. It is still only dropped, not fatal: the asymmetry that matters is between a KEY and a CAPTION, and a reader who loses one platform line keeps a usable catalogue.

Exactly one tool prints these today — which is precisely the argument people use for leaving a field unchecked until something new prints it.

func ValidateProjectName

func ValidateProjectName(p string) error

ValidateProjectName reports whether a name may be used to build a URL, a registry repository or a store path.

func ValidateSpecHash

func ValidateSpecHash(s string) error

ValidateSpecHash refuses a spec that could not be a digest.

WHY THIS IS LOOSER THAN THE VERSION, DELIBERATELY

A version is a KEY: it picks bytes and names a directory, so it is held to a shape. A spec is only ever COMPARED — `bk lock --check` reports it moved — and never becomes a path, a URL or an argument. The whole risk it carries is that it is printed.

Demanding `sha256:` plus exactly 64 hex digits would add nothing against that risk and would refuse locks this project's own tests write with short placeholder digests, as well as any future algorithm prefix. A guard stricter than its reason is a compatibility break with no security to show for it.

Empty is allowed: a lock from a bk that recorded none is still a lock, and the fields are compared only when both sides have them.

func ValidateVersionString

func ValidateVersionString(v string) error

ValidateVersionString refuses anything that is not shaped like a version.

func VerifyRequired

func VerifyRequired() bool

VerifyRequired reports whether fail-closed signature verification is demanded when installing. It is **on by default** (secure by default): only an explicit ZPKG_VERIFY=0/false/no/off opts out. Anything else — unset, 1/true/yes/on, or an unrecognised value — keeps verification on. The value is read through Env, so it may come from a real environment variable or from ~/.zpkg/config.hcl2.

func VerifySignature

func VerifySignature(tarball, payload []byte, b64sig, pubkey string) error

VerifySignature checks a cosign simple-signing signature over an artifact tarball: the base64 sig must verify over payload against pubkey (defaulting to the pinned SigningPublicKey), and payload must commit to this exact tarball (docker-manifest-digest == sha256(tarball)). Every failure returns an error — callers treat that as fail-closed.

func VerifySignatureDigest

func VerifySignatureDigest(digest string, payload []byte, b64sig, pubkey string) error

VerifySignatureDigest is VerifySignature for a tarball identified by its DIGEST rather than by its bytes. The bytes are only ever hashed here, so a caller that streamed the artifact to disk — computing the digest on the way — can verify the signature without reading the tarball back, let alone holding it in memory.

func WithdrawnNote

func WithdrawnNote(project string, builds []Build) string

WithdrawnNote is the one line a command prints under a list when something in it has been withdrawn, or "" when nothing has.

func YAMLToHCL

func YAMLToHCL(src []byte, name string) ([]byte, error)

YAMLToHCL converts an upstream package.yml into package.hcl text.

This is the whole reason the renderer lives in the client. Upstream's pantry is YAML and will stay YAML; converting it AS IT ARRIVES is what lets everything past this point support one format. Nothing is written to disk: the conversion is a step in fetchRecipe, not a migration of upstream.

It must not change what a recipe MEANS. Two places decide that:

  • a list-form `versions:` carries TEXT. Decoding it into a map coerces `5.0` to the number 5, and the distributable URL interpolates {{version.raw}} — a coerced candidate fetches a tarball that does not exist. The raw scalars are read back off the YAML node, which is the only place that text still exists after a decode.
  • every other number stays a NUMBER. `MACOSX_DEPLOYMENT_TARGET: 11.0` reaches the shell as `11` today; emitting it as the string "11.0" would be more faithful to what its author typed and would change what builds. Fidelity to the author is not the requirement here — fidelity to the build is.

Types

type Attestation

type Attestation struct {
	Payload []byte
	Sig     string
	// KeyID is a HINT and nothing more. It orders which trusted key is tried
	// first; it never restricts which may succeed, so a referrer that names
	// the wrong key id costs a few microseconds and still verifies.
	KeyID string
}

Attestation is one candidate signature found on an artifact: the simple-signing payload, the base64 signature over it, and — when the publisher recorded one — the key id that is claimed to have made it.

type BlobFile

type BlobFile struct {
	io.ReadSeeker
	// Digest is the "sha256:…" of the staged bytes, computed as they were
	// written. It is what the signature is checked against, so the tarball
	// never has to be read back — let alone kept — to verify it.
	Digest string
	// contains filtered or unexported fields
}

BlobFile is a staged blob: on disk where there is a filesystem, in memory where there is not. Close releases it — on disk that means REMOVING it, since the caller owns a temporary and not a cache entry.

func (*BlobFile) Close

func (b *BlobFile) Close() error

Close releases the staged blob.

type Build

type Build struct {
	Version Ver
	// Installed is its path in the store, or "" when it is only published.
	Installed string
	// Published says the registry carries it.
	Published bool
	// Spec is what the build IS, read from its manifest. Zero when the
	// registry could not be asked, when the artifact predates identities, or
	// when it was only mirrored — all three are "nothing records what this
	// was built from", which a caller must say rather than show as blank.
	//
	// ⛔ IT IS WHY THE LIST IS USABLE AT ALL. Two hashes with nothing beside
	// them hand the reader the problem the command was asked to solve; see
	// Distinguish.
	Spec Concrete
	// InstalledAt is when this build's directory appeared in THIS store, or
	// the zero time when it is not installed.
	//
	// ⛔ IT IS NOT A BUILD DATE AND MUST NOT BE SHOWN AS ONE. A published
	// artifact has no recoverable build date — the timestamp is pinned so a
	// re-publish is byte-identical. This is when you got it. See BuildOrder.
	InstalledAt time.Time
	// Builder is what made this build, from the in-toto provenance —
	// `https://github.com/go-zpkg/zbk@v0.64.0`. Empty when nothing records
	// it.
	//
	// ⛔ IT IS NOT IN THE HASH, ON PURPOSE, AND THAT IS WHY IT HAS TO BE
	// SHOWN. A zbk release must not re-address every artifact in the registry
	// — Nix and Spack take the same position about their own builders — so
	// two artifacts built by different factories from identical inputs share
	// an identity. The only way a reader can tell them apart is this field,
	// which means a listing that omits it has thrown away the answer to
	// "with what was this built?".
	Builder string
	// Signatures is who vouches for this build, or nil when the registry was
	// not asked or would not answer.
	//
	// ⛔ NIL IS "NOT ASKED", NOT "NOBODY". The two must stay distinguishable
	// all the way to the screen: a column that printed an empty cell for
	// both would turn a network failure into the claim that a build is
	// unsigned, which is how a user ends up rebuilding something that was
	// fine — or trusting something that was not.
	Signatures *SignatureReport
}

Build is one build of one version: what distinguishes it, and where it is.

func BuildsOf

func BuildsOf(project, version, dir, osn, arch string) ([]Build, error)

BuildsOf lists every build of one version of a project: those installed in dir, those published for osn/arch, and the ones that are both.

⛔ ONE ANSWER FOR TWO SOURCES, BECAUSE A USER ASKING "WHICH BUILDS ARE THERE" DOES NOT MEAN ONE OF THEM. A list of only the installed ones cannot answer "what else could I have"; a list of only the published ones cannot answer "what do I already have". Reporting them separately would make the caller join two lists by hash, which is this function.

⛔ A REGISTRY THAT COULD NOT BE REACHED IS AN ERROR, NOT AN EMPTY LIST. An unobtained answer must not read as "nothing is published" — that is how a user concludes a build does not exist and rebuilds it. The installed half is still returned alongside the error, because it is a true answer to half the question and it needs no network.

func MatchBuild

func MatchBuild(builds []Build, prefix string) (Build, error)

MatchBuild finds the one build whose hash starts with prefix.

⛔ EXACTLY ONE, OR AN ERROR NAMING THE CANDIDATES. A prefix that matches two builds is a question, not a choice, and answering it by picking the first would give a different answer as soon as another build is published. The error lists them so the next attempt can be longer.

func SortBuilds

func SortBuilds(project string, builds []Build, by BuildOrder) []Build

SortBuilds orders builds in place and returns them.

⛔ EVERY KEY FALLS BACK TO THE HASH. Without it two builds that tie on the key keep whatever order they arrived in, which is a map walk — so the same command would print a different list twice and a reader would stop believing either.

func (Build) Hash

func (b Build) Hash() string

Hash is the build's address — the short form that appears in its tag and its directory name.

type BuildOrder

type BuildOrder string

Ordering the builds of one version.

⛔⛔ THERE IS NO BUILD DATE, AND THAT IS A DELIBERATE PROPERTY RATHER THAN A GAP TO FILL. A published artifact's timestamp is PINNED — `bk factory` stamps SOURCE_DATE_EPOCH, or its own fixed epoch when that is unset, never the wall clock — so that re-publishing the same artifact yields byte-identical attestations and the same referrer digests. Every artifact the factory publishes therefore carries the SAME instant, and sorting by it would order a list by a constant while looking like it ordered by time.

Taking the wall clock instead would trade that for an ordering nobody needs: it would make every rebuild a new artefact and break the idempotent re-push. So the date is not recoverable from an artifact, by construction, and a reader asking "which is the newest" has to be answered with something that was actually written down.

⛔ AND ORDER IS A PRESENTATION QUESTION, NOT A RESOLUTION ONE. The resolver still refuses an ambiguous version: two builds of one version are siblings, and nothing here gives it permission to pick. What this orders is a LIST a person reads, and every key below is a recorded fact that the listing names out loud — never an inference dressed up as one.

const (
	// ByStatus is the default: the builds that nothing is wrong with first,
	// the withdrawn ones last, each group by hash so the order is stable.
	//
	// It is the only key derived from a judgement, and the judgement was
	// made by a person and written in the withdrawal list — not by us.
	ByStatus BuildOrder = "status"
	// ByHash is the stable, meaningless order: what a script wants, and what
	// two runs are guaranteed to agree on.
	ByHash BuildOrder = "hash"
	// ByInstalled is most recently installed HERE first, then the ones that
	// are not installed at all.
	//
	// ⛔ IT IS A FACT ABOUT THIS MACHINE AND THE LISTING MUST SAY SO. It is
	// when you got the build, not when it was built, and the two are
	// unrelated: pulling a two-year-old artifact today puts it first.
	ByInstalled BuildOrder = "installed"
	// ByClosure is the order a reader actually means by "newer": a build made
	// against a later toolchain comes first.
	//
	// ⛔⛔ IT IS THE ONE ORDER THE ARTIFACTS THEMSELVES CARRY. There is no build
	// date (see above), but the closure IS recorded — so "built with go.dev
	// 1.27.4" against "built with go.dev 1.27.3" is a comparison of two facts
	// rather than an inference, and it is exactly the comparison a CVE
	// rebuild creates.
	//
	// ⛔ AND IT IS A PARTIAL ORDER, HONESTLY TOTALISED. See closureRank: one
	// build is later than another only when it is at least as new on EVERY
	// shared dependency and strictly newer on one. Two builds that each lead
	// on a different dependency are genuinely incomparable, and saying
	// otherwise would be the invented order this codebase keeps refusing.
	ByClosure BuildOrder = "closure"
)

func ParseBuildOrder

func ParseBuildOrder(s string) (BuildOrder, error)

ParseBuildOrder reads a sort key, naming the alternatives when it cannot.

type Catalog

type Catalog struct {
	// Generated is RFC3339 UTC. A catalogue is a snapshot and says so: a
	// reader deciding whether to refresh wants the age, and a reader
	// reporting "no such project" should be able to say how old its answer
	// is rather than implying the registry was asked just now.
	Generated string           `json:"generated"`
	Projects  []CatalogProject `json:"projects"`
	// Dropped counts version strings refused as unreadable while parsing,
	// and is NOT part of the file: it describes this reading of it.
	//
	// It exists so the dropping is not silent. A guard that quietly
	// removes things leaves a reader comparing a short list against their
	// memory, so `pkgx catalog` prints this count when it is not zero —
	// and on a catalogue our own factory published it should never be
	// anything else.
	Dropped int `json:"-"`
}

A CATALOGUE, because a registry you cannot enumerate cannot be browsed.

THE MEASUREMENT THAT MADE THIS NECESSARY

An anonymous client can ask ghcr what VERSIONS a project has and cannot ask what PROJECTS exist. Measured against ghcr.io/go-zpkg/packages:

GET /v2/_catalog                               403  (no token issued)
GET /v2/go-zpkg/packages/_catalog              403  (no token issued)
GET /v2/go-zpkg/packages/zlib.net/tags/list    200  ["1.3.2", …]

The registry API's catalogue endpoint is not served, so `gnu.org/<TAB>` has nothing to read. Nix, Guix and Spack all answer this from a LOCAL index — a flake/eval cache, a channel checkout, a package repo — and the pkgx-native equivalent of a local index is an artefact in the registry itself.

A CATALOGUE IS AN ORDINARY ARTIFACT

The same move `members` made for sets: rather than a second kind of artefact with its own publishing, signing and mirroring, the catalogue is pushed through the path an artifact takes. It is therefore signed, attested, cached by the pull-through registry and verified on install by everything that already does those things — and a scratch image gets it in ONE pull, with no registry API beyond the one it already uses to fetch packages.

func CatalogFromTarball

func CatalogFromTarball(b []byte) (Catalog, error)

CatalogFromTarball is the other direction.

func FetchCatalog

func FetchCatalog(c *OCIClient, osn, arch string) (Catalog, error)

FetchCatalog pulls the catalogue for one platform, and VERIFIES it.

WHY A LIST OF NAMES NEEDS A SIGNATURE

This was the one path that brought artifact bytes onto a machine without the fail-closed check every other artifact gets — which is precisely what the comment above verifyPulled promised could not happen. It was missed because a catalogue "is only a list of names", and that is the wrong way round: a list of names is what a person then TYPES.

Someone able to serve a forged catalogue — a compromised mirror, or a ZPKG_DIST pointed somewhere by a script — cannot make the install path accept an unsigned artifact, because that path still verifies. What they can do is decide what `<TAB>` offers, and a near-miss name offered at the prompt is the whole of a typosquat. The attack is on the reader, not on the installer, and the signature is the only thing that answers it.

It also matters MORE than for an ordinary artifact now, not less: since the catalogue became a file on disk, it is fetched once and then read by every completion for days. An artifact is verified each time it is pulled; this one is verified once and trusted thereafter.

func UnmarshalCatalog

func UnmarshalCatalog(b []byte) (Catalog, error)

UnmarshalCatalog is the other direction, refusing a document that is not one rather than returning an empty catalogue.

An empty catalogue and an unreadable one must not look alike: the first says the registry is empty, the second says we failed to ask, and a browser that confused them would tell a user their package does not exist.

func (*Catalog) Age

func (c *Catalog) Age(now time.Time) string

Age is how old the catalogue is, in words. A browser that cannot reach the registry should still say what it is showing rather than imply it is now.

func (*Catalog) Children

func (c *Catalog) Children(prefix string) []Node

Children returns what is available directly under a node, sorted.

prefix is "" for the roots, or a node path with or without a trailing slash. Only the NEXT path segment is expanded: `github.com` yields `github.com/besser82`, not `github.com/besser82/libxcrypt`, because a completion that jumped two levels would skip the choice the user is making.

func (*Catalog) Complete

func (c *Catalog) Complete(partial string) []Node

Complete returns the completions for a partially typed project name.

It is Children of the enclosing node, filtered by the partial segment. A node with exactly one child still returns that one rather than jumping to its grandchild: a shell completes to the common prefix and the user decides whether to descend.

func (*Catalog) DepTree

func (c *Catalog) DepTree(project string, depth int) []DepNode

DepTree is the dependency subtree under a project, as a browser shows it.

WHY A SEPARATE SHAPE FROM Children

A node has two kinds of thing under it and they are not the same question. `gnu.org` has `gnu.org/bash` under it because of how it is NAMED; `curl.se` has `openssl.org` under it because of what it NEEDS. Nix, Guix and Spack keep the two apart as well — `guix graph` and `nix-tree` walk the dependency graph, and neither is how you find out what a channel contains.

depth 0 means "no limit". `guix graph --max-depth` exists because a full transitive graph of anything interesting is pages long, and the first level is what a person reads.

func (*Catalog) Dependents

func (c *Catalog) Dependents(project string, depth int) []DepNode

Dependents is the SAME walk in the other direction: who needs this.

WHY THE OTHER DIRECTION IS A DIFFERENT QUESTION

`ls --tree` answers "what does this need", which is what you ask before installing something. "Who needs this" is what you ask before CHANGING something, and no amount of descending answers it. Every comparable tool has it, each with its own emphasis:

  • `spack dependents [-t]`: direct, or transitive with -t.
  • `guix refresh --list-dependent`: what would need REBUILDING, and the manual says plainly that it only approximates that.
  • `nix why-depends A B`: not a list at all but a shortest PATH, which is a third question and is WhyDepends below.

WHAT IT COVERS, AND WHAT IT DOES NOT

The catalogue's `deps` are RUNTIME dependencies, reduced for one platform. So this is the runtime blast radius on THIS platform: who would load different bytes if this project changed.

It is NOT the rebuild set. A build dependency is paid once by the factory and is not in anybody's installed closure, so it is not in the catalogue and cannot be in this answer. Guix's command is about rebuilds and says it approximates them; this one is about closures and is exact for what it covers — saying which is better than implying the other.

func (*Catalog) Lookup

func (c *Catalog) Lookup(project string) (CatalogProject, bool)

Lookup returns one project's entry, and whether the catalogue has it.

func (*Catalog) ProvidersOf

func (c *Catalog) ProvidersOf(command string) []string

ProvidersOf returns every project that puts this COMMAND on PATH, sorted.

The inverse of Provides, and the question people actually arrive with: you know `hugo`, not `gohugo.io`. Reported from the field on 2026-10-12 — `pkgx hugo version` answered

no recipe for hugo in …pantry-overlay… or …pkgxdev/pantry…

while `pkgx search hugo` in the same store already printed `gohugo.io hugo`. The mapping was in the catalogue and only the search path consulted it.

⛔ It returns a LIST, and the singular case is the caller's to insist on. Returning one project would mean picking between two providers of the same command, and there is no basis in this data for that choice: `provides` says what a project puts on PATH, not which project owns the name. A caller that wants to run something must refuse an ambiguous word and say which projects it was between — guessing silently would run the wrong program under the right name, which is the one outcome worse than an error.

An empty command returns nothing rather than everything: it is what an empty argument looks like, and `provides` holds no empty strings.

func (*Catalog) Search

func (c *Catalog) Search(query string) []SearchHit

Search ranks the catalogue against a free-text query, best first.

Ranking is by HOW the thing matched, not by how often: a command called exactly what you typed is what you meant, every time, and no amount of substring frequency elsewhere should outrank it.

0  a command named exactly this        rg      → crates.io/ripgrep
1  a project whose last segment is this jq     → stedolan.github.io/jq
2  a command starting with this        py      → python.org (bin/python3)
3  a project segment containing this   ssl     → openssl.org
4  the full project path containing it  gnu    → gnu.org/bash
5  the summary containing it

Ties break on the project name, so the same query gives the same order every time. A catalogue is a Go map nowhere, but the Projects slice comes off a registry and its order is not ours to rely on.

func (*Catalog) WhyDepends

func (c *Catalog) WhyDepends(from, to string) []string

WhyDepends is the shortest path from `from` to `to` through runtime dependencies, including both ends, or nil when there is none.

Nix's question, and it is not answerable from either tree above: a reader looking at a closure wants to know WHICH link put something there, and a transitive list says it is there without saying why. Shortest because that is the explanation a person can hold — `nix why-depends` makes the same choice.

Breadth-first, so the first path found is a shortest one. Ties are broken by name, so two runs on one catalogue give the same answer; an explanation that changes between runs is one nobody can quote.

type CatalogProject

type CatalogProject struct {
	Project string `json:"project"`
	// Versions newest-first, as a reader wants them. Empty is legal and
	// means "named by the pantry, nothing published" — which is the honest
	// answer for most of a pantry on most architectures, and is different
	// from the project not existing.
	//
	// ⛔ IN PRACTICE THE PUBLISHED CATALOGUE CARRIES EXACTLY ONE, and a
	// reader writing against this field needs to know that before they
	// build on the plural.
	//
	// `bk catalog --versions` writes `[]string{v}` — the newest version
	// THIS platform really carries, probed newest-first and stopped at the
	// first hit, because the newest tag is not published for every
	// architecture and the ordinary case then costs one request per project
	// instead of N. Counted 2026-10-08 on the three catalogues published
	// 2026-10-06: 1430, 1371 and 1369 entries, and **not one** of them has
	// a second version.
	//
	// So this answers "what would I get" and not "what can I choose from".
	// The second question goes to PublishedTagFor against the registry —
	// which is also why a catalogue cannot be used to decide whether some
	// OLDER version is installable here, a mistake this field's plural
	// invites.
	Versions []string `json:"versions,omitempty"`
	// Platforms as `os/arch`, sorted. Also legal empty, for the same reason.
	Platforms []string `json:"platforms,omitempty"`
	Summary   string   `json:"summary,omitempty"`
	// Deps are the project's declared RUNTIME dependencies, reduced for the
	// platform this catalogue is for, sorted.
	//
	// They are here so the dependency tree can be walked with NO NETWORK
	// and no pantry: `pkgx --graph` answers the same question by resolving
	// against the registry, which a scratch image cannot do before it has
	// one, and which nobody can do on a train.
	//
	// RUNTIME only, deliberately. A build dependency is paid once by the
	// factory and is not in anybody's installed closure; showing it under a
	// node would answer a question the person browsing did not ask.
	Deps []string `json:"deps,omitempty"`
	// Provides are the COMMAND names this project puts on PATH, without
	// their `bin/` prefix, sorted.
	//
	// Here because it is the only metadata in this pantry a search can
	// work on. Counted in the PUBLISHED catalogue of 2026-10-06, with
	// `go run ./internal/catstat` in go-zpkg/pkgx, and the two platforms
	// agree: of 1908 projects, 1582 declare `provides` — 4592 command
	// names on linux/aarch64, 4498 on darwin/aarch64 — and SIX carry a
	// summary. `nix search` and `guix search` look in the description,
	// which is the right design for a collection that HAS descriptions;
	// here it would find almost nothing.
	//
	// The figures carry their date and their instrument on purpose: an
	// earlier revision of this comment said "1907 recipes, 1592 provides"
	// with neither, and by the time anyone compared, both numbers had
	// moved and nothing said whether the difference was drift or a defect.
	//
	// And it answers the question people actually arrive with. You know
	// the command, not the package: `rg` is `crates.io/ripgrep`, which no
	// amount of guessing at the name reaches.
	Provides []string `json:"provides,omitempty"`
}

CatalogProject is one project as the catalogue knows it.

type Concrete

type Concrete struct {
	// Platform is "<os>/<arch>", eg. "linux/x86-64".
	//
	// ⛔ IN THE HASH, because reuse across operating systems is otherwise
	// silently wrong. Spack carries an `os_compatible` map and it is
	// DIRECTIONAL — sonoma may reuse monterey's builds and not the reverse —
	// which is only expressible if the identity says which one it is. An
	// identity that omitted the platform would let a darwin artifact answer a
	// linux pin, and the error would surface as a missing symbol.
	Platform string

	// Variants are the build options this artifact WAS BUILT WITH: definite,
	// possibly multi-valued, never a constraint. A boolean is ["true"] or
	// ["false"].
	Variants Variants

	// Deps is the resolved closure: every project this build was made
	// against, at the exact version and hash it was made against. glibc is
	// one of these.
	Deps []Dep

	// Recipe is the digest of the build PLAN the recipe produced — never of
	// the recipe file's bytes.
	//
	// ⛔ CANONICALISED, AND THE DISTINCTION IS THE WHOLE VALUE. Nix hashes the
	// derivation — the instantiated plan — not the expression that produced
	// it, and Spack's package_hash strips comments and docstrings from
	// package.py before hashing. Digesting the file's bytes instead would
	// make a reflowed comment rebuild the world: every dependent's hash
	// changes, every artifact is republished, and nothing about any binary
	// differs. That is not a conservative choice, it is a wrong one — it
	// spends the fleet's whole build capacity on whitespace.
	//
	// So the producer owes this field a digest over the RESOLVED build: the
	// script after substitution, the environment, the flags. bk computes it;
	// artifact only carries it, and states the contract here because a field
	// that merely says "digest" invites the cheap, wrong answer.
	Recipe string

	// Graft, when set, is the hash of the spec this artifact was produced from
	// by rewriting references rather than by building.
	//
	// ⛔⛔ THE FIELD EXISTS NOW BECAUSE IT CANNOT BE ADDED LATER. A full-closure
	// identity means a CVE fix in a leaf changes the hash of every dependent,
	// so shipping one would rebuild and re-download the world. Guix's answer
	// is not to weaken the hash: it patches store references in the already
	// built output — "grafting works by patching files, including binary
	// files, directly" — in "time proportional to the size of the package,
	// usually less than a minute", and the result is a NEW store item while
	// the original stays reachable (`guix build bash --no-grafts`).
	//
	// That only works if a grafted artifact has an identity of its own, which
	// means this field is in the canonical form. Adding it after the first
	// artifact is published would change every hash already issued.
	//
	// Guix's restrictions come with it and are not ours to relax: the
	// replaced and replacing name+version must be the SAME LENGTH (the patch
	// is in-place in binaries), and a shared library's replacement must keep
	// the same SONAME and be binary-compatible. The layer that performs
	// grafts enforces those; this field only records that one happened.
	Graft string
}

A CONCRETE SPEC IS WHAT A ARTIFACT IS.

The question this answers

Two artifacts of the same version, compiled with different options, were indistinguishable: the identity of an artifact was

<project> : <version>[_N][-glibc<X>]  ×  <os>/<arch>

and nothing in it names a build option. `openmpi+cuda` and `openmpi~cuda` could not coexist — the only way to publish both was the `_N` rebuild counter, which is an ORDER ("the newer replaces the older"), not a choice. In an HPC pantry that is the difference between expressing a dependency and hoping for one.

Why a hash, and why over everything

Spack is the precedent, including its false starts: it carried THREE hashes — dag_hash, build_hash and full_hash — and in 2022 collapsed them into one dag_hash covering build, link and run dependencies plus the hash of the package file itself. The project that lived this tension longest tried the split and came back from it.

⛔⛔ A CONCRETE SPEC IS NOT A REQUEST

The first version of this type used one map for both, and that was the design error. Spack keeps them apart and must: `fabrics=verbs,ofi` means AT LEAST those, `fabrics:=verbs,ofi` means EXACTLY those, and `++debug` propagates to dependents. All three describe how to CHOOSE a build. None of them describes a build — once concretised, a node has one definite set of values, and the question "was this asked for exactly or at least?" has no answer here because it was already answered.

So: VariantConstraints are what a user writes (see ParseConstraints), and Variants are what an artifact has. Mixing them would put a question into an identity, and two artifacts reached by different questions would hash apart while being the same bytes.

glibc is not a special case

It used to be one: a `-glibc<X>` tag suffix with its own splitter, its own branch in selectVersions and its own host-derived selector. It is a DEPENDENCY whose version changes the ABI, so it belongs in Deps like any other, and the readable suffix was a second spelling of identity — the exact shape of defect that let numericKey drop a published version three times over (see its comment).

Readability is served by the annotation and the local catalogue, which is where `spack find -l` serves it too. The tag carries the address.

func UnmarshalSpec

func UnmarshalSpec(s string) (Concrete, error)

UnmarshalSpec reads a spec from an annotation value.

⛔⛔ IT IS NOT TRUSTED. This value comes from a registry, which is to say from whoever could push to it, and the canonical form had to be made injective for exactly this reason: a validator guards the door it stands at, and the annotation arrives through another one. Callers that care which artifact they have must use VerifySpec, not this.

func VerifySpec

func VerifySpec(annotation, wantHash string) (Concrete, error)

VerifySpec reads the annotation and checks that it hashes to the hash the TAG claims. It returns the spec only when the two agree.

⛔⛔ A SPEC THAT DOES NOT HASH TO ITS OWN ADDRESS IS NOT DESCRIPTION, IT IS A CLAIM. Both halves are supplied by the publisher: the tag says `-h<hash>`, the annotation says what that hash is of. Reading the second without checking it against the first would let an artifact tagged as the plain build describe itself as the CUDA one, or the reverse — and the reverse is the one that matters, because the lie is in the direction of "this is the build you asked for".

An empty wantHash means the tag made no claim (every artifact published before this existed), and then there is nothing to check against: the spec is returned as the unverified description it is, and a caller deciding anything on it should say so.

func (Concrete) Canonical

func (c Concrete) Canonical() string

Canonical renders the spec as the exact bytes that are hashed.

⛔ EVERY FIELD IS SORTED AND DELIMITED. A canonical form that depended on map order would give one build two identities, and one built by concatenation without a delimiter would give two builds one: {"ab","c"} and {"a","bc"} hash alike the moment the separator is "". Both are tested.

⛔ AND THE INSTALL PREFIX IS NOT IN IT. $ZPKG_DIR is where an artifact lands, not what it is. Nix states the consequence of getting this wrong: "One cannot copy a store object to a store with a different store directory", and a cache stops being shareable the moment identity depends on where the reader keeps it. Our having no fixed prefix is only an advantage while the prefix stays out of here.

func (Concrete) Hash

func (c Concrete) Hash() string

Hash is the spec's address: the first hashLen base32 characters of the SHA-256 of its canonical form.

An EMPTY spec hashes to "": an artifact with no platform, no declared variants, no recorded closure and no recipe digest is one we know nothing about, and giving it an address would claim knowledge we do not have. That is also what makes every tag published before this existed readable — it has no hash because nothing recorded one, which is true.

func (Concrete) IsZero

func (c Concrete) IsZero() bool

IsZero reports whether nothing about this build was recorded.

func (Concrete) VariantTag

func (c Concrete) VariantTag() string

VariantTag is the suffix a concrete spec contributes to a registry tag, or "" when there is nothing to say.

type ConflictError

type ConflictError struct {
	Project     string   // the project nobody could agree on
	Constraints []string // every demand on it, in the order collected
	AskedBy     []string // who made each; "requested" for a root
	Err         error    // what the version pick said
}

ConflictError is a closure that has no solution: several demands on one project whose intersection is empty.

Structured rather than only formatted, because the interesting question is never one closure. Asking how much of a pantry a chosen base reaches means resolving hundreds of them and counting WHICH demand did the excluding — libxml2 ~2.13 shutting out 68 recipes is a decision about what to rebuild, and a caller should not have to parse a sentence to learn it.

func (*ConflictError) Error

func (e *ConflictError) Error() string

func (*ConflictError) Unwrap

func (e *ConflictError) Unwrap() error

type Dep

type Dep struct {
	Project string
	Version string
	Hash    string
}

Dep is one resolved edge of a closure: a project at a version, and the hash of the concrete spec that version was itself built from.

The hash is part of the edge, not a lookup: depending on "openssl.org 3.6.4" is ambiguous once openssl itself has variants, and an identity built from the ambiguous half would collide exactly where it matters.

type DepNode

type DepNode struct {
	Project string
	Version string
	// Known is false for a dependency the catalogue does not list. It
	// happens, and saying so is the point: a name with nothing behind it is
	// a hole in the catalogue or a project that resolves from somewhere
	// else, and a browser that printed it like any other node would hide
	// both.
	Known  bool
	Repeat bool // already shown higher up; not descended into
	Under  []DepNode
}

DepNode is one dependency, and what hangs under it.

type Edge

type Edge struct {
	Of         string
	On         string
	Constraint string
}

Edge is one dependency: Of depends On, under Constraint. An Of of "" is a project the operator asked for directly.

type Graph

type Graph struct {
	Roots    []string       // what was asked for, sorted
	Versions map[string]Ver // project -> the version the unifier picked
	// Lines holds every version of a project the closure carries, and is set
	// ONLY where there is more than one: demands that do not intersect but
	// name disjoint sonames are two ABI lines, not a conflict. Versions keeps
	// the leading one, so a reader that does not know about lines still gets
	// an answer rather than an arbitrary half.
	Lines map[string][]Ver
	Deps  map[string][]Edge // project -> the edges leaving it, sorted by On
	Asks  map[string][]Edge // project -> the edges pointing AT it
}

Graph is a resolved closure with its edges kept, so a caller can show WHY a version was chosen and not only WHICH.

func GraphFor

func GraphFor(roots map[string]string, osn, arch string) (*Graph, error)

GraphFor resolves a closure and returns it with its structure intact.

Same walk, same unifier and same determinism as ResolveClosureFor: this is a view of the resolution, not a second implementation of it, so what it shows is what a build or an environment will get.

type Lock

type Lock struct {
	Version   int    // lockfile_version
	Platform  string // the platform it was taken on: it CHANGES the answer
	Generated string // RFC3339, UTC
	ZBK       string // which zbk wrote it
	Roots     []string
	Pantry    string
	Overlay   string
	// Pinned says which question the versions answer: PinsFromRecipes (the
	// newest the recipes can BUILD) or PinsPublished (the newest the factory
	// has PUBLISHED for this platform). They are different sets — measured
	// 2026-10-08, 2 of 5 pins for curl.se on linux/aarch64 — and only the
	// second can be installed by `pkgx --lock`.
	//
	// It is written down rather than inferred because `bk lock --check`
	// re-resolves, and re-resolving by the wrong question reports every
	// pin as having moved. The file answers its own roots and platform for
	// that same reason; the mode is one more of them.
	//
	// Empty means PinsFromRecipes: that is what every lock written before
	// this field existed holds.
	Pinned string
	Pins   []LockPin
}

Lock is a whole lock: what was asked for, what it resolved to, and the facts a later reader needs in order to decide whether the answer still holds.

THE HEADER IS DATA, NOT A COMMENT

The first version of this wrote the platform, the roots and the two revisions as `#` lines. They were the right facts in the wrong place: a comment cannot be read back, so a check would have had to re-derive them from arguments the caller might give differently, and would then be checking a different question from the one the file answers.

Spack's lockfile is the precedent for every field. Its `_meta` carries a `lockfile-version`; it records the Spack version "to track information that should enhance reproducibility"; and its top level holds `roots` beside `concrete_specs`.

func ParseLock

func ParseLock(src []byte, path string) (Lock, error)

ParseLock parses a lock from bytes. path is used only in error messages, so a caller that already has the content — a test, or a tool reading from a pipe — does not have to put it on disk first.

Through HCLToMap, the same reader the pantry overlay's HCL recipes go through, so the lock cannot develop a dialect of its own.

func ReadLock

func ReadLock(path string) (Lock, error)

ReadLock parses a lock from a file.

type LockPin

type LockPin struct{ Project, Version, Spec string }

LockPin is one project as a lock pins it.

Spec is a Merkle hash over the platform, the name, the resolved version, the PARSED recipe and the dependencies' spec hashes — Spack's shape. A version alone calls two builds the same when a build script changed under them.

type NoArtifactHereError

type NoArtifactHereError struct {
	Project     string
	OS, Arch    string
	Constraints []string // what was asked for; empty means "any version"
	Available   []Ver    // every version the pantry knows, oldest first
	Absent      []string // of those, the ones with no artifact here
}

NoArtifactHereError says the project IS published — just not anything this platform can install.

WHY A SECOND TYPE, AND WHY IT IS THE COMMON ONE

⛔ NotPublishedError ALONE COVERED THE RARE HALF. It is returned when the repository does not exist at all, which is what a brand-new package looks like. The ordinary state is the other one: measured 2026-10-23 against the published darwin/aarch64 catalogue, 567 of 1940 projects had a recipe and no artifact — 29% of the pantry, against which `pkgx <project>` answered

no version of aerc-mail.org is published for darwin/aarch64
(available: 0.22.0; not published here: 0.22.0)

A good sentence and a dead end: it names no next step, and the hint that exists to name one could not fire, because this error was a sentence and not a type. Found by RUNNING the shipped binary over a list derived from the catalogue — the unit test passed and the mutation was caught, and neither could see that the covered case was the rare one.

The fields are the message's own parts, so a caller that wants to say something else does not have to parse the sentence back apart.

func (*NoArtifactHereError) Error

func (e *NoArtifactHereError) Error() string

Error keeps the two sentences the field reports settled on, verbatim.

⛔ NO CONSTRAINT MEANS THE CLAUSE IS OMITTED, not printed empty: reported 2026-10-12, when `pkgx hugo version` produced "satisfies AND is published for", two spaces where the reason should be, blaming a constraint that did not exist.

type Node

type Node struct {
	// Name is the FULL project path of a leaf, or the path of the interior
	// node including its trailing segment — never a bare segment, so a
	// caller can complete straight onto it without reassembling anything.
	Name string
	// Leaf is true when Name is itself a published project.
	//
	// Leaf and Interior are NOT exclusive, and that is not a corner case:
	// `curl.se` is a project and `curl.se/ca-certs` lives under it. A tree
	// that made them exclusive would hide one of the two.
	Leaf bool
	// Under is how many projects sit strictly below Name. Zero for a pure
	// leaf.
	Under int
}

Node is one entry under a tree node, as `pkgx ls` and tab-completion show it.

type NotPublishedError

type NotPublishedError struct {
	Project  string
	Registry string
	Repo     string
	Err      error
}

NotPublishedError says that THIS registry does not carry a project, and says which project, so a caller can act rather than parse.

WHY A TYPE AND NOT A SENTENCE

⛔ BECAUSE THE PROSE TEST IS TOO WIDE ONCE IT LEAVES THIS PACKAGE. repoAbsent's last resort matches `not found` anywhere in an error's text, which is safe where it was written — it only ever sees the result of a registry call — and wrong as soon as an exported predicate is pointed at arbitrary errors. Measured 2026-10-23, the first use of the exported IsNotPublished at pkgx's top-level error printer:

pkgx: command not found in the requested packages: true
pkgx: if it should be here: `pkgs request <project>` …

The package was published; the COMMAND was not there. The hint fired on the word "not found" in an unrelated sentence.

A type cannot be matched by accident, and it carries the project — so a caller can name it instead of printing a placeholder and hoping the reader fills it in from the line above.

func (*NotPublishedError) Error

func (e *NotPublishedError) Error() string

func (*NotPublishedError) Unwrap

func (e *NotPublishedError) Unwrap() error

type OCIClient

type OCIClient struct {
	// contains filtered or unexported fields
}

OCIClient talks to an OCI distribution (registry v2) endpoint via ORAS. It is used by the artifact pull path and by the mirror push tool, so the OCI protocol lives in one place (ORAS).

func NewOCIClient

func NewOCIClient(base string) (*OCIClient, error)

NewOCIClient parses an oci:// dist base into a client. Localhost/loopback hosts default to plain HTTP (for a local zot/registry on a port); every other host uses HTTPS. Credentials are read from the environment: OCI_TOKEN (a pre-issued bearer), else OCI_USERNAME / OCI_PASSWORD for token exchange / basic auth. Public repositories work anonymously.

func (*OCIClient) Annotations

func (c *OCIClient) Annotations(project, ver, osn, arch string) (map[string]string, error)

Annotations returns the OCI annotations of a published artifact's per-platform manifest — how a client reads an artifact's self-description (a glibc's minimum kernel, a tool's glibc flavor) without downloading the artifact itself.

func (*OCIClient) ArtifactDigest

func (c *OCIClient) ArtifactDigest(project, ver, osn, arch string) (string, error)

ArtifactDigest is the digest of the tarball published under one tag — the thing a signature commits to, read from the per-platform manifest without downloading a byte of the artifact.

⛔ IT IS WHAT A REBUILDER COMPARES AGAINST. "Did I produce the same artifact?" has to be answerable before deciding whether to sign, and answerable cheaply enough that asking is the normal thing to do.

func (*OCIClient) AttachReferrer

func (c *OCIClient) AttachReferrer(project, ver, osn, arch string, rf Referrer) error

AttachReferrer adds one attestation to an ALREADY PUBLISHED artifact, leaving the artifact itself untouched.

⛔⛔ A SECOND BUILDER MUST NOT HAVE TO REPUBLISH THE ARTIFACT TO VOUCH FOR IT. Every other path here pushes a tarball and its attestations together, which is right for the builder who made it and wrong for everyone else: re-pushing is how you end up with two objects where the point was that there is one. Independent agreement means several signatures on the SAME bytes, so attaching has to be an operation of its own.

func (*OCIClient) EnsureIndexed

func (c *OCIClient) EnsureIndexed(project, ver string, desc ocispec.Descriptor) (repaired bool, err error)

EnsureIndexed puts desc back into the version tag's index when a racing publisher dropped it, and reports whether it had to.

The per-push reconcile in mergePlatformIntoIndex confirms its write twice with a settle window between, which catches a racer landing just behind it. It cannot catch one that lands later still — after this publisher has moved on to the next package, or exited. This is the pass for that: run it once the whole batch is done, when the other publishers have finished too and a repair sticks.

Re-merging (rather than re-writing) is what keeps the repair safe: it starts from the index as it now stands, so a platform some other publisher added in the meantime survives.

func (*OCIClient) FetchAttestations

func (c *OCIClient) FetchAttestations(project, ver, osn, arch string) ([]Referrer, map[string]string, error)

FetchAttestations returns a published artifact's attestations, ready to be pushed to another registry with PushWithReferrersAnnotated, along with the per-platform manifest's annotations (a glibc artifact's minimum kernel, a flavored build's glibc version) so those survive the copy too.

func (*OCIClient) HasPlatform

func (c *OCIClient) HasPlatform(project, ver, osn, arch string) (bool, error)

HasPlatform reports whether (project, ver) is published for os/arch in the registry: true if a per-platform manifest resolves, false if the tag or the platform is absent (a not-yet-published signal), and an error only on a real transport/parse failure. The build factory uses it to skip already-published (project,version,platform) tuples — a pure-Go replacement for the curl+jq ghcr manifest probe.

func (*OCIClient) HasSource

func (c *OCIClient) HasSource(project, sha256hex string) (bool, error)

HasSource reports whether the store already holds a digest, without fetching it — what a populate step asks before uploading megabytes for nothing.

func (*OCIClient) IndexLists

func (c *OCIClient) IndexLists(project, ver string, desc ocispec.Descriptor) (bool, error)

IndexLists reports whether the version tag's index carries desc.

An artifact can be pushed, valid and signed — and absent from the index, because publishing the index is a read-modify-write on one mutable tag that several publishers race on. Nothing about the artifact itself says so; only the index does, which is why this takes the descriptor the push returned rather than trying to discover it. Discovery would go through the very index in question.

func (*OCIClient) ListTags

func (c *OCIClient) ListTags(project string) ([]string, error)

ListTags returns the tags (versions) published for a project, or an error if the repository does not exist.

func (*OCIClient) PinnedDigest

func (c *OCIClient) PinnedDigest(project, uri string) (string, error)

PinnedDigest returns the sha256 the store last recorded for a URL, or ErrSourceAbsent when it has never seen it.

This is the read half of trust-on-first-use. The first fetch of a URL records what it served; every later fetch compares. It does not make an upstream trustworthy — it makes a CHANGE visible, which is the part nobody had.

It reads the digest off the manifest's layer rather than parsing it out of the tag, so a pin can never claim a digest the stored bytes do not have.

func (*OCIClient) PlatformAnnotations

func (c *OCIClient) PlatformAnnotations(project, ver, osn, arch string) (map[string]string, error)

PlatformAnnotations returns the annotations on the per-platform image manifest — the map a publish attaches, e.g. org.go-zpkg.abi.provides.

Separate from PullFile on purpose: the cache may serve the artifact, and a mirror's manifest is not the authority on what the artifact DECLARES. This always asks the dist the caller named.

func (*OCIClient) Platforms

func (c *OCIClient) Platforms(project, ver string) ([]string, error)

Platforms lists the platforms one version tag publishes, in pkgx's spelling (`linux/x86-64`), sorted and deduplicated.

WHY IT EXISTS

A caller could already ask "is THIS platform published" one guess at a time — HasPlatform does that — and could not ask which ones are. ⛔ Measured 2026-10-20: `go-zpkg/packages`' daily CVE lane carried its platform list in the workflow file, the list said two, and the registry published SIX. The four nobody asked about were clean, so the cost was coverage rather than an exposure, but the mechanism guarantees the next platform is missed too (go-zpkg/pkgx#119).

⛔ AN EMPTY LIST IS AN ERROR, NEVER AN ANSWER. fetchOrNewIndex, the other reader of this document, turns every failure into an empty index on purpose — it is composing a push and "nothing published yet" is its normal case. For a LISTING that behaviour is the defect: "this version publishes no platform" and "I could not read the index" would print the same nothing, and this fleet has published that zero before.

func (*OCIClient) ProvenanceOfTag

func (c *OCIClient) ProvenanceOfTag(project, tag, osn, arch string) (SignatureReport, string, error)

ProvenanceOfTag answers both halves of "can I trust this build?" in ONE pass over the artifact's attestations: who signed it, and what built it.

⛔⛔ THE TWO ARE ONE QUESTION AND MUST COST ONE LISTING. A signature says these bytes are somebody's; it says nothing about what produced them. The builder — `https://github.com/go-zpkg/zbk@v0.64.0`, recorded by the factory in the in-toto statement's runDetails.builder.id — is the other half, and it is NOT in the build's identity: a zbk release must not re-address every artifact in the registry. So the only way to learn it is to read the attestation, and a caller that had to ask twice would pay two referrer listings for one screenful.

The builder is "" when the artifact carries no provenance, or one this cannot parse. Empty means NOT RECORDED, never "built by nobody", and a caller that prints it must keep that distinction — the registry is still full of artifacts published before any of this existed.

func (*OCIClient) Pull

func (c *OCIClient) Pull(project, ver, osn, arch string) ([]byte, string, error)

Pull downloads the artifact tarball for a project/version/os/arch and returns it in MEMORY. Prefer PullFile for anything artifact-sized: llvm.org is ~1.7 GiB and this makes the biggest package in the catalogue the memory floor of the caller. Kept for callers that genuinely want the bytes.

func (*OCIClient) PullFile

func (c *OCIClient) PullFile(project, ver, osn, arch string) (*BlobFile, string, error)

PullFile downloads the artifact tarball for a project/version/os/arch: fetch the version-tag manifest (an index, or a single-platform image manifest), pick the platform-matching image manifest, take its single non-config layer, stream the layer blob TO DISK. Returns the staged file (Close removes it) and the extension (".tar.gz"/".tar.xz") derived from the layer mediaType.

func (*OCIClient) PullSource

func (c *OCIClient) PullSource(project, sha256hex string) ([]byte, error)

PullSource fetches the archive stored for a sha256 digest, and VERIFIES that what came back hashes to it.

The verification is not ceremony. A mirror exists so a rebuild need not trust the upstream host; trusting the mirror instead would move the problem rather than solve it. The digest is checked against the one the caller asked for, so a wrong or tampered blob is an error rather than a different build.

func (*OCIClient) Push

func (c *OCIClient) Push(project, ver, osn, arch string, tarball []byte, ext string) error

Push publishes one artifact for a project/version/os/arch: it pushes the tarball as a layer blob, packs a per-platform image manifest (ORAS PackManifest, which also pushes a scratch config blob), then merges the platform into the version-tag image index (fetching any existing index first) and tags it. ext selects the layer mediaType (".tar.gz" or ".tar.xz").

func (*OCIClient) PushSource

func (c *OCIClient) PushSource(project string, data []byte, uri string) (ocispec.Descriptor, error)

PushSource stores one source archive under project:sha256-<digest>.

Idempotent by construction: the tag IS the content, so re-pushing the same bytes writes the same manifest. uri is recorded as an annotation for whoever later asks where this came from.

func (*OCIClient) PushWithReferrers

func (c *OCIClient) PushWithReferrers(project, ver, osn, arch string, tarball []byte, ext string, refs []Referrer) (ocispec.Descriptor, error)

PushWithReferrers is Push plus a set of attestations attached to the pushed per-platform manifest as OCI referrers. It returns that manifest's descriptor so callers can list/verify the referrers. Referrers are pushed before the version index is tagged, so a reader that resolves the index always finds a manifest whose attestations are already present.

func (*OCIClient) PushWithReferrersAnnotated

func (c *OCIClient) PushWithReferrersAnnotated(project, ver, osn, arch string, tarball []byte, ext string, refs []Referrer, annotations map[string]string) (ocispec.Descriptor, error)

PushWithReferrersAnnotated is PushWithReferrers plus annotations merged onto the per-platform image manifest — used to carry descriptive metadata such as org.go-zpkg.glibc.version and org.go-zpkg.glibc.min-kernel so glibc-flavored artifacts are self-describing.

func (*OCIClient) Referrers

func (c *OCIClient) Referrers(project string, subject ocispec.Descriptor) ([]ocispec.Descriptor, error)

Referrers lists the attestations attached to a manifest (its OCI referrers), each descriptor carrying the referrer's ArtifactType.

func (*OCIClient) SignaturesOfDigest

func (c *OCIClient) SignaturesOfDigest(project, ver, osn, arch, digest string) (SignatureReport, error)

SignaturesOfDigest reports who signed an artifact. Its error is reserved for not being able to ASK — a registry that would not answer, or a subject carrying more signatures than this will look at. The verdict is the report's: see SignatureReport.Err.

⛔⛔ EVERY SIGNATURE REFERRER IS COLLECTED, NOT THE FIRST ONE. Taking the first was a defect the moment a second signer existed, and a quiet one: the registry decides the order of a referrers listing, so the same artifact would verify or fail depending on which signature it happened to serve first, and the failure would read as tampering. It also made the threshold unimplementable — "both builders signed this" and "one did" were the same answer.

func (*OCIClient) SignaturesOfTag

func (c *OCIClient) SignaturesOfTag(project, tag, osn, arch string) (SignatureReport, error)

SignaturesOfTag reports who signed the artifact published under one tag, WITHOUT downloading the tarball: the digest the signature commits to is the per-platform manifest's layer digest, which the manifest already carries.

It is what a listing uses — "who vouches for this build?" is a question you must be able to ask about a build you have not installed.

func (*OCIClient) SpecOf

func (c *OCIClient) SpecOf(project, tag, osn, arch string) (Concrete, bool, error)

SpecOf reads a published artifact's concrete spec from its manifest and verifies it against the hash in its tag.

An artifact published before specs existed carries no annotation; that is reported as an absent spec (ok=false) rather than an error, because "we do not know what options this was built with" is the truth about every artifact in the registry today.

func (*OCIClient) VerifyArtifact

func (c *OCIClient) VerifyArtifact(project, ver, osn, arch string, tarball []byte) error

VerifyArtifact checks the cosign-style signatures attached to a pulled artifact as OCI referrers against the configured trust set. It is fail-closed: no signature referrer, malformed ones, or too few that verify (or that do not commit to this tarball) all return an error.

func (*OCIClient) VerifyArtifactDigest

func (c *OCIClient) VerifyArtifactDigest(project, ver, osn, arch, digest string) error

VerifyArtifactDigest is VerifyArtifact for an artifact identified by its DIGEST: the form an install uses, having streamed the tarball to disk and hashed it on the way past.

type Referrer

type Referrer struct {
	ArtifactType string
	MediaType    string
	Blob         []byte
	// Annotations are merged into the referrer manifest — used to carry a cosign
	// signature (dev.cosignproject.cosign/signature) alongside its payload blob.
	Annotations map[string]string
}

Referrer is an attestation to attach to an artifact's per-platform manifest as an OCI referrer (subject-linked artifact): a CycloneDX/SPDX SBOM, an in-toto SLSA provenance statement, or (later) a cosign signature. ArtifactType classifies the referrer manifest; MediaType is the layer blob's media type (usually the same); Blob is the attestation bytes.

type Resolved

type Resolved struct {
	Project string
	Version Ver
}

Resolved is a project pinned to a concrete version.

func CompleteClosure

func CompleteClosure(roots map[string]string, dir string) ([]Resolved, error)

CompleteClosure resolves a package's declared closure, installs it, then iteratively reads the artifacts' ELF DT_NEEDED and pulls whatever artifact provides each unsatisfied soname — the implicit glibc/gcc system libraries and the common undeclared libraries (libz, libbz2, ncurses…) that package.yml graphs omit — until every NEEDED soname is satisfied. This makes the closure complete enough to run on a `FROM scratch` image.

func CompleteClosureFor

func CompleteClosureFor(roots map[string]string, dir, osn, arch string) ([]Resolved, error)

CompleteClosureFor is CompleteClosure for an EXPLICIT pkgx os/arch slug: the version pick, the dependency read, every install and the choice of binary format all happen in the TARGET platform's terms.

WHY IT HAD TO EXIST

⛔ AN AUDIT THAT CAN ONLY ASK ABOUT ITS OWN MACHINE COVERS ONE PLATFORM OF SIX. `bk unresolved` reads every reference in an installed closure and reports the ones the store cannot answer; wired into CI as sonames.yml it found 967 unresolvable references over 259 of the 1407 packaged projects on linux/x86-64 (2026-10-24). The other five platforms had never been asked, because this function resolved and installed for the host and nothing else could be said honestly.

That is the shape vulnscan.yml spent two months in: a coverage gap living in a comment. ResolveClosureFor and InstallFor were already platform-explicit — this is the piece between them that was not.

⛔ CompleteClosure IS NOW THIS FUNCTION AT HostSlug(), so the host path is identical BY CONSTRUCTION rather than by a second reading of the same logic. The existing closure tests passing unchanged is the evidence, including the ones that swap the `goos` seam: HostSlug reads it, so a test pretending to be Windows still reaches the Windows path.

func MergeClosures

func MergeClosures(a, b []Resolved) []Resolved

MergeClosures appends the packages of b not already in a (dedup by project).

func ResolveClosure

func ResolveClosure(roots map[string]string) ([]Resolved, error)

ResolveClosure walks the runtime dependency graph breadth-first.

func ResolveClosureFor

func ResolveClosureFor(roots map[string]string, osn, arch string) ([]Resolved, error)

ResolveClosureFor is ResolveClosure for an EXPLICIT pkgx os/arch slug: the whole walk — version pick and dependency read alike — happens in the target platform's terms. This is what lets a darwin machine stage a complete linux/aarch64 userland, which is how the sovereign builder image is built.

type RevokedKey

type RevokedKey struct {
	// Name is how it is spoken of in the verdict.
	Name string
	// Key is the minisign public key.
	Key string
	// Because is why — "CI secret exposed in run 42067525642", an advisory
	// identifier, a date. ⛔ A REVOCATION WITHOUT A REASON IS A RUMOUR. The
	// person reading "this artifact is signed by a revoked key" has to decide
	// what to do about an artefact they may already be running, and "we
	// revoked it" tells them nothing about whether that matters.
	Because string
}

RevokedKey is a key whose signatures are no longer to be believed.

type SearchHit

type SearchHit struct {
	Project string
	// Why names the field that matched, for a reader deciding whether a
	// result is the one they meant: "command", "name" or "summary". An
	// unexplained list of 40 projects is a list nobody reads to the end.
	Why string
	// Match is the matching value — the command name for a command hit, so
	// `rg` appears beside `crates.io/ripgrep` and the reader sees WHY.
	Match string
	// contains filtered or unexported fields
}

SearchHit is one result, with the reason it matched.

type Service

type Service struct {
	// Name is the block's label: the unit is <Name>@.service, and it is the
	// default account and the name of the state, runtime and configuration
	// directories.
	Name string
	// Description is the unit's Description= and the account's GECOS.
	Description string
	// Documentation is the unit's Documentation=, or "".
	Documentation string
	// Command is the program to run: the base name of one of the recipe's
	// `provides`, found in the installed version's bin/.
	Command string
	// Args follow Command on the command line.
	Args []string
	// User is the static account the service runs as (systemd-sysusers). It
	// defaults to Name.
	User string
	// Capabilities are the only capabilities the service holds, e.g.
	// CAP_NET_BIND_SERVICE for a port below 1024. Usually none.
	Capabilities []string
	// Restart is systemd's Restart=: "on-failure" (the default) or "always".
	Restart string
	// Reload is what `systemctl reload` does: "none" (refused, the default) or
	// "hup" (SIGHUP to the main process).
	Reload string
	// ProcSubset is "pid" (the default: /proc shows only processes) or "all",
	// for a program that reads /proc/sys — Go reads net.core.somaxconn there,
	// and without it every listener's backlog falls from 4096 to 128.
	ProcSubset string
	// StopTimeout is TimeoutStopSec=, or "" for systemd's default.
	StopTimeout string
	// StateDirectory, RuntimeDirectory and ConfigurationDirectory are the
	// versionless directories systemd creates for the service under
	// /var/lib, /run and /etc, named after it. Nil means none.
	StateDirectory         *ServiceDirectory
	RuntimeDirectory       *ServiceDirectory
	ConfigurationDirectory *ServiceDirectory
}

Service is a recipe's `service "<name>" { … }` block: what a supervisor needs to know to run one of the package's programs as a daemon.

It is DATA, not a unit file. The installer (pkgm) renders the systemd unit from it, so the hardening every service shares is written once, in the renderer, and a recipe states only what VARIES between daemons. The fields are the ones that varied between the three daemons this was derived from — authnd, authn-bridge and authn-revokd, whose hand-written units were checked with `systemd-analyze security` — and nothing else: a field nobody needed is a field nobody tested.

service "authn-bridge" {
  description   = "authn-bridge, an OpenID Connect provider in front of a SAML federation"
  documentation = "https://github.com/go-authn/bridge/blob/main/docs/install.md"
  command       = "authn-bridge"
  args          = ["--config", "/etc/authn-bridge"]
  restart       = "on-failure"
  stop-timeout  = "20s"
  state-directory { mode = "0700" }
  runtime-directory { mode = "0700" }
  configuration-directory { mode = "0750" }
}

WHERE IT LIVES, AND WHY OLD READERS DO NOT BREAK ON IT

In package.hcl, beside the rest of the recipe. That was decided by measurement against artifact v0.43.0, the reader every installed pkgm and pkgx carries: HCLToMap is a generic reader, not a schema-checked decode, so a block it has never heard of is read as one more map key and ignored by every consumer — FetchMeta returns the same deps and provides with the block as without it, and `bk lint` (top-level additionalProperties: true) passes.

It DROPS THE LABEL, though, and that has one consequence a recipe must respect: two `service` blocks are two values for the key "service", which that reader refuses as a duplicate key — and the whole recipe with it, so every install of the project fails for every old client. Hence ONE service per recipe, refused here too so the rule holds before a client sees it.

It is read from the HCL source, not from the merged document recipeDoc builds, because that document is HCLToMap's and has already lost the name.

func FetchService

func FetchService(project string) (*Service, error)

FetchService returns a project's service block, or (nil, nil) when its recipe declares none.

It reads the package.hcl of the overlay first and of the base pantry second, following recipeDoc's rule that a key the overlay states replaces that key and a key it omits is inherited. A package.yml is never consulted: YAML has no labelled block, so it cannot spell a service.

func ParseService

func ParseService(src []byte, filename string) (*Service, error)

ParseService reads the `service` block of a package.hcl. A recipe with no such block returns (nil, nil): most packages are not daemons.

type ServiceDirectory

type ServiceDirectory struct {
	// Mode is its octal mode, e.g. "0700".
	Mode string
}

ServiceDirectory is one of the directories systemd makes for a service.

type SignatureReport

type SignatureReport struct {
	// Signers are the distinct trusted keys that vouch for this artifact.
	Signers []string
	// Untrusted counts signatures made by keys outside the trust set.
	//
	// ⛔ THEY ARE NOT AN ERROR. Anybody may attach a signature to a public
	// artefact; if a stranger's signature invalidated an artifact, invalidating
	// any artifact would cost one push.
	Untrusted int
	// Malformed counts signature referrers that could not be read at all.
	// Counted rather than skipped: "unsigned" and "signed, but the referrer
	// is broken" are different facts, and the second names a publishing bug.
	Malformed int
	// Revoked are OUR OWN keys, found on this artifact, that are no longer
	// believed.
	//
	// ⛔⛔ THIS IS NOT "Untrusted WITH A NAME". A stranger's signature is
	// noise anybody can make and is correctly ignored. Our own revoked key
	// on an artifact means either that we published it before the key was
	// compromised — and cannot prove which — or that whoever took the key
	// published it. A reader has to act on that, and they can only act on
	// it if the verdict says it in those words.
	Revoked []RevokedKey
	// Threshold is how many Signers were required.
	Threshold int
}

SignatureReport is what a registry had to say about who signed one artifact: the trusted keys that did, and everything that did not count, kept apart so a caller can tell "nobody signed this" from "we could not read what was attached".

func (SignatureReport) Err

func (r SignatureReport) Err() error

Err is the verdict: nil when the threshold is met, else why not.

func (SignatureReport) OK

func (r SignatureReport) OK() bool

OK reports whether the threshold was met.

func (SignatureReport) RevokedNote

func (r SignatureReport) RevokedNote() string

RevokedNote is the line a command prints when an artifact carries one of our own revoked keys, or "" when it does not.

⛔ IT IS SEPARATE FROM THE VERDICT ON PURPOSE. An artifact can meet the threshold on its current signature AND still carry a revoked one — that is what a re-signed artifact looks like before the old referrer is removed, and what a planted signature looks like too. The verdict says whether to install; this says what else is on the artefact.

func (SignatureReport) Signed

func (r SignatureReport) Signed() bool

Signed reports whether anything at all was attached.

func (SignatureReport) String

func (r SignatureReport) String() string

String is the one cell a listing prints: who vouches for this build.

type Signer

type Signer struct {
	// Name is how this key is spoken of in output. It comes from the
	// operator's configuration — NEVER from the registry.
	//
	// ⛔⛔ A NAME TAKEN FROM THE ARTEFACT WOULD BE THE ATTACKER'S NAME. The
	// whole value of the line "signed by: go-zpkg, alice" is that both
	// words were written by the person reading it. The referrer carries a
	// key id, which is checkable, and no name, which would not be.
	Name string
	// Key is the minisign-format public key.
	Key string
	// contains filtered or unexported fields
}

Signer is one key that may vouch for a build.

func ParseSigner

func ParseSigner(entry string) (Signer, error)

ParseSigner reads one `name:KEY` entry, or a bare `KEY`.

A bare key is named by its own key id: an unnamed key still has to be printable, and printing it as "" would make "signed by: , alice" out of a perfectly good trust set.

func ParseSigners

func ParseSigners(list string) ([]Signer, error)

ParseSigners reads a whitespace-separated list of entries.

func (Signer) KeyID

func (s Signer) KeyID() string

KeyID is the key's 8-byte minisign identifier, hex. It is what a signature referrer names, and what two entries for the same key share.

func (Signer) String

func (s Signer) String() string

String is "name (keyid)".

type Stage

type Stage struct {
	// Dir is where the artifacts are on the machine writing the stubs. Files are
	// checked for existence here.
	Dir string
	// GuestDir is the path those same artifacts will have where the stubs run —
	// "/pkgx" for an image staged under /tmp/rootfs. Empty means Dir: the
	// staging machine IS the running machine (pkgm's case).
	GuestDir string
	// Prefix is where <prefix>/bin receives the stubs, on the staging machine.
	Prefix string
	// OS and Arch are the pkgx slug of the target. Empty means the host's —
	// they select which platform's `provides:` list names the binaries.
	OS, Arch string
}

Stage describes where a closure's artifacts LIVE while stubs are written and where they will live when the stubs RUN. The two differ whenever a rootfs is assembled somewhere other than the machine that will boot it — the whole point of staging.

type StoreEntry

type StoreEntry struct {
	Project string
	Version string
	Bytes   int64
	// Newest is true for the highest version of its project PRESENT here.
	// Not "the newest that exists" — this says nothing about the registry.
	Newest bool
}

StoreEntry is one version directory and what it occupies.

func LiveFromLocks

func LiveFromLocks(entries []StoreEntry, locks []Lock) (live, dead []StoreEntry)

LIVE AND DEAD, against roots the caller names.

WHY A LOCK IS THE ROOT, AND WHY THAT MAKES THIS EXACT

guix's rule is the one to copy: "any file under /gnu/store reachable from a root is considered live and cannot be deleted; any other file is considered dead". The roots are explicit — symlinks under /var/guix/gcroots, the user's profiles — and `guix gc --list-live` reports before `guix gc` removes.

There is no profile here, which is why ScanStore reports and nothing deletes. But a LOCK is a root set already: it pins every project in the closure, not only the roots the person typed, which is the whole reason `pkgx --lock` exists. So reachability needs no graph walk and no network — membership in the lock IS the closure.

An ENVIRONMENT would not do: it names constraints, which have to be resolved against the registry, and that is a network call this cannot make. Saying which of the two is usable here is better than offering both and failing on one.

IT STILL DELETES NOTHING, AND THE WORDING SAYS WHOSE ROOTS

"Dead" here means "not reachable from the roots you named" — not "garbage". Name a different lock and a different half of the store is dead. The caller holds that judgement, and the report has to hand it back rather than imply the store has an opinion.

func ScanStore

func ScanStore(dir string) ([]StoreEntry, error)

ScanStore reports every version directory under an artifact store.

Symlinks are counted NEVER: writeVersionLinks puts `v*`, `v22` and `v22.1` beside `v22.1.8`, all pointing at it, and following them would report llvm.org four times and the store at several times its size. The store on this machine carries 118 such aliases.

type TrustSet

type TrustSet struct {
	Signers   []Signer
	Threshold int
	// Revoked are keys that were ours and are not believed any more.
	//
	// ⛔ THEY ARE NOT SIMPLY ABSENT FROM Signers. Absent means "we do not
	// know this key", which is a stranger and is noise. Present-and-revoked
	// means "this was ours and somebody else may hold it", which is a fact
	// about our own supply chain that a reader must be told in those words.
	Revoked []RevokedKey
}

TrustSet is the whole policy: who may vouch, how many must, and whose word is no longer accepted.

func DefaultTrustSet

func DefaultTrustSet() TrustSet

DefaultTrustSet is the project's CURRENT key plus every key it has retired, threshold 1.

⛔⛔ THE RETIRED ONES ARE IN IT, AND THAT IS WHAT MAKES ROTATION POSSIBLE AT ALL. Rotating is how you answer a lost or compromised key; if rotating also invalidated every artifact the old key signed, the registry would go dark the day it was most needed and nobody would ever rotate. A signature is a statement about bytes; changing the key we sign WITH says nothing about what is already signed.

⛔ THEY ARE NOT RANKED AND NOT DATED. Verify counts DISTINCT keys, so a artifact signed by the current key and an old one counts as two — which is exactly right during an overlap, where the same publisher deliberately signs with both. A scheme that discounted retired keys would make an overlap artifact look like one signer, and a threshold of two unmeetable precisely while rotating.

It reads the variables at call time rather than at init, because SigningPublicKey is the seam tests swap to sign with a throwaway key.

⛔ AN UNPARSEABLE RETIRED ENTRY IS SKIPPED, NOT FATAL. It is a compiled-in typo in a list that grows by hand, and refusing to build any trust set over it would take out verification entirely — including for the current key, which is fine. The current key is the one whose failure returns an empty set, because without it there is nothing to trust.

func LoadTrustSet

func LoadTrustSet() (TrustSet, error)

LoadTrustSet builds the trust set from the environment (or ~/.zpkg/config.hcl2, via Env).

⛔ A THRESHOLD NOBODY COULD MEET IS REFUSED HERE, NOT AT INSTALL TIME. Three signatures required of a one-key trust set is a configuration that fails every install with a message about the artifact. Said at load time it is a message about the configuration, which is where the mistake is.

func (TrustSet) Count

func (t TrustSet) Count(digest string, atts []Attestation) (names []string, untrusted int, err error)

Count names the DISTINCT trusted keys that signed this digest, and says how many signatures were made by keys outside the trust set.

⛔⛔ IT COUNTS, IT DOES NOT JUDGE, AND THE SPLIT IS THE POINT. "Who vouches for this build?" is a question a listing asks about builds it has no intention of installing, and it has a true answer — possibly "nobody" — that is not a failure. Folding the threshold in here would have made every listing of an under-signed build an error, which is how a column ends up printing blanks that read as "unsigned". The verdict is SignatureReport.Err.

⛔ ONE SIGNATURE PER KEY (TUF §5.4.2). Without that rule a threshold of two is met by attaching the same signature twice, which anyone who can push a referrer can do.

Cost: with a key id on each referrer this is one signature verification per signature; without, it is one per (signature, trusted key) pair. Both are bounded by MaxSignatures × MaxTrustedKeys.

Its errors — an unusable trust set, or more signatures than it will look at — name the finding and NOT the artifact, because only the caller knows which artifact was being checked.

func (TrustSet) Names

func (t TrustSet) Names() []string

Names lists the trusted signers' names, in configured order.

type VariantConstraint

type VariantConstraint struct {
	// Values the variant must take. For a boolean, ["true"] or ["false"].
	Values []string
	// Exact distinguishes Spack's two spellings: `fabrics=verbs,ofi` asks for
	// AT LEAST those values, `fabrics:=verbs,ofi` for EXACTLY those. Without
	// the distinction "give me a build with ofi" and "give me a build with
	// ofi and nothing else" are the same request, and one of them gets the
	// wrong artifact.
	Exact bool
	// Propagate carries the constraint down to dependencies — Spack's `++`,
	// `~~` and `==`. It is resolved during concretisation, which is why no
	// such field exists on Variants: by then every node holds the value it
	// was given, and how it came to hold it is not part of what it IS.
	Propagate bool
}

VariantConstraint is what a USER asks for, which is strictly more than what an artifact has.

type VariantConstraints

type VariantConstraints map[string]VariantConstraint

VariantConstraints is a parsed variant request.

func ParseConstraints

func ParseConstraints(s string) (VariantConstraints, error)

ParseConstraints reads the spec spelling of a variant request:

+cuda  ~mpi  ++debug  ~~debug  fabrics=verbs,ofi  fabrics:=verbs,ofi

⛔ `~` IS ALSO OUR TILDE RANGE OPERATOR, and that collision is the reason this parser exists rather than a Split. pkgx spells a version range `~5.44` (verDelims is "@^~<>="), while Spack spells a disabled variant `~cuda`. They are told apart by what FOLLOWS the tilde: a digit begins a version, a letter begins a variant name. `-cuda` is not an option at all — project names contain hyphens (llvm.org/mingw-w64), so it cannot be a boundary.

func (VariantConstraints) Satisfies

func (cs VariantConstraints) Satisfies(have Variants) bool

Satisfies reports whether a built artifact's variants answer this request.

⛔ THE TWO SPELLINGS MEAN DIFFERENT THINGS, which is the whole reason VariantConstraint exists: `=` is a subset test and `:=` an equality. A resolver that treated them alike would hand "built with ofi and verbs" to someone who asked for ofi ALONE — an artifact linked against a fabric library their machine does not have.

type Variants

type Variants map[string][]string

Variants are the definite build options of a built artifact: name to its values, sorted. Multi-valued because Spack's are — `languages=c,c++` is one variant with two values, not two variants — and a single-string map would have had to invent an escape for the comma.

func (Variants) Bool

func (v Variants) Bool(name string) (val, ok bool)

Bool reads a variant as a boolean, reporting whether it was a single boolean value at all.

func (Variants) String

func (v Variants) String() string

String renders variants the way a person writes them, which is the way `pkgx ls` and the catalogue show them: `+cuda ~mpi fabrics=ofi,verbs`.

This is a RENDERING, never an identity. Hash is the identity, and the two must not be confused: a reader that compared these strings would call two builds equal whenever their deps differ and their variants do not.

type Ver

type Ver struct {
	Raw  string
	Nums []int
	Tag  string
	// Rebuild is the N in `1.8.2_N`: the SAME upstream software, built again
	// — for a compiler security release, say, where nothing about the recipe
	// or the version changes and the binary does.
	//
	// Absent means 0, which is Debian's rule for its own revision ("the
	// absence of a debian_revision is equivalent to a debian_revision of 0")
	// and, here, the literal state of the registry: measured 2026-10-10, 0
	// of 1371 published version strings carry one.
	Rebuild int
	// Variant is the concrete spec's hash from the tag's `-h<hash>` suffix:
	// WHICH BUILD of this version, when several were published.
	//
	// It is not ordered and never will be. A rebuild counter says "newer than";
	// a variant hash says "different from", and two builds of one version are
	// siblings rather than a sequence. That is why cmpVer ignores it and
	// numericKey does not: ordering by it would invent a preference the
	// registry never expressed, and dropping it from the identity would
	// collapse two artifacts into one — the defect numericKey already carries a
	// comment about.
	Variant string
}

Ver is a parsed pkgx version: its numeric components plus the raw string.

Tag is the registry tag the version was found under, when that differs from Raw — a glibc-flavored build lives at `<version>-glibc<ver>` while remaining version <version>. Everything version-shaped (comparison, constraints, the installed v<version> prefix) uses Raw; only the registry pull uses the tag.

func FetchVersions

func FetchVersions(project string) ([]Ver, error)

FetchVersions returns the available versions of a project for the host os/arch, ascending.

func InstalledBuilds

func InstalledBuilds(project, dir string) []Ver

InstalledBuilds lists the BUILDS of a project present in a store, newest first, each carrying the variant hash its directory names (empty when the directory does not record one).

InstalledVersions answers "which releases do I have"; this answers "which artifacts do I have", and the two stopped being the same question the moment a path could hold more than one build of a version.

Builds of ONE version keep the order their directories were read in, which is alphabetical by hash: they are siblings, not a sequence, and cmpVer ignores the hash for exactly that reason (see Ver.Variant). Any other order here would invent a preference between them.

func ParseStoreDirName

func ParseStoreDirName(name string) (Ver, bool)

ParseStoreDirName is the other direction: what a directory in the store says about the artifact inside it.

It reports false for anything that is not a version directory, which includes the `v1` / `v1.2` / `v*` aliases writeVersionLinks leaves beside the real ones. A digit after the `v` is what separates a version from a word, and the aliases' second character is not one.

⛔ THE HASH IS CUT BEFORE ParseVer SEES THE NAME. Without that, a variant directory parses as a version whose Raw is `3.3.11-habcdefghij012345` — a string that matches no constraint, compares below every real 3.3.11, and names a release that does not exist. The order here is the correctness.

func ParseVer

func ParseVer(s string) Ver

ParseVer parses a pkgx version string into a Ver, stopping each component at the first non-numeric character (e.g. "1w" in openssl 1.1.1w).

func PickVersion

func PickVersion(project, constraint string) (Ver, error)

PickVersion returns the highest available version satisfying constraint, among those published for the HOST platform.

func PickVersionFor

func PickVersionFor(project, constraint, osn, arch string) (Ver, error)

PickVersionFor is PickVersion for an EXPLICIT pkgx os/arch slug. A project's published version sets differ per platform — a linux/aarch64 artifact can exist where the darwin one does not — so a host that stages a rootfs for another platform must resolve against that platform's catalogue, not its own.

func PickVersionForAll

func PickVersionForAll(project string, constraints []string, osn, arch string) (Ver, error)

PickVersionForAll is PickVersionFor for SEVERAL constraints that must all hold — the case a dependency closure creates when two packages ask for the same project.

Taking one constraint was not a simplification, it was a silent choice. The closure walk kept the FIRST constraint it reached and discarded every later one, so `gnome.org/glib` asking for `gnu.org/gettext: ^0.21` and `freedesktop.org/fontconfig` asking for `^1` never met: whichever was dequeued first won, and the other package could not load. Worse, the queue was seeded from a Go map, so the winner changed between runs — the same closure resolved gettext to 0.26 eight times and 1.0.0 four times out of twelve.

Among the versions that satisfy, one on an end-of-life line (see DeprecatedEnv) ranks LAST: it is chosen only when nothing else satisfies, and the fall back is announced on stderr. With no list the behaviour is exactly the newest satisfier.

func PickVersionForAllWarn

func PickVersionForAllWarn(project string, constraints []string, osn, arch string) (Ver, string, error)

PickVersionForAllWarn is PickVersionForAll returning the deprecation warning as data instead of printing it: "" when the pick is not on an end-of-life line.

func VersionsFor

func VersionsFor(project, osn, arch string) ([]Ver, error)

VersionsFor returns the available versions of a project for an explicit pkgx os/arch slug (e.g. "linux"/"aarch64"), ascending. Used by mirror tooling that spans arches other than the host's.

When DistBase is an OCI registry (the factory default), version listing comes from that registry's tags. That registry is a growing subset of the upstream catalogue, so build-time deps (e.g. llvm.org, perl.org) may not be published there yet and list zero versions. In that case we fall back to the upstream pkgx dist so dep resolution matches the version `pkgx +<pkg>` installs at build time — the actual artifact *pull* still goes through DistBase unchanged, so this only affects version listing, not the install source.

func VersionsForSourced

func VersionsForSourced(project, osn, arch string) ([]Ver, bool, error)

VersionsForSourced is VersionsFor plus WHERE the list came from: true when it is this registry's tag listing, false when it is the upstream dist it falls back to.

A resolver does not care — either list resolves a constraint, and the install still pulls from DistBase. A CATALOGUE does, and this exists because it was about to get it wrong: `bk catalog --versions` would have written UPSTREAM's versions beside a project this registry has never published, and `pkgx ls` would have offered a version that cannot be installed. "Available" is a claim about one registry, so a list from a different one is not a weaker answer to the question — it is an answer to another question.

Pair it with PublishedTagFor to go from "this registry has the tag" to "this registry has it FOR YOU": a tag listing spans every platform, so a mirror wave that lands one architecture publishes a tag the other cannot use.

func VersionsFromTags

func VersionsFromTags(tags []string) []Ver

VersionsFromTags turns a registry tag listing into the versions a client on THIS machine would consider, **oldest first** — so the last is the newest.

WHY A CALLER MUST NOT DO THIS ITSELF

⛔ MEASURED 2026-10-24, in pkgx's `pkgs request`. It filtered a tag listing with IsVersionTag and ordered it with `sort.Strings`, and reported:

nodejs.org
state     published for darwin/aarch64 at 26.8.1

an hour after 26.12.1 was published. Version strings do not sort as text: "26.8.1" > "26.12.1" because "8" > "1". Every fact under that line — which platforms carry it, whether this one does — was then read off the index of the wrong version.

Ordering is only one of the three rules this applies, and the other two are no more guessable:

  • a FLAVOURED tag ("2.44-glibc2.27.0") is a build against some other glibc and is never this client's, so it is dropped unless it matches GlibcFlavor();
  • "1.0.0", "v1.0.0" and "1.0" are ONE version, and our own registry carries several of those spellings — gnu.org/glibc has a 2.44 tag from a factory build and a 2.44.0 tag from the upstream mirror. Left as separate candidates the pick depends on the registry's listing order, and a build once installed both glibc trees side by side and linked against a sysroot its loader had not come from.

This is the same function `ociVersionsFor` has always used; it was simply unexported, which is the third time in two days that a rule this package owns and tests was reimplemented worse one import away.

func (Ver) RegistryTag

func (v Ver) RegistryTag() string

RegistryTag is the tag to pull this version from.

⛔ IT IS NOT ALWAYS Raw. A flavoured artifact's version is "2.44" and its tag is "2.44-glibc2.27.0", so a caller that took `Raw` and asked the registry for it would be told the platform is not in the index — a sentence that reads as "we do not publish that for you" and means "I asked for a tag that does not exist".

func (Ver) Satisfies

func (v Ver) Satisfies(constraint string) bool

Satisfies reports whether the version meets a pkgx constraint ("^1.2", "~2", ">=1.0", "=1.2.3", "*", or "").

type Withdrawal

type Withdrawal struct {
	Version string `json:"version"`
	// Hash is the build's identity. Empty withdraws EVERY build of the
	// version, which is the right shape when the flaw is in the source
	// rather than in how it was compiled.
	Hash string `json:"hash,omitempty"`
	// Because is why — an advisory identifier, e.g. "CVE-2026-1234".
	Because string `json:"because"`
	// SupersededBy is the hash of the build that fixes it, when there is one.
	SupersededBy string `json:"superseded_by,omitempty"`
}

Withdrawal is one recorded reason not to choose a build.

func WithdrawalOf

func WithdrawalOf(project, version, hash string) (Withdrawal, bool)

WithdrawalOf reports whether a build has been withdrawn, and why.

⛔ A WITHDRAWAL WITH NO HASH APPLIES TO EVERY BUILD OF THE VERSION, and an exact one wins over it: a flaw in the source condemns them all, and a later entry naming one build says something more specific about that build. A reader that took the first match would report "the source is bad" for a build whose own entry says it is the FIXED one.

func (Withdrawal) String

func (w Withdrawal) String() string

String renders a withdrawal the way a line of output wants it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL