bottle

package module
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 31, 2026 License: BSD-3-Clause Imports: 41 Imported by: 0

README

bottle

pkg.go.dev Go License

The pure-Go pkgx bottle client — the shared backend of the pure-Go pkgx family. It resolves a package's dependency closure from the pkgx pantry, downloads the bottles from the configured PKGX_DIST (default: the signed oci://ghcr.io/go-pkgx/packages registry; set https://dist.pkgx.dev for the full unsigned upstream pantry), completes the implicit libc/gcc closure a FROM scratch image needs, and execs through the pkgx glibc loader. CGO_ENABLED=0, no runtime dependencies of its own.

Consumed by:

so both tools share one source of truth for the bottle protocol.

Highlights

  • Reads package.yml + versions.txt from the pantry, BFS the runtime dependency closure, streams gzip/xz + tar into PKGX_DIR.
  • Soname-exact FROM-scratch completion: walks each bottle's ELF DT_NEEDED and pulls the provider version that ships the exact soname an ABI needs (not merely the latest), so drifted sonames resolve.
  • Embedded Mozilla CA bundle (net/http with no system trust store), pure-Go DNS. Cross-compiles to six 64-bit targets.

BSD-3-Clause.

Documentation

Overview

Package bottle is the reusable pkgx "bottle backend": it resolves a package's runtime dependency closure from the pkgx pantry, downloads the bottles from the configured PKGX_DIST (default: the signed oci://ghcr.io/go-pkgx/packages registry; PKGX_DIST=https://dist.pkgx.dev for the unsigned upstream), and installs them — with no runtime dependencies of its own (pure Go, CGO_ENABLED=0, runnable on a `FROM scratch` image). It is imported by the pkgm CLI and by sibling tools so there is one source of truth.

Index

Constants

View Source 
const (
	MediaBottleLayerGz = "application/vnd.pkgx.bottle.layer.v1.tar+gzip"
	MediaBottleLayerXz = "application/vnd.pkgx.bottle.layer.v1.tar+xz"
	// +zstd is a media type the OCI image-spec itself defines (unlike, say,
	// brotli, which has none and would make every generic tool — skopeo, oras,
	// registry UIs — treat the layer as opaque).
	MediaBottleLayerZst = "application/vnd.pkgx.bottle.layer.v1.tar+zstd"
	// ArtifactTypeBottle marks a pkgx-bottle image manifest (its manifest.config
	// carries this via ORAS PackManifest v1.1's artifactType).
	ArtifactTypeBottle = "application/vnd.pkgx.bottle"
)

OCI / pkgx media types.

View Source 
const (
	// The bottle tarball extensions; ext also selects the OCI layer media type
	// on push.
	//
	// Measured on real bottle payloads (25 MiB coreutils, 68 MiB llvm/lib/clang),
	// same machine, pure-Go codecs:
	//
	//	codec       ratio        compress   decompress
	//	gzip -6     3.50x/4.45x  1.4s       0.2s
	//	xz          6.85x/7.97x  3.5s       0.9s
	//	zstd -19    7.40x/8.89x  1.1s       ~0.0s
	//
	// zstd beats xz on all three axes, and gzip is the worst of both worlds: the
	// poorest ratio AND a slower decompression than zstd. Every install pays the
	// decompression; the factory pays the compression once.
	ExtTarGz  = ".tar.gz"
	ExtTarXz  = ".tar.xz"
	ExtTarZst = ".tar.zst"

	// GlibcProject is the pkgx project that provides the C library + loader; it
	// is the implicit from-scratch root on linux and the bottle whose manifest
	// carries the glibc min-kernel annotation.
	GlibcProject = "gnu.org/glibc"
)

Shared string symbols for the pkgx bottle ecosystem — named once here so bottle, bk and pkgm reference constants instead of scattering magic literals. (The OCI media types live near their use in oci.go as MediaBottleLayer*, and the manifest annotation keys in verify.go as *Annotation; those are already named constants — this file gathers the remaining recurring literals.)

View Source 
const (
	// ArtifactTypeSignature marks a cosign signature referrer manifest.
	ArtifactTypeSignature = "application/vnd.dev.cosign.artifact.sig.v1+json"
	// MediaSimpleSigning is the media type of the simple-signing payload blob.
	MediaSimpleSigning = "application/vnd.dev.cosign.simplesigning.v1+json"
	// CosignSignatureAnnotation carries the base64 signature on the referrer
	// manifest (cosign's convention).
	CosignSignatureAnnotation = "dev.cosignproject.cosign/signature"

	// GlibcVersionAnnotation / GlibcMinKernelAnnotation describe a glibc-flavored
	// bottle on its per-platform manifest: the exact glibc it was built against,
	// and (for the glibc bottle itself) glibc's min supported kernel from
	// .note.ABI-tag. A glibc-aware resolver/selector reads these.
	GlibcVersionAnnotation   = "org.go-pkgx.glibc.version"
	GlibcMinKernelAnnotation = "org.go-pkgx.glibc.min-kernel"
)

Signature referrer conventions.

Variables

View Source 
var (
	DistBase   = "oci://ghcr.io/go-pkgx/packages"
	PantryBase = "https://raw.githubusercontent.com/pkgxdev/pantry/main/projects"

	// PantryOverlay (PKGX_PANTRY_OVERLAY), when set, is consulted for a project's
	// package.yml BEFORE PantryBase, falling back to PantryBase when the overlay
	// has no recipe for that project. This lets a small curated overlay carry
	// corrected recipes (e.g. a stale `openssl.org: ^1.1` bumped to a modern
	// constraint that matches the published bottles) without forking the whole
	// pantry — everything the overlay does not override resolves upstream.
	PantryOverlay = ""

	// UpstreamDist is the canonical pkgx distribution used to list versions for
	// projects that are not (yet) published to an OCI DistBase — typically
	// build-time deps such as llvm.org, perl.org or qt.io. It mirrors what
	// `pkgx +<pkg>` installs, so a {{deps.<p>.prefix}} token resolves to the same
	// version pkgx actually installs at build time. Overridable in tests.
	UpstreamDist = "https://dist.pkgx.dev"

	// CacheBase is an OCI registry consulted for the BOTTLE ITSELF before
	// DistBase — a pull-through cache next to the builders, so a job does not
	// fetch llvm.org's ~1.7 GiB across the network again.
	//
	// It deliberately takes NO part in resolution. A pull-through cache cannot
	// list the tags of a repository it has not synced yet, so listing against it
	// falls back to the upstream dist, which yields a VERSION where our registry
	// wants a TAG — gnu.org/bash is published as `5.3`, not `5.3.0` — and the
	// pull then 404s on a tag that never existed. Measured; it is why this is a
	// separate variable and not simply PKGX_DIST pointed elsewhere.
	//
	// What the cache serves is still verified against the signature DistBase
	// advertises, so a cache that returns the wrong bytes is rejected rather
	// than trusted.
	CacheBase = ""
)

Base URLs for the pkgx distribution + pantry; overridable in tests, and at runtime via $PKGX_DIST / $PKGX_PANTRY so a consumer can point pkgm/pkgx at a local mirror produced by the `mirror` tool.

DistBase defaults to the **signed** go-pkgx OCI registry so the default install path is verifiable end-to-end (pairs with VerifyRequired defaulting on). That registry is a growing subset of the pantry; for the full upstream catalogue set PKGX_DIST=https://dist.pkgx.dev (unsigned → also set PKGX_VERIFY=0).

View Source 
var ErrInsecurePath = errors.New("bottle: insecure path in archive")

ErrInsecurePath is returned by Extract when an archive entry (or a hard-link source) names an absolute path, or one that after component-stripping would escape the destination directory. It is wrapped with the offending name so callers can test for it with errors.Is.

View Source 
var Exec func(argv0 string, argv []string, env []string) error

Exec runs (or, on UNIX, replaces the current process image with) the target binary; overridable in tests. Its concrete value is platform-specific:

  • UNIX (exec_unix.go): syscall.Exec — replaces the process, never returns on success.
  • Windows (exec_windows.go): spawns the child with inherited stdio, waits, and exits the parent with the child's exit code (Windows has no execve).

The signature is identical on every platform, so callers are unchanged.

View Source 
var HTTPClient = NewHTTPClient()

HTTPClient is an HTTP client that trusts the embedded CA bundle (falling back to the system pool if, somehow, the embed is empty). It is overridable.

View Source 
var SigningPublicKey = "RWQ+rmH+fXy2iYr+gReQAOQtYWtH0A7UlxcAa2hpr+txNBwGqtpFsR6L"

SigningPublicKey is the go-pkgx bottle signing key (minisign format). A signed bottle carries a cosign-style signature referrer whose signature verifies against this key; verification is fail-closed. It is a var (not const) only so tests can substitute a throwaway key.

View Source 
var Warn func(msg string)

Warn receives diagnostics a caller should SEE but that are not fatal: a closure the resolver could not complete, a soname nothing provides. It is nil by default — a library must not write to a process's stderr uninvited — so a command-line tool sets it once at startup: 

bottle.Warn = func(msg string) { fmt.Fprintln(os.Stderr, "pkgx: "+msg) }

Silence here is expensive: an unprovided soname does not fail the install, it fails LATER, when the binary starts and reports "cannot open shared object file" with nothing pointing back at the resolution that gave up.

Functions

func BinNames 

func BinNames(project string, provides []string) []string

BinNames returns the base names of the binaries a package provides, falling back to the project's leaf name when it declares no `provides:`.

func CanonicalLoaderExists 

func CanonicalLoaderExists() bool

CanonicalLoaderExists reports whether /lib{,64}/ld-linux-* is present.

func CertPool added in v0.5.4 

func CertPool() *x509.CertPool

CertPool is the trust store the go-pkgx tools use: the host's, plus the embedded Mozilla bundle. On a FROM-scratch image there is no system store at all, so the embedded bundle is the only thing standing between a tool and "x509: certificate signed by unknown authority" — which is exactly what bk hit when it tried to reach github from inside a scratch builder.

func ConfigError added in v0.5.0 

func ConfigError() error

ConfigError returns the error, if any, from loading ~/.pkgx/config.hcl2. It is nil when the file parsed cleanly or is simply absent, and non-nil only on a real path or parse failure (in which case reads fall back to the environment).

func Dir 

func Dir() string

Dir resolves the bottle store (PKGX_DIR, default ~/.pkgx).

func DownloadBottle added in v0.2.0 

func DownloadBottle(project, ver, osn, arch string) ([]byte, string, error)

DownloadBottle fetches the raw compressed bottle tarball for a specific project/version/os/arch, trying .tar.gz then .tar.xz. It returns the bytes and the extension that succeeded (".tar.gz" or ".tar.xz") — for mirror tooling that copies bottles verbatim without extracting them.

func Env added in v0.5.0 

func Env(name string) string

Env resolves a variable for the go-pkgx tools. A real process environment variable that is set AND non-empty wins; otherwise the value from ~/.pkgx/config.hcl2 is returned; otherwise the empty string.

func Extract added in v0.5.4 

func Extract(tr *tar.Reader, dest string, strip int) error

Extract unpacks every entry of tr into dest, stripping strip leading path components from each entry name (strip == 0 keeps the archive layout). It is the shared archive extractor for the pkgx ecosystem — both bottle's bottle installer and bk's source fetcher route through it, so a single implementation carries the invariants both rely on:

  • Regular files keep the modification time the archive recorded (tar(1) semantics). Recipes DEPEND on this: a release tarball ships generated files NEWER than the sources they derive from so make leaves them alone; stamping every file "now" re-orders them by archive position and make then tries to regenerate — libexpat 2.8.3 dies that way (doc/xmlwf.1 is archived before, hence would be stamped older than, doc/xmlwf.xml → make runs the docbook rule with no docbook2x-man installed → Error 1). A zero recorded time is left as written.
  • Absolute names, and names that escape dest after stripping, are rejected with ErrInsecurePath; a hard-link source is vetted the same way.
  • Directories, regular files, symlinks and hard links are reproduced; unsupported entry types (fifos, devices, ...) are skipped.

tar.ErrInsecurePath from tr.Next is tolerated because Extract performs its own stricter vetting on every entry name.

func FetchMeta 

func FetchMeta(project string) (deps map[string]string, provides []string, err error)

FetchMeta returns the host-relevant runtime dependencies (project -> constraint) and the list of provided paths for a project's package.yml.

func FetchMetaFor added in v0.5.5 

func FetchMetaFor(project, osn, arch string) (deps map[string]string, provides []string, err error)

FetchMetaFor is FetchMeta for an EXPLICIT pkgx os/arch slug. A recipe's dependencies are platform-keyed (`linux: {...}`, `darwin/aarch64: {...}`), so reading them through the host's slug while staging a rootfs for another platform silently produces the WRONG closure — a linux image missing the deps only linux declares, and carrying the ones only darwin needs.

func FetchRuntimeEnv added in v0.5.5 

func FetchRuntimeEnv(project, prefix, version string) (map[string]string, error)

FetchRuntimeEnv returns a project's runtime env declarations with the recipe placeholders resolved against the version actually installed. A project that declares none yields an empty map, not an error.

func FetchRuntimeEnvIn added in v0.17.0 

func FetchRuntimeEnvIn(project string, closure []Resolved, dir string) (map[string]string, error)

FetchRuntimeEnvIn is FetchRuntimeEnv with the resolved closure in hand, so `deps.<project>.prefix` and the dep version tokens resolve.

libpkgx's useMoustaches tokenises, for every installed dependency: 

map.push({ from: `deps.${dep.pkg.project}.prefix`, to: dep.path.string })
map.push(...tokenize.version(dep.pkg.version, `deps.${dep.pkg.project}.version`))

Without them rust-lang.org/cargo declares 

CARGO_HTTP_CAINFO: ${{deps.curl.se/ca-certs.prefix}}/ssl/cert.pem

and cargo builds with no CA bundle at all — `[77] Problem with the SSL CA cert … error setting certificate file: /ssl/cert.pem` — which is where the whole crates.io family stopped.

func FindClosureBin 

func FindClosureBin(closure []Resolved, dir, project, name string) string

FindClosureBin returns the path to <project>/…/bin/<name> if that project is in the installed closure, else "".

func FindLoader 

func FindLoader(dir string) string

FindLoader locates the pkgx glibc dynamic loader in an installed closure.

func FindLoaderFor added in v0.6.0 

func FindLoaderFor(dir, arch string) string

FindLoaderFor is FindLoader for an EXPLICIT architecture: the loader of the rootfs being STAGED, whose ELF name (ld-linux-aarch64.so.1 vs ld-linux-x86-64.so.2) is the target's, not the staging machine's.

func GOARCH 

func GOARCH() string

GOARCH returns the pkgx architecture slug (e.g. "x86-64", "aarch64") for the running machine.

func GOOS 

func GOOS() string

GOOS returns the pkgx OS slug ("linux", "darwin", or "windows") for the running machine.

func GlibcConstraint added in v0.5.4 

func GlibcConstraint() string

GlibcConstraint is the constraint gnu.org/glibc is resolved with when a closure needs the C library: an exact "=<version>" when a glibc has been pinned (PKGX_GLIBC) or chosen for this host's kernel, else "*" (newest).

func GlibcFlavor added in v0.5.4 

func GlibcFlavor() string

GlibcFlavor is the glibc build variant this host asks for: the PKGX_GLIBC pin, or "" for ordinary builds. A pinned host wants BOTH that exact glibc bottle AND, wherever the factory published one, the tool built against it — the `<version>-glibc<ver>` tag.

func GlibcMinKernel added in v0.5.4 

func GlibcMinKernel(path string) (string, error)

GlibcMinKernel reads the minimum supported Linux kernel version baked into a glibc's libc.so.6 (or any glibc-linked ELF) via the GNU `.note.ABI-tag` note, returned as "major.minor.subminor" (e.g. "3.2.0").

glibc records its `--enable-kernel` floor here: the loader+libc refuse to run on an older kernel. So this is exactly the datum a glibc-by-kernel selector needs — pick the newest glibc whose min-kernel <= the host's `uname -r`. Pure Go (debug/elf); no external readelf.

The note layout (ELF spec + GNU ABI tag): namesz(u32) descsz(u32) type(u32), then the name ("GNU\0", padded to 4 bytes), then desc = 4 u32 words [OS, major, minor, subminor] with OS==0 for Linux. type is NT_GNU_ABI_TAG(1).

func HostKernel added in v0.5.4 

func HostKernel() (string, error)

HostKernel returns the running Linux kernel release (e.g. "6.8.0-45-generic"), or "" on a non-Linux host.

func HostSlug 

func HostSlug() (string, string)

HostSlug returns the pkgx (os, arch) slug for the running machine.

func Install 

func Install(r Resolved, pkgxDir string) (bool, error)

Install downloads and extracts one bottle into pkgxDir (skips if the versioned prefix already exists), then writes major/minor convenience links. Extraction is atomic — the bottle is unpacked into a temp dir and the versioned prefix is renamed into place — so concurrent installs sharing one PKGX_DIR never observe a half-extracted prefix. Install materialises a bottle for the HOST platform. Use InstallFor to stage a rootfs for another one -- building a linux image from a mac, say.

func InstallFor added in v0.5.5 

func InstallFor(r Resolved, pkgxDir, osn, arch string) (bool, error)

InstallFor materialises a bottle for an EXPLICIT platform. The extracted tree is identical whatever the host: only the bottle fetched differs, so a builder can stage a linux/aarch64 rootfs from a darwin/arm64 machine.

func IsELF 

func IsELF(p string) bool

IsELF reports whether the file at p starts with the ELF magic.

func IsOCI added in v0.4.0 

func IsOCI(base string) bool

IsOCI reports whether a dist base selects the OCI transport.

func IsPlatformTag added in v0.11.0 

func IsPlatformTag(tag string) bool

IsPlatformTag reports whether a tag names a platform manifest rather than a version. Consumers walking a repository's tags need to skip these.

func LibDirs added in v0.3.0 

func LibDirs(closure []Resolved, dir string) []string

LibDirs returns every library directory in an installed closure (including glibc's versioned sub-libdir on linux). It is the list form of LibPath, used where an OS-native path separator is required (Windows joins with ";" and resolves DLLs from these dirs via PATH, so a ":"-joined string is wrong).

func LibPath 

func LibPath(closure []Resolved, dir string) string

LibPath returns a ":"-joined library path covering every lib dir in an installed closure (including glibc's versioned sub-libdir). It is the value of $LD_LIBRARY_PATH used on linux; on Windows use LibDirs with the native separator instead.

func LoaderName 

func LoaderName() string

LoaderName is the dynamic-loader soname for the current architecture.

func LoaderNameFor added in v0.5.5 

func LoaderNameFor(arch string) string

LoaderNameFor answers LoaderName for an explicit architecture, so a builder can name the loader of the image it is assembling.

func NewHTTPClient 

func NewHTTPClient() *http.Client

NewHTTPClient builds an *http.Client that trusts CertPool.

func PrefixOf 

func PrefixOf(project string, closure []Resolved, dir string) string

PrefixOf returns the installed prefix directory <dir>/<project>/v<version> for the Resolved whose Project equals project in the closure, or "" if the project is not part of the closure.

func PrimaryBin 

func PrimaryBin(project string, provides []string) string

PrimaryBin picks the binary to run for a project: the one whose name matches the project's path leaf (gnu.org/wget -> wget) or its domain's second-level label (perl.org -> perl, not the first-listed corelist), else the first provided binary.

func ResolveBinPath added in v0.3.0 

func ResolveBinPath(bin string) string

ResolveBinPath maps a logical bin path (…/bin/foo) to the file that actually exists on disk. On Windows it prefers "foo.exe" (the PE image the toolchain produces), falling back to the bare name; on UNIX it is the identity. This keeps BinNames/PrimaryBin returning logical, extension-free names while the run/stub sites resolve to the real executable.

func SetupScratchRootfs 

func SetupScratchRootfs(loader, shellPath string)

SetupScratchRootfs makes the pkgx loader and a shell available at their canonical absolute paths (best-effort). On a FROM-scratch image this lets every bottle ELF — the one we exec AND any child processes it spawns — resolve its PT_INTERP=/lib/ld-linux natively, and (when shellPath is set) lets "#!/bin/sh" wrapper scripts resolve to the pkgx bash we installed. On a normal system these paths already exist, so os.Symlink fails and is ignored (no clobbering).

func SetupScratchRootfsAt added in v0.5.5 

func SetupScratchRootfsAt(root, loaderName, loaderTarget, shellTarget string) error

SetupScratchRootfsAt is SetupScratchRootfs against an arbitrary root, for a rootfs being STAGED rather than the one being run on. A builder assembling a FROM-scratch image on a host cannot write /lib and /bin — those belong to the host — so it hands the staging directory here instead.

loaderName is the loader's ELF name for the TARGET architecture, which is not necessarily the host's: LoaderName() answers for the host, LoaderNameFor() for whatever is being built.

func SplitFlavor added in v0.5.4 

func SplitFlavor(tag string) (version, glibc string)

SplitFlavor splits a registry tag into the software version and the glibc flavor it was built against; the flavor is "" for an ordinary tag. 

"8.20.0"             → "8.20.0", ""
"8.20.0-glibc2.27.0" → "8.20.0", "2.27.0"

func StubBins 

func StubBins(closure []Resolved, dir, prefix string) (int, error)

StubBins writes a small env-setting shell stub into <prefix>/bin for every binary in the closure, mirroring the reference pkgm: the stub exports the closure's LD_LIBRARY_PATH and exec's the real bottle binary.

func StubBinsStaged added in v0.6.0 

func StubBinsStaged(closure []Resolved, s Stage) (int, error)

StubBinsStaged is StubBins for a rootfs being assembled elsewhere.

A stub is a shell script holding ABSOLUTE paths — the closure's LD_LIBRARY_PATH and the bottle binary to exec. Written with the staging machine's paths, every one of them is wrong the moment the rootfs boots: they point at /tmp/whatever-the-builder-used, a directory the guest does not have. So the paths baked in come from GuestDir while the existence checks use Dir.

func VerifyRequired added in v0.5.0 

func VerifyRequired() bool

VerifyRequired reports whether fail-closed signature verification is demanded when installing. It is **on by default** (secure by default): only an explicit PKGX_VERIFY=0/false/no/off opts out. Anything else — unset, 1/true/yes/on, or an unrecognised value — keeps verification on. The value is read through Env, so it may come from a real environment variable or from ~/.pkgx/config.hcl2.

func VerifySignature added in v0.5.0 

func VerifySignature(tarball, payload []byte, b64sig, pubkey string) error

VerifySignature checks a cosign simple-signing signature over a bottle tarball: the base64 sig must verify over payload against pubkey (defaulting to the pinned SigningPublicKey), and payload must commit to this exact tarball (docker-manifest-digest == sha256(tarball)). Every failure returns an error — callers treat that as fail-closed.

func VerifySignatureDigest added in v0.7.0 

func VerifySignatureDigest(digest string, payload []byte, b64sig, pubkey string) error

VerifySignatureDigest is VerifySignature for a tarball identified by its DIGEST rather than by its bytes. The bytes are only ever hashed here, so a caller that streamed the bottle to disk — computing the digest on the way — can verify the signature without reading the tarball back, let alone holding it in memory.

Types

type BlobFile added in v0.7.0 

type BlobFile struct {
	io.ReadSeeker
	// Digest is the "sha256:…" of the staged bytes, computed as they were
	// written. It is what the signature is checked against, so the tarball
	// never has to be read back — let alone kept — to verify it.
	Digest string
	// contains filtered or unexported fields
}

BlobFile is a staged blob: on disk where there is a filesystem, in memory where there is not. Close releases it — on disk that means REMOVING it, since the caller owns a temporary and not a cache entry.

func (*BlobFile) Close added in v0.7.0 

func (b *BlobFile) Close() error

Close releases the staged blob.

type OCIClient added in v0.4.0 

type OCIClient struct {
	// contains filtered or unexported fields
}

OCIClient talks to an OCI distribution (registry v2) endpoint via ORAS. It is used by the bottle pull path and by the mirror push tool, so the OCI protocol lives in one place (ORAS).

func NewOCIClient added in v0.4.0 

func NewOCIClient(base string) (*OCIClient, error)

NewOCIClient parses an oci:// dist base into a client. Localhost/loopback hosts default to plain HTTP (for a local zot/registry on a port); every other host uses HTTPS. Credentials are read from the environment: OCI_TOKEN (a pre-issued bearer), else OCI_USERNAME / OCI_PASSWORD for token exchange / basic auth. Public repositories work anonymously.

func (*OCIClient) Annotations added in v0.5.4 

func (c *OCIClient) Annotations(project, ver, osn, arch string) (map[string]string, error)

Annotations returns the OCI annotations of a published bottle's per-platform manifest — how a client reads a bottle's self-description (a glibc's minimum kernel, a tool's glibc flavor) without downloading the bottle itself.

func (*OCIClient) EnsureIndexed added in v0.8.1 

func (c *OCIClient) EnsureIndexed(project, ver string, desc ocispec.Descriptor) (repaired bool, err error)

EnsureIndexed puts desc back into the version tag's index when a racing publisher dropped it, and reports whether it had to.

The per-push reconcile in mergePlatformIntoIndex confirms its write twice with a settle window between, which catches a racer landing just behind it. It cannot catch one that lands later still — after this publisher has moved on to the next package, or exited. This is the pass for that: run it once the whole batch is done, when the other publishers have finished too and a repair sticks.

Re-merging (rather than re-writing) is what keeps the repair safe: it starts from the index as it now stands, so a platform some other publisher added in the meantime survives.

func (*OCIClient) FetchAttestations added in v0.5.4 

func (c *OCIClient) FetchAttestations(project, ver, osn, arch string) ([]Referrer, map[string]string, error)

FetchAttestations returns a published bottle's attestations, ready to be pushed to another registry with PushWithReferrersAnnotated, along with the per-platform manifest's annotations (a glibc bottle's minimum kernel, a flavored build's glibc version) so those survive the copy too.

func (*OCIClient) HasPlatform added in v0.5.4 

func (c *OCIClient) HasPlatform(project, ver, osn, arch string) (bool, error)

HasPlatform reports whether (project, ver) is published for os/arch in the registry: true if a per-platform manifest resolves, false if the tag or the platform is absent (a not-yet-published signal), and an error only on a real transport/parse failure. The build factory uses it to skip already-published (project,version,platform) tuples — a pure-Go replacement for the curl+jq ghcr manifest probe.

func (*OCIClient) IndexLists added in v0.8.1 

func (c *OCIClient) IndexLists(project, ver string, desc ocispec.Descriptor) (bool, error)

IndexLists reports whether the version tag's index carries desc.

A bottle can be pushed, valid and signed — and absent from the index, because publishing the index is a read-modify-write on one mutable tag that several publishers race on. Nothing about the bottle itself says so; only the index does, which is why this takes the descriptor the push returned rather than trying to discover it. Discovery would go through the very index in question.

func (*OCIClient) ListTags added in v0.4.0 

func (c *OCIClient) ListTags(project string) ([]string, error)

ListTags returns the tags (versions) published for a project, or an error if the repository does not exist.

func (*OCIClient) Pull added in v0.4.0 

func (c *OCIClient) Pull(project, ver, osn, arch string) ([]byte, string, error)

Pull downloads the bottle tarball for a project/version/os/arch and returns it in MEMORY. Prefer PullFile for anything bottle-sized: llvm.org is ~1.7 GiB and this makes the biggest package in the catalogue the memory floor of the caller. Kept for callers that genuinely want the bytes.

func (*OCIClient) PullFile added in v0.7.0 

func (c *OCIClient) PullFile(project, ver, osn, arch string) (*BlobFile, string, error)

PullFile downloads the bottle tarball for a project/version/os/arch: fetch the version-tag manifest (an index, or a single-platform image manifest), pick the platform-matching image manifest, take its single non-config layer, stream the layer blob TO DISK. Returns the staged file (Close removes it) and the extension (".tar.gz"/".tar.xz") derived from the layer mediaType.

func (*OCIClient) Push added in v0.4.0 

func (c *OCIClient) Push(project, ver, osn, arch string, tarball []byte, ext string) error

Push publishes one bottle for a project/version/os/arch: it pushes the tarball as a layer blob, packs a per-platform image manifest (ORAS PackManifest, which also pushes a scratch config blob), then merges the platform into the version-tag image index (fetching any existing index first) and tags it. ext selects the layer mediaType (".tar.gz" or ".tar.xz").

func (*OCIClient) PushWithReferrers added in v0.5.0 

func (c *OCIClient) PushWithReferrers(project, ver, osn, arch string, tarball []byte, ext string, refs []Referrer) (ocispec.Descriptor, error)

PushWithReferrers is Push plus a set of attestations attached to the pushed per-platform manifest as OCI referrers. It returns that manifest's descriptor so callers can list/verify the referrers. Referrers are pushed before the version index is tagged, so a reader that resolves the index always finds a manifest whose attestations are already present.

func (*OCIClient) PushWithReferrersAnnotated added in v0.5.4 

func (c *OCIClient) PushWithReferrersAnnotated(project, ver, osn, arch string, tarball []byte, ext string, refs []Referrer, annotations map[string]string) (ocispec.Descriptor, error)

PushWithReferrersAnnotated is PushWithReferrers plus annotations merged onto the per-platform image manifest — used to carry descriptive metadata such as org.go-pkgx.glibc.version and org.go-pkgx.glibc.min-kernel so glibc-flavored bottles are self-describing.

func (*OCIClient) Referrers added in v0.5.0 

func (c *OCIClient) Referrers(project string, subject ocispec.Descriptor) ([]ocispec.Descriptor, error)

Referrers lists the attestations attached to a manifest (its OCI referrers), each descriptor carrying the referrer's ArtifactType.

func (*OCIClient) VerifyBottle added in v0.5.0 

func (c *OCIClient) VerifyBottle(project, ver, osn, arch string, tarball []byte) error

VerifyBottle checks the cosign-style signature attached to a pulled bottle as an OCI referrer, against the pinned SigningPublicKey. It is fail-closed: a missing signature referrer, a malformed one, or a signature that does not verify (or does not commit to this tarball) all return an error.

func (*OCIClient) VerifyBottleDigest added in v0.7.0 

func (c *OCIClient) VerifyBottleDigest(project, ver, osn, arch, digest string) error

VerifyBottleDigest is VerifyBottle for a bottle identified by its DIGEST: the form an install uses, having streamed the tarball to disk and hashed it on the way past.

type Referrer added in v0.5.0 

type Referrer struct {
	ArtifactType string
	MediaType    string
	Blob         []byte
	// Annotations are merged into the referrer manifest — used to carry a cosign
	// signature (dev.cosignproject.cosign/signature) alongside its payload blob.
	Annotations map[string]string
}

Referrer is an attestation to attach to a bottle's per-platform manifest as an OCI referrer (subject-linked artifact): a CycloneDX/SPDX SBOM, an in-toto SLSA provenance statement, or (later) a cosign signature. ArtifactType classifies the referrer manifest; MediaType is the layer blob's media type (usually the same); Blob is the attestation bytes.

type Resolved 

type Resolved struct {
	Project string
	Version Ver
}

Resolved is a project pinned to a concrete version.

func CompleteClosure 

func CompleteClosure(roots map[string]string, dir string) ([]Resolved, error)

CompleteClosure resolves a package's declared closure, installs it, then iteratively reads the bottles' ELF DT_NEEDED and pulls whatever bottle provides each unsatisfied soname — the implicit glibc/gcc system libraries and the common undeclared libraries (libz, libbz2, ncurses…) that package.yml graphs omit — until every NEEDED soname is satisfied. This makes the closure complete enough to run on a `FROM scratch` image.

func MergeClosures 

func MergeClosures(a, b []Resolved) []Resolved

MergeClosures appends the packages of b not already in a (dedup by project).

func ResolveClosure 

func ResolveClosure(roots map[string]string) ([]Resolved, error)

ResolveClosure walks the runtime dependency graph breadth-first.

func ResolveClosureFor added in v0.5.5 

func ResolveClosureFor(roots map[string]string, osn, arch string) ([]Resolved, error)

ResolveClosureFor is ResolveClosure for an EXPLICIT pkgx os/arch slug: the whole walk — version pick and dependency read alike — happens in the target platform's terms. This is what lets a darwin machine stage a complete linux/aarch64 userland, which is how the sovereign builder image is built.

type Stage added in v0.6.0 

type Stage struct {
	// Dir is where the bottles are on the machine writing the stubs. Files are
	// checked for existence here.
	Dir string
	// GuestDir is the path those same bottles will have where the stubs run —
	// "/pkgx" for an image staged under /tmp/rootfs. Empty means Dir: the
	// staging machine IS the running machine (pkgm's case).
	GuestDir string
	// Prefix is where <prefix>/bin receives the stubs, on the staging machine.
	Prefix string
	// OS and Arch are the pkgx slug of the target. Empty means the host's —
	// they select which platform's `provides:` list names the binaries.
	OS, Arch string
}

Stage describes where a closure's bottles LIVE while stubs are written and where they will live when the stubs RUN. The two differ whenever a rootfs is assembled somewhere other than the machine that will boot it — the whole point of staging.

type Ver 

type Ver struct {
	Raw  string
	Nums []int
	Tag  string
}

Ver is a parsed pkgx version: its numeric components plus the raw string.

Tag is the registry tag the version was found under, when that differs from Raw — a glibc-flavored build lives at `<version>-glibc<ver>` while remaining version <version>. Everything version-shaped (comparison, constraints, the installed v<version> prefix) uses Raw; only the registry pull uses the tag.

func FetchVersions 

func FetchVersions(project string) ([]Ver, error)

FetchVersions returns the available versions of a project for the host os/arch, ascending.

func ParseVer 

func ParseVer(s string) Ver

ParseVer parses a pkgx version string into a Ver, stopping each component at the first non-numeric character (e.g. "1w" in openssl 1.1.1w).

func PickVersion 

func PickVersion(project, constraint string) (Ver, error)

PickVersion returns the highest available version satisfying constraint, among those published for the HOST platform.

func PickVersionFor added in v0.5.5 

func PickVersionFor(project, constraint, osn, arch string) (Ver, error)

PickVersionFor is PickVersion for an EXPLICIT pkgx os/arch slug. A project's published version sets differ per platform — a linux/aarch64 bottle can exist where the darwin one does not — so a host that stages a rootfs for another platform must resolve against that platform's catalogue, not its own.

func VersionsFor added in v0.2.0 

func VersionsFor(project, osn, arch string) ([]Ver, error)

VersionsFor returns the available versions of a project for an explicit pkgx os/arch slug (e.g. "linux"/"aarch64"), ascending. Used by mirror tooling that spans arches other than the host's.

When DistBase is an OCI registry (the factory default), version listing comes from that registry's tags. That registry is a growing subset of the upstream catalogue, so build-time deps (e.g. llvm.org, perl.org) may not be published there yet and list zero versions. In that case we fall back to the upstream pkgx dist so dep resolution matches the version `pkgx +<pkg>` installs at build time — the actual bottle *pull* still goes through DistBase unchanged, so this only affects version listing, not the install source.

func (Ver) Satisfies added in v0.2.0 

func (v Ver) Satisfies(constraint string) bool

Satisfies reports whether the version meets a pkgx constraint ("^1.2", "~2", ">=1.0", "=1.2.3", "*", or "").

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.