README
¶
artifacts
Resolve, verify and cache approved artefacts from the phpboyscout artefact channel: fetch by name and version, verify the publisher's signature, return a path on disk.
verifier, err := trust.Estate(releaseKey) // embedded key + WKD, must agree
client, err := artifacts.New(verifier, artifacts.WithCache(cache))
path, err := client.Resolve(ctx, artifacts.Ref{Name: "onnxruntime", Version: "1.28.0"},
"onnxruntime-linux-x64-1.28.0.tgz")
Why it exists
Every tool that loads a model or a runtime was otherwise writing its own download-verify-cache loop. Three hand-rolled clients are three chances to skip the verification step — and the one that skips it fails silently, because a wrong artefact simply loads and runs.
Specification:
specs/0014
D6, in the phpbotscout wiki until the artefact project takes ownership.
What is verified, and in what order
- The signature over the manifest, against a Verifier that requires the key embedded in your binary and the key served over WKD to agree.
- The artefact's digest against that manifest.
The signature is checked before the artefact is fetched, so an artefact whose publisher cannot be established is never downloaded.
A signature proves who published the bytes; a digest proves the bytes have not changed. Neither alone suffices — a digest pinned in source cannot be rotated or revoked and says nothing about origin, and a signature over a manifest nobody checked the files against leaves the gap between manifest and file open.
A client cannot be constructed without a Verifier. A mode that fetches without verifying is a mode that will eventually be the one in use.
What it deliberately does not do
It resolves what it is asked for; it does not choose versions. The channel's
inventory labels each version supported, legacy or untested, and those are
advisory — a human reads them to decide what to pin. This package will not
resolve "give me stable", because a floating pointer resolves to a correctly
signed artefact: verification passes and the change is silent.
Documentation
https://artifacts.phpboyscout.uk — tutorial, how-to guides, reference and explanation, arranged by Diátaxis.
A note on spelling
Identifiers use artifact; prose uses artefact.
They are dialect variants of one word, not two words — there is no semantic
difference. The split is deliberate: code follows the ecosystem it lives in
(GitLab's artifacts: keyword, the package registry, Maven and OCI artifacts),
and prose follows the house voice.
So the module, the repository, artifacts.yaml and every Go identifier spell it
artifact, while comments, documentation and specs spell it artefact. If you
are grepping the Go source for artefact you will find comments only, which is
the intended result rather than an oversight.
Documentation
¶
Overview ¶
Package artifacts resolves approved artefacts from the phpboyscout artefact channel: fetch by name and version, verify the publisher's signature, cache, return a path on disk.
It exists so that every tool needing a model or a runtime does not write its own download-verify-cache loop. Three hand-rolled clients are three chances to skip the verification step, and the one that skips it fails silently — a wrong artefact loads and runs.
What is verified ¶
The channel publishes each artefact-version with a `checksums.txt` manifest, a detached OpenPGP signature over that manifest, and the publisher's public key. Resolution verifies in that order:
- the signature over the manifest, using the Verifier the caller supplied;
- the artefact's digest against the manifest.
A signature proves who published the bytes. A digest proves the bytes did not change. Neither alone is sufficient: a digest pinned in source cannot be rotated or revoked and says nothing about origin, and a signature over an artefact nobody checked the digest of leaves the door open between manifest and file.
Where trust comes from ¶
This package does not decide which keys to trust — it verifies against the Verifier it is given. That is a real boundary, not a hedge: what counts as an acceptable signature is a deployment's decision, and a resolver that hard- coded one answer could not be tested without a key.
The estate's answer lives in the trust subpackage: gitlab.com/phpboyscout/go/artifacts/trust.Estate requires the key embedded in the calling binary and the key served over WKD to agree, and fails closed if either is unavailable. Use it unless you have a specific reason not to. A tool that builds its own Verifier is choosing its own trust anchors, which is exactly the decision that should be hard to make by accident.
What is deliberately NOT here ¶
This package resolves what it is asked for. It does not choose versions. Artefact statuses in the channel's inventory — supported, legacy, untested — are advisory labels a human reads, not a channel this package resolves against: a floating pointer resolves to a correctly signed artefact, so verification passes and the surprise is silent. Pin a version.
Index ¶
Constants ¶
const ( ManifestFile = "checksums.txt" SignatureFile = "checksums.txt.sig" )
Manifest and signature filenames, fixed by the channel's publisher.
The channel also publishes the signing key as release.asc, but this package never fetches it: trust comes from the key embedded in the calling binary and the key served over WKD (see the trust subpackage). A key downloaded from the same place as the signature it validates proves nothing.
const DefaultIndexURL = "https://artifacts.phpboyscout.uk/index.txt"
DefaultIndexURL is where the signed catalogue of approved artefacts lives.
Compiled in, as the embedded key is (spec 0016 OQ2). Together they are the only two trust inputs a consuming binary carries: everything else — which artefacts exist, which are approved, where the bytes live — arrives signed and is checked against these. Redirecting either means rebuilding.
A different provider from the artefacts on purpose. That is control-plane and failure-domain isolation rather than a second authorisation factor: the same key signs both, so this does not raise the cryptographic bar.
const DefaultNamespace = "phpboyscout"
The channel's path shape, which every location in the index is expected to reproduce: `<location>/<artefact>/<version>/<file>`.
There is deliberately no DefaultChannel constant and no WithChannel option any more. Download locations come from the signed index (spec 0016 D4/OQ1), so a compiled-in channel would be a second answer to a question the index already answers — and the one that could not be corrected without a release.
DefaultNamespace is the logical channel the estate publishes under.
A logical name, not a location: it is compared against the namespace inside the signed envelope, so a mirror serving the same channel from a different host still satisfies it. Changing this changes which publisher a client will accept.
Variables ¶
var ( // ErrNotFound means the channel has no such artefact-version. Usually a // typo or a version nobody has approved yet — the inventory is the answer. ErrNotFound = errors.New("artifacts: not published") // ErrUnverified means the bytes arrived but could not be trusted. It is // deliberately not more specific to the caller: a failed signature and a // mismatched digest are the same answer — do not use this. ErrUnverified = errors.New("artifacts: verification failed") // the signature was never actually checked. // // Deliberately NOT ErrUnverified. The two demand different responses: a // failed signature must not be retried, and an unreachable anchor is // exactly the case that should be. Collapsing them meant a tool on a flaky // network received a permanent-looking security failure for a transient // condition — and the documentation told it never to retry. // // Retry it under a context with a bounded number of attempts. It is not a // signal that anything is wrong with the artefact, because nothing about // the artefact was examined. ErrTrustUnavailable = errors.New("artifacts: trust anchors unavailable") // ErrUnsigned means the artefact-version exists but no signature is // published beside its manifest. // // Distinct from ErrNotFound, which reads as "no such version" and sends a // reader to check their spelling. An unsigned publication is an incomplete // or interrupted one, and it is the publisher's problem rather than the // caller's. ErrUnsigned = errors.New("artifacts: published without a signature") // ErrWithdrawn means the channel has withdrawn this artefact-version. // // Its own sentinel, separate from index-unavailable, not-listed, a bad // signature and a stale index. Collapsing those loses the difference // between "this is dangerous" and "I could not check" — which warrant // opposite responses from whoever is on call. // // Terminal: a withdrawn version never returns. A correction is a new // version, so this is not a state to retry through. ErrWithdrawn = errors.New("artifacts: withdrawn") // ErrNotListed means the index does not mention this artefact-version. // // The index is a COMPLETE statement of what is approved, so an absent entry // means not approved rather than not mentioned. Distinct from ErrNotFound, // which is about the channel not holding bytes. ErrNotListed = errors.New("artifacts: not in the approved index") // no approval decision could be made. Like ErrTrustUnavailable it is a // retryable condition rather than a statement about the artefact. ErrIndexUnavailable = errors.New("artifacts: index unavailable") // ErrStaleIndex means the index verified but is too old to act on — past // its expiry, or older than one this client has already accepted. // // A signature proves an index was issued and says nothing about when. This // is the error that stops a replayed index restoring a withdrawn version. ErrStaleIndex = errors.New("artifacts: index is stale") // ErrMalformedIndex means the index verified and is not an index. ErrMalformedIndex = errors.New("artifacts: malformed index") // ErrInvalidIdentifier means a Ref or a manifest names something the // grammar does not admit — uppercase, Unicode, a path separator, a reserved // device name. Rejected rather than sanitised, because sanitising is what // let two distinct refs share one cache slot. ErrInvalidIdentifier = errors.New("artifacts: invalid identifier") // ErrIdentityMismatch means a correctly signed manifest describes a // different artefact-version from the one requested. // // Distinct from ErrUnverified and ErrNotFound on purpose. The signature is // genuine and the file may well be present — what is wrong is that the // publisher signed this manifest for something else, which is the signature // of a substitution or replay. Reporting it as a bad signature or a missing // file loses the only signal that says so. ErrIdentityMismatch = errors.New("artifacts: manifest describes another artefact-version") // ErrMalformedManifest means the manifest's signature was good but its // contents are not a manifest. // // Distinct from ErrUnverified on purpose. Parsing happens only AFTER the // signature is accepted, so reaching this means the publisher signed // something malformed — an operational fault in the channel, not an attack, // and it wants a different response from a failed signature. ErrMalformedManifest = errors.New("artifacts: malformed manifest") )
Errors callers should match on with errors.Is rather than by string.
Functions ¶
func UserCacheDir ¶
UserCacheDir is the conventional location: <user cache>/phpboyscout/artifacts.
Deliberately not named after any one tool. krites' runtime resolver cached under a "krites" directory, so a phpbotscout install populated a directory named after a different tool — the kind of thing nobody notices until they go looking for disk usage.
Types ¶
type Cache ¶
type Cache interface {
// Key validates the components and returns the entry they name, or an
// error wrapping ErrInvalidIdentifier.
Key(namespace string, ref Ref, file string) (Key, error)
// Path returns where an entry lives, whether or not it is present.
Path(key Key) string
// Has reports whether a verified copy is already stored.
Has(key Key) bool
// Put streams from r into the cache, returning the path. The reader
// verifies as it goes and fails before EOF on a mismatch, so nothing is
// installed at the final path unless every byte matched what was signed.
Put(key Key, r io.Reader) (string, error)
// Verify re-hashes a stored entry against a digest from a verified
// manifest. Called on every cache hit: bytes are checked on the way out as
// well as on the way in.
Verify(key Key, digest string) error
}
Cache stores verified artefacts so a second resolution is free.
Entries are named by a Key, which can only be built by validating every component (spec 0015 D3). An implementation therefore never sees an unvalidated path fragment, and cannot be the place a traversal or a collision is introduced.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client resolves artefacts from a channel.
The zero value is not usable; construct with New. Safe for concurrent use.
func New ¶
New builds a Client that verifies with the given Verifier.
The Verifier is required rather than optional. A resolver that can be constructed without one has a configuration in which it fetches and does not verify, and that configuration will eventually be used.
func (*Client) Manifest ¶
Manifest returns the verified digests for an artefact-version, keyed by filename.
Exported because a caller sometimes wants to know what a version contains — which platforms were published, for instance — without downloading any of it.
func (*Client) Resolve ¶
Resolve returns a local path to one verified file of an artefact-version.
The order is not an implementation detail. The manifest's signature is checked BEFORE the artefact is fetched, so an artefact whose publisher cannot be established is never downloaded at all — and a caller cannot end up with unverified bytes on disk because a later step failed.
type DirCache ¶
type DirCache struct {
// contains filtered or unexported fields
}
DirCache stores verified artefacts under a directory.
Laid out as <root>/<namespace>/<artefact>/<version>/<file>, mirroring the channel, so a cache is browsable and a stale entry is obvious to a human deleting it.
func NewDirCache ¶
func NewDirCache(root string, opts ...DirCacheOption) *DirCache
NewDirCache stores under root, creating it on first write.
A caller wanting the conventional location passes UserCacheDir(). This takes an explicit path because a library that decides where to write on a user's disk without being asked is a library that surprises somebody.
func (*DirCache) Has ¶
Has reports whether a stored copy exists.
Lstat rather than Stat, and regular files only: a symlink in the cache is not an artefact this package stored, and following one would return whatever it points at. That is a malformed-cache check rather than a defence against a local attacker, who can replace the file itself — see the trust model.
func (*DirCache) Key ¶
Key validates the components and returns the entry they name.
It REJECTS rather than sanitises, which is the whole of spec 0015 D3. The previous helper took the base name of each component, so "foo/bar" and "bar" produced the same path — and because the cache is consulted before the manifest is verified, resolving one ref could return another ref's artefact.
A validator has no output. There is nothing for two distinct inputs to collide onto, on a case-sensitive filesystem or a case-insensitive one, because the grammar admits exactly one case.
func (*DirCache) Put ¶
Put streams from r into the cache and returns the path.
The reader is expected to VERIFY as it goes and to fail before EOF if the bytes do not match what was signed — see verifyingReader. That is what makes this safe without holding the artefact in memory: bytes land in a quarantine file as they arrive, and the rename that installs them only happens once the reader has completed without complaint.
So an artefact is never written to its final path unverified, and a failure anywhere in the stream leaves the quarantine file to be removed rather than a partial artefact for Has() to report as present. Two processes racing resolve the same version to the same bytes, so whichever rename wins is correct.
The quarantine file is created 0600 by afero.TempFile: unverified bytes on disk should not be readable by anyone who could not already read the cache.
func (*DirCache) Verify ¶
Verify re-hashes a stored file against an expected digest.
The digest is validated before it is used: it arrives from a caller, and the error path formats a prefix of it. Slicing an unvalidated argument is how the previous version panicked on anything shorter than sixteen characters.
type DirCacheOption ¶
type DirCacheOption func(*DirCache)
DirCacheOption configures a DirCache.
func WithFS ¶
func WithFS(fs afero.Fs) DirCacheOption
WithFS supplies the filesystem, defaulting to the real one.
The seam exists so the failure paths can be exercised. Put creates a temporary file, writes, closes and renames, and each of those can fail on a full disk or a revoked permission — none of which is reachable from a test against a real directory without making the machine unwell. An in-memory filesystem reaches them in microseconds.
It is the estate's usual seam: go/repo and go/workspace take afero the same way, and gtb's props.FS hands one to every command.
type Key ¶
type Key struct {
// contains filtered or unexported fields
}
Key is a validated location within a cache.
Constructing one is the only way to name a cache entry, which is what makes the collision impossible rather than merely unlikely: every component has already passed the grammar by the time a path exists.
type MemoryWatermark ¶
type MemoryWatermark struct {
// contains filtered or unexported fields
}
MemoryWatermark keeps the mark for the life of the process.
The default, and honest about what it buys: a long-running service is protected from a rollback after its first successful resolution, and a short-lived command is protected by the expiry alone. Persisting it across runs is a caller's decision, because it means writing state this package otherwise does not own.
func NewMemoryWatermark ¶
func NewMemoryWatermark() *MemoryWatermark
NewMemoryWatermark returns an empty in-process mark.
func (*MemoryWatermark) Highest ¶
func (m *MemoryWatermark) Highest(namespace string) uint64
Highest returns the greatest generation accepted for a namespace.
type Option ¶
type Option func(*Client)
Option configures a Client.
func WithHTTPClient ¶
WithHTTPClient supplies the HTTP client, for callers that need a proxy, a custom transport, or a shorter deadline.
func WithIndexURL ¶
WithIndexURL points the client at a different signed index — a mirror of the whole channel, or a test server.
func WithNamespace ¶
WithNamespace sets the logical channel this client accepts.
Separate from WithChannel on purpose: one says where to fetch, the other says whose signature counts. Pointing a client at a mirror should not change which publisher it trusts, and conflating them would make it do exactly that.
func WithWatermark ¶
WithWatermark supplies where accepted index generations are remembered.
The default remembers for the life of the process. A caller that wants rollback protection to survive restarts persists it, which means owning a small piece of state this package deliberately does not write on its own.
type Ref ¶
type Ref struct {
// Name is the artefact's id in the inventory, e.g. "onnxruntime".
Name string
// Version is the artefact's OWN upstream version, e.g. "1.28.0" — not a
// version of the channel, which does not have one.
Version string
}
Ref names one artefact-version in the channel.
type Verifier ¶
type Verifier interface {
// Verify reports whether sig is a valid signature over manifest by a key
// this verifier accepts. It returns an error describing the failure for
// logs; callers match on ErrUnverified from the resolver instead.
Verify(ctx context.Context, manifest, sig []byte) error
}
Verifier checks a detached signature over a manifest.
An interface rather than a concrete dependency on go/signing, so this package stays testable without a key, so the root module keeps no dependencies, and so a consumer with a different trust posture is not forced through ours. The estate's implementation is trust.Estate.
Named for what it does, not what it holds. go/signing already has a TrustSet, and that one is a set of keys — reusing the word here for "the thing that verifies" made the two impossible to tell apart in the one file that mentions both.
type Watermark ¶
type Watermark interface {
// Highest returns the greatest generation accepted for a namespace, or
// zero if none has been.
Highest(namespace string) uint64
// Record stores a newly accepted generation. Implementations may assume it
// is greater than the current value.
Record(namespace string, generation uint64) error
}
Watermark remembers the highest index generation a client has accepted.
One half of spec 0016 D8. An expiry bounds a replay against a client that has never seen an index; a high-water mark bounds one against a client that has. Neither covers both populations, which is why there are two.
It is the only persistent state this package introduces, and it is safe to lose: losing it degrades to expiry-only, never to accepting a rollback that was already refused.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package trust supplies the estate's artefact artifacts.Verifier: a signature is accepted only when the key embedded in the calling binary and the key published over WKD agree.
|
Package trust supplies the estate's artefact artifacts.Verifier: a signature is accepted only when the key embedded in the calling binary and the key published over WKD agree. |