Documentation
¶
Overview ¶
Package oci speaks the OCI distribution spec to one repository over net/http.
bigoci owns this transport instead of importing an OCI SDK, because the library's value — parallel part transfers, per-part retry, and streaming that never buffers a part — lives in the HTTP layer. The protocol surface it needs is six endpoints: HEAD and GET on a blob, POST and PUT to upload one, and GET and PUT on a manifest. The reasoning is documented at https://imgoci.github.io/bigoci/explanation/design/.
NewRepository parses a reference into the repository it names and binds the manifest adapter to the tag or digest that reference carried. NewDigestPushRepository parses a repository-only reference and puts the adapter in digest-publication mode: Put writes at the digest of the body, and Get is unsupported. Repository.Blobs and Repository.Manifests return the two adapters, which implement the transfer package's Blobs and Manifests ports. Either way the address is fixed at construction, so the core asks for "the manifest" and never learns the reference grammar.
Nothing here buffers blob content: a read hands back the response body unread, and a write streams its reader onto the wire under an explicit Content-Length. Manifests are the exception, because they are small and both directions need the whole document in hand to digest it.
Every failure this package returns is classified for the retry policy before it leaves, and this package never retries anything itself. A 429 or a 5xx, and any request that never got a response, come back marked transient, carrying the wait a Retry-After header asked for; every other unexpected status comes back plain, and so does a request whose own context had already ended — that is the transfer stopping, not the registry failing. A blob read's body is wrapped so a connection that breaks part way through a part is marked the same way, and so is a manifest body that dies mid-read. The core therefore decides whether to try again without knowing that HTTP, status codes, or connections exist, and the attempt budget it spends is the only one there is.
Authentication is a pre-condition of a request rather than a recovery from one. A repository holds what the registry challenged with and the token acquired for each scope, the request builder stamps the Authorization header while it builds a request, and every request — the ordinary ones and the token exchanges alike — goes out through the caller's client, so nothing watching that client is blind to any of them. A registry that never challenges costs nothing at all: no probe, no header, and not one extra request.
Two things make this package send a request a second time, and both are the registry demanding a different request rather than failing to answer one: a challenge, answered when it changed what the request would carry and the standard library says the body can be produced again, and a redirect, re-issued as the paragraph below describes. A blob upload's body cannot be produced again, by construction, so a refusal in the middle of one comes back marked worth repeating and the orchestrator opens the file again. No identical request is ever repeated.
Registries that keep their blob content in object storage answer a read with a location rather than with bytes, and this package follows that location itself instead of letting net/http do it. Automatic following is off for every request; a read is re-issued up to three times, each time as a request built from nothing and carrying three headers and no more. The credential goes along only when the location is the registry itself — same scheme, same host, same port — so a request to signed storage arrives with no Authorization header and no cookie, and a location is never stored: the next attempt asks the registry again and follows whatever fresh location it sends. Beyond the registry, a refusal reads differently: a signature that has expired looks like a 403 or a 404 and is worth another attempt, and never means the caller's credentials or a missing artifact. No error this package returns names a signed location; every one of them names the registry request it started as.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNotFound = errors.New("not found")
ErrNotFound reports that the registry does not hold what a request named: a manifest, or a blob Blobs.Get tried to read. Blobs.Exists never returns it, because "the registry does not hold this blob" is the answer that call asks for rather than a failure. A 404's StatusError matches it through errors.Is; nothing wraps the sentinel in a second layer.
var ErrTooLarge = ociblob.ErrTooLarge
ErrTooLarge reports that the registry refused a request as larger than it accepts. A 413 registry error matches it through errors.Is, the same way a 404 matches ErrNotFound.
A 413 is how a registry states its layer cap. bigoci ships no table of vendor limits — the caps differ per registry, they move, and a stale table is worse than none — so the limit is discovered by being told about it, once, by the registry that enforces it.
The mapping is the status and nothing else, which means a 413 answering a manifest write would match too. That is admitted rather than guarded against: a bigoci manifest is a few hundred kilobytes at the format's own part cap, no registry rejects one as too large, and sniffing the body to tell the two apart would be the vendor table again in a different shape.
ErrUnauthorized reports that the registry refused a request rather than answering it: it wants credentials the request did not carry, or the ones it carried do not reach what the request asked for. A 401 or 403 registry error matches it through errors.Is, the same way a 404 matches ErrNotFound; nothing wraps the sentinel in a second layer.
The sentinel means "your credentials" and nothing else, which is what makes it worth branching on: every failure it names is answered by logging in or by being granted access, and no further attempt changes either.
Admitting the 403 beside the 401 is deliberate — a registry refuses a credential it could not read, or one that falls short of the access asked for, with either status. The cost is that a 403 from something standing in front of the registry, a proxy or a web application firewall, reports as unauthorized too. That is admitted the way ErrTooLarge admits a 413 answering a manifest write: sniffing the body to tell the two apart would be the vendor table again in a different shape.
Functions ¶
This section is empty.
Types ¶
type Blobs ¶
type Blobs struct {
// contains filtered or unexported fields
}
Blobs reads and writes the blobs of one repository. It implements the transfer package's Blobs port and comes from Repository.Blobs.
Blobs is immutable after construction and safe to use concurrently across every part of one transfer.
func (*Blobs) Exists ¶
Exists reports whether the repository holds the blob dgst names.
A 404 is the registry answering the question, so a blob it does not hold is (false, nil) rather than an error. That reading stops at the registry's own origin: a registry that redirects the check to storage has already decided the blob exists, so a 404 from the location it named is a stale signature rather than an answer, and comes back as an error worth another attempt — matching neither ErrNotFound nor ErrUnauthorized. This is the check a push makes for every part before uploading it, and on a first push the answer is "no" every time. Any other unexpected status is an error naming the method, path, and status.
func (*Blobs) Get ¶
func (b *Blobs) Get(ctx context.Context, dgst digest.Digest, off int64) (io.ReadCloser, int64, error)
Get streams the blob dgst names from off, and reports the offset the stream it returns actually starts at. An off of zero reads the whole blob.
The returned reader is the response body itself, unread: no blob content passes through a buffer on the way to the caller, who owns the reader and closes it. The body is wrapped only to classify its failures, so a connection that breaks part way through a blob is reported as worth another attempt even though Get itself has long since returned.
A nonzero off asks for the remainder of the blob with a Range request. The distribution spec does not require registries to honor one, so a registry that answers 200 with the whole blob instead of 206 with the requested range is reported rather than refused: the reported offset is zero, the body is a blob read like any other, and what to do about starting over belongs to the orchestrator that asked. A 206 whose range starts at some third byte is still an error — that is an answer to a question nobody asked.
A blob the registry does not hold is an error wrapping ErrNotFound. A 404 from a storage location the registry redirected to is not that answer: it reads as a stale signature, comes back marked worth another attempt, and matches no sentinel. Public errors use a "<digest>" path label: a digest was selected by the manifest and can itself be a reusable Bearer token.
The request asks for identity coding so the bytes the caller hashes are the stored bytes. A response that arrives under any other content coding is refused before its status or body is read, and the body is closed.
func (*Blobs) Put ¶
func (b *Blobs) Put( ctx context.Context, dgst digest.Digest, size int64, r io.Reader, wire transfer.WireProgress, ) error
Put uploads size bytes read from r as the blob dgst names.
The upload is monolithic: one POST opens a session, and one PUT streams the content into it and names the digest. That pair is the only push primitive every registry implements correctly, which is why bigoci splits a file into parts small enough to push this way instead of reaching for chunked or resumable uploads.
The upstream client keeps memory bounded to its transport staging buffer rather than buffering a part. wire receives only bytes the HTTP transport consumed, including failed attempts, and stops before Put returns.
type Credential ¶
type Credential struct {
// Username is the account name presented to the token endpoint.
Username string
// Password is the secret, or the personal access token, that goes with
// Username.
Password string
// IdentityToken is the OAuth2 refresh token some logins store in place of
// a password. bigoci cannot exchange one, and reads the field so it can
// say so rather than quietly presenting nothing at all.
IdentityToken string
// RegistryToken is a bearer token to present verbatim, with no exchange.
RegistryToken string
}
Credential is what bigoci presents to one registry. It mirrors the shape a Docker configuration file stores, because that file is where a credential usually comes from.
The zero value is the anonymous credential, which is a credential and not the absence of one: bigoci still performs the token exchange with it, because registries that require a bearer token for public reads answer an unauthenticated token request with a public-access token.
func (Credential) Empty ¶
func (c Credential) Empty() bool
Empty reports whether the credential carries nothing to present, which is the anonymous credential.
type Credentials ¶
type Credentials interface {
// Credential returns what bigoci should present to registry.
Credential(ctx context.Context, registry Registry) (Credential, error)
}
Credentials resolves the credential bigoci should present to one registry.
A registry the resolver knows nothing about is the zero Credential and a nil error: anonymous is an answer, not a failure. An error means the lookup itself could not be performed — an unreadable configuration file, a credential helper that would not run — and it ends the transfer, because a transfer that quietly fell back to anonymous would fail later and somewhere less obvious.
Implementations must be safe for concurrent use and must not retry: a lookup runs inside an attempt the orchestrator is already counting.
type ExternalTransportBase ¶
type ExternalTransportBase struct {
// contains filtered or unexported fields
}
ExternalTransportBase is the caller-derived transport and proxy policy shared by every repository one public bigoci client builds. The caller's original transport keeps same-registry behavior and pooling; its concrete clone owns one reusable guarded cross-host connection pool. An explicitly unverified repository uses the original transport for cross-host requests as well, preserving caller-owned behavior that Clone cannot carry.
This type is exported only across bigoci's internal package boundary. The root package prepares it lazily so the public Client's zero value remains usable.
func NewExternalTransportBase ¶
func NewExternalTransportBase(next http.RoundTripper) *ExternalTransportBase
NewExternalTransportBase derives one reusable guarded-transport base from next. A nil next selects http.DefaultTransport.
type Manifests ¶
type Manifests struct {
// contains filtered or unexported fields
}
Manifests reads and writes the manifest of one repository. It implements the transfer package's Manifests port and comes from Repository.Manifests.
How the endpoints are addressed is fixed when the repository is built rather than passed on every call, which keeps the reference grammar out of the core: the core asks for "the manifest" and this adapter knows which one that is. A repository built by NewRepository is bound to the tag or digest the reference carried. A repository built by NewDigestPushRepository publishes at the digest of the body Put is given and has no bound fetch target.
func (*Manifests) Get ¶
Get fetches the manifest the bound reference resolves to and returns its raw bytes with a descriptor for them.
The descriptor's digest is computed from the bytes that arrived. Registries also send the digest in a header, but a digest a caller verifies content against cannot come from the same response the content did. When the repository was built from a digest reference, Get also checks the bytes against that digest and fails when they disagree.
The body is read under a 4 MiB limit rather than into whatever the far end sends, and a manifest above the limit is an error. The media type is the response's Content-Type and the size is the length of the returned bytes.
The request asks for identity coding so the bytes hashed here are the stored bytes. A response that arrives under any other content coding is refused before its status or body is read.
A reference the registry does not resolve is an error wrapping ErrNotFound.
A repository built by NewDigestPushRepository has no bound tag or digest to fetch. Get is a misuse of that repository and fails without talking to the registry.
func (*Manifests) Put ¶
Put writes body as the manifest at the bound reference and returns the digest of body.
The digest is the one the registry stores the manifest under, and because it is computed here from the same bytes that went on the wire, a caller can bind a signature or an index entry to it without trusting the registry to report it back.
A repository built by NewDigestPushRepository has no bound tag or digest. Put computes the digest of body before constructing the request and writes to manifests/<digest>. A repository built by NewRepository still writes at the tag or digest the reference named.
type Option ¶
type Option func(*settings)
Option configures a Repository as it is built. The set is deliberately small: everything else about a repository comes from the reference it is built from.
func WithCredentials ¶
func WithCredentials(c Credentials) Option
WithCredentials answers a registry's challenges with what c resolves for the registry this repository is on. A nil Credentials is ignored, so a caller may pass one through unconditionally.
Leaving it unset does not turn authentication off. A registry that challenges still gets the full bearer exchange, made with the anonymous credential, because that is what registries which require a token for public reads expect. It only means bigoci has no user name or secret to offer when the exchange asks for one.
func WithExternalTransportBase ¶
func WithExternalTransportBase(base *ExternalTransportBase) Option
WithExternalTransportBase shares base with this repository's registry-selected external requests. The root package supplies it so one public Client keeps one external connection pool across transfers.
func WithHTTPClient ¶
WithHTTPClient sends the repository's requests with client instead of the default one. A nil client is ignored, so a caller may pass one through unconditionally. The public package's option of the same name carries the rationale.
func WithPlainHTTP ¶
func WithPlainHTTP() Option
WithPlainHTTP talks http:// to the registry instead of https://, for local registries and test fixtures. Nothing else should use it.
func WithUnverifiedExternalTransport ¶
func WithUnverifiedExternalTransport() Option
WithUnverifiedExternalTransport authorizes registry-selected cross-host requests through a custom dial hook, proxy, or opaque transport whose final destination this adapter cannot verify. The public option of the same name documents the security boundary callers opt out of.
type Registry ¶
type Registry string
Registry names one registry by host, with a port when the reference that named it carried one: "ghcr.io", "127.0.0.1:5000".
It is the key a credential is looked up under, and it is always the host bigoci dialed. A bearer challenge names the same registry again in the issuer's own vocabulary, in its service parameter, and that name is never used as a lookup key: a registry that could choose which credential bigoci presents would be choosing which secret leaves the machine.
type Repository ¶
type Repository struct {
// contains filtered or unexported fields
}
Repository is one repository on one registry: the endpoint prefix every request shares, plus how the manifest endpoints are addressed.
Built by NewRepository, the endpoints are bound to the tag or digest the reference carried. Built by NewDigestPushRepository, Put publishes at the digest of the body it is given and Get is a misuse.
A Repository is immutable once a constructor returns it and is safe to use from several goroutines at once, which is what lets one repository carry a transfer's parts in parallel.
func NewDigestPushRepository ¶ added in v0.2.0
func NewDigestPushRepository(ref string, opts ...Option) (*Repository, error)
NewDigestPushRepository returns a repository that publishes manifests at the digest of the body Manifests.Put is given.
ref is a reference in the registry/repo grammar, parsed by github.com/distribution/reference, the canonical implementation of that grammar. The registry is required and the name must already be canonical: bigoci pushes where it is told, so a familiar Docker Hub short name such as "ubuntu" is rejected rather than quietly expanded. The reference must carry neither a tag nor a digest. A tag would bind the write to a name NewRepository already handles, and a digest would claim a body that does not exist until Put computes it. The digest the registry stores the manifest under is computed from the bytes on the wire, before the PUT is constructed, and the write is addressed at manifests/<digest>.
Manifests.Get is a misuse of a repository built this way and fails without talking to the registry: there is no bound tag or digest to fetch. Blob requests use neither a tag nor a digest, and are unchanged.
The repository talks https with a client built on net/http.DefaultTransport. WithPlainHTTP and WithHTTPClient change that. Authentication and the two derived clients are the same as NewRepository.
func NewRepository ¶
func NewRepository(ref string, opts ...Option) (*Repository, error)
NewRepository returns the repository ref names.
ref is a reference in the registry/repo[:tag][@digest] grammar, parsed by github.com/distribution/reference, the canonical implementation of that grammar. The registry is required and the name must already be canonical: bigoci pushes and pulls where it is told, so a familiar Docker Hub short name such as "ubuntu:latest" is rejected rather than quietly expanded.
The reference must carry a tag or a digest, because the manifest endpoints address one of the two and every push and pull names a manifest. A reference carrying both is bound to its digest: the digest names exactly one manifest, while the tag beside it is only a claim about where that tag pointed. A reference that names a digest also makes Manifests.Get verify what it fetched against it. Blob requests use neither. A repository that publishes by digest, with no tag or digest in the reference, is built by NewDigestPushRepository.
The repository talks https with a client built on net/http.DefaultTransport. WithPlainHTTP and WithHTTPClient change that.
Two clients are derived from whichever one is in force, and the caller's own is never touched: net/http.Client is four exported fields and no hidden state, so copying the struct copies all of it. One turns redirect following off, which is how this package rather than net/http decides what a re-issued request may carry; the other drops the cookie jar as well and sends requests to token realms, redirect targets, and off-origin upload sessions. The external client binds the caller-derived guarded transport to this registry, preserving its TLS, dial, proxy, and pooling settings while enforcing the destination boundary at the actual connection. Repositories built by one public Client share that transport's external connection pool.
func (*Repository) Blobs ¶
func (r *Repository) Blobs() *Blobs
Blobs returns the adapter for this repository's blob endpoints.
func (*Repository) Manifests ¶
func (r *Repository) Manifests() *Manifests
Manifests returns the adapter for this repository's manifest endpoints. A repository built by NewRepository is bound to the tag or digest the reference carried. A repository built by NewDigestPushRepository publishes at the digest of the body Put is given.
type StatusError ¶
type StatusError struct {
// Method is the HTTP method of the request that failed.
Method string
// Path is the safe structural request path on the registry. A blob read
// replaces its manifest-selected digest with "<digest>". A token endpoint
// failure uses the fixed label "token endpoint". A redirected request is
// reported against the original registry path, never against the location
// it was sent to.
Path string
// Status is the HTTP status code the registry answered with.
Status int
// RetryAfter is how long the registry asked the caller to wait before
// trying again, read raw from the Retry-After header a 429 or a 503
// carries. It is zero when the response sent no header or sent one that
// cannot be read. Nothing is trimmed or dropped here: this field is what
// the registry said, and deciding how much of it to obey belongs to the
// retry policy, which bounds every wait by its own cap.
RetryAfter time.Duration
// Detail is the bounded start of the response body, when it carried one.
// It is retained for programmatic diagnosis through [errors.As] but never
// rendered by [StatusError.Error]: a registry can reflect the Authorization
// header it just received, making this field live credential material.
Detail string
}
StatusError reports a response whose HTTP status the repository adapter did not expect. Internal callers that must react to a specific status read StatusError.Status through errors.As instead of parsing the message.
func (*StatusError) Error ¶
func (e *StatusError) Error() string
Error renders only the method, path, and status. StatusError.Detail is deliberately excluded because it is peer-controlled and can contain a reusable credential reflected from the request. A wait the registry asked for is also left out: it is bookkeeping for the retry loop, not something a person reading a failure needs.
func (*StatusError) Is ¶
func (e *StatusError) Is(target error) bool
Is makes a 404 match ErrNotFound, a 401 or a 403 match ErrUnauthorized, and a 413 match ErrTooLarge under errors.Is without a second wrapping layer, so a not-found failure says "not found" once — in the status text the message already carries — instead of stacking the phrase at every boundary.
Source Files
Directories