nixoci

README

nixoci

Push and pull a Nix store as an OCI image layer. nixoci serializes a populated /nix into a single layer, ships it to a registry like GHCR, and restores it on the other side. The point is caching: rebuilding a Nix environment from scratch takes minutes to hours, pulling a layer takes seconds.

It talks to registries through go-containerregistry directly rather than shelling out to crane. That started as a fix (crane's stdin handling re-gzipped pre-gzipped input and buffered to disk, which broke CI in creative ways) and ended up as the better design: deterministic encoding, true streaming, and one code path shared by tests and CI.

nixoci is used by devcell to cache container Nix volumes between CI runs. The API is Push, Pull, and ResolveImage; pin a version if you build on it.

go get github.com/devcell-sh/go-nixoci

Documentation

Overview

Package nixoci implements push/pull of a nix-store tarball as an OCI image layer, against a remote registry like GHCR.

Background: the CI workflow caches the populated `/nix` Docker volume between runs by serializing it as a single OCI layer. CELL-292 made this work via the `crane` CLI, but `crane append`'s stdin handler had behavior that caused six iterations of CI breakage (re-gzipping pre-gzipped input, buffering uncompressed stdin to disk, …). This package replaces those CLI calls with direct uses of `github.com/google/go-containerregistry/pkg/v1/{stream,tarball}`, giving deterministic encoding + true streaming + a single Go code path shared between local tests and CI.

Index

• Variables
• func ParseSize(s string) (int64, error)
• func Pull(ctx context.Context, srcRef, dstDir string, stripComponents int) error
• func PullToDockerVolume(ctx context.Context, srcRef, volName string, stripComponents int) error
• func Push(ctx context.Context, baseRef, dstRef string, r io.ReadCloser) error
• func PushFromVolume(ctx context.Context, volume, baseRef, dstRef string, opts PushOpts) error
• func ResolveImage(ctx context.Context, candidates ...string) (string, error)
• func WithRetry(ctx context.Context, maxAttempts int, fn func() error) error
• type PushOpts

Variables

var ChunkSize int64 = 512 << 20 // 512 MB

ChunkSize is the soft cap (in uncompressed tar bytes) at which Push starts a new OCI layer. Entries are never split mid-file — a single entry larger than ChunkSize produces a one-entry chunk. Set to 0 to disable chunking (single layer, original behavior).

var Debug bool

Debug enables verbose HTTP-level logging for registry operations. Set via DEVCELL_NIX_PUSH_DEBUG=1.

var ProgressTick = 5 * time.Second

ProgressTick controls how often Push emits an in-flight progress line. Set short in tests; CI uses the default.

var ProgressWriter io.Writer = os.Stderr

ProgressWriter receives periodic progress lines from Push. Defaults to os.Stderr so CI logs surface upload activity; tests can swap it for a buffer. Concurrent writes are serialized inside Push.

var RetryBaseDelay = 15 * time.Second

RetryBaseDelay is the base delay between retry attempts. Actual delay is attempt * RetryBaseDelay (linear backoff matching the Taskfile's sleep $((attempt * 15))). Tests set this short.

var StallTimeout = 120 * time.Second

StallTimeout is the maximum duration Push tolerates zero byte progress before aborting the upload. Set to 0 to disable. GHCR occasionally throttles multi-GB layer uploads to near-zero; without this the process blocks until externally SIGTERMed.

var TotalSizeHint int64

TotalSizeHint is the estimated uncompressed size (in bytes) of the tar being pushed. When > 0, progress lines show "sent X / Y" so the operator can gauge completion. Set from DEVCELL_NIX_TOTAL_SIZE.

var UploadJobs = 4

UploadJobs controls how many OCI layer blobs are uploaded concurrently by remote.Write. Higher values improve throughput but risk GHCR throttling; 16 caused session kills, 4 is a safe default.

Functions

func ParseSize

func ParseSize(s string) (int64, error)

ParseSize parses a human-readable size string (e.g. "1GB", "500MB", "1024KB") into bytes. Case-insensitive.

func Pull

func Pull(ctx context.Context, srcRef, dstDir string, stripComponents int) error

Pull resolves srcRef, downloads all non-base layers of the image, and extracts each gzipped tarball into dstDir. For multi-layer (chunked) images, layers[1:] are extracted in order; for single-layer images, layers[0] is extracted (backward compat with legacy cache images). Archive entries are written relative to dstDir, with stripComponents leading path elements stripped per `tar --strip-components=N` semantics. Symlinks and file modes are preserved.

func PullToDockerVolume

func PullToDockerVolume(ctx context.Context, srcRef, volName string, stripComponents int) error

PullToDockerVolume streams all non-base layers of srcRef into the named Docker volume by spawning `docker run -i alpine sh -c 'cd /dest && tar -x --strip-components=N'` for each layer and feeding the (gunzipped) tar stream over stdin. For multi-layer (chunked) images, layers[1:] are extracted in order; for single-layer images, layers[0] is extracted (backward compat).

stripComponents has the same meaning as Pull: leading path elements to strip from each archive entry.

func Push

func Push(ctx context.Context, baseRef, dstRef string, r io.ReadCloser) error

Push streams an uncompressed tar from r through to dstRef as one or more OCI tar+gzip layers atop baseRef.

When ChunkSize > 0 (default 512 MB), the input tar is split on entry boundaries into ~ChunkSize temp files. All chunks are then uploaded in parallel (UploadJobs concurrent blob streams, default 4) via a single remote.Write call — one manifest write at the end. This eliminates per-chunk manifest churn and overlaps gzip+upload across layers. Temp files are removed after the push completes.

When ChunkSize <= 0, the original single-layer streaming behavior is preserved: bytes flow r → gzip → registry with no disk staging.

func PushFromVolume

func PushFromVolume(ctx context.Context, volume, baseRef, dstRef string, opts PushOpts) error

PushFromVolume pushes the contents of a Docker volume to a registry as OCI layers. Handles size checking, retry with backoff, and tag aliasing — replacing the ~48-line shell pipeline in Taskfile's nix-cache:publish task.

func ResolveImage

func ResolveImage(ctx context.Context, candidates ...string) (string, error)

ResolveImage probes each candidate image reference in order and returns the first one whose manifest exists in the registry. Returns "" if none resolve. Empty strings in candidates are skipped.

func WithRetry

func WithRetry(ctx context.Context, maxAttempts int, fn func() error) error

WithRetry calls fn up to maxAttempts times. On failure, waits attempt * RetryBaseDelay before the next attempt (linear backoff). Returns the last error if all attempts fail. Respects context cancellation between attempts.

Types

type PushOpts

type PushOpts struct {
	TagAlias string // create alias tag after push via docker buildx imagetools create
	Retries  int    // total push attempts (0 or 1 = single attempt, no retry)
	MinSize  int64  // skip push if volume content < MinSize bytes (0 = always push)
}

PushOpts configures PushFromVolume behavior.