Documentation
¶
Overview ¶
Package oci pulls OCI image references and materializes their merged layers as a bootable ext4 disk suitable for use as a machine.RawDisk.
The conversion is a single stream — registry layers are flattened with full OCI whiteout semantics (explicit and opaque; see flatten.go) and fed straight into a user-space ext4 writer — so no privileged mounts, no e2fsprogs, and no intermediate unpacked tree on disk. Ownership, setuid/setgid bits, xattrs and device nodes survive, which an unprivileged untar-to-directory pipeline silently loses.
The pipeline is digest-keyed: calling Build twice with the same image digest reuses the cached disk without re-pulling. The cache layout is stable:
<CacheDir>/
layers/ # uncompressed layer blobs, shared across images
<digest>_<platform>_<size>/ # one per (image, platform, size floor)
disk.ext4 # the built rootfs
done # marker; absent until disk.ext4 is complete
Layer blobs are content-addressed, so images that share base layers (e.g. several toolchains on the same debian base) download and store each layer once.
Index ¶
- Constants
- func Key(ctx context.Context, opts Options) (string, error)
- func LocalTarballPath(ref string) string
- func Materialize(ctx context.Context, r machine.RootFS, dstPath string) (machine.RawDisk, error)
- func Resolve(ctx context.Context, r machine.RootFS) (machine.RawDisk, error)
- func ResolveDigest(ctx context.Context, ref, platform string) (string, error)
- func WriteDirDisk(srcDir, dst string, sizeBytes int64) error
- type LayerStatus
- type Options
- type Phase
- type ProgressUpdate
- type Result
Constants ¶
const ImageEnvPath = "/etc/clawk/image-env"
ImageEnvPath is where the image config's Env lands inside the built filesystem, one KEY=VALUE per line. Guest agents read it to give spawned processes the environment the image was built to run with.
Variables ¶
This section is empty.
Functions ¶
func Key ¶
Key returns the cache-directory name a Build with opts would use — without building anything. Registry refs resolve their digest over the network; tarball refs read it locally. `clawk image gc` computes the keep-set with this.
func LocalTarballPath ¶
LocalTarballPath reports whether ref names a local `docker save` tarball rather than a registry reference, returning the path (or "" for registry refs). Path shapes (absolute, ./relative) can never be valid registry refs, and a .tar suffix on a bare name is unambiguous enough in practice.
func Materialize ¶
Materialize produces a per-caller writable RawDisk at dstPath by resolving r and copy-on-writing the source into dstPath. Use this instead of Resolve when the VM may write to the disk and the source may be shared across multiple Machines.
If src and dstPath are the same file, no copy is performed — safe for callers that conditionally pre-materialize their rootfs.
func Resolve ¶
Resolve turns any machine.RootFS into the canonical on-disk path for its content. RawDisk values pass through unchanged; OCIImage values are materialized by calling Build.
The returned path may be shared across multiple Machines (an OCI digest cache, or a user-owned master image). Backends wanting a per-VM writable disk should use Materialize instead, which reflinks on top.
func ResolveDigest ¶
ResolveDigest resolves ref to its content digest: a registry HEAD for registry refs, a local manifest hash for tarballs. Diagnostics (clawk doctor) use it to answer "is this sandbox's image still reachable" without building anything.
func WriteDirDisk ¶
WriteDirDisk builds a writable ext4 image at dst holding the tree rooted at srcDir, padded to sizeBytes so the guest has room to allocate (the padding is a sparse hole; it costs no physical disk until written). The tree lands at the image root: srcDir/foo is /foo in the image.
Ownership, mode bits and symlinks are preserved as-is — including srcDir's own, so the mounted root belongs to whoever owns the source rather than to root — which keeps the semantics of the `cp -a` this replaced. Sockets and devices are skipped: a worktree has none, and neither survives a copy meaningfully. Hardlinks are written as independent regular files: git's object store has none (objects are distinct files), and duplicating is safer than emitting a link whose target the walk hasn't reached yet.
The image is built under a temporary name and renamed into place, so an interrupted build never leaves a plausible-looking partial disk behind.
Types ¶
type LayerStatus ¶
type LayerStatus struct {
// Index is the layer's 1-based position in the image manifest.
Index int
// CompressedSize is the layer's blob size from the manifest.
CompressedSize int64
// Downloaded is the compressed bytes pulled from the registry so far.
Downloaded int64
// Cached is true when the layer was already present locally and
// needed no download (Downloaded reads full from the first frame).
Cached bool
// Done is true once the layer is fully in the cache.
Done bool
}
LayerStatus is one layer's download progress during PhaseDownload.
type Options ¶
type Options struct {
// Ref is the OCI image reference (e.g. "docker.io/library/alpine:3.20").
Ref string
// CacheDir holds digest-keyed built disks and shared layer blobs.
// Required.
CacheDir string
// MinSizeMiB is the minimum filesystem size in MiB; the gap between
// content and this floor becomes free space the guest can write into
// without resize2fs or growpart. Content larger than the floor grows
// the filesystem past it. The padding is a sparse hole, so a large
// floor costs no physical disk. Default 1024.
MinSizeMiB int
// Platform forces a specific OCI platform ("linux/amd64", "linux/arm64").
// Empty picks the registry's default for the current arch.
Platform string
// Inject is a list of host files written into the filesystem on top of
// the image content (last-wins, like an extra layer). The injected
// content is hashed into the disk cache key.
Inject []machine.InjectFile
// Progress, if non-nil, reports build progress: once at PhaseStart,
// then per-layer during the parallel PhaseDownload, then roughly
// every 8 MiB of converted content during PhaseUnpack. Never called
// on a cache hit.
Progress func(ProgressUpdate)
}
Options control a Build.
func OptionsForImage ¶
OptionsForImage maps a machine.OCIImage rootfs spec to build Options. Every consumer of the spec must use this one mapping, or cache keys drift between the provider's pre-build and the backend's Materialize.
type Phase ¶
type Phase int
Phase identifies which stage of a Build a ProgressUpdate describes.
const ( // PhaseStart is reported once when a real build begins (never on a // cache hit), carrying the compressed-size scale and layer count. PhaseStart Phase = iota // PhaseDownload covers the concurrent pull of layer blobs into the // cache; its updates carry per-layer Downloads. PhaseDownload // PhaseUnpack covers flattening the cached layers into the ext4 disk, // which proceeds one layer at a time. PhaseUnpack )
type ProgressUpdate ¶
type ProgressUpdate struct {
// Phase is the build stage this update describes.
Phase Phase
// UnpackedBytes is the cumulative flattened content converted into
// the disk image so far. Reported during PhaseUnpack.
UnpackedBytes int64
// CompressedTotal is the sum of the image's compressed layer sizes
// from the manifest.
CompressedTotal int64
// Downloads holds one entry per image layer during PhaseDownload —
// the concurrent download's per-layer progress. Nil in other phases.
Downloads []LayerStatus
// Layer and Layers locate the layer currently being flattened
// (1-based) during PhaseUnpack. Zero before the first layer starts.
Layer, Layers int
}
ProgressUpdate is one build progress report.
type Result ¶
type Result struct {
// DiskPath is the built ext4 image. Use it as machine.RawDisk.Path.
DiskPath string
// Digest is the resolved image digest (the cache key).
Digest string
// UnpackedBytes is the size of the flattened filesystem tar that was
// converted. Zero on a cache hit.
UnpackedBytes int64
}
Result describes a built rootfs.
func Build ¶
Build pulls Opts.Ref, flattens its layers, and produces an ext4 disk. It is idempotent: a successful prior Build for the same digest is reused without re-pulling or re-creating the disk.
The disk in the cache is shared; callers handing it to a VM that may write must copy-on-write it first (see Materialize).
Source Files