oci

README

oci

Freeze a block-volume image into an immutable, content-addressed OCI artifact (push to a registry) and re-open it as a read-only block backing (pull from a registry). Pure Go, CGO_ENABLED=0, standard library only (plus the github.com/go-volumes/interface contract).

A volume is sliced into fixed-size chunks; each chunk is sha256-addressed and pushed once. Identical chunks — most importantly the all-zero holes of a sparse image — collapse to a single blob, so a mostly-empty disk freezes to a handful of blobs regardless of its logical size. The recovered image serves reads by pulling chunks on demand (LRU byte cache) and rejects every write, so it plugs straight into github.com/go-volumes/pool's OpenWith as a genuinely immutable backing.

Packages

• registry — a minimal, dependency-free OCI Distribution v2 client over net/http: monolithic blob upload, HEAD-based dedup, manifest get/put, and anonymous / HTTP Basic / Docker–OCI bearer-token auth. sha256 digests; every pulled blob is verified.
• oci (root) — Freeze, OpenReadOnly, and Overlay (a writable layer over a frozen image).

Use

// Freeze a read-only volume into a registry as an OCI artifact.
mfDigest, err := oci.Freeze(ctx, src /* volume.ReadOnly */, client, "myimage:v1",
    oci.Options{ChunkSize: 4 << 20})

// Re-open it as a read-only block backing and mount a pool on it.
img, err := oci.OpenReadOnly(ctx, client, "myimage:v1")
p, err := pool.OpenWith(img) // img is volume.ReadOnly AND pool.Backing

Writable overlay

OpenReadOnly gives an immutable *Image; NewOverlay wraps it in a read-write layer that buffers writes in memory (the base is never mutated) and merges them over base reads. pool.OpenWith(overlay) therefore yields a fully read-write pool over a frozen, content-addressed base — including the pool's own metadata writes, which the overlay absorbs like any other write.

img, err := oci.OpenReadOnly(ctx, client, "myimage:v1")
ov := oci.NewOverlay(img)
p, err := pool.OpenWith(ov)

// ... mutate through p ...

// Snapshot only the changed chunks into a new, versioned artifact; unchanged
// chunks are reused by digest from the base — no re-pull, no re-push.
newDigest, err := ov.Commit(ctx, client, "myimage:v2")

Overlay.Sync is a no-op — durability is the explicit Overlay.Commit, which pushes blobs only for the chunks that changed since the base and returns the new manifest digest. Reopen the new tag with OpenReadOnly to branch another overlay from it.

Artifact shape

• Manifest (application/vnd.oci.image.manifest.v1+json): artifactType and config.mediaType are application/vnd.go-volumes.pool-image.v1+json; layers are the deduped set of chunk-blob descriptors (application/vnd.go-volumes.pool-image.chunk.v1).

• Config blob (JSON):

  {"mediaType":"application/vnd.go-volumes.pool-image.v1+json",
   "size":N,"chunkSize":C,"chunks":["sha256:…","",...]}
  

  chunks is the ordered per-chunk digest index; "" marks an all-zero hole that is never stored as a blob.

License

BSD-3-Clause — see LICENSE.

Documentation

Overview

Package oci freezes a block volume image into an immutable, content-addressed OCI artifact (Freeze, push to a registry) and re-opens it as a read-only block backing (OpenReadOnly, pull from a registry).

A volume is sliced into fixed-size chunks; each chunk is sha256-addressed and pushed once. Identical chunks — most importantly the all-zero holes of a sparse image — collapse to a single blob, so a mostly-empty disk freezes to a handful of blobs regardless of its logical size. The recovered *Image serves reads by pulling chunks on demand (with an LRU byte cache) and rejects every write, so it can be handed to pool.OpenWith as a genuinely immutable backing.

Index

• Constants
• Variables
• func Freeze(ctx context.Context, src volume.ReadOnly, dst *registry.Client, ref string, ...) (string, error)
• type Config
• type Image
  • func OpenReadOnly(ctx context.Context, src *registry.Client, ref string) (*Image, error)
  • func (im *Image) Close() error
  • func (im *Image) ReadAt(p []byte, off int64) (int, error)
  • func (im *Image) Size() (int64, error)
  • func (im *Image) Sync() error
  • func (im *Image) Truncate(size int64) error
  • func (im *Image) WriteAt(p []byte, off int64) (int, error)
• type Options
• type Overlay
  • func NewOverlay(base *Image) *Overlay
  • func (o *Overlay) Close() error
  • func (o *Overlay) Commit(ctx context.Context, dst *registry.Client, ref string) (string, error)
  • func (o *Overlay) ReadAt(p []byte, off int64) (int, error)
  • func (o *Overlay) Size() (int64, error)
  • func (o *Overlay) Sync() error
  • func (o *Overlay) Truncate(size int64) error
  • func (o *Overlay) WriteAt(p []byte, off int64) (int, error)

Constants

const ChunkMediaType = "application/vnd.go-volumes.pool-image.chunk.v1"

ChunkMediaType is the media type of each chunk layer descriptor.

const ConfigMediaType = "application/vnd.go-volumes.pool-image.v1+json"

ConfigMediaType is the artifact type carried by the manifest config blob, marking this image as a go-volumes frozen pool image rather than a runnable container.

const DefaultChunkSize = 4 << 20 // 4 MiB

DefaultChunkSize is the chunk window used when Options.ChunkSize is zero.

Variables

var ErrReadOnly = errors.New("oci: read-only frozen image")

ErrReadOnly is returned by every mutating method of an *Image: it is a frozen, content-addressed artifact and cannot be written through.

Functions

func Freeze

func Freeze(ctx context.Context, src volume.ReadOnly, dst *registry.Client, ref string, opts Options) (string, error)

Freeze reads src in fixed-size chunk windows, pushes each UNIQUE chunk to dst once (deduplicating by content — all-zero holes collapse to nothing stored), builds a config blob carrying the geometry and ordered chunk-digest index, and pushes an OCI image manifest tagged ref whose layers are the deduped chunk blobs. It returns the manifest digest.

Types

type Config

type Config struct {
	// MediaType identifies the payload format.
	MediaType string `json:"mediaType"`
	// Size is the logical image size in bytes.
	Size int64 `json:"size"`
	// ChunkSize is the fixed chunk window in bytes (a power of two).
	ChunkSize int64 `json:"chunkSize"`
	// Chunks is the ordered per-chunk digest index, one entry per chunk window
	// covering [0, Size). "" means an all-zero (hole) chunk.
	Chunks []string `json:"chunks"`
}

Config is the JSON config blob: the image geometry plus the ordered list of per-chunk digests. Reading chunk i means fetching Chunks[i]; an empty string denotes an all-zero chunk that need not be stored at all.

type Image

type Image struct {
	// contains filtered or unexported fields
}

Image is a read-only block backing recovered from a frozen OCI artifact. It serves ReadAt by pulling the covering chunk blobs on demand (an LRU byte cache avoids re-pulling hot chunks); hole chunks read as zeros. It satisfies both volume.ReadOnly and pool.Backing, and rejects all writes with ErrReadOnly so pool.OpenWith gets a genuinely immutable store.

func OpenReadOnly

func OpenReadOnly(ctx context.Context, src *registry.Client, ref string) (*Image, error)

OpenReadOnly pulls and parses the manifest tagged ref, recovers the config (size, chunk size, ordered chunk-digest index) and returns an *Image serving reads from src.

func (*Image) Close

func (im *Image) Close() error

Close releases the image. The underlying registry client is the caller's.

func (*Image) ReadAt

func (im *Image) ReadAt(p []byte, off int64) (int, error)

ReadAt assembles len(p) bytes at off from the covering chunks, pulling each by digest (cached) and treating hole chunks as zeros. A read past the end returns the bytes available and io.EOF, matching io.ReaderAt semantics.

func (*Image) Size

func (im *Image) Size() (int64, error)

Size reports the logical image size in bytes.

func (*Image) Sync

func (im *Image) Sync() error

Sync always fails: a frozen image is read-only, so any attempt to commit buffered state is rejected. A read-only consumer that never mutates never calls Sync.

func (*Image) Truncate

func (im *Image) Truncate(size int64) error

Truncate always fails: a frozen image is read-only.

func (*Image) WriteAt

func (im *Image) WriteAt(p []byte, off int64) (int, error)

WriteAt always fails: a frozen image is read-only.

type Options

type Options struct {
	// ChunkSize is the fixed chunk window in bytes; it must be a power of two.
	// Zero selects DefaultChunkSize.
	ChunkSize int64
}

Options configures Freeze.

type Overlay

type Overlay struct {
	// contains filtered or unexported fields
}

Overlay is a read-write block backing layered over an immutable OCI base Image. Writes are buffered in memory — the base is never mutated — and reads merge the dirty overlay over the base (a never-written region falls through to the base, or reads zero past it). Overlay.Sync is a no-op: durability is the explicit Overlay.Commit, which snapshots the current state into a new versioned OCI artifact and pushes blobs ONLY for the chunks that changed.

Because the overlay absorbs every write — including a pool's own metadata writes — pool.OpenWith(overlay) yields a fully read-write pool over a frozen, content-addressed base: edit, then Commit to a new tag.

func NewOverlay

func NewOverlay(base *Image) *Overlay

NewOverlay wraps an immutable base Image in a writable overlay, starting at the base's size and chunk size.

func (*Overlay) Close

func (o *Overlay) Close() error

Close releases the overlay; the base Image and its registry client remain the caller's to close.

func (*Overlay) Commit

func (o *Overlay) Commit(ctx context.Context, dst *registry.Client, ref string) (string, error)

Commit snapshots the overlay's current state into a NEW immutable OCI artifact tagged ref on dst, pushing blobs ONLY for the chunks that changed since the base: an unchanged chunk reuses the base's digest verbatim — no pull, no re-hash, no re-push — so a commit costs the size of the delta, and unchanged chunks are shared by digest across every tag. It returns the new manifest digest. The overlay and its base are left unchanged; reopen ref with OpenReadOnly (e.g. to branch a new overlay from it).

dst may be the base's registry (versioned tags in one repo) or a different one (push the snapshot elsewhere); when it is the base's registry the unchanged chunk blobs are already present, so only the delta uploads.

func (*Overlay) ReadAt

func (o *Overlay) ReadAt(p []byte, off int64) (int, error)

ReadAt assembles len(p) bytes at off, merging the dirty overlay over the base. A read past the current size returns the available bytes and io.EOF.

func (*Overlay) Size

func (o *Overlay) Size() (int64, error)

Size reports the current logical size, which may exceed the base after a grow.

func (*Overlay) Sync

func (o *Overlay) Sync() error

Sync is a no-op: the overlay is in memory and durability is provided by Overlay.Commit.

func (*Overlay) Truncate

func (o *Overlay) Truncate(size int64) error

Truncate sets the logical size, dropping dirty chunks wholly beyond it on a shrink (base chunks past the size become invisible too). A grow is sparse.

func (*Overlay) WriteAt

func (o *Overlay) WriteAt(p []byte, off int64) (int, error)

WriteAt writes p at off into the in-memory overlay (read-modify-write per chunk), growing the logical size when it writes past the current end.