clipperoci

Documentation

Index

• Constants
• func ConvertLayer(ctx context.Context, data []byte, mediaType string, progress Progress) (*TOC, []Blob, error)
• type Blob
• type Progress
• type TOC
  • func ConvertLayerToChunked(ctx context.Context, compressedData []byte, mediaType string, ...) (*TOC, error)
  • func ParseTOC(data []byte) (*TOC, error)
  • func (t *TOC) ChunkDigests() []digest.Digest
  • func (t *TOC) UncompressedSize() int64
• type TOCChunk
• type TOCEntry

Constants

const (
	// MediaTypeLayerTOC is the media type for chunked layer TOC blobs.
	MediaTypeLayerTOC = "application/vnd.oci.image.layer.toc.v1+json"

	// OriginalDigestAnnotation is the annotation key for the original layer digest.
	OriginalDigestAnnotation = "org.opencontainers.image.layer.original-digest"

	// UncompressedSizeAnnotation is the annotation key for the total uncompressed
	// size of all regular files in the layer, in bytes.
	UncompressedSizeAnnotation = "org.opencontainers.image.layer.uncompressed-size"
)

const (
	// SmallFileThreshold is the maximum file size eligible for tar compaction.
	// Files smaller than this are aggregated into a single compressed tar blob.
	SmallFileThreshold = 1 << 20 // 1 MiB
)

Functions

func ConvertLayer

func ConvertLayer(ctx context.Context, data []byte, mediaType string, progress Progress) (*TOC, []Blob, error)

ConvertLayer converts a compressed layer to chunked format, collecting all chunk blobs in memory. Duplicate blobs are deduplicated by digest. For streaming conversions (e.g. pushing directly to a blob store), use ConvertLayerToChunked with a custom blobPusher instead. If progress is nil, progress events are discarded.

Types

type Blob

type Blob struct {
	Digest digest.Digest
	Data   []byte
}

Blob is a content-addressed chunk blob produced during layer conversion.

type Progress

type Progress interface {
	// ConvertingLayer is called once when layer decompression begins.
	ConvertingLayer(mediaType string, compressedBytes int64)
	// LargeFile is called for each file ≥ SmallFileThreshold that becomes its own blob.
	LargeFile(name string, size int64, digest digest.Digest)
	// CompactTar is called after the small-files tar blob is created.
	CompactTar(fileCount int, uncompressedBytes int64, compressedBytes int64, digest digest.Digest)
	// Done is called when conversion is complete.
	Done(totalEntries int, largeFiles int, smallFiles int, hardlinks int)
}

Progress receives conversion progress events. Implementations must be safe for use from a single goroutine (no concurrent calls are made).

type TOC

type TOC struct {
	Version int        `json:"version"`
	Entries []TOCEntry `json:"entries"`
}

TOC is the top-level table of contents for a chunked layer.

func ConvertLayerToChunked

func ConvertLayerToChunked(ctx context.Context, compressedData []byte, mediaType string,
	blobPusher func(ctx context.Context, data []byte) (digest.Digest, error), progress Progress) (*TOC, error)

ConvertLayerToChunked decompresses a gzip or zstd tar layer and converts it to the chunked format. Large files (≥ SmallFileThreshold) become individual chunk blobs. Small files are aggregated into a single zstd-compressed tar blob appended at the end of the TOC.

blobPusher is called for each blob to store; it must return the blob's digest. If progress is nil, progress events are discarded. Returns the completed TOC.

func ParseTOC

func ParseTOC(data []byte) (*TOC, error)

ParseTOC parses a TOC from JSON-encoded bytes.

func (*TOC) ChunkDigests

func (t *TOC) ChunkDigests() []digest.Digest

ChunkDigests returns all blob digests that must exist for this TOC to be valid. For "reg" entries: the file digest (or per-chunk digests for multi-chunk files). For "tar" entries: the compressed blob digest (or uncompressed digest if no compression).

func (*TOC) UncompressedSize

func (t *TOC) UncompressedSize() int64

UncompressedSize returns the total uncompressed size of all file content in the TOC.

type TOCChunk

type TOCChunk struct {
	// Digest is the content-addressed digest of this chunk blob.
	Digest digest.Digest `json:"digest"`

	// Size is the byte length of this chunk.
	Size int64 `json:"size"`
}

TOCChunk describes one chunk blob of a multi-chunk regular file.

type TOCEntry

type TOCEntry struct {
	// Type is the entry type: "reg", "dir", "symlink", "hardlink", "char", "block",
	// "fifo", "whiteout", "opaque", "tar".
	Type string `json:"type"`

	// Name is the path of the entry within the layer. Not present for "tar" entries.
	Name string `json:"name,omitempty"`

	// Size is the uncompressed size in bytes. For "reg" entries, the file size.
	// For "tar" entries without compression, the tar blob size.
	// For "tar" entries with compression, the compressed blob size.
	Size int64 `json:"size,omitempty"`

	// Mode is the file mode and permission bits.
	Mode uint32 `json:"mode,omitempty"`

	// UID is the numeric user ID of the owner.
	UID int `json:"uid,omitempty"`

	// GID is the numeric group ID of the owner.
	GID int `json:"gid,omitempty"`

	// Uname is the user name of the owner.
	Uname string `json:"uname,omitempty"`

	// Gname is the group name of the owner.
	Gname string `json:"gname,omitempty"`

	// ModTime is the modification time of the entry.
	ModTime time.Time `json:"modtime,omitempty"`

	// LinkName is the target of a symlink or the source name of a hardlink.
	LinkName string `json:"linkName,omitempty"`

	// DevMajor is the major device number for char/block device entries.
	DevMajor int64 `json:"devMajor,omitempty"`

	// DevMinor is the minor device number for char/block device entries.
	DevMinor int64 `json:"devMinor,omitempty"`

	// Xattrs holds extended attributes for the entry.
	Xattrs map[string][]byte `json:"xattrs,omitempty"`

	// Digest is the content-addressed digest of the uncompressed content.
	// For "reg" entries: digest of the file content (also the blob digest for single-chunk files).
	// For "tar" entries: digest of the uncompressed tar blob (used for local dedup).
	Digest digest.Digest `json:"digest,omitempty"`

	// Chunks holds chunk descriptors when a file is split across multiple chunk blobs.
	Chunks []TOCChunk `json:"chunks,omitempty"`

	// Description is an informational string for "tar" entries (e.g., directory subtree
	// covered). Not used for extraction.
	Description string `json:"description,omitempty"`

	// Compression is the compression algorithm for "tar" entries (e.g., "zstd").
	// If absent, the tar blob is stored uncompressed.
	Compression string `json:"compression,omitempty"`

	// CompressedDigest is the digest of the compressed tar blob in the registry.
	// This is the blob digest used for fetching. Only present when Compression is set.
	CompressedDigest digest.Digest `json:"compressedDigest,omitempty"`

	// UncompressedSize is the uncompressed size of the tar blob in bytes.
	// Only present for "tar" entries when Compression is set.
	// When set, Size refers to the compressed size.
	UncompressedSize int64 `json:"uncompressedSize,omitempty"`
}

TOCEntry represents a single entry in the chunked layer TOC.