Documentation
¶
Overview ¶
Package imagesource implements the container-image Source (ARCHITECTURE.md §7, decision D11): go-containerregistry resolves a v1.Image through the remote → daemon → tarball → OCI-layout fallback chain, and the squashed tar from mutate.Extract is streamed exactly once. Header reads and spool decisions execute in the walker goroutine because tar entry content is only valid during traversal; workers consume spooled buffers (≤4 MiB in memory, ≤64 MiB tmpfile, else header-only) — never the live stream.
The union of phase-2 ProjectDetector selectors is folded into the spool policy so cross-file detectors see their files in image scans, and large model files are tee-hashed during the mandatory discard copy, making content-hash identity for in-image weights free (§9.1). Net effect: a 40 GB GGUF inside an image costs a 32 KB header parse plus a hashing discard-copy — zero memory growth, zero disk.
Package imagesource implements the container-image Source (ARCHITECTURE.md §7). It resolves a squashed root filesystem from a container image and streams its files through the standard source contract.
Spooling and memory bounds (invariant P2) ¶
The squashed filesystem is produced by applying an image's layers top→base with whiteout/opaque resolution, exactly once. Because the underlying tar streams are consume-once, each effective file's bytes are captured (spooled) during that single pass so Entry.Open and the phase-2 Resolver can serve them afterward. Spooling is strictly bounded:
- Files up to MaxMemPerFile (default 4 MiB) are held in memory, subject to a global MemBudget (default 256 MiB).
- Larger files, or files that would exceed MemBudget, spill to a temp file up to MaxDiskPerFile (default 64 MiB), subject to a global DiskBudget (default 2 GiB).
- Files exceeding those caps are surfaced header-only: Entry.Open returns just the first HeaderCap bytes (default 64 KiB), enough for magic/header detectors, with the remainder discarded. Content-based detectors see a truncated read.
Memory and disk therefore stay bounded regardless of image size or content. Entry.ReaderAt always reports ErrNotSeekable — spooled streams are not randomly seekable through the contract.
Deviation from the original design ¶
The design called for go-containerregistry (remote.Image / daemon.Image / mutate.Extract). That library's transitive dependencies (opencontainers/*, klauspost/compress, ...) are absent from this module's go.sum, and the build constraints for this work forbid `go get`/`go mod`. This package therefore reads images with the standard library instead: docker-save archives, OCI image-layout archives, and OCI layout directories, with gzip and uncompressed layers. Consequences:
- New(ref) for a live registry or docker-daemon pull is NOT implemented; it accepts a local archive/layout path (delegating to NewFromTar) and otherwise returns a clear error.
- zstd-compressed layers are unsupported (recorded as an Unknown).
Once go-containerregistry is wired into go.mod, resolution can be swapped to mutate.Extract without changing the spooling/Walk/Resolver machinery below.
Index ¶
- Variables
- type Options
- type Source
- func (s *Source) Clean() error
- func (s *Source) Close() error
- func (s *Source) ID() string
- func (s *Source) Info() source.Info
- func (s *Source) Kind() source.Kind
- func (s *Source) Name() string
- func (s *Source) Resolver() source.Resolver
- func (s *Source) Walk(ctx context.Context, fn source.WalkFunc) error
- func (s *Source) WalkStats() source.WalkStats
- func (s *Source) WalkUnknowns() []source.Unknown
Constants ¶
This section is empty.
Variables ¶
var ErrNotSeekable = errors.New("imagesource: entries are not seekable (stream-backed)")
ErrNotSeekable is returned by Entry.ReaderAt: image files are served from a consume-once spool and are not randomly seekable through the contract.
Functions ¶
This section is empty.
Types ¶
type Options ¶
type Options struct {
// IgnoreGlobs are user --ignore doublestar patterns over image-root paths.
IgnoreGlobs []string
// Spool bounds; any zero field takes its default.
MaxMemPerFile int64
MaxDiskPerFile int64
MemBudget int64
DiskBudget int64
HeaderCap int64
// TmpDir is where spill/temporary files are created (default os.TempDir()).
TmpDir string
}
Options configures an image source.
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source is the image implementation of source.Source.
func New ¶
New prepares an image source from ref.
Because live registry/daemon resolution is not wired in (see the package doc), ref must be a local path to a container-image archive (docker-save or OCI image-layout tar) or an OCI layout directory. Anything else returns an error directing the caller to supply such a path.
func NewFromTar ¶
NewFromTar prepares an image source from a docker-save/OCI-archive tar file or an OCI image-layout directory at path.
func (*Source) Resolver ¶
Resolver returns the phase-2 query API. It serves fully-spooled files; header-only (oversized) files are visible to Stat/FilesByGlob but Open yields only their header prefix.
func (*Source) Walk ¶
Walk streams the squashed filesystem's regular files. Materialization (squash + spool) happens once here or via the Resolver; a materialization failure is a source-acquisition error (fatal), while per-file/per-layer problems degrade to Unknowns (invariant P6).
func (*Source) WalkStats ¶ added in v0.4.2
WalkStats returns the zero value: this source applies no ignore rules, so nothing is ever deliberately excluded from its walk.
func (*Source) WalkUnknowns ¶
WalkUnknowns returns the Unknown records accumulated during materialization (unreadable/whiteout-inconsistent/zstd layers, oversized truncations).
Source Files