cache

Documentation

Overview

Package cache provides content-addressed caching for blob archives.

This package is designed as an optional enhancement to the core blob library, adding caching capabilities for improved performance when reading files from remote archives.

The cache uses SHA256 hashes of uncompressed file content as keys, providing automatic deduplication across archives and implicit integrity verification on cache hits.

Index

• Constants
• type BlockCache
• type ByteSource
• type Cache
• type RangeReader
• type WrapConfig
  • func DefaultWrapConfig() WrapConfig
• type WrapOption
  • func WithBlockSize(n int64) WrapOption
  • func WithMaxBlocksPerRead(n int) WrapOption

Constants

const DefaultBlockSize int64 = 64 << 10

DefaultBlockSize is the default block size used by block caches.

const DefaultMaxBlocksPerRead = 4

DefaultMaxBlocksPerRead caps cached blocks per ReadAt to avoid large sequential reads.

Types

type BlockCache

type BlockCache interface {
	// Wrap returns a ByteSource that caches reads from src in fixed-size blocks.
	// The returned ByteSource also implements RangeReader if the underlying
	// cache supports it.
	Wrap(src ByteSource, opts ...WrapOption) (ByteSource, error)

	// MaxBytes returns the configured cache size limit (0 = unlimited).
	MaxBytes() int64

	// SizeBytes returns the current cache size in bytes.
	SizeBytes() int64

	// Prune removes cached entries until the cache is at or below targetBytes.
	// Returns the number of bytes freed.
	Prune(targetBytes int64) (int64, error)
}

BlockCache wraps ByteSources with block-level caching.

Block caching is most effective for random, non-contiguous reads (e.g. scattered ReadFile/Open calls over remote sources). For large sequential reads (CopyDir/CopyTo), caching can add overhead; DefaultMaxBlocksPerRead provides a conservative bypass to avoid caching large ranges.

type ByteSource

type ByteSource interface {
	io.ReaderAt

	// Size returns the total size of the data source in bytes.
	Size() int64

	// SourceID returns a unique identifier for this data source.
	// The ID is used as part of the cache key, so it must be stable
	// across calls and unique across different sources.
	SourceID() string
}

ByteSource provides random access to data for block caching.

type Cache

type Cache interface {
	// Get returns an fs.File for reading cached content.
	// Returns nil, false if content is not cached.
	// Each call returns a new file handle (safe for concurrent use).
	Get(hash []byte) (fs.File, bool)

	// Put stores content by reading from the provided fs.File.
	// The cache reads the file to completion; caller still owns/closes the file.
	Put(hash []byte, f fs.File) error

	// Delete removes cached content for the given hash.
	// Implementations should treat missing entries as a no-op.
	Delete(hash []byte) error

	// MaxBytes returns the configured cache size limit (0 = unlimited).
	MaxBytes() int64

	// SizeBytes returns the current cache size in bytes.
	SizeBytes() int64

	// Prune removes cached entries until the cache is at or below targetBytes.
	// Returns the number of bytes freed.
	Prune(targetBytes int64) (int64, error)
}

Cache provides content-addressed file storage.

Keys are SHA256 hashes of uncompressed file content. Because keys are content hashes, cache hits are implicitly verified—no additional integrity check is needed.

Implementations should handle their own size limits and eviction policies. Implementations must be safe for concurrent use.

type RangeReader

type RangeReader interface {
	// ReadRange returns a ReadCloser for reading length bytes starting at off.
	// The caller is responsible for closing the returned ReadCloser.
	ReadRange(off, length int64) (io.ReadCloser, error)
}

RangeReader provides range reads for block cache fetches. Types implementing both ByteSource and RangeReader allow the block cache to use more efficient range-based fetching instead of ReadAt.

type WrapConfig

type WrapConfig struct {
	// BlockSize is the size in bytes of each cached block.
	// Smaller blocks improve cache hit rates for random reads but increase
	// metadata overhead. Larger blocks are more efficient for sequential reads.
	BlockSize int64

	// MaxBlocksPerRead is the maximum number of blocks that will be cached
	// for a single ReadAt call. Reads spanning more blocks than this limit
	// bypass the cache entirely. Use 0 to disable the limit.
	MaxBlocksPerRead int
}

WrapConfig controls block cache wrapping behavior.

func DefaultWrapConfig

func DefaultWrapConfig() WrapConfig

DefaultWrapConfig returns the default block cache configuration.

type WrapOption

type WrapOption func(*WrapConfig)

WrapOption configures block cache wrapping behavior.

func WithBlockSize

func WithBlockSize(n int64) WrapOption

WithBlockSize sets the block size used for caching.

func WithMaxBlocksPerRead

func WithMaxBlocksPerRead(n int) WrapOption

WithMaxBlocksPerRead bypasses caching when a ReadAt spans more than n blocks. Values <= 0 disable the limit.