residency

Documentation

Overview

Package residency implements a bounded in-process cache for lazily decoded, immutable build artifacts — render plans and packaged static entries — resident between requests, bounded by entry count and by estimated decoded bytes.

The cache is a segmented LRU (SLRU): entries land in a probation segment on first load and move to a protected segment on their next hit, so a one-off scan of cold routes cannot flush the working set. Concurrent loads for the same key are coalesced into a single decode, and total in-flight cold-load work is bounded by a weighted semaphore over estimated peak decode weight. A canceled waiter returns promptly without canceling the shared decode; the finished result still populates the cache for the next request.

Eviction, idle expiration, and Trim drop only the cache's own reference. Values already returned to callers remain valid for as long as those callers hold them; reclamation is left entirely to the garbage collector and the cache never forces a collection.

The package deliberately knows nothing about the pack container format. PlanStore and StaticEntries implementations wrap a Cache and supply the decode step as a LoadFunc.

Index

• Constants
• Variables
• func ImmutableError(err error) error
• func IsImmutable(err error) bool
• type Cache
  • func New[V any](opts Options) *Cache[V]
  • func (c *Cache[V]) Close() error
  • func (c *Cache[V]) Get(ctx context.Context, key string, estimatedDecoded, estimatedPeak int64, ...) (V, error)
  • func (c *Cache[V]) Stats() Stats
  • func (c *Cache[V]) Trim(targetBytes int64)
• type LoadFunc
• type Options
• type Stats

Constants

const (
	// DefaultMaxEntries bounds resident entries when Options.MaxEntries is
	// zero. Plans use 64 and static entries use 128; the plan
	// figure is the package default.
	DefaultMaxEntries = 64
	// DefaultMaxResidentBytes bounds total estimated decoded bytes when
	// Options.MaxResidentBytes is zero.
	DefaultMaxResidentBytes = 32 << 20
	// DefaultIdleExpiry drops entries that have not been read for this long.
	DefaultIdleExpiry = 10 * time.Minute
	// DefaultDecodeSemaphoreBytes bounds the summed estimated peak weight of
	// concurrent cold loads when Options.DecodeSemaphoreBytes is zero.
	DefaultDecodeSemaphoreBytes = 32 << 20
	// DefaultNegativeMaxEntries bounds the negative cache when
	// Options.NegativeMaxEntries is zero.
	DefaultNegativeMaxEntries = 256
	// DefaultNegativeTTL bounds how long an immutable failure is remembered
	// when Options.NegativeTTL is zero.
	DefaultNegativeTTL = 5 * time.Minute
)

Defaults are per-cache: a plan store and a static entry store each get their own budgets.

const ProtectedSegmentFraction = 0.8

ProtectedSegmentFraction is the share of the entry and byte budgets reserved for the protected SLRU segment. It is a stable runtime contract, not an option.

Variables

var ErrClosed = errors.New("residency: cache closed")

ErrClosed is returned by Get after Close.

Functions

func ImmutableError

func ImmutableError(err error) error

ImmutableError marks err as an immutable failure: one that cannot heal within the current build, such as a record digest mismatch or a decode error over immutable bytes. Only errors marked immutable enter the negative cache; transient failures (I/O, deadline) are always retried. The returned error unwraps to err.

func IsImmutable

func IsImmutable(err error) bool

IsImmutable reports whether err (or an error it wraps) is marked as an immutable failure, either via ImmutableError or by implementing ImmutableFailure() bool returning true.

Types

type Cache

type Cache[V any] struct {
	// contains filtered or unexported fields
}

Cache is a bounded SLRU residency cache. It is safe for concurrent use.

func New

func New[V any](opts Options) *Cache[V]

New builds a Cache from opts.

func (*Cache[V]) Close

func (c *Cache[V]) Close() error

Close releases the cache's references and stops its janitor. Loads already in flight complete and return to their waiters but are not retained. Close is idempotent; Get after Close returns ErrClosed.

func (*Cache[V]) Get

func (c *Cache[V]) Get(ctx context.Context, key string, estimatedDecoded, estimatedPeak int64, load LoadFunc[V]) (V, error)

Get returns the value for key, loading it with load on a miss. Concurrent callers for the same key share one load. estimatedDecoded and estimatedPeak come from the caller's pack index weights and are used for the decode semaphore before the load has run; the entry's resident weight uses the decodedWeight the load itself reports, falling back to estimatedDecoded when the load reports nothing.

If ctx is canceled while waiting, Get returns ctx.Err() promptly; the shared load keeps running and its result still populates the cache.

func (*Cache[V]) Stats

func (c *Cache[V]) Stats() Stats

Stats returns a snapshot of cache state and counters.

func (*Cache[V]) Trim

func (c *Cache[V]) Trim(targetBytes int64)

Trim evicts least-recently-used entries (probation first) until estimated resident bytes are at or below targetBytes. Trim(0) clears every evictable entry. Values already returned to callers remain valid.

type LoadFunc

type LoadFunc[V any] func(ctx context.Context) (value V, decodedWeight, peakWeight int64, err error)

LoadFunc performs one cold load: read, verify, and decode a record. It returns the decoded value, the estimated resident weight of the decoded value in bytes, and the estimated peak transient weight of the decode itself. The context it receives is detached from any single caller's cancellation because the result is shared; loads should be bounded by their own I/O deadlines, not by the first requester's patience.

type Options

type Options struct {
	// MaxEntries bounds the number of resident entries. Zero or negative
	// yields DefaultMaxEntries.
	MaxEntries int
	// MaxResidentBytes bounds the summed estimated decoded weight of
	// resident entries. It also fixes the oversized threshold: a value whose
	// decoded weight exceeds MaxResidentBytes/8 is returned to the caller
	// but never inserted. Zero or negative yields DefaultMaxResidentBytes.
	MaxResidentBytes int64
	// IdleExpiry drops entries not read for this long. Zero yields
	// DefaultIdleExpiry; negative disables idle expiration.
	IdleExpiry time.Duration
	// DecodeSemaphoreBytes bounds the summed estimated peak weight of
	// concurrent cold loads. Zero yields DefaultDecodeSemaphoreBytes;
	// negative disables the bound.
	DecodeSemaphoreBytes int64
	// NegativeMaxEntries bounds the negative cache. Zero yields
	// DefaultNegativeMaxEntries; negative disables negative caching.
	NegativeMaxEntries int
	// NegativeTTL bounds how long an immutable failure is remembered. Zero
	// yields DefaultNegativeTTL; negative disables negative caching.
	NegativeTTL time.Duration
	// Clock overrides time.Now, for tests.
	Clock func() time.Time
}

Options configures a Cache. The zero value is valid and yields the defaults above. For the duration and count fields, zero means "use the default" and a negative value disables the mechanism entirely.

type Stats

type Stats struct {
	// Entries and ResidentBytes cover both SLRU segments.
	Entries       int
	ResidentBytes int64
	// ProtectedEntries and ProtectedBytes cover the protected segment only.
	ProtectedEntries int
	ProtectedBytes   int64

	Hits   uint64
	Misses uint64
	// Evictions counts entries dropped for capacity, including by Trim.
	Evictions uint64
	// IdleExpirations counts entries dropped for exceeding IdleExpiry.
	IdleExpirations uint64
	// OversizedLoads counts successful loads returned but not inserted
	// because their decoded weight exceeded MaxResidentBytes/8.
	OversizedLoads uint64

	NegativeEntries int
	NegativeHits    uint64

	// InFlight is the number of loads currently running or being awaited.
	InFlight int
}

Stats is a point-in-time snapshot of cache state and counters.