Documentation
¶
Overview ¶
Package lrucache provides a bounded, generation-guarded LRU cache with single-flight fills, built for in-memory mirrors of database rows that are kept consistent by explicit eviction rather than TTLs.
Design points, all load-bearing for correctness:
- Fills are single-flight: concurrent callers for the same key wait on one leader and share its result and error. Errors are propagated to every waiter but never cached, so a failed load is retried by the next caller. A panicking loader releases its waiters with an error and re-raises on the leader's own stack.
- A generation counter increments on every explicit eviction or flush. A fill records the generation before running its loader and its install is discarded if the generation moved, so a fill racing an eviction can never re-install the value the eviction just removed. Capacity (LRU) eviction does not increment it: dropping a still-valid entry for space is not an invalidation.
- An optional secondary index maps a caller-chosen string (typically a row's primary key) to the cache key holding it, so write paths that only know the row identity can evict without recomputing the key.
- An optional validator runs on every hit; an entry it rejects is removed and reported as a miss, so callers fall through to their full load path (used, for example, to treat expired credentials as misses while keeping all expiry policy in the caller).
The cache owns its own locking and never holds a lock across a loader call, so loaders are free to perform database reads or network I/O. It must not be guarded by any caller-side lock that is itself held across I/O; give the cache exclusive ownership of its consistency instead.
All methods are safe on a nil receiver: reads miss, evictions no-op, and Fill degrades to calling the loader directly. This lets consumers treat an absent cache as "caching disabled" without branching.
Index ¶
- func DecodeKey(key string, n int) (parts []string, ok bool)
- func EncodeKey(parts ...string) string
- type Cache
- func (c *Cache[V]) Evict(key string)
- func (c *Cache[V]) EvictByIndex(indexKey string)
- func (c *Cache[V]) EvictWhere(pred func(key string) bool)
- func (c *Cache[V]) Fill(ctx context.Context, key string, load Loader[V]) (V, error)
- func (c *Cache[V]) Flush()
- func (c *Cache[V]) Get(key string) (V, bool)
- func (c *Cache[V]) Len() int
- type Loader
- type Option
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DecodeKey ¶
DecodeKey is EncodeKey's inverse: it parses exactly n length-prefixed parts out of key, in the order EncodeKey wrote them. ok is false for a key that isn't in the length-prefixed form EncodeKey builds for n parts (impossible for keys a well-behaved caller produces with EncodeKey).
func EncodeKey ¶
EncodeKey builds a collision-free composite cache key from parts, for callers whose key is a tuple of caller-controlled strings (e.g. (auth mode, identity, mcp client ID) — identity is frequently a caller-asserted string with no charset restriction). A naive separator-joined key lets one part's content forge a boundary — e.g. join("\x00", "a\x00b", "c") and join("\x00", "a", "b\x00c") would build the identical string, aliasing two distinct tuples onto one cache entry and skewing any eviction predicate that parses the key back apart. Length-prefixing each part makes that forgery impossible regardless of what bytes a part contains. Pair with DecodeKey to parse it back.
Types ¶
type Cache ¶
type Cache[V any] struct { // contains filtered or unexported fields }
Cache is a bounded LRU cache with single-flight fills. Construct with New; the zero value is not usable (but a nil *Cache is, see the package doc).
func New ¶
New constructs a Cache bounded to capacity entries. A non-positive capacity panics: the bound is part of the type's contract, and silently defaulting it would hide a caller bug.
func (*Cache[V]) EvictByIndex ¶
EvictByIndex removes the entry registered under the given secondary-index key, if any.
func (*Cache[V]) EvictWhere ¶
EvictWhere removes every entry whose cache key matches pred. A linear sweep under the lock; intended for rare bulk invalidations, not hot paths.
func (*Cache[V]) Fill ¶
Fill runs load for key with single-flight deduplication: concurrent callers for the same key wait for one leader and share its result and error. A successful result is cached unless an eviction or flush landed while the load was in flight; errors are propagated but never cached.
A follower waits on its own ctx as well as the leader's completion: if ctx is canceled first, Fill returns ctx.Err() immediately rather than blocking until the (possibly unrelated, possibly slow) leader finishes. The leader itself is unaffected by ctx here — canceling it is load's own responsibility, since load is the one actually doing the I/O.
Source Files