Documentation
¶
Overview ¶
Package lru provides high-performance, concurrent, zero-dependency LRU cache implementations adapted from Google Cloud Storage FUSE (GCSFuse).
The package defines a unified Cache interface satisfied by three specialized engines:
- MapCache: A standard doubly-linked list + map implementation with O(1) operations.
- RadixCache: A Left-Child Right-Sibling (LCRS) radix tree with intrusive LRU pointers.
- ArenaRadixCache: A flat-slice arena-allocated radix tree with 32-bit node indices that eliminate internal tree pointer traversal during garbage collection, while stored values and prefixes retain standard Go GC properties.
All cache constructors require maxSize > 0 and unconditionally panic if maxSize == 0.
Index ¶
- Constants
- Variables
- type Backend
- type BytesValue
- type Cache
- type Option
- func WithBackend(backend Backend) Option
- func WithCompactionThreshold(threshold float64) Option
- func WithEvictionRetentionRatio(ratio float64) Option
- func WithEvictionThreshold(threshold float64) Option
- func WithInvariantChecking(enabled bool) Option
- func WithMemoryBudget(bytes uint64) Option
- func WithPressureFunc(fn PressureFunc) Option
- type Options
- type PressureAwareCache
- type PressureFunc
- type SizedValue
- type StringValue
- type ValueType
Examples ¶
Constants ¶
const ( // DefaultCompactionThreshold is the normalized memory pressure [0.0, 1.0+] // at or above which Moderate Pressure (Tier 1) lossless arena & map compaction is triggered. DefaultCompactionThreshold = 0.75 // DefaultEvictionThreshold is the normalized memory pressure [0.0, 1.0+] // at or above which Critical Pressure (Tier 2) proactive LRU shedding + compaction is triggered. DefaultEvictionThreshold = 0.90 // DefaultEvictionRetentionRatio is the target fraction [0.0, 1.0] of maxSize // to retain when shedding LRU entries under Critical Pressure. DefaultEvictionRetentionRatio = 0.50 )
Default thresholds for two-tier memory-pressure reclamation.
Variables ¶
var ( // ErrInvalidEntrySize is returned when the size of an entry exceeds the cache's maxSize. ErrInvalidEntrySize = errors.New("size of the entry is more than the cache's maxSize") // ErrInvalidEntry is returned when attempting to insert or update a nil value into the cache. ErrInvalidEntry = errors.New("nil values are not supported") // ErrInvalidUpdateEntrySize is returned by UpdateWithoutChangingOrder when the new value's // size differs from the existing entry's size, or by UpdateSize when sizeDelta causes uint64 overflow. ErrInvalidUpdateEntrySize = errors.New("size of entry to be updated is not same as existing size") // ErrEntryNotExist is returned when attempting to update or modify an entry that does not exist in the cache. ErrEntryNotExist = errors.New("entry with given key does not exist") )
Predefined sentinel errors returned by Cache operations.
Functions ¶
This section is empty.
Types ¶
type Backend ¶
type Backend uint8
Backend identifies the underlying cache data structure engine constructed by New.
const ( // BackendMap selects the hash-map + doubly-linked list LRU engine (MapCache). // This is the default backend when WithBackend is not specified. BackendMap Backend = iota // BackendRadix selects the pointer-based Left-Child Right-Sibling (LCRS) radix tree // LRU engine (RadixCache), optimized for hierarchical keys and fast prefix eviction. BackendRadix // BackendArenaRadix selects the contiguous-slice 32-bit index arena-backed radix tree // LRU engine (ArenaRadixCache) with two-tier memory-pressure reclamation. BackendArenaRadix )
type BytesValue ¶
type BytesValue struct {
// contains filtered or unexported fields
}
BytesValue is an immutable, comparable ValueType wrapper for a byte slice whose Size() is its byte length (len(b)).
func NewBytesValue ¶
func NewBytesValue(b []byte) BytesValue
NewBytesValue clones b and wraps it as an immutable BytesValue implementing ValueType. Cloning prevents caller mutations from racing with cached readers and prevents sub-slices from pinning large underlying backing arrays in memory.
func (BytesValue) Bytes ¶
func (b BytesValue) Bytes() []byte
Bytes returns a defensive copy of the underlying []byte slice.
func (BytesValue) Size ¶
func (b BytesValue) Size() uint64
Size returns the byte length of the slice (uint64(len(b))).
type Cache ¶
type Cache interface {
// Insert inserts or updates the given key and value in the cache.
// If key already exists, its value is replaced and moved to the most recently used (MRU) position.
// If the cache exceeds capacity after insertion, least recently used (LRU) entries are evicted
// and returned in the slice.
//
// On error, no entries are evicted, the cache state remains unmodified, and (nil, err) is returned.
// Returns ErrInvalidEntry if value is nil.
// Returns ErrInvalidEntrySize if value.Size() exceeds the cache's maximum size.
Insert(key string, value ValueType) ([]ValueType, error)
// Erase removes the entry associated with the given key from the cache.
// Returns the value of the erased entry, or nil if the key was not found.
Erase(key string) (value ValueType)
// LookUp retrieves the value associated with key and updates its position to MRU.
// Returns nil if key is not found in the cache.
LookUp(key string) (value ValueType)
// LookUpWithoutChangingOrder retrieves the value associated with key without altering its LRU position.
// Returns nil if key is not found in the cache.
LookUpWithoutChangingOrder(key string) (value ValueType)
// UpdateWithoutChangingOrder updates the value of an existing key without modifying its LRU position.
//
// Returns ErrInvalidEntry if value is nil.
// Returns ErrEntryNotExist if key is not present in the cache.
// Returns ErrInvalidUpdateEntrySize if value.Size() does not match the existing entry's size.
UpdateWithoutChangingOrder(key string, value ValueType) error
// UpdateSize adjusts the size accounting for an existing key by sizeDelta without altering its LRU position.
// Useful for entries whose size grows incrementally (e.g. sparse files).
// If the entry's updated size (existingSize + sizeDelta) exceeds maxSize (or cannot fit alongside
// entries more recent than key), the entry itself is evicted immediately without evicting older entries.
// Otherwise, if total cache capacity is exceeded, least recently used (LRU) entries are evicted
// immediately to maintain capacity invariants.
//
// Returns ErrEntryNotExist if key is not present in the cache.
// Returns ErrInvalidUpdateEntrySize if sizeDelta causes uint64 integer overflow.
UpdateSize(key string, sizeDelta uint64) error
// EraseEntriesWithGivenPrefix deletes all entries whose keys begin with prefix.
// If prefix is an empty string (""), all entries in the cache are deleted.
EraseEntriesWithGivenPrefix(prefix string)
}
Cache defines the unified interface for LRU cache engines. All implementations must be safe for concurrent access by multiple goroutines.
func New ¶
New creates and returns a new LRU Cache bounded by maxSize. By default, New constructs a MapCache (BackendMap). Callers can select an alternative engine via WithBackend(BackendRadix) or WithBackend(BackendArenaRadix), or invoke NewMapCache, NewRadixCache, or NewArenaRadixCache directly.
All returned Cache instances also implement PressureAwareCache.
maxSize must be greater than zero; otherwise New panics.
Example ¶
ExampleNew demonstrates creating an LRU cache with the unified constructor and caching common types using StringValue, BytesValue, and NewValue.
package main
import (
"fmt"
"github.com/google/go-lru"
)
func main() {
cache := lru.New(1024)
_, _ = cache.Insert("greeting", lru.StringValue("hello, world"))
_, _ = cache.Insert("payload", lru.NewBytesValue([]byte{0xDE, 0xAD, 0xBE, 0xEF}))
_, _ = cache.Insert("inode-42", lru.NewValue(42, 64))
if v := cache.LookUp("greeting"); v != nil {
fmt.Printf("greeting=%s (size=%d)\n", v.(lru.StringValue), v.Size())
}
if v := cache.LookUp("inode-42"); v != nil {
fmt.Printf("inode=%d (size=%d)\n", v.(lru.SizedValue[int]).Value, v.Size())
}
}
Output: greeting=hello, world (size=12) inode=42 (size=64)
Example (BackendsAndPrefixErase) ¶
ExampleNew_backendsAndPrefixErase demonstrates selecting the RadixCache backend via WithBackend and performing fast hierarchical prefix erasure.
package main
import (
"fmt"
"github.com/google/go-lru"
)
func main() {
cache := lru.New(4096, lru.WithBackend(lru.BackendRadix))
_, _ = cache.Insert("bucket/dirA/file1.txt", lru.StringValue("data-1"))
_, _ = cache.Insert("bucket/dirA/file2.txt", lru.StringValue("data-2"))
_, _ = cache.Insert("bucket/dirB/file3.txt", lru.StringValue("data-3"))
// Purge only the "bucket/dirA/" subtree.
cache.EraseEntriesWithGivenPrefix("bucket/dirA/")
fmt.Printf("dirA/file1=%v, dirB/file3=%v\n",
cache.LookUp("bucket/dirA/file1.txt") != nil,
cache.LookUp("bucket/dirB/file3.txt") != nil,
)
}
Output: dirA/file1=false, dirB/file3=true
Example (Eviction) ¶
ExampleNew_eviction demonstrates size-based capacity tracking and LRU eviction order.
package main
import (
"fmt"
"github.com/google/go-lru"
)
func main() {
// Create a cache with a 10-byte capacity.
cache := lru.New(10)
_, _ = cache.Insert("a", lru.StringValue("1234")) // size 4 (total 4)
_, _ = cache.Insert("b", lru.StringValue("5678")) // size 4 (total 8)
// Access "a" so "a" becomes MRU and "b" becomes LRU.
_ = cache.LookUp("a")
// Inserting "c" (size 4) exceeds capacity (8 + 4 > 10), evicting "b".
evicted, err := cache.Insert("c", lru.StringValue("9012"))
if err != nil {
panic(err)
}
fmt.Printf("evicted count=%d, first=%s\n", len(evicted), evicted[0].(lru.StringValue))
fmt.Printf("a present=%v, b present=%v, c present=%v\n",
cache.LookUp("a") != nil,
cache.LookUp("b") != nil,
cache.LookUp("c") != nil,
)
}
Output: evicted count=1, first=5678 a present=true, b present=false, c present=true
Example (MemoryPressure) ¶
ExampleNew_memoryPressure demonstrates configuring the ArenaRadixCache backend with a custom memory-pressure callback and triggering two-tier reclamation.
package main
import (
"fmt"
"github.com/google/go-lru"
)
func main() {
pressure := 0.50 // Normal pressure
cache := lru.New(
100,
lru.WithBackend(lru.BackendArenaRadix),
lru.WithPressureFunc(func() float64 { return pressure }),
lru.WithCompactionThreshold(0.75),
lru.WithEvictionThreshold(0.90),
lru.WithEvictionRetentionRatio(0.50),
)
_, _ = cache.Insert("dir/item1", lru.NewValue("v1", 40))
_, _ = cache.Insert("dir/item2", lru.NewValue("v2", 40))
// Simulate Critical Pressure (>= 0.90) to shed LRU entries down to 50% of maxSize (50 bytes).
pressure = 0.95
paCache := cache.(lru.PressureAwareCache)
shed := paCache.EvaluateMemoryPressure()
fmt.Printf("shed entries=%d, item1 remaining=%v, item2 remaining=%v\n",
len(shed),
cache.LookUpWithoutChangingOrder("dir/item1") != nil,
cache.LookUpWithoutChangingOrder("dir/item2") != nil,
)
}
Output: shed entries=1, item1 remaining=false, item2 remaining=true
func NewArenaRadixCache ¶
NewArenaRadixCache returns a new arena-backed radix LRU Cache bounded by maxSize. Optional configuration parameters can be passed via opts (e.g. WithInvariantChecking).
maxSize must be greater than zero; otherwise NewArenaRadixCache panics.
func NewMapCache ¶
NewMapCache returns a new map-based LRU Cache initialized with the supplied maxSize. Optional configuration parameters can be passed via opts (e.g. WithInvariantChecking).
maxSize must be greater than zero; otherwise NewMapCache panics.
func NewRadixCache ¶
NewRadixCache creates a new RadixCache instance bounded by maxSize (in bytes). maxSize must be greater than zero; otherwise NewRadixCache panics.
type Option ¶
type Option func(*Options)
Option is a functional option for configuring a Cache instance.
func WithBackend ¶
WithBackend configures the cache engine backend constructed by New. Supported backends: BackendMap (default), BackendRadix, BackendArenaRadix.
func WithCompactionThreshold ¶
WithCompactionThreshold configures the Moderate Pressure threshold for lossless arena/map compaction. Passing a non-positive, NaN, or infinite value resets CompactionThreshold to DefaultCompactionThreshold.
func WithEvictionRetentionRatio ¶
WithEvictionRetentionRatio configures the target retention ratio [0.0, 1.0] of maxSize retained during Critical Pressure LRU shedding.
func WithEvictionThreshold ¶
WithEvictionThreshold configures the Critical Pressure threshold for proactive LRU shedding. Passing a non-positive, NaN, or infinite value resets EvictionThreshold to DefaultEvictionThreshold.
func WithInvariantChecking ¶
WithInvariantChecking returns an Option that enables or disables internal invariant checking.
func WithMemoryBudget ¶
WithMemoryBudget configures a process-wide runtime memory budget ceiling in bytes for the built-in runtime/metrics probe when GOMEMLIMIT is unset or higher than the desired process ceiling.
func WithPressureFunc ¶
func WithPressureFunc(fn PressureFunc) Option
WithPressureFunc configures a custom memory-pressure probe function.
type Options ¶
type Options struct {
// Backend selects the underlying cache engine when calling New.
// Defaults to BackendMap.
Backend Backend
// EnableInvariantChecking enables internal data structure integrity and invariant validation.
// When enabled, cache operations execute comprehensive validation checks (e.g. bidirectional pointer
// consistency, tree structure validity, size accounting parity) and panic if corruption is detected.
// Intended primarily for testing and debugging.
// Note: Fundamental constructor preconditions (such as requiring maxSize > 0) are enforced
// unconditionally regardless of this setting.
EnableInvariantChecking bool
// PressureFunc is an optional custom callback returning normalized memory pressure in [0.0, 1.0+].
// If nil, DefaultRuntimePressureFunc(MemoryBudget) is used.
PressureFunc PressureFunc
// MemoryBudget specifies an optional memory limit in bytes for the default runtime/metrics
// pressure probe. If 0, the probe falls back to the Go runtime's GOMEMLIMIT (/gc/gomemlimit:bytes).
MemoryBudget uint64
// CompactionThreshold specifies the normalized pressure threshold for Tier 1 lossless compaction.
// Defaults to DefaultCompactionThreshold (0.75) if <= 0, NaN, or Inf.
// If CompactionThreshold > EvictionThreshold, thresholds are reconciled to preserve ordering.
CompactionThreshold float64
// EvictionThreshold specifies the normalized pressure threshold for Tier 2 LRU shedding + compaction.
// Defaults to DefaultEvictionThreshold (0.90) if <= 0, NaN, or Inf.
EvictionThreshold float64
// EvictionRetentionRatio specifies the fraction [0.0, 1.0] of cache maxSize to retain
// when Critical Pressure (EvictionThreshold) is reached.
// Defaults to DefaultEvictionRetentionRatio (0.50) if unset, negative, or NaN.
EvictionRetentionRatio float64
// contains filtered or unexported fields
}
Options contains configuration parameters for Cache instances. Memory-pressure reclamation options (PressureFunc, MemoryBudget, CompactionThreshold, EvictionThreshold, EvictionRetentionRatio) configure both automatic amortized foreground reclamation (on Insert, Erase, UpdateSize, and EraseEntriesWithGivenPrefix) and explicit EvaluateMemoryPressure() / Compact() calls across ArenaRadixCache, MapCache, and RadixCache.
func ApplyOptions ¶
ApplyOptions parses and applies the provided slice of Option functions onto a default Options configuration.
type PressureAwareCache ¶
type PressureAwareCache interface {
Cache
// Compact performs lossless compaction of the internal node arena and/or lookup index.
Compact()
// EvaluateMemoryPressure samples the configured memory-pressure probe and executes
// Tier 1 (lossless compaction) or Tier 2 (LRU tail shedding + compaction) reclamation,
// returning any values evicted during Tier 2 shedding.
EvaluateMemoryPressure() []ValueType
}
PressureAwareCache extends Cache with explicit arena/map compaction and memory-pressure reclamation. All three cache backends (MapCache, RadixCache, and ArenaRadixCache) implement this interface and perform both automatic amortized foreground reclamation (on Insert, Erase, UpdateSize, and EraseEntriesWithGivenPrefix) and explicit reclamation via EvaluateMemoryPressure() and Compact().
type PressureFunc ¶
type PressureFunc func() float64
PressureFunc returns a normalized memory pressure reading in [0.0, 1.0+], where 0.0 indicates no memory pressure, values in [CompactionThreshold, EvictionThreshold) indicate moderate pressure, and values >= EvictionThreshold indicate critical pressure.
func DefaultRuntimePressureFunc ¶
func DefaultRuntimePressureFunc(memoryBudget uint64) PressureFunc
DefaultRuntimePressureFunc returns a lock-free, zero-allocation, STW-free PressureFunc backed by runtime/metrics. It compares active Go runtime memory (/memory/classes/total:bytes - (/memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes)) against memoryBudget (if > 0) or /gc/gomemlimit:bytes (if configured < math.MaxInt64).
type SizedValue ¶
SizedValue wraps an arbitrary value of type T with an explicit logical or byte size, allowing any type to be cached without defining a custom struct implementing ValueType.
func NewSizedValue ¶
func NewSizedValue[T any](val T, size uint64) SizedValue[T]
NewSizedValue wraps val with the specified logical or byte size as a SizedValue[T].
func NewValue ¶
func NewValue[T any](val T, size uint64) SizedValue[T]
NewValue is a shorthand constructor alias for NewSizedValue.
func (SizedValue[T]) Size ¶
func (v SizedValue[T]) Size() uint64
Size returns the logical or byte size configured for this entry.
func (SizedValue[T]) Unwrap ¶
func (v SizedValue[T]) Unwrap() T
Unwrap returns the underlying value of type T.
type StringValue ¶
type StringValue string
StringValue is a ValueType wrapper for a standard Go string whose Size() is its byte length (len(s)).
func NewStringValue ¶
func NewStringValue(s string) StringValue
NewStringValue clones s and wraps it as a StringValue implementing ValueType. Cloning prevents substring values from pinning large underlying caller backing arrays in memory.
func (StringValue) Size ¶
func (s StringValue) Size() uint64
Size returns the byte length of the string (uint64(len(s))).
func (StringValue) String ¶
func (s StringValue) String() string
String returns the underlying string.
type ValueType ¶
type ValueType interface {
// Size returns the logical or byte size of the cache entry.
Size() uint64
}
ValueType represents an entry stored in a Cache that reports its logical or memory size. The cache uses Size() to calculate total capacity and trigger LRU eviction when capacity is exceeded.