goache

README

goache

Fast golang based cache system. Generic, sharded, goroutine-safe. Optimized for low latency, zero-allocation hot paths, and low memory overhead.

When goache is the right choice

goache is built for concurrent, write-heavy caching where every write must land. Every claim below is a measured number from the benchmarks further down, against five other Go cache libraries at working sets from 1,000 to 1,000,000 entries.

Reach for goache when:

• Many goroutines hit the cache at once, and the process has at least two cores. This is the case goache is built around: at 100,000 entries a concurrent Get costs 4.58 ns/op vs go-cache's 36.61, an 8x difference that comes from sharding instead of one global lock. go-cache stays pinned near 37 ns/op at every size — its bottleneck was never cache size, only lock contention. Among the sharded libraries goache is fastest at every size but the smallest, where otter leads (3.26 vs 5.67 ns/op at 1,000 entries). The core-count condition is not a formality: on a single core sharding buys nothing, and New is the wrong constructor there — use NewSingleCore instead.
• The workload writes as much as it reads. goache leads plain Set at every size measured (32.63 ns/op at 100,000 entries; next best go-cache 39.11, then freecache 86.50, otter 137.6, theine 200.4, ristretto 287.4) and leads SetWithTTL at every size from 5,000 entries up (go-cache edges it by 0.9 ns/op at 1,000).
• A dropped write would be a bug. Every Set is applied synchronously before it returns. ristretto — the fastest library here at very large bounded sizes — explicitly may discard writes under pressure; goache never does.
• You need a hard memory ceiling. WithMaxSize(n) is an exact upper bound, not an approximation: the budget is split so per-shard limits sum to exactly n, and Len() can never exceed it (docs/adr/0020).
• Bounded caches up to ~50,000 entries. goache leads eviction cost through that range (47.52 / 61.31 / 69.12 ns/op at 1k / 5k / 50k). See the caveat below for larger bounds.
• Bulk writes against a live cache. Repeated SetMany/DeleteMany are completely allocation-free (docs/adr/0022).
• You want typed keys and values with no dependencies. Full generics, no interface{} boxing, and goache's own module has zero external dependencies — the comparison libraries live in a separate module so they never reach your build.
• The process runs on exactly one core — a Kubernetes pod at limits.cpu: 1000m or below, under Go 1.25+. Use NewSingleCore, not New. At 100,000 entries it is the fastest of the seven caches measured here in all eight benchmark categories, though the margin is honest about its shape: -37% to -48% on writes and bounded eviction, only -4.9% to -6.5% on reads, and a dead tie with go-cache on delete-then-reinsert churn.

Reach for something else when:

• The cache is read almost exclusively from one goroutine. New's sharding pays for a shard hash and one pointer hop that only earn their keep once goroutines compete: go-cache's Get beats it at every size (17.40 vs 23.84 ns/op at 100,000), and the same trade shows up in delete-then-reinsert churn at every size and in GetWithTTL up to 100,000 entries (past that goache edges ahead, 93.24 vs 94.21 at 1,000,000). This is an argument against New, not against goache — NewSingleCore drops the sharding and wins that comparison instead.
• Hit ratio under heavily skewed access matters more than throughput. goache uses CLOCK (second-chance), deliberately chosen to keep Get write-lock-free (docs/adr/0016). theine and otter use W-TinyLFU, which keeps a better hit ratio on Zipf-like workloads. goache makes no hit-ratio claim; if that is your bottleneck, measure them.
• You need a bounded cache of ~100,000+ entries under constant eviction. ristretto's admission scales better there (89.94 ns/op at 100,000 and 76.34 at 1,000,000, vs goache's 110.4 and 269.8) — if you accept that it may drop writes to get it. Note ristretto's numbers in this category moved substantially between two clean runs, so treat them as directional (docs/adr/0017).
• You need loading/stampede protection, hit-miss statistics, persistence, or a disk tier today. None are implemented — they are tracked in docs/roadmap.md, and theine, otter or sturdyc cover them now.
• You cache raw bytes at a scale where GC pressure dominates. freecache stores everything off-heap in a byte ring buffer, which goache's typed on-heap design does not attempt.

docs/competitor-analysis.md has the qualitative comparison behind these numbers.

Install

go get github.com/Nerzal/goache

Requires Go 1.25 or later, and goache has no dependencies of its own. The floor is 1.25 for two reasons: hash/maphash.Comparable (1.24) is how keys are hashed without reflection or per-key allocation, and 1.25 is where the runtime began deriving GOMAXPROCS from the cgroup CPU quota — the behaviour NewSingleCore exists for.

Usage

c := goache.New[string, int]()

c.Set("a", 1)
v, ok := c.Get("a") // 1, true

c.SetMany([]goache.Entry[string, int]{
    {Key: "b", Value: 2},
    {Key: "c", Value: 3},
})

// Know roughly how many items you're loading upfront? Pre-size to skip
// Go's incremental map growth entirely — see Benchmarks below.
c2 := goache.New[string, int](goache.WithCapacity(10000))

// Entries can optionally expire. Plain Set/Get never touch the clock —
// only entries actually given a TTL pay for one, see Benchmarks below.
c.SetWithTTL("request-count", 1, 5*time.Minute)

c.SetMany([]goache.Entry[string, int]{
    {Key: "d", Value: 4, TTL: time.Minute}, // expires in a minute
    {Key: "e", Value: 5},                   // no TTL: TTL zero value = never expires
})

// Expired entries are hidden by Get automatically, but not reclaimed until
// something overwrites the key or Purge is called — goache runs no
// background goroutines of its own. Call this periodically if you use TTLs
// and want expired keys' memory reclaimed promptly.
removed := c.Purge()

// Remove entries explicitly, single or bulk.
c.Delete("a")
c.DeleteMany([]string{"b", "c"})

// Drop everything at once; the cache is still usable afterwards.
c.Clear()

// Bound the cache to at most n entries instead of growing forever. Once a
// shard is full, Set evicts via CLOCK (a second-chance approximation of
// LRU) — see Benchmarks below and docs/adr/0016-clock-eviction.md for why
// CLOCK and what it costs.
c3 := goache.New[string, int](goache.WithMaxSize(100_000))

Running on a single core — a Kubernetes pod at limits.cpu: 1000m or below, where Go 1.25+ sets GOMAXPROCS=1 — use NewSingleCore instead. Same API, same options, no sharding:

c := goache.NewSingleCore[string, int]()          // same methods as above
c2 := goache.NewSingleCore[string, int](goache.WithMaxSize(100_000))

// Deciding at startup from the runtime's own view of the CPU budget needs
// one variable that can hold either type — that's what Cacher is for. Its
// dynamic dispatch costs ~2 ns per call, so prefer keeping the concrete
// type when you can.
var cache goache.Cacher[string, int]
if runtime.GOMAXPROCS(0) == 1 {
    cache = goache.NewSingleCore[string, int]()
} else {
    cache = goache.New[string, int]()
}

WithShardCount is meaningless to NewSingleCore and ignored, so one shared []goache.Option can be passed to either constructor. See Single-core mode for the numbers and the crossover point.

Every snippet above has a runnable counterpart in example_test.go — ExampleNew, ExampleNew_withCapacity, ExampleNew_withMaxSize, ExampleCache_SetMany, ExampleCache_Purge, ExampleCache_DeleteMany, ExampleNewSingleCore, ExampleCacher and two more. They run under make test and are checked against their // Output: comments, so a change that breaks a documented usage pattern fails CI instead of quietly rotting here and on pkg.go.dev.

Architecture

Cache[K comparable, V any] shards keys across N independently-locked segments (sync.RWMutex per shard, hash/maphash.Comparable for routing) instead of one global lock or sync.Map. See the package doc comment in cache.go for the full reasoning, including why Go's experimental arena package was evaluated and rejected for this phase, and a log of optimizations that were tried and measured — some kept (cache-line-padded contiguous shard storage, entry recycling on bounded shards), several reverted and documented so they aren't re-attempted blind (a custom open-addressed per-shard table replacing Go's map, and three more in docs/adr/0018).

SingleCoreCache[K comparable, V any] (via NewSingleCore) is a second, independent implementation of the same operations for processes that run on one core: one map behind one sync.RWMutex, no shard routing, no per-entry eviction metadata unless WithMaxSize is set. It exists because a branch inside Cache was built, measured, and found insufficient — the sharded path is left completely untouched by it, so it cannot regress. See docs/adr/0026, which also records why returning an interface from New was rejected on measurement.

Phase 1 scope: Set, SetMany, Get, Delete, DeleteMany, Clear, WithCapacity (pre-sizing). Phase 2: optional per-entry TTL via SetWithTTL/Entry.TTL, lazily enforced in Get, reclaimed via the caller-driven Purge — no background goroutine (see docs/adr/0011) — plus bounded automatic eviction via WithMaxSize, using a per-shard CLOCK (second-chance) policy chosen specifically so Get never needs a write lock to track recency (see docs/adr/0016). Remaining roadmap items (loading cache, stampede protection, stats) are tracked in docs/roadmap.md.

Benchmarks

The full record — goache's own numbers, every competitor, every category, at five working-set sizes and five core counts — is in benchmarks/README.md. This section keeps the three comparisons that decide which cache to reach for.

Run them with make bench-compare (24 threads), make bench-compare-cpu (across core counts) and make bench-compare-singlecore (at one core). Machine for every number below: AMD Ryzen AI 9 HX 370 (24 threads), Go 1.26.2, ns/op, lower is better.

Read How these numbers are measured before comparing any two of them — two comparisons in this project's history came out backwards before being caught.

Concurrent reads, by available cores

The comparison that decides the choice, because the answer changes with the CPU budget. Concurrent Get, 100,000 entries. The goache row is the sharded Cache at every column, so the core-count effect is visible on one implementation:

Library	1 core	2	4	8	24
go-cache	16.88	25.50	17.56	27.29	37.41
goache (New)	24.51	14.08	7.399	7.587	4.687
otter	34.38	24.86	9.695	5.899	3.924
ristretto	37.31	32.37	16.10	15.99	8.211
freecache	103.7	52.92	27.16	26.02	16.18
theine	133.5	81.79	75.17	12.57	6.443

New is not the fastest choice on a single core — go-cache is, by 45%. Its one global mutex is uncontended when only one goroutine can run, so it pays nothing for locking and nothing for a shard hash. That inverts immediately: at two cores New costs 45% less, and by 24 cores it is 8x cheaper, because go-cache's single lock gets worse with every core added while goache's cost keeps falling. For the leftmost column, use NewSingleCore — the next table.

At one core: NewSingleCore against the whole field

n=100,000, -cpu=1, -count=10, benchstat medians. Every category NewSingleCore has an equivalent for, against whichever library is fastest in it:

Benchmark	NewSingleCore	Best competitor	Lead
Bounded (limit = n/2)	54.84 ± 0%	ristretto 105.6 ± 6%	-48%
Set	23.90 ± 1%	go-cache 38.20 ± 3%	-37%
SetWithTTL	33.27 ± 1%	go-cache 52.56 ± 2%	-37%
ParallelGetSet (90/10)	17.14 ± 0%	go-cache 19.37 ± 1%	-11.5%
Get	16.17 ± 0%	go-cache 17.29 ± 2%	-6.5%
GetWithTTL	21.03 ± 0%	go-cache 22.41 ± 1%	-6.2%
ParallelGet	16.11 ± 1%	go-cache 16.94 ± 1%	-4.9%
Delete (churn)	75.11 ± 0%	go-cache 75.39 ± 1%	-0.4% (tie)

Fastest of the seven in all eight categories. Two caveats, both against goache: the read leads are thin (4.9-6.5%, not the 20-25% an earlier -count=3 measurement claimed), and Delete is a tie rather than a win. The large leads are on writes, where go-cache boxes every value as interface{}. One documented exception, and it is against goache's own sharded cache rather than a competitor: with WithMaxSize under ~10,000 entries, New evicts faster even at one core. All of it in benchmarks/README.md and ADR 0027.

Concurrent reads at 24 threads, by working-set size

Where sharding pays. ParallelGet, ns/op:

Library	1,000	5,000	50,000	100,000	1,000,000
goache	5.673	4.573	4.235	4.578	9.250
otter	3.262	6.722	7.479	5.980	9.934
theine	5.258	5.220	5.978	6.324	10.52
ristretto	9.079	8.217	7.534	8.674	10.88
freecache	14.59	14.95	15.39	15.66	17.59
go-cache	36.74	37.01	37.05	36.61	38.42

goache leads at every size except n=1,000, after padding each shard to a cache line to remove false sharing between adjacent shards' mutexes (ADR 0018). go-cache's single global lock is flat regardless of size — it was never bottlenecked by cache size, only by contention, and that does not change however big the map gets.

What is not on this page

goache's own numbers (Set/Get/SetMany/Purge/Clear, WithCapacity pre-sizing, TTL overhead, eviction cost), the remaining six cross-library categories at all five sizes, the per-library single-core matrix, the bounded-eviction crossover, and the decomposition of where a Get's 16 ns actually goes — all in benchmarks/README.md.

Documentation

Overview

Package goache implements a low-latency, low-allocation, goroutine-safe in-memory cache built on generics.

Architecture

The cache is sharded (lock-striped): keys are hashed and distributed across a fixed number of independent shards, each guarded by its own sync.RWMutex. This was chosen over two simpler alternatives:

• A single global sync.RWMutex serializes every writer against every other writer, regardless of key. Under concurrent Set/Get load from many goroutines this becomes the bottleneck long before the map itself does.
• sync.Map is optimized for two specific access patterns: keys written once and read many times ("append-only" growth), or many goroutines operating on disjoint key sets. A general-purpose cache with mixed read/write/overwrite traffic on shared keys does not fit that profile and sync.Map falls back to its slower, mutex-guarded "dirty" path, plus it boxes keys/values as any, adding allocation and losing type safety.

Sharding keeps lock contention proportional to 1/shardCount: goroutines touching different keys typically land on different shards and never block each other, while the per-shard map stays a plain Go map (no interface boxing for K/V, since both are generic type parameters).

Shard selection uses hash/maphash.Comparable, which hashes any comparable type using the runtime's built-in hash function — no reflection, no per-key allocation, and no requirement that callers supply a hash function.

Instead, allocation pressure is kept low by: storing values inline in the map (no interface{} boxing, since V is a concrete generic type param), pre-sizing shard maps via WithCapacity when the final size is known upfront (see New), and batching multi-key writes (SetMany) so the map only rehashes/grows as needed rather than once per key.

A hand-rolled open-addressed per-shard table (avoiding Go's map entirely, to skip the redundant internal re-hash of a key already hashed once for shard routing) was tried and measured 2.4-3.5x *slower* across Set/Get/ parallel-Get than the plain Go map used here. Go's map runtime uses SIMD-friendly grouped buckets with a tophash pre-filter and incremental (non-stop-the-world) resizing — reproducing that level of engineering in pure Go isn't worth attempting again without a much larger effort than the one redundant hash call saves. Don't re-attempt this without new evidence it can actually win.

Shards are stored as a contiguous []shard value slice, each padded to a full cache line, rather than a []*shard pointer slice. An earlier version of this cache used the pointer-slice form specifically to avoid false sharing, since packing every shard's sync.RWMutex back-to-back with no padding measured ~9% slower on concurrent Get. Adding padding instead of giving up the contiguous layout measured ~15-20% *faster* on BenchmarkParallelGet/BenchmarkParallelGetSet than the pointer-slice version — the padding prevents the false sharing the pointer-slice was working around, while the value slice still saves one pointer dereference and one heap object per shard access. See docs/adr/0018-gemini-analysis-experiments.md for the measurements; it supersedes docs/adr/0004's pointer-slice conclusion now that padding is part of the comparison.

Optional per-entry TTL

Entries may optionally expire: SetWithTTL and Entry.TTL (for SetMany) set an absolute deadline (time.Now().Add(ttl).UnixNano()) stored in entry's expiresAt int64 field — 0 means "never expires" and is Go's zero value, so entries created via plain Set/SetMany without a TTL cost nothing extra anywhere. Expiry is checked lazily in Get: the clock is read only when the found entry's expiresAt is non-zero, so looking up a non-TTL entry never calls time.Now() and is exactly as fast as before TTL support existed (verified by benchmark — see cache_bench_test.go). An expired entry is treated as a miss but is not removed from the map by Get, since Get only holds a read lock; active reclamation is Purge, which callers invoke themselves (e.g. from their own ticker) — goache does not start any background goroutine, timer, or ticker of its own. See docs/adr/0011-lazy-ttl-no-background-janitor.md for why an internal auto-janitor was considered and rejected.

Automatic eviction (bounded cache size)

WithMaxSize(n) bounds the cache to at most n total entries and turns on per-shard eviction: when a shard is full, inserting a new key evicts an existing one first. The budget is split so the per-shard limits sum to exactly n (the first n%shardCount shards get one extra slot), making n a hard ceiling on Len rather than an approximation; when n is below the shard count, the shard count is lowered to the largest power of two <= n so every shard still gets at least one slot. See docs/adr/0020-shard-count-does-not-scale-eviction.md. The policy is CLOCK (a second-chance approximation of LRU), chosen deliberately over two alternatives:

• True LRU (move-to-front on every access) would require Get to take a write lock to update recency on every hit, defeating the sharding design's whole point of letting concurrent readers proceed without blocking each other. Rejected for that reason alone.
• W-TinyLFU-style frequency-sketch admission (what theine-go and otter/v2 use, see docs/competitor-analysis.md) gives a better hit ratio under skewed/Zipf workloads, but costs real CPU on every Set (updating a frequency estimate, comparing admission candidates) — exactly the bookkeeping goache's Set currently beats those libraries on. Rejected to keep Set's cost close to its current numbers; may be revisited later if hit-ratio-under-skew becomes a measured problem.

CLOCK keeps Get's cost close to zero: each entry carries a single atomic "referenced" bit, set by Get with a plain atomic store — no write lock, no ring-structure mutation, just one bit flip safe under concurrent readers. Eviction (which does need the write lock, since it mutates the shard's map and ring) starts at a per-shard "hand" pointer and walks forward, clearing the referenced bit on anything it finds set and evicting the first entry it finds already clear — approximating "evict what hasn't been touched since we last passed by" without ever taking a write lock on the read path.

That walk is short in practice, which is worth knowing before trying to make it faster: instrumenting it measured 0.00-2.54 hand steps per eviction across churn and read-heavy workloads (1.00 even at nine reads per write). Entries near the hand are the oldest ones, whose bits an earlier pass already cleared, while reads land on recently-inserted keys far from it. A contiguous per-shard bit map replacing this ring was designed and rejected on that evidence — see docs/adr/0023-reject-clock-bitmap.md, which also shows the growth in bounded-cache cost with size is mostly working set versus CPU cache (unbounded Set scales 4.56x from 1,000 to 1,000,000 entries with no eviction at all), not eviction bookkeeping.

This does cost something even when WithMaxSize isn't used: to let Get flip that bit without a write lock, entries must be individually addressable, heap-allocated objects rather than plain values inlined in the map — so shards store map[K]*entry[K, V] instead of the pre-eviction map[K]entry[V]. That means every *new* key (not overwrites of existing keys) now costs one heap allocation, whether or not WithMaxSize is ever called. This is a real, deliberate trade-off, not an oversight — see docs/adr/0016-clock-eviction.md for the full reasoning and the measured cost. Ring-maintenance work (linking/unlinking entries, the eviction sweep itself) is skipped entirely when a shard has no configured limit, so non-eviction users pay the one allocation per new key but no extra CPU beyond that.

The obvious fix — give unbounded shards their own inline map[K]entry storage and keep pointers only for WithMaxSize shards — was implemented in full and reverted. It delivered exactly the wins it promised on allocation count and full-map sweeps (Purge -51%, Clear -25% and 100,000 fewer allocs/op, SetMany/Delete -17%), but cost 20% of concurrent mixed read/write throughput (BenchmarkParallelGetSet): storing values inline makes an overwrite a mapassign *into the bucket array concurrent readers are probing*, where pointer storage writes through the pointer and leaves the buckets clean. The predicted single-threaded Get win never appeared either. See docs/adr/0021-reject-inline-storage-unbounded.md — the allocation wins are real and will keep looking tempting in a profile, so don't re-attempt this without first solving that co-location problem.

Shards with a configured WithMaxSize limit recycle entries through a sync.Pool: every eviction and every Delete/DeleteMany/Clear/Purge removal returns its entry to the pool instead of abandoning it to the GC, and a subsequent set for a new key takes one from the pool before falling back to a fresh allocation. This measurably cuts both allocation count and ns/op on sustained eviction churn (WithMaxSize Set, always-evicting churn) — see docs/adr/0018-gemini-analysis-experiments.md. Unbounded shards (no WithMaxSize) deliberately never touch the pool: nothing ever calls evict() to reclaim into it, so a cold pool.Get() is pure overhead over a plain allocation, and Delete/Clear/Purge stay exactly as cheap on unbounded caches as they always were.

Index

• type Cache
  • func New[K comparable, V any](opts ...Option) *Cache[K, V]
  • func (c *Cache[K, V]) Clear()
  • func (c *Cache[K, V]) Delete(key K)
  • func (c *Cache[K, V]) DeleteMany(keys []K)
  • func (c *Cache[K, V]) Get(key K) (V, bool)
  • func (c *Cache[K, V]) Len() int
  • func (c *Cache[K, V]) Purge() int
  • func (c *Cache[K, V]) Set(key K, value V)
  • func (c *Cache[K, V]) SetMany(entries []Entry[K, V])
  • func (c *Cache[K, V]) SetWithTTL(key K, value V, ttl time.Duration)
• type Cacher
• type Entry
• type Option
  • func WithCapacity(n int) Option
  • func WithMaxSize(n int) Option
  • func WithShardCount(n int) Option
• type SingleCoreCache
  • func NewSingleCore[K comparable, V any](opts ...Option) *SingleCoreCache[K, V]
  • func (c *SingleCoreCache[K, V]) Clear()
  • func (c *SingleCoreCache[K, V]) Delete(key K)
  • func (c *SingleCoreCache[K, V]) DeleteMany(keys []K)
  • func (c *SingleCoreCache[K, V]) Get(key K) (V, bool)
  • func (c *SingleCoreCache[K, V]) Len() int
  • func (c *SingleCoreCache[K, V]) Purge() int
  • func (c *SingleCoreCache[K, V]) Set(key K, value V)
  • func (c *SingleCoreCache[K, V]) SetMany(entries []Entry[K, V])
  • func (c *SingleCoreCache[K, V]) SetWithTTL(key K, value V, ttl time.Duration)

Examples

• Cache.DeleteMany
• Cache.Purge
• Cache.SetMany
• Cacher
• New
• New (StructKey)
• New (WithCapacity)
• New (WithMaxSize)
• New (WithShardCount)
• NewSingleCore

Types

type Cache

type Cache[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Cache is a goroutine-safe, sharded key/value cache.

func New

func New[K comparable, V any](opts ...Option) *Cache[K, V]

New creates an empty Cache.

Example

The common case: a sharded, goroutine-safe cache with no configuration.

package main

import (
	"fmt"

	"github.com/Nerzal/goache"
)

func main() {
	cache := goache.New[string, int]()

	cache.Set("answer", 42)

	value, ok := cache.Get("answer")
	fmt.Println(value, ok)

	_, ok = cache.Get("missing")
	fmt.Println(ok)

}

Output:
42 true
false

Example (StructKey)

Any comparable type works as a key, including structs. Keys are hashed with hash/maphash.Comparable, so there is no reflection and no per-key allocation.

package main

import (
	"fmt"

	"github.com/Nerzal/goache"
)

func main() {
	type coord struct{ X, Y int }

	cache := goache.New[coord, string]()
	cache.Set(coord{1, 2}, "origin-ish")

	value, ok := cache.Get(coord{1, 2})
	fmt.Println(value, ok)

}

Output:
origin-ish true

Example (WithCapacity)

WithCapacity pre-sizes every shard map when the final size is known upfront, which skips most of Go's incremental map growth during a bulk load. The hint is ignored if n <= 0.

package main

import (
	"fmt"

	"github.com/Nerzal/goache"
)

func main() {
	cache := goache.New[int, int](goache.WithCapacity(10_000))

	for i := range 10_000 {
		cache.Set(i, i*i)
	}

	fmt.Println(cache.Len())

}

Output:
10000

Example (WithMaxSize)

WithMaxSize bounds the cache. Once the limit is reached, each insert evicts one entry using CLOCK (second-chance), so Len stops at the limit instead of growing without end.

package main

import (
	"fmt"

	"github.com/Nerzal/goache"
)

func main() {
	cache := goache.New[int, int](goache.WithMaxSize(100))

	for i := range 1_000 {
		cache.Set(i, i)
	}

	fmt.Println(cache.Len() <= 100)

}

Output:
true

Example (WithShardCount)

WithShardCount overrides the default of 256 shards. The count is rounded up to a power of two so shard selection stays a bitmask rather than a modulo.

package main

import (
	"fmt"

	"github.com/Nerzal/goache"
)

func main() {
	// 100 rounds up to 128.
	cache := goache.New[string, int](goache.WithShardCount(100))

	cache.Set("k", 1)
	fmt.Println(cache.Len())

}

Output:
1

func (*Cache[K, V]) Clear

func (c *Cache[K, V]) Clear()

Clear removes every entry from the cache, across all shards. Each shard's underlying map storage is retained (via the built-in clear) rather than replaced, so a Clear followed by refilling the cache to roughly its previous size doesn't pay for map growth again. Any CLOCK ring a shard was maintaining is discarded along with it (its entries are only reachable from each other after this, and Go's tracing GC collects that cycle normally once nothing outside references it).

func (*Cache[K, V]) Delete

func (c *Cache[K, V]) Delete(key K)

Delete removes key from the cache, if present. Deleting a key that isn't present is a no-op.

func (*Cache[K, V]) DeleteMany

func (c *Cache[K, V]) DeleteMany(keys []K)

DeleteMany removes 1..N keys. Keys are grouped by destination shard up front so each shard's lock is acquired at most once, instead of once per key — same pattern as SetMany. Keys not present are silently skipped.

Example

DeleteMany skips keys that are not present, so callers need not check first.

package main

import (
	"fmt"

	"github.com/Nerzal/goache"
)

func main() {
	cache := goache.New[string, int]()
	cache.SetMany([]goache.Entry[string, int]{
		{Key: "a", Value: 1},
		{Key: "b", Value: 2},
		{Key: "c", Value: 3},
	})

	cache.DeleteMany([]string{"a", "c", "never-existed"})

	fmt.Println(cache.Len())

	_, ok := cache.Get("b")
	fmt.Println(ok)

}

Output:
1
true

func (*Cache[K, V]) Get

func (c *Cache[K, V]) Get(key K) (V, bool)

Get returns the value stored for key, and whether it was found. An entry past its TTL is treated as a miss without being removed from the shard — see Purge for active reclamation. The clock is only read when the found entry actually has a TTL (expiresAt != 0), so looking up entries created via Set/SetMany without a TTL costs exactly what it did before TTL support existed. When the cache has a configured WithMaxSize limit, a hit also flips the entry's CLOCK "referenced" bit via a single atomic store — no write lock is taken to do this, so concurrent Get calls never block each other.

func (*Cache[K, V]) Len

func (c *Cache[K, V]) Len() int

Len returns the number of items physically stored in the cache. This includes entries whose TTL has passed but haven't been reclaimed yet by Purge — Len is O(shard count), not O(item count), specifically so it stays cheap regardless of cache size; Get is the source of truth for whether any individual key is still alive. Call Purge first if you need an exact live count.

func (*Cache[K, V]) Purge

func (c *Cache[K, V]) Purge() int

Purge actively removes all expired entries and returns how many were removed. Nothing calls this automatically — goache starts no background goroutines, timers, or tickers of its own (see the package doc comment and docs/adr/0011-lazy-ttl-no-background-janitor.md for why). Callers using TTLs and who need bounded memory for keys that expire and are never looked up again should call Purge periodically themselves (e.g. from their own ticker).

Example

An entry past its TTL reads as a miss immediately, but its memory is not reclaimed until Purge runs — goache starts no background goroutine, timer or ticker. Len reports what is physically stored, so it still counts the expired entry until then.

package main

import (
	"fmt"
	"time"

	"github.com/Nerzal/goache"
)

func main() {
	cache := goache.New[string, int]()

	cache.Set("keep", 1)
	cache.SetWithTTL("expire", 2, 10*time.Millisecond)

	time.Sleep(50 * time.Millisecond)

	_, ok := cache.Get("expire")
	fmt.Println("readable:", ok)
	fmt.Println("stored:", cache.Len())

	fmt.Println("purged:", cache.Purge())
	fmt.Println("stored:", cache.Len())

}

Output:
readable: false
stored: 2
purged: 1
stored: 1

func (*Cache[K, V]) Set

func (c *Cache[K, V]) Set(key K, value V)

Set adds or overwrites a single key/value pair with no expiry. Identical cost to before TTL support existed — it never touches the clock.

func (*Cache[K, V]) SetMany

func (c *Cache[K, V]) SetMany(entries []Entry[K, V])

SetMany adds or overwrites 1..N key/value pairs, each optionally expiring per its own Entry.TTL. Entries are grouped by destination shard up front so each shard's lock is acquired at most once, instead of once per entry. time.Now() is called at most once per SetMany call (only if at least one entry has a positive TTL), not once per entry.

Example

SetMany groups entries by destination shard before locking, so each shard mutex is acquired at most once per call regardless of batch size. Entries may mix expiring and non-expiring values; a zero TTL means no expiry.

package main

import (
	"fmt"
	"time"

	"github.com/Nerzal/goache"
)

func main() {
	cache := goache.New[string, string]()

	cache.SetMany([]goache.Entry[string, string]{
		{Key: "permanent", Value: "stays"},
		{Key: "temporary", Value: "goes", TTL: 10 * time.Millisecond},
	})

	fmt.Println(cache.Len())

	time.Sleep(50 * time.Millisecond)

	_, ok := cache.Get("permanent")
	fmt.Println("permanent:", ok)

	_, ok = cache.Get("temporary")
	fmt.Println("temporary:", ok)

}

Output:
2
permanent: true
temporary: false

func (*Cache[K, V]) SetWithTTL

func (c *Cache[K, V]) SetWithTTL(key K, value V, ttl time.Duration)

SetWithTTL adds or overwrites a single key/value pair that expires after ttl. ttl <= 0 is treated as "no expiry", same as Set — consistent with WithCapacity's "n <= 0 is ignored" convention elsewhere in this package.

type Cacher

type Cacher[K comparable, V any] interface {
	// Set adds or overwrites a single key/value pair with no expiry.
	Set(key K, value V)
	// SetWithTTL adds or overwrites a single key/value pair that expires
	// after ttl. ttl <= 0 means no expiry.
	SetWithTTL(key K, value V, ttl time.Duration)
	// SetMany adds or overwrites 1..N pairs, each optionally expiring per its
	// own Entry.TTL.
	SetMany(entries []Entry[K, V])
	// Get returns the value stored for key and whether it was found. An entry
	// past its TTL reports as a miss.
	Get(key K) (V, bool)
	// Delete removes key if present.
	Delete(key K)
	// DeleteMany removes 1..N keys, skipping any that aren't present.
	DeleteMany(keys []K)
	// Clear removes every entry.
	Clear()
	// Len returns the number of items physically stored, including expired
	// entries not yet reclaimed by Purge.
	Len() int
	// Purge removes all expired entries and returns how many were removed.
	Purge() int
}

Cacher is the operation set shared by Cache and SingleCoreCache. It exists for callers that must pick between the two at run time — typically from runtime.GOMAXPROCS(0) at startup — and therefore need one variable that can hold either.

Using it is opt-in and not free. Calls through an interface are indirect and cannot be inlined, which measured roughly 2 ns per operation on this package's benchmarks: +8.8% on a single-core Get (23.40 to 25.46 ns/op, -count=10 benchstat medians) and +2-5% on Cache's concurrent Get at 24 cores (4.52 to 4.71 ns/op). That is exactly why New and NewSingleCore return their concrete types instead of this interface — nobody pays the dispatch unless they ask for it.

If the choice can be made at compile time, or if the two branches can each keep their own concrete variable, prefer that and skip this entirely.

Example

Picking the implementation from the process's actual core count requires one variable that can hold either type, which is what Cacher is for. Its dynamic dispatch costs roughly 2 ns per call (+8.8% on a single-core Get), so prefer a concrete type whenever the choice can be made at compile time.

package main

import (
	"fmt"
	"runtime"

	"github.com/Nerzal/goache"
)

func main() {
	var cache goache.Cacher[string, int]

	if runtime.GOMAXPROCS(0) == 1 {
		cache = goache.NewSingleCore[string, int]()
	} else {
		cache = goache.New[string, int]()
	}

	cache.Set("answer", 42)

	value, ok := cache.Get("answer")
	fmt.Println(value, ok)

}

Output:
42 true

type Entry

type Entry[K comparable, V any] struct {
	Key   K
	Value V
	TTL   time.Duration
}

Entry is a key/value pair used for bulk operations. TTL is optional: zero (the default) means the entry never expires. A positive TTL makes the entry expire ttl after SetMany is called, same as SetWithTTL.

type Option

type Option func(*config)

Option configures a Cache created with New.

func WithCapacity

func WithCapacity(n int) Option

WithCapacity pre-sizes every shard's underlying map for roughly n total items (split evenly across shards), so bulk-loading close to n items avoids Go's incremental map growth/rehashing entirely. Use this whenever the approximate final size is known upfront — e.g. loading a fixed data set at startup — since it turns many small map growths into zero. Ignored if n <= 0.

func WithMaxSize

func WithMaxSize(n int) Option

WithMaxSize bounds the cache to at most n total entries by evicting via CLOCK (a second-chance approximation of LRU) once a shard is full — see the package doc comment's "Automatic eviction" section for why CLOCK was chosen over true LRU or W-TinyLFU-style admission. Ignored if n <= 0 (the default: no limit, no eviction), same convention as WithCapacity.

n is a hard upper bound, not an approximation: the budget is split across shards so the per-shard limits sum to exactly n, and Len never exceeds it. Because keys are distributed by hash rather than perfectly evenly, a shard can reach its own share and start evicting while others still have room, so a cache under churn typically settles somewhat below n — n bounds memory, it doesn't reserve it.

If n is smaller than the configured shard count, the shard count is lowered to the largest power of two <= n (every shard needs at least one slot). A cache that small doesn't need many shards to avoid contention, so this costs nothing in practice.

func WithShardCount

func WithShardCount(n int) Option

WithShardCount sets the number of shards the cache is split into. The value is rounded up to the next power of two. More shards reduce lock contention under high concurrency at the cost of a small amount of per-shard memory overhead. The default is 256.

Raising this is not a way to make a WithMaxSize-bounded cache evict faster — measured across 1,000 to 1,000,000 entries, more shards left eviction cost flat at best and up to 25% worse at large sizes (see docs/adr/0020-shard-count-does-not-scale-eviction.md). It is also capped at maxSize when WithMaxSize is smaller — see WithMaxSize.

type SingleCoreCache

type SingleCoreCache[K comparable, V any] struct {
	// contains filtered or unexported fields
}

SingleCoreCache is a goroutine-safe key/value cache for processes that run on a single core, where Cache's sharded design is pure overhead.

When to use this instead of Cache

Cache is lock-striped: keys are hashed and routed to one of N independently locked shards so concurrent goroutines on different cores don't block each other. That only pays off when goroutines actually run at the same time. A Kubernetes pod with limits.cpu at or below 1000m runs with GOMAXPROCS=1 under Go 1.25+, which means exactly one goroutine executes at a time — and there Cache pays for machinery that returns nothing:

• a hash/maphash.Comparable call per operation purely to pick a shard, on top of the hash Go's map does internally anyway,
• an indirection through the shard slice plus a cache-line-padded shard struct,
• an s.limit > 0 check on every read and write even when eviction is never configured,
• per-entry CLOCK metadata (key, referenced, prev, next) that inflates an entry from 16 bytes to roughly 56 for a string key, tripling the working set a lookup walks.

SingleCoreCache drops all four. It is one map behind one sync.RWMutex, with entries holding only what the configured feature set needs. Measured at GOMAXPROCS=1 against a 100,000-entry working set, it is the fastest of the seven caches bench/ compares in all eight benchmark categories — but the size of that lead varies enormously by category, and quoting one number for it would be misleading:

• Writes win big: Set 23.90 vs go-cache's 38.20 ns/op (-37%), and bounded Set 54.84 vs ristretto's 105.6 (-48%). go-cache boxes every value as interface{} and allocates; goache does neither.
• Reads win narrowly: Get 16.17 vs go-cache's 17.29 (-6.5%). About 70% of a Get is Go's own map lookup, which every competitor built on map[K]V pays identically.
• Delete-then-reinsert churn is a tie (75.11 vs 75.39). Pointer storage must look a key up to recycle its entry where go-cache, storing values inline, blindly overwrites. That lookup costs ~9.6 ns here and saves ~2.9 ns on every Get — a deliberate trade, not an oversight.

Against the sharded Cache at one core the margin is wider and more uniform (-14% to -89%). See docs/adr/0027-single-core-field-claim.md for the full matrix and the cost decompositions, docs/adr/0026-single-core-cache.md for why this type exists, and docs/adr/0025-cpu-constrained-benchmarks.md for the crossover point.

The trade is exact and one-directional: with two or more cores available, Cache pulls ahead immediately and keeps improving as cores are added, while SingleCoreCache's one lock becomes the bottleneck — the same shape go-cache has. **If you declare single-core and then run on many cores, you get one global lock and you will feel it.** When in doubt, use New.

What it does not change

Locking is not weakened. GOMAXPROCS=1 does not make a mutex unnecessary: goroutines still interleave via preemption, and this type must stay correct on any machine regardless of what the caller claimed. Nothing here starts a background goroutine, timer, or ticker either — TTL reclamation is Purge, called by the caller, exactly as with Cache (see docs/adr/0011-lazy-ttl-no-background-janitor.md).

Storage

Two entry layouts, chosen once in NewSingleCore and never mixed: without WithMaxSize, entries are scEntry (value + deadline, 16 bytes for an int value) held in data; with WithMaxSize, they are scRingEntry (value + deadline + key + CLOCK bookkeeping) held in clock, since eviction needs the key to delete by and a ring to sweep. Exactly one of the two maps is non-nil for the lifetime of the cache, so the unbounded path never carries the bounded path's per-entry cost.

Values are stored behind a pointer rather than inline in the map. Inline storage was measured here too and is slower on every axis at one core (Get 19.3 vs 16.4 ns/op, Set 27.7 vs 24.3, mixed read/write 20.9 vs 19.1): a 16-byte map value doubles the bucket payload the map probes, which costs more than the saved dereference returns. See docs/adr/0021-reject-inline-storage-unbounded.md, which reached the same conclusion for Cache by a different route.

func NewSingleCore

func NewSingleCore[K comparable, V any](opts ...Option) *SingleCoreCache[K, V]

NewSingleCore creates an empty SingleCoreCache.

It accepts the same Options as New. WithCapacity pre-sizes the single underlying map and WithMaxSize bounds the cache and turns on CLOCK eviction, both exactly as they do for Cache. WithShardCount is meaningless here — this cache has no shards — and is ignored rather than treated as an error, so one shared []Option can be passed to either constructor.

Typical use, deciding at startup from the runtime's own view of the CPU budget:

if runtime.GOMAXPROCS(0) == 1 {
	cache = goache.NewSingleCore[string, User]()
} else {
	cache = goache.New[string, User]()
}

Note that binding the two types to one variable like that requires a Cacher[K, V] interface, whose dynamic dispatch costs roughly 2 ns per call — see Cacher.

Example

NewSingleCore is a second, independent implementation for processes pinned to one core — a Kubernetes pod with limits.cpu of 1000m or less runs at GOMAXPROCS=1 under Go 1.25+. It drops the shard routing hash and the shard slice indirection for one map behind one mutex, and has the same API as Cache. Above one core it loses to Cache and is the wrong choice.

package main

import (
	"fmt"
	"time"

	"github.com/Nerzal/goache"
)

func main() {
	cache := goache.NewSingleCore[string, int]()

	cache.Set("answer", 42)
	cache.SetWithTTL("session", 7, time.Hour)

	value, ok := cache.Get("answer")
	fmt.Println(value, ok)
	fmt.Println(cache.Len())

}

Output:
42 true
2

func (*SingleCoreCache[K, V]) Clear

func (c *SingleCoreCache[K, V]) Clear()

Clear removes every entry. The underlying map's storage is retained (via the builtin clear) rather than replaced, so refilling the cache to roughly its previous size doesn't pay for map growth again. Any CLOCK ring is discarded along with it — its entries only reference each other afterwards, which Go's tracing GC collects normally.

func (*SingleCoreCache[K, V]) Delete

func (c *SingleCoreCache[K, V]) Delete(key K)

Delete removes key from the cache, if present. Deleting a key that isn't present is a no-op.

func (*SingleCoreCache[K, V]) DeleteMany

func (c *SingleCoreCache[K, V]) DeleteMany(keys []K)

DeleteMany removes 1..N keys, taking the lock once. Keys not present are silently skipped. As with SetMany there is no shard grouping to do.

func (*SingleCoreCache[K, V]) Get

func (c *SingleCoreCache[K, V]) Get(key K) (V, bool)

Get returns the value stored for key, and whether it was found. An entry past its TTL is reported as a miss without being removed — see Purge for active reclamation. The clock is only read when the found entry actually carries a TTL, so looking up entries written without one costs nothing extra. On a bounded cache a hit also flips the entry's CLOCK referenced bit with a single atomic store, taken under the read lock so concurrent readers never block each other.

func (*SingleCoreCache[K, V]) Len

func (c *SingleCoreCache[K, V]) Len() int

Len returns the number of items physically stored, including entries whose TTL has passed but which Purge hasn't reclaimed yet — same contract as Cache.Len. Unlike Cache.Len, which is O(shard count), this is O(1).

func (*SingleCoreCache[K, V]) Purge

func (c *SingleCoreCache[K, V]) Purge() int

Purge actively removes all expired entries and returns how many were removed. Nothing calls this automatically — see the type doc comment.

func (*SingleCoreCache[K, V]) Set

func (c *SingleCoreCache[K, V]) Set(key K, value V)

Set adds or overwrites a single key/value pair with no expiry.

func (*SingleCoreCache[K, V]) SetMany

func (c *SingleCoreCache[K, V]) SetMany(entries []Entry[K, V])

SetMany adds or overwrites 1..N key/value pairs, each optionally expiring per its own Entry.TTL.

Unlike Cache.SetMany this does no shard grouping — there is one shard, so there is nothing to group. The lock is taken once and the entries are applied in order, which skips the per-entry routing hash, the per-call bucket-slice bookkeeping and the scratch-space handoff that docs/adr/0022-bulk-bucket-scratch-reuse.md exists to amortize for Cache. time.Now() is still called at most once per call, and only if at least one entry carries a positive TTL.

func (*SingleCoreCache[K, V]) SetWithTTL

func (c *SingleCoreCache[K, V]) SetWithTTL(key K, value V, ttl time.Duration)

SetWithTTL adds or overwrites a single key/value pair that expires after ttl. ttl <= 0 is treated as "no expiry", same as Set and same as Cache.