lingcache

README

lingcache

English | 中文

A lightweight, high-performance, thread-safe in-memory cache for Go. The name is 岭 (ling, “ridge”, from Bagualing) plus cache.

lingcache aims to be small enough to embed and fast enough for hot paths: generics, per-entry TTL, adaptive W-TinyLFU eviction, and snapshots that keep absolute expiration times.

go get github.com/bagualing/lingcache

Features

• Generic Cache[K comparable, V any]
• Thread-safe sharded hashmap
• Adaptive W-TinyLFU (window LRU + TinyLFU admission + segmented main LRU)
• TTL via a hierarchical timing wheel, plus lazy expiry on Get
• Persistence (Save / Load) that stores ExpireAt (wall-clock unix nano), so remaining TTL survives restarts
• Standard library only (no third-party dependencies)

Quick start

package main

import (
    "time"

    "github.com/bagualing/lingcache"
)

func main() {
    c, err := lingcache.New[string, string](lingcache.Options[string, string]{
        Capacity:   10_000,
        DefaultTTL: 10 * time.Minute, // optional; 0 = no expiry
    })
    if err != nil {
        panic(err)
    }
    defer c.Close()

    c.Set("user:1", "alice")
    c.SetWithTTL("otp", "123456", 30*time.Second)

    if v, ok := c.Get("user:1"); ok {
        _ = v
    }
    c.Delete("user:1")
}

Persistence

Keys and values must be gob-encodable (exported struct fields).

var buf bytes.Buffer
if err := c.Save(&buf); err != nil {
    panic(err)
}

c2 := lingcache.Must(lingcache.Options[string, string]{Capacity: 10_000})
defer c2.Close()
if err := c2.Load(&buf); err != nil {
    panic(err)
}

Semantics:

• The snapshot stores ExpireAt, not remaining duration.
• After five minutes of downtime, remaining TTL is five minutes shorter; the deadline does not move.
• Entries already expired at load time are skipped.
• Entries are written hottest-first; loading into a smaller cache keeps the hot set.

Call Load immediately after New on an empty cache.

The snapshot CRC is an integrity check for local files, not a security boundary. Load refuses snapshots larger than 256 MiB (ErrTooLarge). Entries skipped because they are already expired increment Stats.LoadsExpired; entries skipped because the destination is full increment Stats.LoadsDropped. Overwriting an existing key on Load invokes OnEvict with ReasonDeleted.

New starts a background janitor. Always Close the cache when finished. After Close, Get/Set/Delete are no-ops and Save/Load return ErrClosed. Peek/Has/TTL/Range observe expiry but only Get (and the janitor) reap expired entries.

API

Method	Description
Get / Peek / Has	Lookup. Peek does not record an access or a hit.
Set / SetWithTTL	Insert or replace. ttl <= 0 means no expiration.
Delete / Clear / Close	Remove / empty / stop the janitor
TTL	Remaining TTL. ok && rem == 0 means no expiration.
Len / Capacity / Stats	Size and counters
Range	Visit live entries (do not re-enter the cache from the callback)
Save / Load	Snapshot persistence
Sync	Drain policy buffers (tests)

Design

Informed by otter, theine-go, and Caffeine, kept deliberately small.

Concern	Choice
Eviction	Adaptive W-TinyLFU
Concurrency	Sharded RWMutex maps + BP-Wrapper buffers (lossy reads, lossless writes)
Expiration	Hierarchical timing wheel + lazy Get + 1s janitor
Persistence	Magic + gob + CRC32, absolute ExpireAt
Hashing	hash/maphash.Comparable with a per-cache seed

Not included on purpose: loading cache / singleflight, weighted cost, secondary cache, async refresh.

Version

Version	Notes
v0.1.0	First cut: sharded map, W-TinyLFU, timing wheel, gob snapshots
v0.2.0	Tighter memory and hot path: sentinel-free wheel, lazy TinyLFU sketch, skip access recording below 50% occupancy
v0.2.2	Review fixes: Clear/Save races, never-expire vs wheel, climber misses, snapshot size cap

Requirements

Go 1.24+

License

MIT

Documentation

Overview

Package lingcache is a lightweight, high-performance, thread-safe in-memory cache.

Design is informed by Otter, Theine, and Caffeine:

• Adaptive W-TinyLFU eviction (window LRU + TinyLFU admission into a segmented main LRU) for high hit ratio across recency- and frequency-biased workloads.
• Sharded hashmap with per-shard RWMutex for concurrent Get/Set.
• BP-Wrapper style buffers: lossy striped read buffer and lossless write buffer, so the eviction policy is updated in batches under a single lock.
• Hierarchical timing wheel for O(1) TTL expiration, plus lazy checks on Get. Wheel buckets are circular lists (no entry[K,V] sentinels).
• Persistence via encoding/gob that stores absolute expiration timestamps, so remaining TTL is preserved across Save/Load (including process downtime).
• Fast path below 50% occupancy: the frequency sketch is lazy and access recording is skipped until the cache is actually competing for space.

The library uses only the Go standard library.

Example

package main

import (
	"bytes"
	"fmt"
	"time"

	"github.com/bagualing/lingcache"
)

func main() {
	c, err := lingcache.New[string, string](lingcache.Options[string, string]{
		Capacity: 10_000,
	})
	if err != nil {
		panic(err)
	}
	defer c.Close()

	c.Set("user:1", "alice")
	c.SetWithTTL("session", "tok", time.Hour)

	if v, ok := c.Get("user:1"); ok {
		fmt.Println(v)
	}

	var buf bytes.Buffer
	if err := c.Save(&buf); err != nil {
		panic(err)
	}
	c2 := lingcache.Must(lingcache.Options[string, string]{Capacity: 10_000})
	defer c2.Close()
	if err := c2.Load(&buf); err != nil {
		panic(err)
	}
	fmt.Println(c2.Has("session"))
}

Output:
alice
true

Index

• Variables
• type Cache
  • func Must[K comparable, V any](opts Options[K, V]) *Cache[K, V]
  • func New[K comparable, V any](opts Options[K, V]) (*Cache[K, V], error)
  • func (c *Cache[K, V]) Capacity() int
  • func (c *Cache[K, V]) Clear()
  • func (c *Cache[K, V]) Close()
  • func (c *Cache[K, V]) Delete(key K) bool
  • func (c *Cache[K, V]) Get(key K) (V, bool)
  • func (c *Cache[K, V]) Has(key K) bool
  • func (c *Cache[K, V]) Len() int
  • func (c *Cache[K, V]) Load(r io.Reader) error
  • func (c *Cache[K, V]) Peek(key K) (V, bool)
  • func (c *Cache[K, V]) Range(fn func(key K, value V) bool)
  • func (c *Cache[K, V]) Save(w io.Writer) error
  • func (c *Cache[K, V]) Set(key K, value V)
  • func (c *Cache[K, V]) SetWithTTL(key K, value V, ttl time.Duration)
  • func (c *Cache[K, V]) Stats() Stats
  • func (c *Cache[K, V]) Sync()
  • func (c *Cache[K, V]) TTL(key K) (time.Duration, bool)
• type Options
• type Reason
  • func (r Reason) String() string
• type Stats
  • func (s Stats) HitRatio() float64

Examples

• Package

Variables

var (
	// ErrClosed is returned by Save/Load after Close.
	ErrClosed = errors.New("lingcache: closed")
	// ErrInvalidSnapshot is returned when the stream is not a cache snapshot
	// or the format version is unsupported.
	ErrInvalidSnapshot = errors.New("lingcache: invalid snapshot")
	// ErrCorrupt is returned when the checksum does not match.
	ErrCorrupt = errors.New("lingcache: corrupt snapshot")
	// ErrTooLarge is returned by Load when the snapshot exceeds the size cap.
	ErrTooLarge = errors.New("lingcache: snapshot too large")
)

Types

type Cache

type Cache[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Cache is a thread-safe in-memory cache with generic keys and values, optional per-entry TTL, W-TinyLFU eviction, and snapshot persistence.

func Must

func Must[K comparable, V any](opts Options[K, V]) *Cache[K, V]

Must is like New but panics on invalid options.

func New

func New[K comparable, V any](opts Options[K, V]) (*Cache[K, V], error)

New constructs a cache from opts. Capacity must be positive. New starts a background janitor; the caller must Close the cache when finished to stop it. Mutations after Close are no-ops; Save and Load return ErrClosed.

func (*Cache[K, V]) Capacity

func (c *Cache[K, V]) Capacity() int

Capacity returns the configured maximum number of entries.

func (*Cache[K, V]) Clear

func (c *Cache[K, V]) Clear()

Clear removes all entries. OnEvict is invoked with ReasonDeleted for each.

func (*Cache[K, V]) Close

func (c *Cache[K, V]) Close()

Close stops the background janitor, clears the cache, and rejects further mutations. Close is required to avoid leaking the janitor goroutine. After Close, Get/Set/Delete are no-ops; Save and Load return ErrClosed. Close is idempotent.

func (*Cache[K, V]) Delete

func (c *Cache[K, V]) Delete(key K) bool

Delete removes key. It returns whether the key was present.

func (*Cache[K, V]) Get

func (c *Cache[K, V]) Get(key K) (V, bool)

Get returns the value stored for key. Expired entries are treated as misses and removed. A successful Get counts as an access for the eviction policy.

func (*Cache[K, V]) Has

func (c *Cache[K, V]) Has(key K) bool

Has reports whether key is present and not expired.

func (*Cache[K, V]) Len

func (c *Cache[K, V]) Len() int

Len returns an estimate of the number of resident entries, including entries that have expired but not yet been reaped.

func (*Cache[K, V]) Load

func (c *Cache[K, V]) Load(r io.Reader) error

Load restores entries from r, skipping those whose absolute deadline has already passed. Existing entries with the same key are overwritten and the previous value is reported via OnEvict as ReasonDeleted. Load should typically be called on an empty cache immediately after New. The snapshot is treated as local/trusted (CRC32 is integrity, not security).

func (*Cache[K, V]) Peek

func (c *Cache[K, V]) Peek(key K) (V, bool)

Peek returns the value without recording an access, updating stats, or removing an expired entry. Expired entries are still reaped by Get and the janitor.

func (*Cache[K, V]) Range

func (c *Cache[K, V]) Range(fn func(key K, value V) bool)

Range calls fn for each live (non-expired) entry. If fn returns false, iteration stops. fn must not call methods on c.

func (*Cache[K, V]) Save

func (c *Cache[K, V]) Save(w io.Writer) error

Save writes a snapshot to w. Absolute expiration timestamps are stored, so remaining TTL is preserved (wall-clock deadlines survive process downtime). Entries are written hottest-first; Load into a smaller cache keeps the hottest.

Key and value types must be gob-encodable. Save may be called concurrently with other cache operations.

func (*Cache[K, V]) Set

func (c *Cache[K, V]) Set(key K, value V)

Set stores value for key using DefaultTTL (no expiration if DefaultTTL is 0).

func (*Cache[K, V]) SetWithTTL

func (c *Cache[K, V]) SetWithTTL(key K, value V, ttl time.Duration)

SetWithTTL stores value for key with the given TTL. A non-positive ttl means the entry does not expire. If the key already exists, its value and TTL are replaced.

func (*Cache[K, V]) Stats

func (c *Cache[K, V]) Stats() Stats

Stats returns a snapshot of counters.

func (*Cache[K, V]) Sync

func (c *Cache[K, V]) Sync()

Sync applies pending read/write buffers to the eviction policy. Useful in tests; production code does not need to call it.

func (*Cache[K, V]) TTL

func (c *Cache[K, V]) TTL(key K) (time.Duration, bool)

TTL returns the remaining lifetime of key. ok is false if the key is missing or expired. A zero remaining duration with ok == true means the entry has no expiration.

type Options

type Options[K comparable, V any] struct {
	// Capacity is the maximum number of entries. Must be > 0.
	Capacity int

	// DefaultTTL is applied by Set. Zero means the entry does not expire.
	// SetWithTTL ignores this and uses the provided duration.
	DefaultTTL time.Duration

	// OnEvict is invoked after an entry is removed, without holding cache locks.
	// The callback must not block and must not call back into the same Cache.
	OnEvict func(key K, value V, reason Reason)

	// ShardCount overrides the number of hashmap shards. Must be a power of two.
	// Zero selects a default based on GOMAXPROCS.
	ShardCount int

	// Clock overrides the time source. Used in tests. Nil uses time.Now.
	// The function must be safe for concurrent use; the cache calls it from
	// Get/Set as well as the background janitor.
	Clock func() time.Time
}

Options configures a Cache. Capacity is the only required field.

type Reason

type Reason uint8

Reason explains why an entry left the cache.

const (
	// ReasonDeleted means the entry was removed by Delete or Clear.
	ReasonDeleted Reason = iota + 1
	// ReasonEvicted means the entry was evicted by the size-based policy.
	ReasonEvicted
	// ReasonExpired means the entry's TTL elapsed.
	ReasonExpired
)

func (Reason) String

func (r Reason) String() string

type Stats

type Stats struct {
	Hits         uint64
	Misses       uint64
	Sets         uint64
	Deletes      uint64
	Evictions    uint64
	Expirations  uint64
	LoadsExpired uint64 // skipped on Load because ExpireAt is already due
	LoadsDropped uint64 // skipped on Load because the cache was at capacity
	LoadsSkipped uint64 // LoadsExpired + LoadsDropped
}

Stats is a point-in-time snapshot of cache counters.

func (Stats) HitRatio

func (s Stats) HitRatio() float64

HitRatio returns hits / (hits + misses). Returns 0 if no lookups.