smartcache

package module
v0.0.0-...-0d21b7a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 19, 2026 License: MIT Imports: 15 Imported by: 0

README

type: Overview title: "smartcache"

smartcache

A small Go cache library whose one generic, type-safe Cache[T] does both read-through and write-through over a pluggable, backend-agnostic store — so you stop re-implementing the miss → load → populate → invalidate dance for every entity, and you can swap Redis for in-memory (or your own backend) without touching a single call site.

Install

go get github.com/Bytonomics/smartcache

Why smartcache

Most Go caching options are either a raw store (a Redis client, an in-memory map) that hands you get/set and leaves you to build — and repeat — the read-through, write-through, and invalidation logic at every call site, or an in-memory-only cache with no way to put a real backend behind the same API. smartcache is the missing middle: one primitive that owns that orchestration for you, generically and type-safely, over whatever backend you inject.

The core value:

  • One Cache[T] for both directions — read-through (Get + a loader) and write-through (Put + a writer) in the same type-safe primitive; no interface{}, no manual casts.
  • Backend-agnostic — Cache[T] talks only to a small CacheStore interface. redisstore and memstore ship in the box; anything else (an LRU, ristretto, bigcache, your own backend) is a tiny adapter, and swapping backends never changes a call site.

Correct by default — the choices that prevent the classic caching bugs are made for you, not left as footguns:

  • Required TTL backstop — a value can never be cached forever by accident, so a missed or failed invalidation self-heals within a bounded window (opt into no-expiry explicitly with AllowInfinite).
  • Downward-only TTL jitter — every cached value's effective TTL is shaved down by a small random amount by default, so a batch of keys written together doesn't all expire at the exact same instant and stampede the database together.
  • Delete-on-write — writes evict rather than overwrite in place, avoiding the concurrent stale-set race.
  • A cache failure never fails your operation — if writing to the backend fails, your Get/Put/GetMany still returns the real value; the cache stays a performance layer, not a hard dependency.
  • Cache-stampede protection — concurrent reads that miss the same key are coalesced into a single load (see below).
  • Optional negative caching — briefly remember "not found" so repeated probes of bad or non-existent ids don't reach your database. Off by default (NegativeTTL has no default duration — it stays disabled until you explicitly set one).

Cache stampede

A cache stampede — also called a thundering herd (or "dog-piling") — happens when a hot key is missing or has just expired and many concurrent requests all miss it at the same instant, so they all fall through to the database at once and can overwhelm it. smartcache prevents this on reads: concurrent Get calls for the same key are coalesced so that only one runs your loader and performs the write-back, while the rest wait and share that single result. This is on by default (disable with DisableSingleflight). Writes are never coalesced — each Put/PutValue is a distinct intended write, and GetMany's batched load is not coalesced across concurrent GetMany calls either.

Background reading: Cache stampede (Wikipedia).

Core Concepts

  • Manager + Register — the only way to build a Cache[T]. A Manager holds the injected CacheStore, a set of default options, and (optionally) an OpenTelemetry metrics exporter; Register[T] creates one named Cache[T] on it, inheriting the manager's defaults unless you override them for that entity.
  • CacheStore interface — the injected cache backend (never the application's own database). memstore (in-memory) and redisstore (go-redis) are provided; bring your own implementation for any other system.
  • Generic Cache[T] — read-through Get/GetMany with a loader function; write-through Put with a writer function; PutValue to cache a value you already hold, with no external write; Evict and EvictMany for delete-on-write.
  • RegisterAliasGroup + alias groups — an opt-in second constructor for a value reachable by several lookup keys (id, email, slug, …), where a write or delete through any one of them keeps the rest consistent — see RegisterAliasGroup.
  • Required positive TTL backstop — the resolved TTL must be positive. Opt in to no expiry via AllowInfinite: true.
  • Downward-only TTL jitter — both TTL and NegativeTTL are independently shaved down by a random fraction (default 10%, configurable, 0 disables) so keys written together don't all expire in sync.
  • Optional negative caching, off by default — cache "not found" results for a separate duration via NegativeTTL. There is no built-in default duration; negative caching only activates once you set NegativeTTL to a positive value yourself, so its behavior is never a surprise.
  • Singleflight de-duplication — concurrent cold loads for the same key are serialized (on by default; disable via DisableSingleflight: true).
  • Pluggable Codec — customize serialization. JSON is the default.
  • Optional OpenTelemetry metrics — configure WithOTLP on the Manager to export per-cache counters and a load-latency histogram, fire-and-forget on a background interval.

Failure Semantics

  • Populate failures — if the backend Set call fails after a loader runs, Get still returns the loaded value and reports Outcome == LoadedNotCached. If the backend Set call fails after a writer runs, Put still returns the written value and reports Outcome == WrittenNotCached. GetMany applies the same rule per key. Neither the read nor the write fails — only caching the result failed.
  • Writer failures — if writer returns a non-nil error, Put returns that error unchanged and the cache is left untouched. If writer returns (nil, nil), Put returns ErrNilWrite and the cache is left untouched — the cache is never set to a nil value.
  • GetMany load failures — if loadMissing returns a non-nil error, GetMany returns that error wrapped, with a nil result map — even the keys that were already cache hits are discarded, matching Get's "a failed load fails the call" behavior.
  • GetMany batch-read failures — if the backend's batch read fails, every requested key is treated as a miss and loaded via loadMissing, exactly as if none of them were cached; the batch-read failure itself never surfaces to the caller.
  • Evict failures — if a backend Delete call fails, the error is returned to the caller so it can retry or alert.
  • TTL backstop — the TTL bounds how long any missed or failed eviction can leave a stale value in the cache.

Outcomes

Get, GetMany, and Put each report an Outcome so callers can meter cache-hit rate and alert on populate failures. Get/GetMany never return a Put-only outcome, and Put never returns a Get-only outcome — the two are read-side and write-side vocabularies, not interchangeable.

Get and GetMany return (or, for GetMany, meter per key) one of:

  • Hit — value was served from the cache (and not expired).
  • Loaded — a miss occurred, the loader ran, and the value was cached. GetMany also reports Loaded for a key confirmed not-found in the source of truth (whether or not negative caching is enabled).
  • LoadedNotCached — a miss occurred, the loader ran, but the backend Set failed; the value is still returned, uncached.
  • NegativeHit — a previously cached "not found" was served.

Put returns one of:

  • Written — the writer ran and the value it returned was cached.
  • WrittenNotCached — the writer ran, but the backend Set failed; the value is still returned, uncached.

PutValue returns only an error — it has no Outcome, since it performs no external write to characterize.

Usage

Setup — one Manager and one Cache[User] registered on it, shared by every snippet below:

import (
	"context"
	"fmt"
	"time"

	"github.com/Bytonomics/smartcache"
	"github.com/Bytonomics/smartcache/redisstore"
	"github.com/redis/go-redis/v9"
)

type User struct {
	ID   string
	Name string
}

func main() {
	rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"})
	store := redisstore.New(rdb)

	mgr, err := smartcache.NewManager(store)
	if err != nil {
		panic(err)
	}

	ttl := time.Hour
	users, err := smartcache.Register[User](mgr, "user", &smartcache.EntityOptions{TTL: &ttl})
	if err != nil {
		panic(err)
	}
	ctx := context.Background()

	// ... the snippets below continue here, using ctx and users.
}

Manager + Register — the only way to build a Cache[T]

mgr, err := smartcache.NewManager(store,
	smartcache.WithDefaultTTL(time.Hour),
	smartcache.WithDefaultNegativeTTL(30*time.Second),
)
if err != nil {
	panic(err)
}

// name is required and unique per manager: it doubles as the metric name and
// the default key prefix. EntityOptions fields are pointers: nil inherits the
// manager default, non-nil overrides it for this cache only.
teamTTL := 15 * time.Minute
teams, err := smartcache.Register[Team](mgr, "team", &smartcache.EntityOptions{TTL: &teamTTL})
if err != nil {
	panic(err)
}

Register is a package-level generic function, not a method on Manager — Go methods cannot have type parameters, so mgr.Register[Team](...) is not possible. Manager.Shutdown(ctx) flushes and stops the OTLP exporter (see Telemetry below); it is a no-op when WithOTLP was never configured.

Get — read-through

user, outcome, err := users.Get(ctx, "u_123", func(ctx context.Context) (*User, error) {
	return loadUserFromDB(ctx, "u_123") // your own database call
})
if err != nil {
	panic(err)
}
fmt.Println(user.Name, outcome)

Get checks the cache for "u_123" first. On a miss, it calls the loader function you passed in, caches exactly the value the loader returned, and returns that value. On a hit, the loader is never called at all — this is what makes repeat reads for a hot key stop touching your database. outcome reports which of these happened: Hit, Loaded, LoadedNotCached, or NegativeHit — see Outcomes for what each one means.

Signaling and detecting "not found". To cache a "not found" — and to enable negative caching — your loader returns smartcache.ErrNotFound. smartcache then reports Outcome == Loaded (cold) or NegativeHit (warm) and returns smartcache.ErrNotFound to you. Any other error from your loader is treated as transient: it is returned to you unchanged and is never cached. A caller therefore tells the two apart with errors.Is(err, smartcache.ErrNotFound) — that is a definite not-found; any other non-nil error is a transient failure. (GetMany's loadMissing signals not-found differently: simply omit the id from the returned map — see below.)

GetMany — batch read-through

result, err := users.GetMany(ctx, []string{"u_1", "u_2", "u_3"}, func(ctx context.Context, missing []string) (map[string]*User, error) {
	return loadUsersFromDB(ctx, missing) // your own batched database call — one query for all missing ids
})
if err != nil {
	panic(err)
}
for id, user := range result {
	fmt.Println(id, user.Name)
}

GetMany reads every key from the cache in one batch round trip (a Redis MGET when the backend supports it; otherwise one Get per key). Cached keys never touch your loader at all. Every key that misses is collected and loaded in exactly one call to loadMissing — never one call per missing key — and each returned value is cached individually. An id loadMissing does not return (present as a missing key in the input but absent from its result map) is treated as confirmed not-found: it is negative-cached when NegativeTTL > 0, and is never present in the returned map either way. Unlike Get, GetMany is not deduplicated via singleflight — two concurrent GetMany calls for overlapping missing keys may each call loadMissing.

Put — write-through

newUser := &User{ID: "u_456", Name: "Ada Lovelace"}
saved, outcome, err := users.Put(ctx, "u_456", func(ctx context.Context) (*User, error) {
	if err := saveUserToDB(ctx, newUser); err != nil { // your own database call
		return nil, err
	}
	return newUser, nil
})
if err != nil {
	panic(err)
}
fmt.Println(saved.Name, outcome)

Put is for when the write itself should go through smartcache. It calls your writer function to persist the value to your own source of truth, then caches exactly the value writer returned — so a Get for "u_456" right after this Put is served from cache with no database round trip. If writer returns a non-nil error, Put returns that error unchanged and the cache stays untouched. If writer returns (nil, nil), Put returns ErrNilWrite, because the cache is never set to a nil value. Unlike Get's loader, writer is never deduplicated: two concurrent Put calls for the same key are two distinct writes, and singleflight would silently drop one of them. outcome is Written or WrittenNotCached — see Outcomes.

PutValue — direct cache write

if err := users.PutValue(ctx, "u_456", newUser); err != nil {
	panic(err)
}

PutValue writes a value you already hold straight into the cache — no external write happens, no writer function is called. Use it when your own code already performed the real write (e.g. you just ran the INSERT yourself) and you only need the cache updated to match it. PutValue returns only an error; it has no Outcome, since there is no external write for one to characterize.

Evict — delete-on-write

// After updating the user elsewhere (e.g. via Put's writer, or your own code):
if err := users.Evict(ctx, "u_123"); err != nil {
	panic(err)
}

Evict removes the cached entry for "u_123" right away. Call it after the source of truth changes outside of Put/PutValue, so the next Get for that key is a clean read-through instead of serving stale data. EvictMany does the same for several keys at once, joining any errors.

RegisterAliasGroup — one value, many lookup keys

type User struct {
	ID    string
	Name  string
	Email string
}

func (u User) CachePrimaryKey() string { return u.ID }

users, err := smartcache.RegisterAliasGroup[User](mgr, "user", &smartcache.EntityOptions{TTL: &ttl})
if err != nil {
	panic(err)
}

RegisterAliasGroup is Register's alias-aware counterpart: it builds a Cache[T] whose one cached value can be reachable by several lookup keys — e.g. a user by id, by email, and by slug — where writing or deleting through any key keeps the rest consistent. User must implement PrimaryKeyed (one method, CachePrimaryKey() string, returning its own primary key); RegisterAliasGroup panics at startup — not at first use — if User doesn't implement it, or if the injected CacheStore doesn't support alias groups. Both bundled stores, memstore and redisstore, support alias groups.

PutAliased / PutAliasedValue — register an alias

_, _, err = users.PutAliased(ctx, "u_123", smartcache.AliasRef{Field: "email", Value: "ada@example.com"},
	func(ctx context.Context) (*User, error) {
		return loadUserFromDB(ctx, "u_123")
	})

PutAliased is Put's alias-aware counterpart — it runs your writer and registers the given alias for the result, so a later GetByAlias finds it. PutAliasedValue is PutValue's counterpart: it registers an alias for a value you already hold, with no writer call. Call either once per alias, as your code first needs each lookup path. A field holds at most one value per record: registering email again for the same primary replaces the old one, and if that email was already registered to a different primary, it is moved rather than shared.

GetByAlias — read-through by alias

user, outcome, err := users.GetByAlias(ctx, smartcache.AliasRef{Field: "email", Value: "ada@example.com"},
	func(ctx context.Context) (*User, error) {
		return loadUserByEmailFromDB(ctx, "ada@example.com")
	})

GetByAlias is Get's alias-aware counterpart. On a hit it resolves the alias straight to the value, same as Get. On a miss it runs your loader, reads the loaded value's CachePrimaryKey(), and registers this alias for it automatically — so a first-ever login by email warms the cache exactly like a first Get by id would.

EvictByAlias — delete-on-write by alias

if err := users.EvictByAlias(ctx, smartcache.AliasRef{Field: "email", Value: "ada@example.com"}); err != nil {
	panic(err)
}

EvictByAlias is Evict's alias-aware counterpart: deleting through an alias removes the value and every alias for it, exactly as deleting through the primary key does — so no lookup path is left pointing at stale or deleted data. There is no separate "update by alias" method: a primary Put/PutValue already updates every alias's view.

TTL jitter

fraction := 0.10 // default; 0 disables jitter for this cache
users, err := smartcache.Register[User](mgr, "user", &smartcache.EntityOptions{
	TTL:            &ttl,
	JitterFraction: &fraction,
})

Every positive and negative TTL is shaved down — never extended — by a random amount up to JitterFraction of the base TTL before being applied on a cache Set. This means a large batch of keys written around the same time (e.g. warming a cache on deploy) does not all expire at the exact same instant and cause a synchronized cache stampede when they do. The configured TTL always remains a hard upper bound on staleness; jitter only ever shortens it. Set JitterFraction: ptrFloat64(0) to disable jitter for a specific cache, or smartcache.WithDefaultJitterFraction(0) on the Manager to disable it everywhere.

Telemetry — OpenTelemetry metrics

url := "localhost:4317"
mgr, err := smartcache.NewManager(store, smartcache.WithOTLP(smartcache.OTLPConfig{
	URL: &url, // required; every other field is optional with a sensible default
}))
if err != nil {
	panic(err)
}
defer func() { _ = mgr.Shutdown(context.Background()) }()

When WithOTLP is configured, every cache registered on the manager gets its own set of counters — smartcache.<name>.hit, .loaded, .loaded_not_cached, .negative_hit, .load_error, .written, .written_not_cached, .evict — plus a smartcache.<name>.load_latency histogram (seconds), exported over OTLP gRPC on a background interval (default 15s), fire-and-forget. Metrics are never labeled by cache key — only by the cache's registered name — so cardinality stays bounded regardless of traffic. Call Manager.Shutdown on process exit to flush the final batch; it is a no-op when WithOTLP was never configured, so callers that don't need metrics pay no lifecycle cost.

Configuration

Every option is set at one of two levels: on the manager (a default inherited by every cache it builds), or on an individual cache at Register time (which overrides the manager default for that one cache). Per-cache options are pointers — a nil field inherits the manager default; a non-nil field overrides it.

Manager options — NewManager(store, ...ManagerOption)

Option Sets Default when the option is omitted
WithDefaultTTL(d) Default positive TTL for caches that do not set their own 0 — unset; each cache must then set TTL or AllowInfinite, or Register returns ErrInvalidTTL
WithDefaultJitterFraction(f) Default downward TTL-jitter fraction 0.10
WithDefaultNegativeTTL(d) Default negative-cache TTL 0 — negative caching off
WithDefaultDisableSingleflight(b) Default singleflight toggle false — singleflight on
WithDefaultCodec(c) Default serialization codec nil → JSON
WithOTLP(cfg) Turn on OpenTelemetry metric export not set — metrics disabled

OTLPConfig — the value passed to WithOTLP

Pointer fields; a nil field uses the default shown.

Field Type Default
URL *string required — WithOTLP returns an error if nil; no default
FlushInterval *time.Duration 15s
Timeout *time.Duration 10s
Insecure *bool false (TLS)
ServiceName *string "smartcache"

Per-cache options — EntityOptions in Register[T](mgr, name, opts)

Pointer fields; nil inherits the manager default, non-nil overrides it.

Field Type Default when nil
Prefix *string the registered name
TTL *time.Duration manager default TTL (0 unless WithDefaultTTL set); TTL <= 0 without AllowInfinite → ErrInvalidTTL
AllowInfinite *bool false (per-cache only — there is no manager-level default)
JitterFraction *float64 manager default (0.10 unless changed); 0 disables jitter; must be in [0, 1) or Register returns ErrInvalidJitterFraction
NegativeTTL *time.Duration manager default (0 = negative caching off)
DisableSingleflight *bool manager default (false = singleflight on)
Codec Codec manager default codec, else JSON

name (the second argument to Register) is required and unique per manager: an empty name returns ErrEmptyName, a duplicate returns ErrDuplicateName. Registering a pointer type (Register[*T]) panics with ErrPointerType.

Examples

Runnable, end-to-end examples live in examples/, demonstrating every method above — Get (miss then hit), negative caching, PutValue, Put, GetMany, and Evict — against both backends:

cd examples
make deps   # first time only — downloads examples' own dependencies

# No infrastructure needed:
make run-memstore

# Needs a local Redis:
docker run --rm -p 6379:6379 redis:7
make run-redisstore

examples is its own Go module (own go.mod/go.sum, replaced to the local checkout) so trying things out — or adding a new example with its own dependencies — never touches the root module's dependency graph.

Implementation Notes

The backend is an interface, so any store — or a fake, for tests — can replace Redis or in-memory storage without touching call sites. The optional OpenTelemetry metrics exporter (see Telemetry) means every consumer of this module — even one that only uses the in-memory memstore and never configures WithOTLP — pulls the OpenTelemetry API and SDK as a dependency; this is a deliberate trade-off for a zero-friction "just pass an endpoint" telemetry story.

Shared values under singleflight

When singleflight is enabled (the default), a Get call that returns Loaded or LoadedNotCached may hand back the exact same *T given to every other concurrent caller deduped onto that same loader call — this is singleflight.Do's own contract: all duplicate callers receive the one result the leader produced. Treat a Loaded/LoadedNotCached result as read-only; copy it before mutating. A Hit result is different: Get unmarshals a fresh value from the cache on every call, so it is never shared with another caller.

The CacheStore interface

To back Cache[T] with your own storage system, implement CacheStore:

type CacheStore interface {
	// Get returns the raw bytes for key, or ErrStoreMiss if the key is absent.
	Get(ctx context.Context, key string) ([]byte, error)
	// Set stores val under key with the given ttl. A ttl <= 0 means no expiry.
	Set(ctx context.Context, key string, val []byte, ttl time.Duration) error
	// Delete removes key. Deleting an absent key is not an error.
	Delete(ctx context.Context, key string) error
	// Exists reports whether key is present (and not expired).
	Exists(ctx context.Context, key string) (bool, error)
}

Cache[T] owns all serialization — CacheStore only ever sees raw bytes, never the cached type T. Pass your implementation to smartcache.NewManager(store, ...) in place of memstore.New() or redisstore.New(rdb).

Optionally, also implement BatchCacheStore (embeds CacheStore and adds one GetMany(ctx, keys) (map[string][]byte, error) method) so Cache[T].GetMany can read several keys in a single round trip instead of falling back to one Get call per key. redisstore implements it via Redis MGET; memstore implements it with an in-process loop.

AliasCacheStore — backing RegisterAliasGroup

To back an alias-group cache (see RegisterAliasGroup) with your own storage system, implement AliasCacheStore in addition to CacheStore:

type AliasCacheStore interface {
	CacheStore
	GetByAlias(ctx context.Context, pointerKey string) ([]byte, error)
	PutByAlias(ctx context.Context, spec *AliasWriteSpec) error
	EvictByPrimary(ctx context.Context, valueKey, membersKey string) error
	EvictByAlias(ctx context.Context, pointerKey, valueKeyPrefix, membersKeyPrefix string) error
}

type AliasWriteSpec struct {
	ValueKey, MembersKey, PointerKey, FieldPrefix, ValueKeyPrefix, MembersKeyPrefix string
	Value []byte
	TTL   time.Duration
}

RegisterAliasGroup detects this interface the same way Cache[T].GetMany detects BatchCacheStore — a type assertion at registration time, so a store that doesn't implement it simply can't be used with RegisterAliasGroup (see the Error reference for the resulting panic). redisstore and memstore both implement it out of the box.

Custom serialization (Codec)

By default Cache[T] serializes values as JSON. To use a different encoding — for speed, a compact binary format, compression, or encryption-at-rest — supply a Codec, either per cache (EntityOptions.Codec) or as a manager default (WithDefaultCodec):

type Codec interface {
	Marshal(v any) ([]byte, error)
	Unmarshal(data []byte, v any) error
}

A compressing or encrypting codec simply wraps another: on Marshal, encode then compress/encrypt the bytes; on Unmarshal, decrypt/decompress then decode. This is how "compressed" and "encrypted-at-rest" cached values are achieved with no change to smartcache itself.

Adapting another cache library

To back Cache[T] with any existing Go cache (an LRU, ristretto, bigcache, patrickmn/go-cache, …), write a small adapter type that implements CacheStore by calling into that library — the same pattern memstore and redisstore already use. Get must return ErrStoreMiss (not the underlying library's own miss value) so Cache[T] recognizes it as a read-through miss.

Outcome reference

Get, GetMany, and Put return an Outcome (an int enum with a String() method) describing how the call was served. Get/GetMany use the read-side values; Put uses the write-side values; the two never mix.

Outcome Reported by Meaning
Hit Get, GetMany Served from the cache (present and not expired).
Loaded Get, GetMany A miss occurred, the loader ran, and the value was cached. GetMany also reports Loaded for a key confirmed not-found in the source.
LoadedNotCached Get, GetMany A miss occurred and the loader ran, but the backend Set failed; the value is still returned, uncached.
NegativeHit Get, GetMany A previously cached "not found" was served.
Written Put The writer ran and the value it returned was cached.
WrittenNotCached Put The writer ran, but the backend Set failed; the value is still returned, uncached.

PutValue, Evict, and EvictMany return only an error — they have no Outcome.

Error reference

All sentinels are exported from the smartcache package; check them with errors.Is. ErrPointerType is the only one that panics (a programming error caught at construction) rather than being returned.

Sentinel Raised by When
ErrNotFound Get / GetMany return it; your loader returns it A cacheable "not found". Your loader returns it to mark a not-found (this is what enables negative caching); Get returns it to you. Detect with errors.Is(err, smartcache.ErrNotFound).
ErrNilWrite Put (returned) The writer returned (nil, nil); the cache is never set to a nil value.
ErrInvalidTTL Register (returned) The resolved TTL <= 0 and AllowInfinite is false.
ErrInvalidJitterFraction Register (returned) The resolved jitter fraction is outside [0, 1).
ErrEmptyName Register (returned) The cache name is empty.
ErrDuplicateName Register (returned) A cache with that name is already registered on the manager.
ErrPointerType Register / RegisterAliasGroup (panics) T is itself a pointer type (e.g. Register[*User]) — a programming error, caught at construction.
ErrNilStore NewManager (returned) The CacheStore passed to NewManager is nil.
ErrAliasingNotSupported RegisterAliasGroup (panics) The injected CacheStore doesn't support alias groups.
ErrNotAliasGroup GetByAlias / PutAliased / PutAliasedValue / EvictByAlias (returned) Called on a cache built with Register instead of RegisterAliasGroup.
ErrStoreMiss CacheStore.Get (backend contract) The key is absent. A backend's Get returns it and Cache[T] treats it as a miss; it is never surfaced to callers. Relevant only when writing a custom backend.

License

MIT

Documentation

Overview

Package smartcache is a small, type-safe caching library.

It provides a generic read-through / write-through cache (Cache[T]) over a pluggable byte key-value backend (the CacheStore interface). Cache[T] owns serialization (JSON by default), enforces a positive TTL backstop so a missed or failed eviction can only cause bounded staleness, applies downward-only TTL jitter by default so keys written together don't expire in sync, optionally negative-caches "not found" results, and de-duplicates concurrent cold loads with singleflight.

Every Cache[T] is built through a Manager: NewManager creates one over a CacheStore and a set of defaults, and the package-level generic function Register creates a named, type-safe Cache[T] on it (Go methods cannot have type parameters, so registration cannot be a method on Manager). GetMany batches a read-through lookup for several keys in one round trip, calling the caller's batch loader at most once for whatever keys are not already cached.

The library performs no logging itself: Get, GetMany, and Put return an Outcome so the caller can meter cache-hit rate and detect populate failures without any of the library's own log lines. A Manager can optionally be configured with WithOTLP to export per-cache metrics (hit/miss/load/evict counters and a load-latency histogram) over OpenTelemetry OTLP, fire-and- forget on a background interval.

The backend is injected as an interface, so Redis (see subpackage redisstore) can be swapped for any other store (see subpackage memstore) without touching call sites.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrStoreMiss is returned by CacheStore.Get when the key is absent. It is an
	// internal miss signal that Cache[T] handles; it is never surfaced to callers.
	ErrStoreMiss = errors.New("smartcache: store miss")

	// ErrNotFound is the sentinel a Loader returns (or wraps) to signal a cacheable
	// "does not exist". When negative caching is enabled Cache[T] remembers it
	// briefly and returns it from Get.
	ErrNotFound = errors.New("smartcache: not found")

	// ErrInvalidTTL is returned by Register when the resolved TTL <= 0 and
	// AllowInfinite is false.
	ErrInvalidTTL = errors.New("smartcache: TTL must be > 0 (set AllowInfinite for no expiry)")

	// ErrPointerType is the value Register panics with when T is itself a pointer
	// type. Cache[T] already returns *T from every method; T being a pointer
	// too makes that a double pointer (e.g. **User), which opens a hole where
	// a non-nil outer pointer can wrap a nil inner one — silently violating
	// the "a successful Get/Put never returns nil" guarantee the rest of this
	// package relies on. This is a programming error at the call site
	// (Register[*User] instead of Register[User]), not a runtime condition, so
	// Register panics with it instead of returning it.
	ErrPointerType = errors.New("smartcache: T must not itself be a pointer type")

	// ErrNilWrite is returned by Put when writer succeeds (nil error) but
	// returns a nil value. The cache is never set to nil, so this is treated
	// as a contract violation, not a cacheable state.
	ErrNilWrite = errors.New("smartcache: writer returned a nil value with no error")

	// ErrNilStore is returned by NewManager when the CacheStore is nil.
	ErrNilStore = errors.New("smartcache: store must not be nil")

	// ErrEmptyName is returned by Register when the cache name is empty. The name
	// is required: it is both the metric name and the default key prefix.
	ErrEmptyName = errors.New("smartcache: cache name must not be empty")

	// ErrDuplicateName is returned by Register when a cache name is already
	// registered on the manager. Each registered cache must have a unique name.
	ErrDuplicateName = errors.New("smartcache: cache name already registered")

	// ErrEmptyPrefix is returned by Register when EntityOptions.Prefix is explicitly
	// set to an empty string. Prefix namespaces every key this cache stores; an empty
	// prefix would collide with any other cache that also opts out of namespacing.
	ErrEmptyPrefix = errors.New("smartcache: prefix must not be empty")

	// ErrInvalidJitterFraction is returned by Register when the resolved jitter
	// fraction is outside [0, 1). Zero disables jitter.
	ErrInvalidJitterFraction = errors.New("smartcache: jitter fraction must be in [0, 1)")

	// ErrNotAliasGroup is returned when an alias-only method (GetByAlias, PutAliased,
	// PutAliasedValue, EvictByAlias) is called on a cache that was not created with
	// RegisterAliasGroup.
	ErrNotAliasGroup = errors.New("smartcache: cache is not an alias group")

	// ErrAliasingNotSupported is used in the RegisterAliasGroup panic when the manager's
	// store does not implement AliasCacheStore.
	ErrAliasingNotSupported = errors.New("smartcache: store does not implement AliasCacheStore")
)

Functions

This section is empty.

Types

type AliasCacheStore 

type AliasCacheStore interface {
	CacheStore

	// GetByAlias resolves pointerKey -> value key -> value bytes. It returns ErrStoreMiss when
	// the pointer or the value it points to is absent.
	GetByAlias(ctx context.Context, pointerKey string) ([]byte, error)

	// PutByAlias writes spec.Value at spec.ValueKey and, when spec.PointerKey is non-empty,
	// upserts the alias pointer (one-alias-per-field replacement plus cross-primary steal
	// cleanup) and adds it to the members set — then refreshes every current group key to
	// spec.TTL. It is one atomic operation on the {ns} slot.
	PutByAlias(ctx context.Context, spec *AliasWriteSpec) error

	// EvictByPrimary deletes the value key, every pointer listed in the members set, and the
	// members set itself.
	EvictByPrimary(ctx context.Context, valueKey, membersKey string) error

	// EvictByAlias resolves pointerKey to its primary, then cascades exactly like EvictByPrimary.
	// valueKeyPrefix and membersKeyPrefix let the store derive the primary and members key.
	EvictByAlias(ctx context.Context, pointerKey, valueKeyPrefix, membersKeyPrefix string) error
}

AliasCacheStore is an optional CacheStore extension for backends that can maintain atomic key groups (a value key, its alias pointers, and a members set) in one operation. It is detected once at RegisterAliasGroup via a comma-ok type assertion, mirroring BatchCacheStore. The store is a dumb executor: Cache[T] builds every key string via keyspace.go and passes them in, so the keyspace stays single-source and the store never constructs keys.

type AliasRef 

type AliasRef struct {
	Field string
	Value string
}

AliasRef names one secondary lookup key for an alias-group cache: a field ("email") and a value ("foo@bar.com").

type AliasWriteSpec 

type AliasWriteSpec struct {
	ValueKey         string // bc:{ns}:<primary>
	MembersKey       string // bc:memb:{ns}:<primary>
	PointerKey       string // bc:grp:{ns}:<field>:<value>   ("" => primary-only write)
	FieldPrefix      string // bc:grp:{ns}:<field>:          ("" => primary-only write)
	ValueKeyPrefix   string // bc:{ns}:      (steal-cleanup: parse old primary from old value key)
	MembersKeyPrefix string // bc:memb:{ns}: (steal-cleanup: rebuild old primary's members key)
	Value            []byte
	TTL              time.Duration // the single jittered TTL, computed once by Cache[T]
}

AliasWriteSpec carries the pre-built key strings (produced by keyspace.go on the Cache[T] side) for a single grouped write. The alias-related fields are empty for a primary-only value write, which still refreshes the TTL of every existing group key.

type BatchCacheStore 

type BatchCacheStore interface {
	CacheStore
	// GetMany returns the raw bytes for the keys that are present. Keys that are
	// absent (or expired) are omitted from the returned map — a miss is never an
	// error here.
	GetMany(ctx context.Context, keys []string) (map[string][]byte, error)
}

BatchCacheStore is an optional extension of CacheStore for backends that can read many keys in one round trip. Cache[T].GetMany uses it when the injected store implements it (redisstore does, via MGET); stores that do not are handled transparently by GetMany's per-key fallback.

type Cache 

type Cache[T any] struct {
	// contains filtered or unexported fields
}

Cache is a generic, type-safe read-through / delete-on-write cache over a CacheStore. It is constructed only via Register on a Manager, never directly.

func Register 

func Register[T any](m *Manager, name string, opts *EntityOptions) (*Cache[T], error)

Register creates a Cache[T] on m under name (required, unique; it doubles as the metric name and the default key prefix). It panics with ErrPointerType if T is a pointer type. It returns ErrEmptyName, ErrEmptyPrefix, ErrDuplicateName, ErrInvalidTTL, or ErrInvalidJitterFraction on invalid input. A failed Register never consumes the name.

func RegisterAliasGroup 

func RegisterAliasGroup[T any](m *Manager, name string, opts *EntityOptions) (*Cache[T], error)

RegisterAliasGroup registers an alias-group cache: one cached value reachable by several alias keys, with all bookkeeping maintained atomically by the store. It behaves like Register but additionally (a) requires the manager's store to implement AliasCacheStore and (b) requires T to implement PrimaryKeyed (so GetByAlias read-through can learn a value's primary key). Like Register's pointer-type check, it panics on a misconfiguration that must fail at init: a pointer T, a store without AliasCacheStore, or a T that is not PrimaryKeyed.

func (*Cache[T]) Evict 

func (c *Cache[T]) Evict(ctx context.Context, key string) error

Evict deletes key (delete-on-write). It returns the delete error so the caller can retry or alarm; the TTL backstop bounds staleness if it fails.

func (*Cache[T]) EvictByAlias 

func (c *Cache[T]) EvictByAlias(ctx context.Context, alias AliasRef) error

EvictByAlias deletes the whole group reachable through alias (value + every pointer + members set). Only valid on an alias-group cache.

func (*Cache[T]) EvictMany 

func (c *Cache[T]) EvictMany(ctx context.Context, keys ...string) error

EvictMany deletes several keys, joining any errors.

func (*Cache[T]) Get 

func (c *Cache[T]) Get(ctx context.Context, key string, loader Loader[T]) (*T, Outcome, error)

Get reads through to loader on a cache miss. See Outcome for the result modes.

Sharing note: when singleflight is enabled (the default), a Loaded or LoadedNotCached result may be the exact same *T handed to every concurrent caller deduped onto the same loader call — that is singleflight.Do's own contract. Treat a Loaded/LoadedNotCached result as read-only; copy it before mutating. A Hit result is always freshly unmarshaled per call and is never shared with another caller.

func (*Cache[T]) GetByAlias 

func (c *Cache[T]) GetByAlias(ctx context.Context, alias AliasRef, loader Loader[T]) (*T, Outcome, error)

GetByAlias reads a value by one of its alias keys. It is read-through: on a miss it runs loader, learns the loaded value's primary key via PrimaryKeyed, and rebuilds the group (value + this alias pointer + members) under one TTL. Only valid on an alias-group cache.

func (*Cache[T]) GetMany 

func (c *Cache[T]) GetMany(
	ctx context.Context,
	keys []string,
	loadMissing func(ctx context.Context, missing []string) (map[string]*T, error),
) (map[string]*T, error)

GetMany reads several keys in one batch: cache hits (and warm negative hits) are served without touching loadMissing; keys not found in the cache are collected and loaded in ONE loadMissing call, then populated back into the cache (with per-key downward jitter, on both the positive TTL and NegativeTTL). Keys loadMissing does not return are negative-cached (when NegativeTTL > 0) and omitted from the result map. Unlike Get, GetMany is never deduplicated via singleflight.

func (*Cache[T]) Put 

func (c *Cache[T]) Put(ctx context.Context, key string, writer Writer[T]) (*T, Outcome, error)

Put performs a write-through: it calls writer to persist the value to your source of truth, then caches exactly the value writer returned. See Outcome for the result modes.

If writer fails, its error is returned unchanged and the cache is untouched. If writer succeeds but returns a nil value, Put returns ErrNilWrite and the cache is untouched. If the cache-side write fails after writer succeeded, Put still returns the value with Outcome == WrittenNotCached — the real write already happened; only caching it failed, and that must never look like a failed write to the caller.

writer is never deduplicated the way Get's loader is: two concurrent Put calls for the same key are two distinct writes, and singleflight would silently drop one of them.

func (*Cache[T]) PutAliased 

func (c *Cache[T]) PutAliased(ctx context.Context, primaryKey string, alias AliasRef, writer Writer[T]) (*T, Outcome, error)

PutAliased write-throughs writer's value under primaryKey and registers alias as one of its lookup keys (one-alias-per-field: re-registering a field replaces its old value). Only valid on an alias-group cache.

func (*Cache[T]) PutAliasedValue 

func (c *Cache[T]) PutAliasedValue(ctx context.Context, primaryKey string, alias AliasRef, val *T) error

PutAliasedValue caches a value you already hold under primaryKey and registers alias for it, without running a writer. Only valid on an alias-group cache.

func (*Cache[T]) PutValue 

func (c *Cache[T]) PutValue(ctx context.Context, key string, val *T) error

PutValue writes a value you already hold directly into the cache — no external write happens. Use this when you performed the real write yourself and only need the cache updated to match it.

type CacheStore 

type CacheStore interface {
	// Get returns the raw bytes for key, or ErrStoreMiss if the key is absent.
	Get(ctx context.Context, key string) ([]byte, error)
	// Set stores val under key with the given ttl. A ttl <= 0 means no expiry.
	Set(ctx context.Context, key string, val []byte, ttl time.Duration) error
	// Delete removes key. Deleting an absent key is not an error.
	Delete(ctx context.Context, key string) error
	// Exists reports whether key is present (and not expired).
	Exists(ctx context.Context, key string) (bool, error)
}

CacheStore is the backend abstraction Cache[T] depends on: a byte key-value cache store — never the application's own database. Cache[T] owns all (de)serialization, so CacheStore never sees the cached type T. Swapping the backend (Redis, in-memory, anything) means providing a different CacheStore implementation; the Cache[T] API is unchanged.

type Codec 

type Codec interface {
	Marshal(v any) ([]byte, error)
	Unmarshal(data []byte, v any) error
}

Codec serializes cached values to and from bytes. The default (when Options.Codec is nil) is encoding/json.

type EntityOptions 

type EntityOptions struct {
	Prefix              *string
	TTL                 *time.Duration
	AllowInfinite       *bool
	JitterFraction      *float64
	NegativeTTL         *time.Duration
	DisableSingleflight *bool
	Codec               Codec
}

EntityOptions overrides manager defaults for one registered cache. Every field is an optional pointer: nil inherits the manager default, non-nil overrides it.

type Loader 

type Loader[T any] func(ctx context.Context) (*T, error)

Loader loads a value from the source of truth on a cache miss.

Contract: return (val, nil) on success, (nil, ErrNotFound) for a cacheable not-found, or (nil, err) for a transient error (which is never cached).

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

Manager owns the shared backend store, global defaults, and (optionally) the OTLP meter provider. Caches are created against it with Register.

func NewManager 

func NewManager(store CacheStore, opts ...ManagerOption) (*Manager, error)

NewManager builds a Manager over store. It returns ErrNilStore if store is nil. When WithOTLP was supplied (and no meter was injected for tests), it stands up the OTLP exporter + periodic reader + meter provider.

func (*Manager) Shutdown 

func (m *Manager) Shutdown(ctx context.Context) error

Shutdown flushes and stops the OTLP meter provider. It is a no-op when WithOTLP was not configured.

type ManagerOption 

type ManagerOption func(*Manager) error

ManagerOption configures a Manager in NewManager.

func WithDefaultCodec 

func WithDefaultCodec(c Codec) ManagerOption

WithDefaultCodec sets the default codec (caches fall back to JSON when nil).

func WithDefaultDisableSingleflight 

func WithDefaultDisableSingleflight(b bool) ManagerOption

WithDefaultDisableSingleflight sets the default singleflight toggle.

func WithDefaultJitterFraction 

func WithDefaultJitterFraction(f float64) ManagerOption

WithDefaultJitterFraction sets the default downward-jitter fraction (0 disables).

func WithDefaultNegativeTTL 

func WithDefaultNegativeTTL(d time.Duration) ManagerOption

WithDefaultNegativeTTL sets the default negative-cache TTL (0 disables negative caching).

func WithDefaultTTL 

func WithDefaultTTL(d time.Duration) ManagerOption

WithDefaultTTL sets the default positive TTL inherited by caches that do not override EntityOptions.TTL.

func WithOTLP 

func WithOTLP(cfg OTLPConfig) ManagerOption

WithOTLP enables OpenTelemetry OTLP metric export. It returns an error if the config's URL is nil.

type OTLPConfig 

type OTLPConfig struct {
	// URL is the OTLP gRPC endpoint as host:port (e.g. "localhost:4317"). Required.
	URL *string
	// FlushInterval is how often the background PeriodicReader exports. Default 15s.
	FlushInterval *time.Duration
	// Timeout bounds a single export. Default 10s.
	Timeout *time.Duration
	// Insecure uses plaintext gRPC when true. Default false.
	Insecure *bool
	// ServiceName becomes the resource service.name attribute. Default "smartcache".
	ServiceName *string
}

OTLPConfig configures the optional OpenTelemetry OTLP metric exporter on a Manager (via WithOTLP). Every field is an optional pointer; a nil field falls back to its documented default. URL is required — WithOTLP returns an error if it is nil.

type Options 

type Options struct {
	// Prefix namespaces keys: the stored key is bc:<Prefix>:<key> for a normal
	// cache and bc:{<Prefix>}:<key> for an alias-group cache (built in keyspace.go).
	Prefix string
	// TTL is the positive backstop expiry applied to cached values. Required unless
	// AllowInfinite is true.
	TTL time.Duration
	// AllowInfinite opts in to TTL <= 0 (entries never expire).
	AllowInfinite bool
	// NegativeTTL, when > 0, enables negative caching of ErrNotFound for that
	// duration. Zero disables negative caching.
	NegativeTTL time.Duration
	// Codec overrides serialization. Nil means JSON.
	Codec Codec
	// DisableSingleflight turns off de-duplication of concurrent cold loads.
	DisableSingleflight bool
}

Options is the resolved configuration for a Cache[T], produced by Register.

type Outcome 

type Outcome int

Outcome describes how Cache[T].Get or Cache[T].Put served a request, so the caller can meter hit rate and alarm on populate failures without the library logging anything. Hit, Loaded, LoadedNotCached, and NegativeHit are Get-only (they name a load that happened on a read). Written and WrittenNotCached are Put-only (they name a write that happened, not a load) — Put never returns a Get-only value and Get never returns a Put-only value. 

const (
	// Hit means Get served the value from cache.
	Hit Outcome = iota
	// Loaded means Get missed, the loader ran, and the value was cached.
	Loaded
	// LoadedNotCached means Get missed, the loader ran, but writing the value
	// back to the store failed. The value is still returned; the read never fails.
	LoadedNotCached
	// NegativeHit means Get served a cached "not found" marker.
	NegativeHit
	// Written means Put's writer succeeded and the value was cached.
	Written
	// WrittenNotCached means Put's writer succeeded, but writing the value to
	// the store failed. The value is still returned; the write never fails on
	// account of the cache.
	WrittenNotCached
)

func (Outcome) String 

func (o Outcome) String() string

String returns a human-readable name for the outcome.

type PrimaryKeyed 

type PrimaryKeyed interface {
	CachePrimaryKey() string
}

PrimaryKeyed is implemented by the value type T (or *T) cached in an alias-group cache. It lets the library learn a value's primary key when rebuilding the group on a GetByAlias read-through miss. It returns the primary key VALUE (e.g. "5"); the value key is bc:{ns}:<value>.

type Writer 

type Writer[T any] func(ctx context.Context) (*T, error)

Writer persists a value to the source of truth and returns exactly what was written, so Put can cache that same value.

Contract: return (val, nil) with val != nil on success. Any non-nil error is returned to the caller unchanged and nothing is cached. Returning (nil, nil) violates the contract — Put returns ErrNilWrite, because the cache is never set to a nil value.

Source Files

View all Source files

Directories

Path Synopsis
Package memstore provides an in-memory smartcache.CacheStore for unit tests and light use.
 Package memstore provides an in-memory smartcache.CacheStore for unit tests and light use.
Package redisstore provides a go-redis-backed smartcache.CacheStore.
 Package redisstore provides a go-redis-backed smartcache.CacheStore.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.