lastcache

README

LastCache

v2 is a generics rewrite with built-in single-flight. On the v1 API? See Migrating from v1.

LastCache is a generic, concurrency-safe in-memory cache implementing stale-if-error and stale-while-revalidate, with built-in single-flight so a burst of concurrent requests for the same key triggers at most one fetch.

go get github.com/mbrostami/lastcache/v2

stale-if-error (Get / GetStale)

When a fetch fails and a previous value is still around, the cache serves that stale value for up to Config.StaleTTL instead of returning the error.

stale-while-revalidate (GetAsync)

An expired value is returned immediately while a single background goroutine refreshes it.

Usage

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/mbrostami/lastcache/v2"
)

func main() {
	cache := lastcache.New[string, string](lastcache.Config{
		TTL:      time.Minute,
		StaleTTL: 10 * time.Second, // serve stale up to 10s when a refresh fails
	})

	fetch := func(ctx context.Context, key string) (string, error) {
		// load from db / upstream / etc.
		return "value-for-" + key, nil
	}

	// Common case: fresh value, or transparently a stale one on error.
	v, err := cache.Get(context.Background(), "user:42", fetch)
	fmt.Println(v, err)

	// When you need to know it was served stale:
	res, err := cache.GetStale(context.Background(), "user:42", fetch)
	fmt.Println(res.Value, res.Stale, res.Err, err)

	// stale-while-revalidate: returns immediately, refreshes in the background.
	res = cache.GetAsync(context.Background(), "user:42", fetch)
	fmt.Println(res.Value, res.Stale)
}

API

func New[K comparable, V any](config Config) *Cache[K, V]

func (c *Cache[K, V]) Get(ctx, key, fetch) (V, error)            // fresh or stale-on-error, single-flighted
func (c *Cache[K, V]) GetStale(ctx, key, fetch) (Result[V], error) // same, but reports staleness
func (c *Cache[K, V]) GetAsync(ctx, key, fetch) Result[V]        // serve now, refresh in background
func (c *Cache[K, V]) Set(key, value)
func (c *Cache[K, V]) Delete(key)
func (c *Cache[K, V]) TTL(key) time.Duration
func (c *Cache[K, V]) Range(func(key K, value V, ttl time.Duration) bool)

fetch is a plain func(ctx context.Context, key K) (V, error).

type Result[V any] struct {
	Value V
	Stale bool  // value is expired but served (error or in-progress refresh)
	Err   error // underlying fetch error when stale; nil for fresh values
}

Config

Field	Meaning
TTL	How long a fetched value stays fresh. Defaults to 1 minute.
StaleTTL	How long a stale value may be served after expiry when a refresh fails. 0 disables serving stale.
MaxConcurrentRefresh	Caps concurrent background refreshes (GetAsync) across all keys. Defaults to 1.
OnError	Optional func(key any, err error) called when a background refresh fails.
Context	Base context for background refreshes (they outlive the request). Defaults to context.Background().

Migrating from v1

v2 is a rewrite:

• Generic Cache[K, V] instead of any (no more type assertions).
• Get / GetStale / GetAsync replace LoadOrStore / AsyncLoadOrStore and the WithCtx variants; ctx is now a parameter.
• The fetch callback is (ctx, key) (V, error) - the useStale bool return is gone; serving stale is controlled by StaleTTL.
• GetAsync returns a Result instead of a chan error; background errors go to Config.OnError.
• Config: GlobalTTL -> TTL, ExtendTTL -> StaleTTL, AsyncSemaphore -> MaxConcurrentRefresh.
• Built-in single-flight: concurrent requests for the same key share one fetch.

Documentation

Overview

Package lastcache implements an in-memory cache with stale-if-error and stale-while-revalidate strategies, plus per-key single-flight so a burst of concurrent requests for the same key triggers at most one fetch.

stale-if-error (Get / GetStale)
When a fetch fails and a previous value is still around, the cache serves
that stale value for up to Config.StaleTTL instead of returning the error.

stale-while-revalidate (GetAsync)
An expired value is returned immediately while a single background goroutine
refreshes it.

Index

• type Cache
  • func New[K comparable, V any](config Config) *Cache[K, V]
  • func (c *Cache[K, V]) Delete(key K)
  • func (c *Cache[K, V]) Get(ctx context.Context, key K, fetch Fetch[K, V]) (V, error)
  • func (c *Cache[K, V]) GetAsync(ctx context.Context, key K, fetch Fetch[K, V]) Result[V]
  • func (c *Cache[K, V]) GetStale(ctx context.Context, key K, fetch Fetch[K, V]) (Result[V], error)
  • func (c *Cache[K, V]) Range(f func(key K, value V, ttl time.Duration) bool)
  • func (c *Cache[K, V]) Set(key K, value V)
  • func (c *Cache[K, V]) TTL(key K) time.Duration
• type Config
• type Fetch
• type Result

Types

type Cache

type Cache[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Cache is a generic, concurrency-safe cache. Use New to construct one; it must not be copied after first use.

func New

func New[K comparable, V any](config Config) *Cache[K, V]

New returns a new Cache. A zero Config is valid.

func (*Cache[K, V]) Delete

func (c *Cache[K, V]) Delete(key K)

Delete removes key from the cache.

func (*Cache[K, V]) Get

func (c *Cache[K, V]) Get(ctx context.Context, key K, fetch Fetch[K, V]) (V, error)

Get returns the value for key, fetching it if missing or expired. On a fetch error it transparently serves a stale value when one is available and Config.StaleTTL > 0; otherwise it returns the error. Concurrent calls for the same key share a single fetch.

func (*Cache[K, V]) GetAsync

func (c *Cache[K, V]) GetAsync(ctx context.Context, key K, fetch Fetch[K, V]) Result[V]

GetAsync returns the current value immediately. If it is expired, the stale value is returned (Result.Stale == true) and a single background goroutine refreshes the key. On a cold miss the fetch runs synchronously; if it fails, Result.Err is set. Background refresh errors are reported via Config.OnError.

func (*Cache[K, V]) GetStale

func (c *Cache[K, V]) GetStale(ctx context.Context, key K, fetch Fetch[K, V]) (Result[V], error)

GetStale is like Get but reports whether the value was served stale via the returned Result. The error is non-nil only when nothing could be served.

func (*Cache[K, V]) Range

func (c *Cache[K, V]) Range(f func(key K, value V, ttl time.Duration) bool)

Range calls f for each key with its value and remaining TTL. Iteration stops if f returns false. It follows sync.Map.Range semantics (no consistent snapshot).

func (*Cache[K, V]) Set

func (c *Cache[K, V]) Set(key K, value V)

Set stores value for key with the configured TTL.

func (*Cache[K, V]) TTL

func (c *Cache[K, V]) TTL(key K) time.Duration

TTL returns the remaining time before key expires. A negative value means the item is expired; zero means the key is not present.

type Config

type Config struct {
	// TTL is how long a fetched value stays fresh. Values <= 0 use defaultTTL.
	TTL time.Duration

	// StaleTTL is how long a stale value may be served after expiry when a
	// refresh fails (stale-if-error). 0 disables serving stale: every expired
	// Get then re-runs the fetch until it succeeds.
	StaleTTL time.Duration

	// MaxConcurrentRefresh bounds the number of background refreshes running at
	// once across all keys (GetAsync). Values <= 0 use 1.
	MaxConcurrentRefresh int

	// OnError, if set, is called with the underlying error when a background
	// refresh (GetAsync) fails. Foreground errors are returned to the caller.
	OnError func(key any, err error)

	// Context is the base context used for background refreshes, which outlive
	// the request that triggered them. Defaults to context.Background().
	Context context.Context
}

Config configures a Cache. The zero value is valid and uses defaults.

type Fetch

type Fetch[K comparable, V any] func(ctx context.Context, key K) (V, error)

Fetch retrieves the value for a key. It is called by the cache on a miss or when a value has expired.

type Result

type Result[V any] struct {
	// Value is the cached or freshly fetched value. It is the zero value of V
	// when Err is set and nothing could be served.
	Value V

	// Stale is true when Value is an expired value served because a refresh
	// failed (GetStale) or is still in progress (GetAsync).
	Stale bool

	// Err holds the underlying fetch error when a stale value was served, or
	// the fetch error on a cold miss. It is nil for fresh values.
	Err error
}

Result carries a value plus whether it was served stale.