fillcache

README

fillcache

An in-process cache with single-flight filling semantics.

In short: Given a function that computes the value to be cached for a key, it will ensure that the function is called only once per key no matter how many concurrent cache gets are issued for a key.

This might be useful if, say, you find yourself reaching for the singleflight package and you want to cache the resulting values in memory.

Usage

See example_test.go for example usage.

Testing

make test

Credits

If you like this package, all credit should go to @jphines, who suggested the initial design as we were working through an in-process DNS caching mechanism.

If you don't like its design or its implementation, all blame lies with @mccutchen.

Documentation

Overview

Package fillcache is an in-process cache with single-flight filling semantics.

In short: Given a function that computes the value a cache key, it will ensure that the function is called only once per key no matter how many concurrent cache gets are issued for a key.

Example

Example demonstrates basic usage of fillcache by simulating a "thundering herd" of 10 concurrent cache gets that result in only a single call to the cache filling function.

package main

import (
	"context"
	"fmt"
	"sync"
	"time"

	"github.com/mccutchen/fillcache"
)

func main() {
	var (
		key       = "foo"
		val       = 42
		fillDelay = 250 * time.Millisecond
		wg        sync.WaitGroup
	)

	// A simple cache filling function that just waits a constant amount of
	// time before returning a value
	filler := func(ctx context.Context, key string) (int, error) {
		<-time.After(fillDelay)
		fmt.Printf("computed value for key %q: %d\n", key, val)
		return val, nil
	}

	// Create our cache with the simple cache filling function above.
	cache := fillcache.New(filler, nil)

	// Launch a thundering herd of 10 concurrent cache gets
	for i := 0; i < 10; i++ {
		wg.Add(1)
		go func(i int) {
			defer wg.Done()
			result, _ := cache.Get(context.Background(), key)
			fmt.Printf("got value for key %q: %d\n", key, result)
		}(i)
	}

	wg.Wait()

}

Output:

computed value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42

Index

• type Cache
  • func New[T any](filler Filler[T], cfg *Config) *Cache[T]
  • func (c *Cache[T]) Get(ctx context.Context, key string) (T, error)
  • func (c *Cache[T]) Size() int
  • func (c *Cache[T]) Update(ctx context.Context, key string) (T, error)
• type Config
• type Filler

Examples

• Package
• Config

Types

type Cache

type Cache[T any] struct {
	// contains filtered or unexported fields
}

Cache is a cache whose entries are calculated and filled on-demand.

func New

func New[T any](filler Filler[T], cfg *Config) *Cache[T]

New creates a Cache whose entries will be computed by the given Filler.

func (*Cache[T]) Get

func (c *Cache[T]) Get(ctx context.Context, key string) (T, error)

Get returns the cache value for the given key, computing it as necessary.

func (*Cache[T]) Size

func (c *Cache[T]) Size() int

Size returns the number of entries in the cache

func (*Cache[T]) Update

func (c *Cache[T]) Update(ctx context.Context, key string) (T, error)

Update recomputes, stores, and returns the value for the given key. If an error occurs, the cache is not updated.

Update can be used to proactively update cache entries without waiting for a Get.

type Config

type Config struct {
	TTL        time.Duration
	ServeStale bool
}

Config configures a Cache.

Example

ExampleNew shows how to customize a cache instance with configuration options.

package main

import (
	"context"
	"fmt"
	"sync/atomic"
	"time"

	"github.com/mccutchen/fillcache"
)

func main() {
	var (
		key       = "foo"
		val       = 42
		fillDelay = 250 * time.Millisecond
		calls     = &atomic.Int64{}
	)

	// A fallible cache filling function that waits a constant amount of time
	// before returning a value or an error.
	filler := func(ctx context.Context, key string) (int, error) {
		<-time.After(fillDelay)
		if calls.Add(1)%2 == 0 {
			fmt.Printf("error filling cache for key %q, should serve stale\n", key)
			return 0, fmt.Errorf("error")
		}
		fmt.Printf("computed value for key %q: %d\n", key, val)
		return val, nil
	}

	// Create a cache configured with a 50ms TTL and configured to serve stale
	// values in the face of a fill error.
	cache := fillcache.New(filler, &fillcache.Config{
		// Cache entries will be re-filled after 50ms
		TTL: 50 * time.Millisecond,
		// If an error occurs during re-fill but we have a previously cached
		// value, it will be returned instead of the error.
		ServeStale: true,
	})

	// Fetch the key every 20ms for 100ms to ensure that the value is recomputed
	// after the 50ms TTL
	for i := 0; i < 5; i++ {
		result, _ := cache.Get(context.Background(), key)
		fmt.Printf("got value for key %q: %d\n", key, result)
		<-time.After(20 * time.Millisecond)
	}

}

Output:

computed value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
got value for key "foo": 42
error filling cache for key "foo", should serve stale
got value for key "foo": 42
computed value for key "foo": 42
got value for key "foo": 42

type Filler

type Filler[T any] func(ctx context.Context, key string) (val T, err error)

Filler is a function that computes the value to cache for a given key.