wfcache

README

Waterfall Cache

wfcache is a multi-layered cache with waterfall hit propagation and built-in storage adapters for DynamoDB, Redis, BigCache (in-memory), and GoLRU (in-memory)

wfcache is effective for read-heavy workloads and it can be used both as a side-cache or a read-through/write-through cache.

This libary is used in production by an application that receives 8 Million hits a month.

Built-in Storage Adapters

Package	Description	Eviction strategy
DynamoDB	DynamoDB	TTL
Redis	Redis	TTL/LRU
BigCache	Performant on heap storage with minimal GC	TTL (enforced on add)
GoLRU	In-memory storage with approximated LRU similar to Redis	TTL/LRU
Basic	Basic in-memory storage (not recommended)	TTL (enforced on get)

Installation

To retrieve wfcache, run:

$ go get github.com/juliaqiuxy/wfcache

Usage

import (
  "github.com/juliaqiuxy/wfcache"
  basic "github.com/juliaqiuxy/wfcache/basic"
  bigcache "github.com/juliaqiuxy/wfcache/bigcache"
  dynamodb "github.com/juliaqiuxy/wfcache/dynamodb"
)

c, err := wfcache.New(
  basic.Create(5 * time.Minute),
  bigcache.Create(2 * time.Hour),
  dynamodb.Create(dynamodbClient, "my-cache-table", 24 * time.Hour),
)

items, err := c.BatchGet(keys)
if err == wfcache.ErrPartiallyFulfilled {
  fmt.Println("Somethings are missing")
}

Usage with hooks

You can configure wfcache to notify you when each storage operation starts and finishes. This is useful when you want to do performance logging, tracing etc.

import (
  "context"
  "github.com/juliaqiuxy/wfcache"
  "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
  basic "github.com/juliaqiuxy/wfcache/basic"
)

func onStartStorageOp(ctx context.Context, opName string) interface{} {
  span, _ := tracer.StartSpanFromContext(ctx, opName)
  return span
}

func onFinishStorageOp(span interface{}) {
  span.(ddtrace.Span).Finish()
}

wfcache.NewWithHooks(
  onStartStorageOp,
  onFinishStorageOp,
  basic.Create(5 * time.Minute),
)

How it works

The following steps outline how reads from wfcache work:

• When getting a value, wfcache tries to read it from the first storage layer (e.g. BigCache).
• If the storage layer is not populated with the requested key-value pair (cache miss), transparent to the application, wfcache notes the missing key and moves on to the next layer. This continues until all configured storage options are exhausted.
• When there is a cache hit, wfcache then primes each storage layer with a previously reported cache miss to make the data available for any subsequent reads.
• wfcache returns the key-value pair back to the application

If you want to use wfcache as read-through cache, you can implement a custom adapter for your source database and configure it as the last storage layer. In this setup, a cache miss only ever happens in intermediate storage layers (which are then primed as your source storage resolves values) but wfcache would always yield data.

When mutating wfcache, key-value pairs are written and removed from all storage layers. To mutate a specific storage layer in isolation, you can ask wfcache to provide you with a reference to the underlying slice of storages by calling its Storages() method.

Cache eviction

wfcache leaves it up to each storage layer to implement their eviction strategy. Built-in adapters use a combination of Time-to-Live (TTL) and Least Recently Used (LRU) algorithm to decide which items to evict.

Also note that the built-in Basic storage is not meant for production use as the TTL enforcement only happens if and when a "stale" item is requested form the storage layer.

Implementing Custom Adapters

For use cases where:

• you require a stroge adapter which is not included in wfcache, or
• you want to use wfcache as a read-through/write-through cache

it is trivial to extend wfcache by implementing the following adapter interface:

type Storage interface {
  Get(ctx context.Context, key string) *CacheItem
  BatchGet(ctx context.Context, keys []string) []*CacheItem
  Set(ctx context.Context, key string, value []byte) error
  BatchSet(ctx context.Context, pairs map[string][]byte) error
  Del(ctx context.Context, key string) error
  BatchDel(ctx context.Context, keys []string) error
}

Documentation

Index

• Variables
• type Cache
  • func New(maker StorageMaker, otherMakers ...StorageMaker) (*Cache, error)
  • func NewWithHooks(sop StartStorageOp, fop FinishStorageOp, maker StorageMaker, ...) (*Cache, error)
  • func (c *Cache) BatchGet(keys []string) ([]*CacheItem, error)
  • func (c *Cache) BatchGetWithContext(ctx context.Context, keys []string) ([]*CacheItem, error)
  • func (c *Cache) BatchSet(pairs map[string]interface{}) error
  • func (c *Cache) BatchSetWithContext(ctx context.Context, pairs map[string]interface{}) error
  • func (c *Cache) Del(key string) error
  • func (c *Cache) DelWithContext(ctx context.Context, key string) error
  • func (c *Cache) Get(key string) (*CacheItem, error)
  • func (c *Cache) GetWithContext(ctx context.Context, key string) (*CacheItem, error)
  • func (c *Cache) Set(key string, value interface{}) error
  • func (c *Cache) SetWithContext(ctx context.Context, key string, value interface{}) error
  • func (c *Cache) Storages() ([]Storage, error)
• type CacheItem
• type FinishStorageOp
• type Future
  • func Promise(f func() (interface{}, error)) Future
• type StartStorageOp
• type Storage
• type StorageMaker

Variables

var (
	ErrNotFulfilled       = errors.New("look up not fulfilled")
	ErrPartiallyFulfilled = errors.New("look up only partially fulfilled")
)

Types

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

func New

func New(maker StorageMaker, otherMakers ...StorageMaker) (*Cache, error)

func NewWithHooks

func NewWithHooks(sop StartStorageOp, fop FinishStorageOp, maker StorageMaker, otherMakers ...StorageMaker) (*Cache, error)

func (*Cache) BatchGet

func (c *Cache) BatchGet(keys []string) ([]*CacheItem, error)

func (*Cache) BatchGetWithContext

func (c *Cache) BatchGetWithContext(ctx context.Context, keys []string) ([]*CacheItem, error)

func (*Cache) BatchSet

func (c *Cache) BatchSet(pairs map[string]interface{}) error

func (*Cache) BatchSetWithContext

func (c *Cache) BatchSetWithContext(ctx context.Context, pairs map[string]interface{}) error

func (*Cache) Del

func (c *Cache) Del(key string) error

func (*Cache) DelWithContext

func (c *Cache) DelWithContext(ctx context.Context, key string) error

func (*Cache) Get

func (c *Cache) Get(key string) (*CacheItem, error)

func (*Cache) GetWithContext

func (c *Cache) GetWithContext(ctx context.Context, key string) (*CacheItem, error)

func (*Cache) Set

func (c *Cache) Set(key string, value interface{}) error

func (*Cache) SetWithContext

func (c *Cache) SetWithContext(ctx context.Context, key string, value interface{}) error

func (*Cache) Storages

func (c *Cache) Storages() ([]Storage, error)

type CacheItem

type CacheItem struct {
	Key       string `json:"key"`
	Value     []byte `json:"value"`
	ExpiresAt int64  `json:"expiresAt"`
}

type FinishStorageOp

type FinishStorageOp func(interface{})

type Future

type Future interface {
	Await() (interface{}, error)
	AwaitWithContext(context.Context) (interface{}, error)
}

func Promise

func Promise(f func() (interface{}, error)) Future

type StartStorageOp

type StartStorageOp func(ctx context.Context, opName string) interface{}

type Storage

type Storage interface {
	TimeToLive() time.Duration

	Get(ctx context.Context, key string) *CacheItem
	BatchGet(ctx context.Context, keys []string) []*CacheItem
	Set(ctx context.Context, key string, value []byte) error
	BatchSet(ctx context.Context, pairs map[string][]byte) error
	Del(ctx context.Context, key string) error
}

type StorageMaker

type StorageMaker func() (Storage, error)