lazywritercache

README

In memory GOLANG Lazy Writer Cache

A lazy writer cache is useful in situations where you don't want to block a low latency operation with something expensive, such as a database write, but you want the consistency of a single version of truth if you need to look for the object again. A simple goroutine spun off write satisfies the first condition of unblocking the operation, but doesn't provide any mechanism to resolve the lookup.

A lazy writer cache has the following properties:

• An in memory cache of objects
• Store operations are fast, objects are marked as dirty for lazy writing
• Read objets are also fast as they are coming from the in memory cache of objects
• A goroutine provides background services to write the cache to some durable store (Postgres etc).
• A separate goroutine provides cache pruning to the configured size
• There is no guarantee of object persistence between lazy writes. If your system crashes in between, your data is lost. You must be comfortable with this tradeoff.
• To keep writes fast, eviction is also a lazy operation so the size limit is a soft limit that can be breached. The lazy purge will do a good job of keeping memory usage under control as long as you don't create objects faster than it can purge them.

Dependencies

The basic lazywritercache has no dependencies outside of core go packages. The dependencies in go.mod are for the GORM example in /example.

A second lock free implementation which performs slightly better under highly parallel read workloads is in the lockfree sub-folder as lockfree.LazyWriterCacheLF and has a dependence on xsync.MapOf.

Why not just use REDIS?

Good question. In fact, if you need to live in a distributed world then REDIS is probably a good solution. But it's still much slower than an in memory map. You've got to make hops to a server etc. This in memory cache is much faster even than REDIS - roughly 100x. The benchmark results below (and in the tests) show performance nearly as good as an in memory map. Keep in mind that the lazy writer db time is still a practical limit on how fast you can update the cache. A network based solution will have times measured in 10's of microseconds at best. A database write is another 2 orders of magnitude slower.

If you are really sensitive about nonblocking performance then you could conceivably put a lazy write cache in front of redis with a fairly quick sync loop.

Benchmark results for cache size of 20k items and 100k items on macbook apple silicon.

Benchmark	ns/op	b/op	allocs/op
CacheWrite 20k	167	147	3
CacheWrite 100k	168	146	4
CacheRead 20k	48	0	0
CacheRead 100k	54	1	0
CacheRead 100k 5 threads	677	4	0

Benchmark Lock Free	ns/op	b/op	allocs/op
CacheWrite 20k	299	109	6
CacheWrite 100k	270	113	7
CacheRead 20k	39	0	0
CacheRead 100k	52	1	0
CacheRead 100k 5 threads	637	4	0

How do I use it?

You need to be at least on go1.18 as we use generics. If you've not tried out generics, now is great time to do it.

go get github.com/pilotso11/lazywritercache

You will need to provide 2 structures to the cache.

1. CacheReaderWriter

You will need a CacheReaderWriter implementation. This interface abstracts the cache away from any specific persistence mechanism. It has functions to look up cache items, store cache items and manage transactions. Any of these can be no-op operations if you don't need them. A GORM implementation is provided.

import "github.com/pilotso11/lazywritercache"

type CacheReaderWriter[K,T] interface {
	Find(key K, tx any) (T, error)
	Save(item T, tx any) error
	BeginTx() (tx any, err error)
	CommitTx(tx any)
	Info(msg string)
	Warn(msg string)
}

2. A Cachable structure

Cacheable strucs must implement a function to return their key and a function to sync up their data post storage. If your database schema as dynamically allocated primary keys you need this because you won't know the key at save time (I'm assuming you have a secondary unique key that you are using to lookup the items).

type Cacheable interface {
	Key() any
	CopyKeyDataFrom(from Cacheable) Cacheable // This should copy in DB only ID fields.  If gorm.Model is implement this is ID, creationTime, updateTime, deleteTime
}

For example, if you were using GORM and your type implements gorm.Model your copy function would look like:

func (data MyData) CopyKeyDataFrom(from lazywritercache.Cacheable) lazywritercache.CacheItem {
    fromData := from.(MyData)  
    
    data.ID = fromData.ID
    data.CreatedAt = fromData.CreatedAt
    data.UpdatedAt = fromData.UpdatedAt
    data.DeletedAt = fromData.DeletedAt
    
    return data
}

Two sample CacheReaderWriter implementations are provided. A no-op implementation for unit testing and a simple GORM implementation that looks up items with a key field and saves them back.

The test code is an illustration of the no-op implementation. The example folder has as sample that use GORM and postgres.

Assuming you have a CacheReaderWriter implementation, and you've created your Cacheable, then creating your cache is straightforward.

readerWriter := lazywritercache.NewGormCacheReaderWriter[string, Person](db, NewEmptyPerson)
cacheConfig := lazywritercache.NewDefaultConfig[string, Person](readerWriter)
cache := lazywritercache.NewLazyWriterCache[string, Person](cacheConfig)

Documentation

Index

• type CacheReaderWriter
• type CacheStats
  • func (s *CacheStats) JSON() string
  • func (s *CacheStats) String() string
• type Cacheable
• type Config
  • func NewDefaultConfig[K comparable, T Cacheable](handler CacheReaderWriter[K, T]) Config[K, T]
• type EmptyCacheable
  • func (i EmptyCacheable) CopyKeyDataFrom(from Cacheable) Cacheable
  • func (i EmptyCacheable) Key() any
• type GormCacheReaderWriter
  • func NewGormCacheReaderWriter[K comparable, T Cacheable](db *gorm.DB, itemTemplate func(key K) T) GormCacheReaderWriter[K, T]
  • func (g GormCacheReaderWriter[K, T]) BeginTx() (tx any, err error)
  • func (g GormCacheReaderWriter[K, T]) CommitTx(tx any)
  • func (g GormCacheReaderWriter[K, T]) Find(key K, tx any) (T, error)
  • func (g GormCacheReaderWriter[K, T]) Info(msg string)
  • func (g GormCacheReaderWriter[K, T]) Save(item T, tx any) error
  • func (g GormCacheReaderWriter[K, T]) Warn(msg string)
• type LazyWriterCache
  • func NewLazyWriterCache[K comparable, T Cacheable](cfg Config[K, T]) *LazyWriterCache[K, T]
  • func (c *LazyWriterCache[K, T]) ClearDirty()
  • func (c *LazyWriterCache[K, T]) Flush()
  • func (c *LazyWriterCache[K, T]) GetAndLock(key K) (T, bool)
  • func (c *LazyWriterCache[K, T]) GetAndRelease(key K) (T, bool)
  • func (c *LazyWriterCache[K, T]) GetFromLocked(key K) (T, bool)
  • func (c *LazyWriterCache[K, T]) Invalidate()
  • func (c *LazyWriterCache[K, T]) Lock()
  • func (c *LazyWriterCache[K, T]) Range(action func(k K, v T) bool) (n int)
  • func (c *LazyWriterCache[K, T]) Release()
  • func (c *LazyWriterCache[K, T]) Save(item T)
  • func (c *LazyWriterCache[K, T]) Shutdown()
• type NoOpReaderWriter
  • func NewNoOpReaderWriter[T Cacheable](itemTemplate func(key any) T, forcePanics ...bool) NoOpReaderWriter[T]
  • func (g NoOpReaderWriter[T]) BeginTx() (tx interface{}, err error)
  • func (g NoOpReaderWriter[T]) CommitTx(_ interface{})
  • func (g NoOpReaderWriter[T]) Find(key string, _ interface{}) (T, error)
  • func (g NoOpReaderWriter[T]) Info(msg string)
  • func (g NoOpReaderWriter[T]) Save(_ T, _ interface{}) error
  • func (g NoOpReaderWriter[T]) Warn(msg string)

Types

type CacheReaderWriter

type CacheReaderWriter[K comparable, T Cacheable] interface {
	Find(key K, tx any) (T, error)
	Save(item T, tx any) error
	BeginTx() (tx any, err error)
	CommitTx(tx any)
	Info(msg string)
	Warn(msg string)
}

type CacheStats

type CacheStats struct {
	Hits        atomic.Int64
	Misses      atomic.Int64
	Stores      atomic.Int64
	Evictions   atomic.Int64
	DirtyWrites atomic.Int64
}

func (*CacheStats) JSON

func (s *CacheStats) JSON() string

func (*CacheStats) String

func (s *CacheStats) String() string

type Cacheable

type Cacheable interface {
	Key() any
	CopyKeyDataFrom(from Cacheable) Cacheable // This should copy in DB only ID fields.  If gorm.Model is implement this is ID, creationTime, updateTime, deleteTime
}

type Config

type Config[K comparable, T Cacheable] struct {
	Limit        int
	LookupOnMiss bool // If true, a cache miss will query the DB, with associated performance hit!
	WriteFreq    time.Duration
	PurgeFreq    time.Duration
	// contains filtered or unexported fields
}

func NewDefaultConfig

func NewDefaultConfig[K comparable, T Cacheable](handler CacheReaderWriter[K, T]) Config[K, T]

type EmptyCacheable

type EmptyCacheable struct {
}

EmptyCacheable - placeholder used as a return value if the cache can't find anything

func (EmptyCacheable) CopyKeyDataFrom

func (i EmptyCacheable) CopyKeyDataFrom(from Cacheable) Cacheable

func (EmptyCacheable) Key

func (i EmptyCacheable) Key() any

type GormCacheReaderWriter

type GormCacheReaderWriter[K comparable, T Cacheable] struct {
	UseTransactions     bool
	PreloadAssociations bool
	// contains filtered or unexported fields
}

GormCacheReaderWriter is the GORM implementation of the CacheReaderWriter. It should work with any DB GORM supports. It's been tested with Postgres and Mysql.

UseTransactions should be set to true unless you have a really good reason not to. If set to true t find and save operation is done in a single transaction which ensures no collisions with a parallel writer. But also the flush is done in a transaction which is much faster. You don't really want to set this to false except for debugging.

If PreloadAssociations is true then calls to db.Find are implement as db.Preload(clause.Associations).Find which will cause GORM to eagerly fetch any joined objects.

func NewGormCacheReaderWriter

func NewGormCacheReaderWriter[K comparable, T Cacheable](db *gorm.DB, itemTemplate func(key K) T) GormCacheReaderWriter[K, T]

NewGormCacheReaderWriter creates a GORM Cache Reader Writer supply a new item creator and a wrapper to db.Save() that first unwraps item Cacheable to your type. THe itemTemplate function is used to create new items with only the key, which are then used by db.Find() to find your item by key.

func (GormCacheReaderWriter[K, T]) BeginTx

func (g GormCacheReaderWriter[K, T]) BeginTx() (tx any, err error)

func (GormCacheReaderWriter[K, T]) CommitTx

func (g GormCacheReaderWriter[K, T]) CommitTx(tx any)

func (GormCacheReaderWriter[K, T]) Find

func (g GormCacheReaderWriter[K, T]) Find(key K, tx any) (T, error)

func (GormCacheReaderWriter[K, T]) Info

func (g GormCacheReaderWriter[K, T]) Info(msg string)

func (GormCacheReaderWriter[K, T]) Save

func (g GormCacheReaderWriter[K, T]) Save(item T, tx any) error

func (GormCacheReaderWriter[K, T]) Warn

func (g GormCacheReaderWriter[K, T]) Warn(msg string)

type LazyWriterCache

type LazyWriterCache[K comparable, T Cacheable] struct {
	Config[K, T]

	CacheStats
	// contains filtered or unexported fields
}

LazyWriterCache This cache implementation assumes this process OWNS the database There is no synchronisation on Save or any error handling if the DB is in an inconsistent state To use this in a distributed mode, we'd need to replace it with something like REDIS that keeps a distributed cache for update, and then use a single writer to persist to the DB - with some clustering strategy

func NewLazyWriterCache

func NewLazyWriterCache[K comparable, T Cacheable](cfg Config[K, T]) *LazyWriterCache[K, T]

NewLazyWriterCache creates a new cache and starts up its lazy db writer ticker. Users need to pass a DB Find function and ensure their objects implement lazywritercache.Cacheable which has two functions, one to return the Key() and the other to copy key variables into the cached item from the DB loaded item. (i.e. the number ID, update time etc.) because the lazy write cannot just "Save" the item back to the DB as it might have been updated during the lazy write as its asynchronous.

func (*LazyWriterCache[K, T]) ClearDirty

func (c *LazyWriterCache[K, T]) ClearDirty()

ClearDirty forcefully empties the dirty queue, for example if the cache has just been forcefully loaded from the db, and you want to avoid the overhead of retrying to write it all, then ClearDirty may be useful. ClearDirty will fail if the cache is not locked

func (*LazyWriterCache[K, T]) Flush

func (c *LazyWriterCache[K, T]) Flush()

Flush forces all dirty items to be written to the database. Flush should be called before exiting the application otherwise dirty writes will be lost. As the lazy writer is set up with a timer this should only need to be called at exit.

func (*LazyWriterCache[K, T]) GetAndLock

func (c *LazyWriterCache[K, T]) GetAndLock(key K) (T, bool)

GetAndLock will lock and load an item from the cache. It does not release the lock so always call Release after calling GetAndLock, even if nothing is found Useful if you are checking to see if something is there and then planning to update it.

func (*LazyWriterCache[K, T]) GetAndRelease

func (c *LazyWriterCache[K, T]) GetAndRelease(key K) (T, bool)

GetAndRelease will lock and load an item from the cache and then release the lock.

func (*LazyWriterCache[K, T]) GetFromLocked

func (c *LazyWriterCache[K, T]) GetFromLocked(key K) (T, bool)

GetFromLocked will load an item from a previously locked cache.

func (*LazyWriterCache[K, T]) Invalidate

func (c *LazyWriterCache[K, T]) Invalidate()

Invalidate flushes and empties the cache forcing reloads

func (*LazyWriterCache[K, T]) Lock

func (c *LazyWriterCache[K, T]) Lock()

Lock the cache. This will panic if the cache is already locked when the mutex is entered.

func (*LazyWriterCache[K, T]) Range

func (c *LazyWriterCache[K, T]) Range(action func(k K, v T) bool) (n int)

Range over all the keys and maps. The cache is locked for the duration of the range function to avoid synchronous access issues. If the Range action is expensive, consider using the lock free implementation in lockfree/LazyWriterCacheLF.

As with other Range functions return true to continue iterating or false to stop.

func (*LazyWriterCache[K, T]) Release

func (c *LazyWriterCache[K, T]) Release()

Release the Lock. It will panic if not already locked

func (*LazyWriterCache[K, T]) Save

func (c *LazyWriterCache[K, T]) Save(item T)

Save updates an item in the cache. The cache must already have been locked, if not we will panic.

The expectation is GetAndLock has been called first, and a Release has been deferred.

func (*LazyWriterCache[K, T]) Shutdown

func (c *LazyWriterCache[K, T]) Shutdown()

Shutdown signals to the cache it should stop any running goroutines. This does not Flush the cache first, so it is recommended call Flush beforehand.

type NoOpReaderWriter

type NoOpReaderWriter[T Cacheable] struct {
	// contains filtered or unexported fields
}

func NewNoOpReaderWriter

func NewNoOpReaderWriter[T Cacheable](itemTemplate func(key any) T, forcePanics ...bool) NoOpReaderWriter[T]

func (NoOpReaderWriter[T]) BeginTx

func (g NoOpReaderWriter[T]) BeginTx() (tx interface{}, err error)

func (NoOpReaderWriter[T]) CommitTx

func (g NoOpReaderWriter[T]) CommitTx(_ interface{})

func (NoOpReaderWriter[T]) Find

func (g NoOpReaderWriter[T]) Find(key string, _ interface{}) (T, error)

func (NoOpReaderWriter[T]) Info

func (g NoOpReaderWriter[T]) Info(msg string)

func (NoOpReaderWriter[T]) Save

func (g NoOpReaderWriter[T]) Save(_ T, _ interface{}) error

func (NoOpReaderWriter[T]) Warn

func (g NoOpReaderWriter[T]) Warn(msg string)