lastcache

README

LastCache

LastCache is a go module that implements stale-while-revalidate and stale-if-error in-memory cache strategy.

stale-if-error

In the event of an error when fetching fresh data, the cache serves stale (expired) data for a specified period (Config.ExtendTTL). This ensures a fallback mechanism to provide some data even when the retrieval process encounters errors.
LoadOrStore function is based on this strategy.

stale-while-revalidate

Stale (expired) data is served to caller while a background process runs to refresh the cache.
AsyncLoadOrStore function is based on this strategy.

Examples

package main

import (
	"errors"
	"fmt"
	"time"
	"context"

	"github.com/mbrostami/lastcache"
)

func main() {


	var callbackErr error
	lc := lastcache.New(lastcache.Config{
		GlobalTTL:      1 * time.Nanosecond, // 1 * time.Minute,
		ExtendTTL:      1 * time.Nanosecond, // 10 * time.Second,
		AsyncSemaphore: 1,
	})
	/////////////////////////////////////////////////////
	////////////////// stale-if-error ///////////////////
	// successful callback
	val, err := lc.LoadOrStore("key", func(ctx context.Context, key any) (value any, useStale bool, err error) {
		return "value", false, nil
	})
	fmt.Printf("sync, \tValue: %s, \tStale: %v, \tCallbackErr: %v, \terr: %v\n", val.Value, val.Stale, val.Err, err)

	// failed callback - use stale
	val, err = lc.LoadOrStore("key", func(ctx context.Context, key any) (value any, useStale bool, err error) {
		return nil, true, errors.New("connection lost")
	})
	fmt.Printf("sync, \tValue: %s, \tStale: %v, \tCallbackErr: %v, \terr: %v\n", val.Value, val.Stale, val.Err, err)

	// failed callback - do not use stale
	val, err = lc.LoadOrStore("key", func(ctx context.Context, key any) (value any, useStale bool, err error) {
		return nil, false, errors.New("resource not found")
	})
	fmt.Printf("sync, \tValue: %+v, \terr: %v\n", val, err)

	/////////////////////////////////////////////////////
	///////////////// stale-while-revalidate ////////////
	// successful callback
	val, errChannel, err := lc.AsyncLoadOrStore("key_2", func(ctx context.Context, key any) (value any, err error) {
		return "value", nil
	})

	if errChannel != nil { // check callback error
		callbackErr = <-errChannel
	}
	fmt.Printf("async, \tValue: %s, \tStale: %v, \tCallbackErr: %v, \terr: %v\n", val.Value, val.Stale, callbackErr, err)

	// failed callback
	val, errChannel, err = lc.AsyncLoadOrStore("key_2", func(ctx context.Context, key any) (value any, err error) {
		return nil, errors.New("some query error")
	})
	if errChannel != nil { // check callback error
		callbackErr = <-errChannel
	}
	fmt.Printf("async, \tValue: %s, \tStale: %v, \tCallbackErr: %v, \terr: %v\n", val.Value, val.Stale, callbackErr, err)
}

Output:

sync, 	Value: value, 	Stale: false, 	CallbackErr: <nil>, 	err: <nil>
sync, 	Value: value, 	Stale: true, 	CallbackErr: connection lost, 	err: <nil>
sync, 	Value: <nil>, 	err: resource not found
async, 	Value: value, 	Stale: false, 	CallbackErr: <nil>, 	err: <nil>
async, 	Value: value, 	Stale: true, 	CallbackErr: some query error, 	err: <nil>

Documentation

Overview

Package lastcache implements stale-while-revalidate and stale-if-error in-memory cache strategy.

stale-if-error
In the event of an error when fetching fresh data, the cache serves stale (expired) data for a specified period (Config.ExtendTTL). This ensures a fallback mechanism to provide some data even when the retrieval process encounters errors.
`LoadOrStore` function is based on this strategy.

stale-while-revalidate
Stale (expired) data is served to caller while a background process runs to refresh the cache.
`AsyncLoadOrStore` function is based on this strategy.

Index

• type AsyncCallback
• type Cache
  • func New(config Config) *Cache
  • func (c *Cache) AsyncLoadOrStore(key any, callback AsyncCallback) (Entry, chan error, error)
  • func (c *Cache) AsyncLoadOrStoreWithCtx(ctx context.Context, key any, callback AsyncCallback) (Entry, chan error, error)
  • func (c *Cache) Delete(key any)
  • func (c *Cache) LoadOrStore(key any, callback SyncCallback) (Entry, error)
  • func (c *Cache) LoadOrStoreWithCtx(ctx context.Context, key any, callback SyncCallback) (Entry, error)
  • func (c *Cache) Range(f func(key, value any, ttl time.Duration) bool)
  • func (c *Cache) Set(key, value any)
  • func (c *Cache) TTL(key any) time.Duration
• type Config
• type Entry
• type SyncCallback

Types

type AsyncCallback

type AsyncCallback func(ctx context.Context, key any) (value any, err error)

AsyncCallback given a key, should return the value This will be called in a goroutine, considering the AsyncSemaphore

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache use New function to construct a new Cache Must not be copied after first use

func New

func New(config Config) *Cache

New returns new Cache, zero value Config can be passed to use default values

func (*Cache) AsyncLoadOrStore

func (c *Cache) AsyncLoadOrStore(key any, callback AsyncCallback) (Entry, chan error, error)

AsyncLoadOrStore loads the key from cache with respect to the ttl and runs the callback in background

	There will be three cases:

	1. If key exists and is not expired, the value will be returned as Entry
	2. If key doesn't exist, callback will be called to store the value.
	   2.1 If SyncCallback returns error, the error will be returned
	   2.2 If SyncCallback returns no error, the value will be stored and returned
	3. If key is expired, callback will be called in background to replace the value,
	   and existing cache will be returned immediately
	   a buffered error channel size 1 will be returned if cache is stale,
       nil or error will be sent to the error channel

func (*Cache) AsyncLoadOrStoreWithCtx

func (c *Cache) AsyncLoadOrStoreWithCtx(ctx context.Context, key any, callback AsyncCallback) (Entry, chan error, error)

AsyncLoadOrStoreWithCtx check AsyncLoadOrStore

func (*Cache) Delete

func (c *Cache) Delete(key any)

Delete deletes the value for a key.

func (*Cache) LoadOrStore

func (c *Cache) LoadOrStore(key any, callback SyncCallback) (Entry, error)

LoadOrStore loads the key from cache with respect to the ttl.

	There will be three cases:

	1. If key exists and is not expired, the value will be returned as Entry
	2. If key doesn't exist, SyncCallback will be called to store the value.
	   2.1 If SyncCallback returns error, the error will be returned
	   2.2 If SyncCallback returns no error, the value will be stored and returned
	3. If key is expired, SyncCallback will be called to replace the value,
	   3.1 if SyncCallback returns no error, key will be updated with new value and returned
       3.2 if SyncCallback returns error with true useStale,
			cached value will be added to the entry.Value,
   			SyncCallback error will be added to the entry.Err,
			ttl will be extended,
		   	entry and nil will be returned
       3.3 if SyncCallback returns error with false useStale,
			error will be returned

func (*Cache) LoadOrStoreWithCtx

func (c *Cache) LoadOrStoreWithCtx(ctx context.Context, key any, callback SyncCallback) (Entry, error)

LoadOrStoreWithCtx check LoadOrStore

func (*Cache) Range

func (c *Cache) Range(f func(key, value any, ttl time.Duration) bool)

Range calls f sequentially for each key and value and ttl present in the map. If f returns false, range stops the iteration.

Range does not necessarily correspond to any consistent snapshot of the Map's contents: no key will be visited more than once, but if the value for any key is stored or deleted concurrently (including by f), Range may reflect any mapping for that key from any point during the Range call. Range does not block other methods on the receiver; even f itself may call any method on Cache.

Range may be O(N) with the number of elements in the map even if f returns false after a constant number of calls.

func (*Cache) Set

func (c *Cache) Set(key, value any)

Set sets the value and ttl for a key.

func (*Cache) TTL

func (c *Cache) TTL(key any) time.Duration

TTL returns ttl in duration format. The returned value can be negative as well, which in that case means item is already expired. Positive values are valid items in the cache.

type Config

type Config struct {
	// Will be used to set expire time for all the keys
	// If set to negative or 0 the defaultTTL will be used
	GlobalTTL time.Duration

	// Will be used to extend the ttl if cache is stale and callback is failed
	// If set to 0 ttl will not be extended and evey call to LoadOrStore for stale cache will execute the callback
	// Until the callback can return new value with no error
	// In most cases this should be set to the same value as GlobalTTL,
	// Unless the GlobalTTL is too high, or the callback is expensive to be called
	ExtendTTL time.Duration

	// Number of background callbacks allowed in AsyncLoadOrStore
	// If set to 0 the default value defaultSemaphore will be used
	// If you want to use AsyncLoadOrStore this will limit the number of callback calls while cache is expired
	// If callback is too expensive to run, it's better to set to low value (e.g. 1)
	// If you are using different callback processes for different keys, you might want to optimize this value or use another instance of LastCache
	AsyncSemaphore int

	// Context to be used in lifetime of the Cache instance
	// Default is context.TODO()
	Context context.Context
}

Config configuration to construct LastCache

type Entry

type Entry struct {
	// Value retrieved from callback
	Value any

	// Either the cache entry is stale or not
	Stale bool

	// Holds the underlying error if stale cache is used when using LoadOrStore
	// In case of using AsyncLoadOrStore this always will be nil and the underlying error will be returned in channel
	Err error
}

Entry cache entry

type SyncCallback

type SyncCallback func(ctx context.Context, key any) (value any, useStale bool, err error)

SyncCallback given key, should return the value true useStale can be used to retrieve the stale cache