goswarm

README

goswarm

Go Stale While Asynchronously Revalidate Memoization

DESCRIPTION

goswarm is a library written in Go for storing the results of expensive function calls and returning the cached result when the same input key occurs again.

In addition to the examples provided below, documentation can be found at .

    simple, err := goswarm.NewSimple(nil)
    if err != nil {
        log.Fatal(err)
    }

    // you can store any Go type in a Swarm
    simple.Store("someKeyString", 42)
    simple.Store("anotherKey", struct{}{})
    simple.Store("yetAnotherKey", make(chan interface{}))

    // but when you retrieve it, you are responsible to perform type assertions
    key := "yetAnotherKey"
    value, ok := simple.Load(key)
    if !ok {
        panic(fmt.Errorf("cannot find %q", key))
    }
    value = value.(chan interface{})

    simple.Delete("anotherKey")

As seen above, goswarm's API is similar to that of any associative array, also known as a map in Go terminology: It allows storing a value to be retrieved using a specified key, and overwriting any value that might already be present in the map. It allows loading the value associated with the specified key already stored in the map. It allows deleting a specified key and its associated value stored in the map.

goswarm differs from most traditional associative array APIs in that it provides a method to load the value associated with a specified key, and if that key is not found in the map, to invoke a specified lookup function to fetch the value for that key.

    simple, err := goswarm.NewSimple(&goswarm.Config{
        Lookup: func(key string) (interface{}, error) {
            // TODO: do slow calculation or make a network call
            result := key // example
            return result, nil
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    defer func() { _ = simple.Close() }()

    value, err := simple.Query("%version")
    if !err {
        panic(fmt.Errorf("cannot retrieve value for key %q: %s", key, err))
    }
    fmt.Printf("The value is: %v\n", value)

Stale-While-Revalidate and Stale-If-Error

In addition, goswarm provides stale-while-revalidate and stale-if-error compatible features in its simple API.

Stale-While-Revalidate

When the user requests the value associated with a particular key, goswarm determines whether that key-value pair is present, and if so, whether that value has become stale. If the value is stale, goswarm will return the previously stored value to the client, but spawn an asynchronous routine to fetch a new value for that key and store that value in the map to be used for future queries.

When the user requests the value associated with a particular key that has expired, goswarm will not return that value, but rather synchronously fetch an updated value to store in the map and return to the user.

Stale-If-Error

When fetching a new value to replace a value that has become stale, the lookup callback funciton might return an error. Perhaps the remote network resource used to fetch responses is offline. In these cases, goswarm will not overwrite the stale value with the error, but continue to serve the stale value until the lookup callback function returns a new value rather than an error, or the value expires, in which case, the error is returned to the user.

Periodic Removal Of Expired Keys

If GCPeriodicty configuration value is greater than the zero-value for time.Duration, goswarm spawns a separate go-routine that invokes the GC method periodically, removing all key-value pairs from the data map that have an expired time. When this feature is used, the Close method must be invoked to stop and release that go-routine.

    simple, err := goswarm.NewSimple(&goswarm.Config{
        GoodExpiryDuration: 24 * time.Hour,
        BadExpiryDuration:  5 * time.Minute,
        GCPeriodicity:      time.Hour,
        Lookup:             func(key string) (interface{}, error) {
            // TODO: do slow calculation or make a network call
            result := key // example
            return result, nil
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    defer func() { _ = simple.Close() }()

Note About Choosing Stale and Expiry Durations

Different applications may require different logic, however if your application needs to continue processing data and serving requests even when a downstream dependency is subject to frequent high latency periods or faults, it is recommended to set the GoodStaleDuration period to a low enough value to ensure data is reasonably up-to-date, but extend the GoodExpiryDuration to be long enough that your application can still operate using possibly stale data. The goswarm library will repeatedly attempt to fetch a new value from the downstream service, but until a defined very long period of time transpires, your service will be relatively insulated from these type of downstream faults and latencies.

    simple, err := goswarm.NewSimple(&goswarm.Config{
        GoodStaleDuration:  time.Minute,
        GoodExpiryDuration: 24 * time.Hour,
        BadStaleDuration:   time.Minute,
        BadExpiryDuration:  5 * time.Minute,
        GCPeriodicity:      time.Hour,
        Lookup:             func(key string) (interface{}, error) {
            // TODO: do slow calculation or make a network call
            result := key // example
            return result, nil
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    defer func() { _ = simple.Close() }()

    value, err := simple.Query("%version")
    if !err {
        panic(fmt.Errorf("cannot retrieve value for key %q: %s", key, err))
    }
    fmt.Printf("The value is: %v\n", value)

Documentation

Index

• type Config
• type Querier
• type Simple
  • func NewSimple(config *Config) (*Simple, error)
  • func (s *Simple) Close() error
  • func (s *Simple) Delete(key string)
  • func (s *Simple) GC()
  • func (s *Simple) Load(key string) (interface{}, bool)
  • func (s *Simple) LoadTimedValue(key string) *TimedValue
  • func (s *Simple) Query(key string) (interface{}, error)
  • func (s *Simple) Range(callback func(key string, value *TimedValue))
  • func (s *Simple) RangeBreak(callback func(key string, value *TimedValue) bool) bool
  • func (s *Simple) Stats() Stats
  • func (s *Simple) Store(key string, value interface{})
  • func (s *Simple) Update(key string)
• type Stats
• type TimedValue
  • func (tv *TimedValue) IsExpired() bool
  • func (tv *TimedValue) IsExpiredAt(when time.Time) bool
  • func (tv *TimedValue) IsStale() bool
  • func (tv *TimedValue) IsStaleAt(when time.Time) bool
  • func (tv *TimedValue) Status() TimedValueStatus
  • func (tv *TimedValue) StatusAt(when time.Time) TimedValueStatus
• type TimedValueStatus

Types

type Config

type Config struct {
	// GoodStaleDuration specifies how long a value remains fresh in the data map. A zero-value
	// time.Duration value implies the value never stales, and updates will only be fetched
	// after the value expires.
	GoodStaleDuration time.Duration

	// GoodExpiryDuration specifies how long a value is allowed to be served from the data
	// map. A zero-value time.Duration value implies the value never expires and can always be
	// served.
	GoodExpiryDuration time.Duration

	// BadStaleDuration specifies how long an error result remains fresh in the data map. A
	// zero-value time.Duration value implies that the error result is stale as soon as it's
	// inserted into the data map, and Querying the key again will trigger an asynchronous
	// lookup of the key to try to replace the error value with a good value.
	BadStaleDuration time.Duration

	// BadExpiryDuration specifies how long an error result is allowed to be served from the
	// data map. A zero-value time.Duration value implies the value never expires and can always
	// be served.
	BadExpiryDuration time.Duration

	// Lookup specifies the user callback function to invoke when looking up the value to be
	// associated with a stale key, expired key, or a key that has yet to be loaded into the
	// data map.
	Lookup func(string) (interface{}, error)

	// GCPeriodicity specifies how frequently the data map purges expired entries.
	GCPeriodicity time.Duration

	// GCTimeout specifies how long the GC should wait for values to unlock.
	GCTimeout time.Duration
}

Config specifies the configuration parameters for a Swarm instance.

type Querier

type Querier interface {
	Query(string) (interface{}, error)
}

Querier specifies a type that provides memoizing the results of expensive function calls and returning the cached result when the same input key occurs again.

type Simple

type Simple struct {
	// contains filtered or unexported fields
}

Simple memoizes responses from a Querier, providing very low-level time-based control of how values go stale or expire. When a new value is stored in the Simple instance, if it is a TimedValue item--or a pointer to a TimedValue item)--the data map will use the provided Stale and Expiry values. If the new value is not a TimedValue instance or pointer to a TimedValue instance, then the Simple instance wraps the value in a TimedValue struct, and adds the Simple instance's stale and expiry durations to the current time and stores the resultant TimedValue instance.

func NewSimple

func NewSimple(config *Config) (*Simple, error)

NewSimple returns Swarm that attempts to respond to Query methods by consulting its TTL cache, then directing the call to the underlying Querier if a valid response is not stored. Note this function accepts a pointer so creating an instance with defaults can be done by passing a nil value rather than a pointer to a Config instance.

simple, err := goswarm.NewSimple(&goswarm.Simple{
    GoodStaleDuration:  time.Minute,
    GoodExpiryDuration: 24 * time.Hour,
    BadStaleDuration:   time.Minute,
    BadExpiryDuration:  5 * time.Minute,
    Lookup:             func(key string) (interface{}, error) {
        // TODO: do slow calculation or make a network call
        result := key // example
        return result, nil
    },
})
if err != nil {
    log.Fatal(err)
}
defer func() { _ = simple.Close() }()

func (*Simple) Close

func (s *Simple) Close() error

Close releases all memory and go-routines used by the Simple swarm. If during instantiation, GCPeriodicity was greater than the zero-value for time.Duration, this method may block while completing any in progress GC run.

func (*Simple) Delete

func (s *Simple) Delete(key string)

Delete removes the key and associated value from the data map.

func (*Simple) GC

func (s *Simple) GC()

GC examines all key value pairs in the Simple swarm and deletes those whose values have expired.

func (*Simple) Load

func (s *Simple) Load(key string) (interface{}, bool)

Load returns the value associated with the specified key, and a boolean value indicating whether or not the key was found in the map.

func (*Simple) LoadTimedValue

func (s *Simple) LoadTimedValue(key string) *TimedValue

LoadTimedValue returns the TimedValue associated with the specified key, or false if the key is not found in the map.

func (*Simple) Query

func (s *Simple) Query(key string) (interface{}, error)

Query loads the value associated with the specified key from the data map. When a stale value is found on Query, at most one asynchronous lookup of a new value is triggered, and the current value is returned from the data map. When no value or an expired value is found on Query, a synchronous lookup of a new value is triggered, then the new value is stored and returned.

func (*Simple) Range

func (s *Simple) Range(callback func(key string, value *TimedValue))

Range invokes specified callback function for each non-expired key in the data map. Each key-value pair is independently locked until the callback function invoked with the specified key returns. This method does not block access to the Simple instance, allowing keys to be added and removed like normal even while the callbacks are running.

func (*Simple) RangeBreak

func (s *Simple) RangeBreak(callback func(key string, value *TimedValue) bool) bool

RangeBreak invokes specified callback function for each non-expired key in the data map. Each key-value pair is independently locked until the callback function invoked with the specified key returns. This method does not block access to the Simple instance, allowing keys to be added and removed like normal even while the callbacks are running. When the callback returns true, this function performs an early termination of enumerating the cache, returning true it its caller.

func (*Simple) Stats

func (s *Simple) Stats() Stats

Stats returns a snapshot of the cache's statistics. Note all statistics will be reset when this method is invoked, allowing the client to determine the number of each respective events that have taken place since the previous time this method was invoked.

func (*Simple) Store

func (s *Simple) Store(key string, value interface{})

Store saves the key-value pair to the cache, overwriting whatever was previously stored.

func (*Simple) Update

func (s *Simple) Update(key string)

Update forces an update of the value associated with the specified key.

type Stats

type Stats struct {
	Count        int64 // Count represents the number of items in the cache.
	Creates      int64 // Creates represents how many new cache items were created since the previous Stats call.
	Deletes      int64 // Deletes represents how many Delete calls were made since the previous Stats call. Note this counts when Delete is called with a key that is not in the cache.
	Evictions    int64 // Evictions represents how many key-value pairs were evicted since the previous Stats call.
	Hits         int64 // Hits represents how many Load and Query calls found and returned a Fresh item.
	LookupErrors int64 // LookupErrors represents the number of Lookup invocations that returned an error since the previous Stats call.
	Misses       int64 // Misses represents how many Load and Query calls either did not find the item, or found an expired item.
	Queries      int64 // Queries represents how many Load and Query calls were made since the previous Stats call.
	Stales       int64 // Stales represents how many Load and Query calls found and returned a Stale item.
	Stores       int64 // Stores represents how many Store calls were made since the previous Stats call.
	Updates      int64 // Updates represents how many Update calls were made since the previous Stats call.
}

Stats contains various cache statistics.

type TimedValue

type TimedValue struct {
	// Value stores the datum returned by the lookup function.
	Value interface{}

	// Err stores the error returned by the lookup function.
	Err error

	// Stale stores the time at which the value becomes stale. On Query, a stale
	// value will trigger an asynchronous lookup of a replacement value, and the
	// original value is returned. A zero-value for Stale implies the value
	// never goes stale, and querying the key associated for this value will
	// never trigger an asynchronous lookup of a replacement value.
	Stale time.Time

	// Expiry stores the time at which the value expires. On Query, an expired
	// value will block until a synchronous lookup of a replacement value is
	// attempted. Once the lookup returns, the Query method will return with the
	// new value or the error returned by the lookup function.
	Expiry time.Time

	// Created stores the time at which the value was created.
	Created time.Time
}

TimedValue couples a value or the error with both a stale and expiry time for the value and error.

func (*TimedValue) IsExpired

func (tv *TimedValue) IsExpired() bool

IsExpired returns true when the value is expired.

A value is expired when its non-zero expiry time is before the current time, or when the value represents an error and expiry time is the time.Time zero-value.

func (*TimedValue) IsExpiredAt

func (tv *TimedValue) IsExpiredAt(when time.Time) bool

IsExpiredAt returns true when the value is expired at the specified time.

A value is expired when its non-zero expiry time is before the specified time, or when the value represents an error and expiry time is the time.Time zero-value.

func (*TimedValue) IsStale

func (tv *TimedValue) IsStale() bool

IsStale returns true when the value is stale.

A value is stale when its non-zero stale time is before the current time, or when the value represents an error and stale time is the time.Time zero-value.

func (*TimedValue) IsStaleAt

func (tv *TimedValue) IsStaleAt(when time.Time) bool

IsStaleAt returns true when the value is stale at the specified time.

A value is stale when its non-zero stale time is before the specified time, or when the value represents an error and stale time is the time.Time zero-value.

func (*TimedValue) Status

func (tv *TimedValue) Status() TimedValueStatus

Status returns Fresh, Stale, or Exired, depending on the status of the TimedValue item at the current time.

func (*TimedValue) StatusAt

func (tv *TimedValue) StatusAt(when time.Time) TimedValueStatus

StatusAt returns Fresh, Stale, or Exired, depending on the status of the TimedValue item at the specified time.

type TimedValueStatus

type TimedValueStatus int

TimedValueStatus is an enumeration of the states of a TimedValue: Fresh, Stale, or Expired.

const (
	// Fresh items are not yet Stale, and will be returned immediately on Query
	// without scheduling an asynchronous Lookup.
	Fresh TimedValueStatus = iota

	// Stale items have exceeded their Fresh status, and will be returned
	// immediately on Query, but will also schedule an asynchronous Lookup if a
	// Lookup for this key is not yet in flight.
	Stale

	// Expired items have exceeded their Fresh and Stale status, and will be
	// evicted during the next background cache eviction loop, controlled by
	// GCPeriodicity parameter, or upon Query, which will cause the Query to
	// block until a new value can be obtained from the Lookup function.
	Expired
)