goswarm

README

goswarm

Go Stale While Asynchronously Revalidate Memoization

DESCRIPTION

goswarm is a library written in Go for storing the results of expensive function calls and returning the cached result when the same input key occurs again.

In addition to the examples provided below, documentation can be found at .

    simple, err := goswarm.NewSimple(nil)
    if err != nil {
        log.Fatal(err)
    }

    // you can store any Go type in a Swarm
    simple.Store("someKeyString", 42)
    simple.Store("anotherKey", struct{}{})
    simple.Store("yetAnotherKey", make(chan interface{}))

    // but when you retrieve it, you are responsible to perform type assertions
    key := "yetAnotherKey"
    value, ok := simple.Load(key)
    if !ok {
        panic(fmt.Errorf("cannot find %q", key))
    }
    value = value.(chan interface{})

    simple.Delete("anotherKey")

As seen above, goswarm's API is similar to that of any associative array, also known as a map in Go terminology: It allows storing a value to be retrieved using a specified key, and overwriting any value that might already be present in the map. It allows loading the value associated with the specified key already stored in the map. It allows deleting a specified key and its associated value stored in the map.

goswarm differs from most traditional associative array APIs in that it provides a method to load the value associated with a specified key, and if that key is not found in the map, to invoke a specified lookup function to fetch the value for that key.

    simple, err := goswarm.NewSimple(&goswarm.Config{
        Lookup: func(key string) (interface{}, error) {
            // TODO: do slow calculation or make a network call
            result := key // example
            return result, nil
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    defer func() { _ = simple.Close() }()

    value, err := simple.Query("%version")
    if !err {
        panic(fmt.Errorf("cannot retrieve value for key %q: %s", key, err))
    }
    fmt.Printf("The value is: %v\n", value)

Stale-While-Revalidate and Stale-If-Error

In addition, goswarm provides stale-while-revalidate and stale-if-error compatible features in its simple API.

Stale-While-Revalidate

When the user requests the value associated with a particular key, goswarm determines whether that key-value pair is present, and if so, whether that value has become stale. If the value is stale, goswarm will return the previously stored value to the client, but spawn an asynchronous routine to fetch a new value for that key and store that value in the map to be used for future queries.

When the user requests the value associated with a particular key that has expired, goswarm will not return that value, but rather synchronously fetch an updated value to store in the map and return to the user.

Stale-If-Error

When fetching a new value to replace a value that has become stale, the lookup callback funciton might return an error. Perhaps the remote network resource used to fetch responses is offline. In these cases, goswarm will not overwrite the stale value with the error, but continue to serve the stale value until the lookup callback function returns a new value rather than an error, or the value expires, in which case, the error is returned to the user.

Periodic Removal Of Expired Keys

If GCPeriodicty configuration value is greater than the zero-value for time.Duration, goswarm spawns a separate go-routine that invokes the GC method periodically, removing all key-value pairs from the data map that have an expired time. When this feature is used, the Close method must be invoked to stop and release that go-routine.

    simple, err := goswarm.NewSimple(&goswarm.Config{
        GoodExpiryDuration: 24 * time.Hour,
        BadExpiryDuration:  5 * time.Minute,
        GCPeriodicity:      time.Hour,
        Lookup:             func(key string) (interface{}, error) {
            // TODO: do slow calculation or make a network call
            result := key // example
            return result, nil
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    defer func() { _ = simple.Close() }()

Note About Choosing Stale and Expiry Durations

Different applications may require different logic, however if your application needs to continue processing data and serving requests even when a downstream dependency is subject to frequent high latency periods or faults, it is recommended to set the GoodStaleDuration period to a low enough value to ensure data is reasonably up-to-date, but extend the GoodExpiryDuration to be long enough that your application can still operate using possibly stale data. The goswarm library will repeatedly attempt to fetch a new value from the downstream service, but until a defined very long period of time transpires, your service will be relatively insulated from these type of downstream faults and latencies.

    simple, err := goswarm.NewSimple(&goswarm.Config{
        GoodStaleDuration:  time.Minute,
        GoodExpiryDuration: 24 * time.Hour,
        BadStaleDuration:   time.Minute,
        BadExpiryDuration:  5 * time.Minute,
        GCPeriodicity:      time.Hour,
        Lookup:             func(key string) (interface{}, error) {
            // TODO: do slow calculation or make a network call
            result := key // example
            return result, nil
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    defer func() { _ = simple.Close() }()

    value, err := simple.Query("%version")
    if !err {
        panic(fmt.Errorf("cannot retrieve value for key %q: %s", key, err))
    }
    fmt.Printf("The value is: %v\n", value)

Documentation

Index

• type Config
• type Querier
• type Simple
  • func NewSimple(config *Config) (*Simple, error)
  • func (s *Simple) Close() error
  • func (s *Simple) Delete(key string)
  • func (s *Simple) GC()
  • func (s *Simple) Load(key string) (interface{}, bool)
  • func (s *Simple) Query(key string) (interface{}, error)
  • func (s *Simple) Range(callback func(key string, value *TimedValue))
  • func (s *Simple) Stats() Stats
  • func (s *Simple) Store(key string, value interface{})
  • func (s *Simple) Update(key string)
• type Stats
• type TimedValue
  • func (tv *TimedValue) IsExpired() bool
  • func (tv *TimedValue) IsStale() bool

Types

type Config

type Config struct {
	// GoodStaleDuration specifies how long a value remains fresh in the data map. A zero-value
	// time.Duration value implies the value never stales, and updates will only be fetched
	// after the value expires.
	GoodStaleDuration time.Duration

	// GoodExpiryDuration specifies how long a value is allowed to be served from the data
	// map. A zero-value time.Duration value implies the value never expires and can always be
	// served.
	GoodExpiryDuration time.Duration

	// BadStaleDuration specifies how long an error result remains fresh in the data map. A
	// zero-value time.Duration value implies that the error result is stale as soon as it's
	// inserted into the data map, and Querying the key again will trigger an asynchronous
	// lookup of the key to try to replace the error value with a good value.
	BadStaleDuration time.Duration

	// BadExpiryDuration specifies how long an error result is allowed to be served from the
	// data map. A zero-value time.Duration value implies the value never expires and can always
	// be served.
	BadExpiryDuration time.Duration

	// Lookup specifies the user callback function to invoke when looking up the value to be
	// associated with a stale key, expired key, or a key that has yet to be loaded into the
	// data map.
	Lookup func(string) (interface{}, error)

	// GCPeriodicity specifies how frequently the data map purges expired entries.
	GCPeriodicity time.Duration

	// GCTimeout specifies how long the GC should wait for values to unlock.
	GCTimeout time.Duration
}

Config specifies the configuration parameters for a Swarm instance.

type Querier

type Querier interface {
	Query(string) (interface{}, error)
}

Querier specifies a type that provides memoizing the results of expensive function calls and returning the cached result when the same input key occurs again.

type Simple

type Simple struct {
	// contains filtered or unexported fields
}

Simple memoizes responses from a Querier, providing very low-level time-based control of how values go stale or expire. When a new value is stored in the Simple instance, if it is a TimedValue item--or a pointer to a TimedValue item)--the data map will use the provided Stale and Expiry values. If the new value is not a TimedValue instance or pointer to a TimedValue instance, then the Simple instance wraps the value in a TimedValue struct, and adds the Simple instance's stale and expiry durations to the current time and stores the resultant TimedValue instance.

func NewSimple

func NewSimple(config *Config) (*Simple, error)

NewSimple returns Swarm that attempts to respond to Query methods by consulting its TTL cache, then directing the call to the underlying Querier if a valid response is not stored. Note this function accepts a pointer so creating an instance with defaults can be done by passing a nil value rather than a pointer to a Config instance.

simple, err := goswarm.NewSimple(&goswarm.Simple{
    GoodStaleDuration:  time.Minute,
    GoodExpiryDuration: 24 * time.Hour,
    BadStaleDuration:   time.Minute,
    BadExpiryDuration:  5 * time.Minute,
    Lookup:             func(key string) (interface{}, error) {
        // TODO: do slow calculation or make a network call
        result := key // example
        return result, nil
    },
})
if err != nil {
    log.Fatal(err)
}
defer func() { _ = simple.Close() }()

func (*Simple) Close

func (s *Simple) Close() error

Close releases all memory and go-routines used by the Simple swarm. If during instantiation, GCPeriodicity was greater than the zero-value for time.Duration, this method may block while completing any in progress GC run.

func (*Simple) Delete

func (s *Simple) Delete(key string)

Delete removes the key and associated value from the data map.

func (*Simple) GC

func (s *Simple) GC()

GC examines all key value pairs in the Simple swarm and deletes those whose values have expired.

func (*Simple) Load

func (s *Simple) Load(key string) (interface{}, bool)

Load returns the value associated with the specified key, and a boolean value indicating whether or not the key was found in the map.

func (*Simple) Query

func (s *Simple) Query(key string) (interface{}, error)

Query loads the value associated with the specified key from the data map. When a stale value is found on Query, at most one asynchronous lookup of a new value is triggered, and the current value is returned from the data map. When no value or an expired value is found on Query, a synchronous lookup of a new value is triggered, then the new value is stored and returned.

func (*Simple) Range

func (s *Simple) Range(callback func(key string, value *TimedValue))

Range invokes specified callback function for each non-expired key in the data map. Each key-value pair is independently locked until the callback function invoked with the specified key returns. This method does not block access to the Simple instance, allowing keys to be added and removed like normal even while the callbacks are running.

func (*Simple) Stats

func (s *Simple) Stats() Stats

Stats returns a snapshot of the cache's statistics. Note that except for the Size statistic, all other statistics will be reset when this method is invoked, allowing the client to determine the number of each respective events that have taken place since the previous time this method was invoked.

func (*Simple) Store

func (s *Simple) Store(key string, value interface{})

Store saves the key-value pair to the cache, overwriting whatever was previously stored.

func (*Simple) Update

func (s *Simple) Update(key string)

Update forces an update of the value associated with the specified key.

type Stats

type Stats struct {
	Creates      int64 // Creates represents how many new cache items were created since the previous Stats call.
	Deletes      int64 // Deletes represents how many Delete calls were made since the previous Stats call.
	Evictions    int64 // Evictions represents how many evictions took place since the previous Stats call.
	Loads        int64 // Loads represents how many Load calls were made since the previous Stats call.
	LookupErrors int64 // LookupErrors represents the number of Lookups invocations that returned an error since the previous Stats call.
	Queries      int64 // Queries represents how many Query calls were made since the previous Stats call.
	QueriesFresh int64 // QueriesFresh represents how many Query calls were answered with fresh data.
	QueriesMiss  int64 // QueriesMiss represents how many Query calls were
	QueriesStale int64 // QueriesStale represents how many Query calls were answered with stale data.
	Size         int64 // Size represents the number of items in the cache.
	Stores       int64 // Stores represents how many Store calls were made since the previous Stats call.
	Updates      int64 // Updates represents how many Update calls were made since the previous Stats call.
}

Stats contains various cache statistics.

type TimedValue

type TimedValue struct {
	// Value stores the datum returned by the lookup function.
	Value interface{}

	// Err stores the error returned by the lookup function.
	Err error

	// Stale stores the time at which the value becomes stale. On Query, a stale value will
	// trigger an asynchronous lookup of a replacement value, and the original value is
	// returned. A zero-value for Stale implies the value never goes stale, and querying the key
	// associated for this value will never trigger an asynchronous lookup of a replacement
	// value.
	Stale time.Time

	// Expiry stores the time at which the value expires. On Query, an expired value will block
	// until a synchronous lookup of a replacement value is attempted. Once the lookup returns,
	// the Query method will return with the new value or the error returned by the lookup
	// function.
	Expiry time.Time
}

TimedValue couples a value or the error with both a stale and expiry time for the value and error.

func (*TimedValue) IsExpired

func (tv *TimedValue) IsExpired() bool

IsExpired returns true if and only if value is expired. A value is expired when its non-zero expiry time is before the current time, or when the value represents an error and expiry time is the time.Time zero-value.

func (*TimedValue) IsStale

func (tv *TimedValue) IsStale() bool

IsStale returns true if and only if value is stale. A value is stale when its non-zero stale time is before the current time, or when the value represents an error and stale time is the time.Time zero-value.