svcache

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 17, 2023 License: MIT Imports: 7 Imported by: 0

README

SVCache

Go Reference

SVCache is a threadsafe, single-value cache with a simple but flexible API.

When there is no fresh value in the cache, an attempt to retrieve the value will block until a new value is loaded. Only a single goroutine will load a new value; other goroutines will block as necessary until the loading is complete.

SVCache also supports aynchronously loading a new value before a value has expired.

See the GoDoc for details.

Example

Here's a brief example demonstrate how it can be used:

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/softwaretechnik-berlin/svcache"
)

func main() {
	// You can create a full-featured cache like this:
	cache := svcache.NewInMemory(

		// This context will be used for all calls to the loader, and is respected when a caller waits for a fresh cache value.
		// E.g. you can cancel the context to signal to the load that it should stop and to prevent readers from blocking on updates that aren't coming.
		context.Background(),

		// The backoff updater
		svcache.NewBackoffUpdater(

			// The backoff updater We'll automatically add timestamps to the values returned by the loader.
			// Alternatively
			svcache.NewTimestampedLoader(fetchNewValue),

			// Define a backoff strategy.
			// Here we use our balanced backoff strategy which exponentially scales between two values at the rate of the Fibonacci sequence with some jitter.
			// This is a good default strategy for many cases, but you might choose to use a strategy tailored to your use case.
			svcache.NewBalancedBackoffStrategy(100*time.Millisecond, time.Minute),

			// The error handler is called whenever the loader returns an error, allowing you to log or otherwise handle the error.
			func(ctx context.Context, previous svcache.Timestamped[string], consecutiveErrors uint, err error) {
				fmt.Printf(
					"error retrieving new value for cache (number %v in a row, current value has timestamp %v): %v",
					consecutiveErrors, previous.Timestamp, err,
				)
			},
		),
	)

	// Now you can use the cache.
	// Each time you access it, you can use whatever retrieval strategy is appropriate for that call.
	// You'll probably want to instantiate the strategies once and reuse them, but you can also create them on the fly.

	// some retrieval strategies:
	alwaysTrigger := svcache.AlwaysTrigger[svcache.Timestamped[string]]
	waitForAnyValue := svcache.WaitForNonZeroTimestamp(alwaysTrigger)
	triggerIfTooOld := svcache.TriggerIfAged[string](5 * time.Minute)

	// some accesses:
	value, err := cache.Get(context.Background(), waitForAnyValue)
	fmt.Println(value, err)
	value = cache.GetImmediately(triggerIfTooOld)
	fmt.Println(value)
	value = cache.GetImmediately(alwaysTrigger)
	fmt.Println(value)

	// the strategy in the last access always returns a value, and for that we can use the simpler Peek method:
	fmt.Println(cache.Peek())
}

Documentation

Overview

Package svcache provides a threadsafe in-memory single-value cache.

Example 
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"time"

	"github.com/softwaretechnik-berlin/svcache"
)

func main() {
	// You can create a full-featured cache like this:
	cache := svcache.NewInMemory(

		// This context will be used for all calls to the loader, and is respected when a caller waits for a fresh cache value.
		// E.g. you can cancel the context to signal to the load that it should stop and to prevent readers from blocking on updates that aren't coming.
		context.Background(),

		// The backoff updater
		svcache.NewBackoffUpdater(

			// The backoff updater We'll automatically add timestamps to the values returned by the loader.
			// Alternatively
			svcache.NewTimestampedLoader(fetchNewValue),

			// Define a backoff strategy.
			// Here we use our balanced backoff strategy which exponentially scales between two values at the rate of the Fibonacci sequence with some jitter.
			// This is a good default strategy for many cases, but you might choose to use a strategy tailored to your use case.
			svcache.NewBalancedBackoffStrategy(100*time.Millisecond, time.Minute),

			// The error handler is called whenever the loader returns an error, allowing you to log or otherwise handle the error.
			func(ctx context.Context, previous svcache.Timestamped[string], consecutiveErrors uint, err error) {
				fmt.Printf(
					"error retrieving new value for cache (number %v in a row, current value has timestamp %v): %v",
					consecutiveErrors, previous.Timestamp, err,
				)
			},
		),
	)

	// Now you can use the cache.
	// Each time you access it, you can use whatever retrieval strategy is appropriate for that call.
	// You'll probably want to instantiate the strategies once and reuse them, but you can also create them on the fly.

	// some retrieval strategies:
	alwaysTrigger := svcache.AlwaysTrigger[svcache.Timestamped[string]]
	waitForAnyValue := svcache.WaitForNonZeroTimestamp(alwaysTrigger)
	triggerIfTooOld := svcache.TriggerIfAged[string](5 * time.Minute)

	// some accesses:
	value, err := cache.Get(context.Background(), waitForAnyValue)
	fmt.Println(value, err)
	value = cache.GetImmediately(triggerIfTooOld)
	fmt.Println(value)
	value = cache.GetImmediately(alwaysTrigger)
	fmt.Println(value)

	// the strategy in the last access always returns a value, and for that we can use the simpler Peek method:
	fmt.Println(cache.Peek())
}

func fetchNewValue(ctx context.Context) (string, error) {
	// documentation: https://httpbin.org/#/Dynamic_data/get_uuid
	resp, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://httpbin.org/uuid", nil)
	if err != nil {
		return "", fmt.Errorf("initating GET request: %w", err)
	}
	defer resp.Body.Close()
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return "", fmt.Errorf("error reading response body: %w", err)
	}
	var response uuidResponse
	err = json.Unmarshal(body, &response)
	if err != nil {
		return "", fmt.Errorf("error unmarshalling response body: %w", err)
	}
	return response.UUID, nil
}

type uuidResponse struct {
	UUID string `json:"uuid"`
}

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func AlwaysTrigger added in v0.2.1 

func AlwaysTrigger[V any](current V) bool

AlwaysTrigger is a RefreshStrategy that always triggers an update.

func NeverTrigger added in v0.2.1 

func NeverTrigger[V any](current V) bool

NeverTrigger is a RefreshStrategy that never triggers an update.

func NewBackoffUpdaterAndConsecutiveErrorsCounter added in v0.3.0 

func NewBackoffUpdaterAndConsecutiveErrorsCounter[V any](
	loader Loader[Timestamped[V]],
	backoffStrategy BackoffStrategy,
	errorHandler ErrorHandler[Timestamped[V]],
) (Updater[Timestamped[V]], ConsecutiveErrorsCounter)

NewBackoffUpdater returns an updater that uses a backoff strategy to determine how long to wait after an error before retrying, and also returns a counter of the number of consecutive errors that have occurred.

Types

type AccessStrategy added in v0.3.0 

type AccessStrategy[V any] func(currentValue V) Action

AccessStrategy is a function that determines what should be done when accessing the value in the cache using the potentially blocking `Get` function.

It can either return `UseCachedValue` to return the current value immediately, or `TriggerLoadAndUseCachedValue` to trigger an asynchronous load of a new value but immediately return the current value, or `WaitForNewlyLoadedValue` to block until a new value is loaded and then return that.

It is given the currently cached value, in case this decision needs to be value-dependent.

func AccessStrategyFromRefreshStrategyAndWaitPredicate added in v0.3.0 

func AccessStrategyFromRefreshStrategyAndWaitPredicate[V any](trigger RefreshStrategy[V], shouldWait func(currentValue V) bool) AccessStrategy[V]

func AsNonBlockingAccessStrategy added in v0.3.0 

func AsNonBlockingAccessStrategy[V any](refreshStrategy RefreshStrategy[Timestamped[V]]) AccessStrategy[Timestamped[V]]

AsNonBlockingAccessStrategy promotes a non-blocking refresh strategy to a refresh strategy.

func DoNotWaitAfterTooManyErrors added in v0.3.0 

func DoNotWaitAfterTooManyErrors[V any](
	attemptsBeforeGivingUp uint,
	consecutiveErrors ConsecutiveErrorsCounter,
	underlying AccessStrategy[V],
) AccessStrategy[V]

but will not wait for a new value to be loaded if the given number of consecutive errors has been reached.

This can be used with the ConsecutiveErrorsCounter returned by NewBackoffUpdaterAndConsecutiveErrorsCounter.

func TriggerOrWaitIfAged added in v0.2.0 

func TriggerOrWaitIfAged[V any](triggerThreshold, waitThreshold time.Duration) AccessStrategy[Timestamped[V]]

TriggerOrWaitIfAged returns a AccessStrategy for Timestamped values that waits if the value's age is at least the waitThreashold, and triggers a refresh if the value's age is at least the triggerThreshold.

func TriggerOrWaitUnlessNewerThan added in v0.2.1 

func TriggerOrWaitUnlessNewerThan[V any](triggerThreshold, waitThreshold time.Time) AccessStrategy[Timestamped[V]]

TriggerOrWaitUnlessNewThan returns a AccessStrategy for Timestamped values that waits unless the value is newer than the waitThreshold, and triggers a refresh unless the value's timestamp is newer than the given time.

func WaitForNonZeroTimestamp added in v0.2.0 

func WaitForNonZeroTimestamp[V any](nonBlockingStrategy RefreshStrategy[Timestamped[V]]) AccessStrategy[Timestamped[V]]

WaitForNonZeroTimestamp returns a AccessStrategy for Timestamped values that waits for the timestamp to be non-zero, then uses the given strategy.

func WaitIfAged added in v0.2.1 

func WaitIfAged[V any](threshold time.Duration, nonBlockingStrategy RefreshStrategy[Timestamped[V]]) AccessStrategy[Timestamped[V]]

WaitIfAged returns a AccessStrategy for Timestamped values that waits if the value's age is at least the waitThreashold, and otherwise uses the given non-blocking strategy to determine whether to trigger a reload.

func WaitUnlessNewerThan added in v0.2.1 

func WaitUnlessNewerThan[V any](threshold time.Time, nonBlockingStrategy RefreshStrategy[Timestamped[V]]) AccessStrategy[Timestamped[V]]

WaitUnlessNewThan returns a AccessStrategy for Timestamped values that waits unless the value is newer than the waitThreshold, and otherwise uses the given non-blocking strategy to determine whether to trigger a reload.

type Action added in v0.2.0 

type Action uint8
const (
	UseCachedValue Action = iota
	TriggerLoadAndUseCachedValue
	WaitForNewlyLoadedValue
)

func JustReturn added in v0.2.0 

func JustReturn[V any](current V) Action

JustReturn is a AccessStrategy that always returns the current value without triggering an update.

type BackoffStrategy added in v0.2.0 

type BackoffStrategy func(consecutiveErrors uint) time.Duration

BakcoffStrategy is a function that takes the number of consecutive errors that have occurred so far, and returns the minimum amount of time that should have passed since the previous attempt before retrying.

func NewBalancedBackoffStrategy added in v0.2.0 

func NewBalancedBackoffStrategy(initial, final time.Duration) BackoffStrategy

NewBalancedBackoffStrategy returns a BackoffStrategy that exponentially scales between two values at the rate of the Fibonacci sequence with some jitter.

func NewClampedExponentialBackoffWithJitter added in v0.2.0 

func NewClampedExponentialBackoffWithJitter(initial time.Duration, base float64, final time.Duration, jitter float64) BackoffStrategy

NewClampedExponentialBackoffWithJitter returns a BackoffStrategy that exponentially scales with the given base between two values.

The jitter parameter is a fraction indicating how much random jitter can be added; e.g., if the jitter is 0.1 and the raw duration is 3 minutes, then the duration with jitter will be somewhere between 2min 42s and 3min 18s.

type Cache added in v0.3.0 

type Cache[V any] interface {
	// Get returns a value from the cache.
	//
	// The value currently in the cache is passed to the given refresh strategy to determine what should be done.
	//
	// If the refresh strategy returns `Return`, the value currently in the cache will be returned immediately.
	//
	// If the refresh strategy returns `TriggerLoadAndReturn`,
	// the cache will trigger asynchronous loading of a new value if this is not already in progress,
	// and the value currently in the cache will be returned immediately.
	//
	// If the refresh strategy returns `WaitForLoad`, the cache will block waiting for a new value to be loaded;
	// the process then begins again with the new value being passed to the refresh strategy to determine what should be done with it.
	// During the waiting, if the context is cancelled, the context error will be returned and the last considered value will be returned.
	Get(context.Context, AccessStrategy[V]) (V, error)
}

Cache is a cache for a single value of type V.

type ConsecutiveErrorsCounter added in v0.3.0 

type ConsecutiveErrorsCounter struct {
	// contains filtered or unexported fields
}

func (ConsecutiveErrorsCounter) Value added in v0.3.0 

func (c ConsecutiveErrorsCounter) Value() uint

type ErrorHandler added in v0.2.0 

type ErrorHandler[V any] func(ctx context.Context, previous V, consecutiveErrors uint, err error)

ErrorHandler is a function that is called when an error occurs while loading a value, which can be used e.g. for logging.

type InMemory 

type InMemory[V any] struct {
	// contains filtered or unexported fields
}

InMemory is a threadsafe in-memory implementation of SingleValueCache.

Example 
// Given a loader function that returns a new value, like this one:

type uuidResponse struct {
	UUID string `json:"uuid"`
}

fetchNewValue := func(ctx context.Context) (value string, ok bool) {
	// documentation: https://httpbin.org/#/Dynamic_data/get_uuid
	resp, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://httpbin.org/uuid", nil)
	if err != nil {
		return
	}
	defer resp.Body.Close()
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return
	}
	var response uuidResponse
	ok = json.Unmarshal(body, &response) == nil
	return response.UUID, ok
}

// We can create an InMemory cache like this:
ctx := context.Background()
cache := NewInMemory(ctx, func(ctx context.Context, previous string) string {
	value, ok := fetchNewValue(ctx)
	if !ok {
		return previous
	}
	return value
})

// block for a non-empty value
blockIfEmpty := func(current string) Action {
	if current == "" {
		return WaitForNewlyLoadedValue
	}
	return UseCachedValue
}
if value, err := cache.Get(context.Background(), blockIfEmpty); err == nil {
	println(value)
}

// get current value without blocking
println(cache.Peek())

func NewInMemory 

func NewInMemory[V any](loaderCtx context.Context, loader Updater[V]) *InMemory[V]

NewInMemory returns a new InMemory SingleValueCache using the given Loader.

There will only ever be one goroutine invoking the Loader at any given time. If multiple threads call Get concurrently, a single one of them will invoke the Loader, and the others will wait for it to finish.

func NewInMemoryWithInitialValue added in v0.2.0 

func NewInMemoryWithInitialValue[V any](updateCtx context.Context, initialValue V, updater Updater[V]) *InMemory[V]

NewInMemoryWithInitialValue returns a new InMemory SingleValueCache using the given initial value and Updater.

There will only ever be one goroutine invoking the Updater at any given time. If multiple threads call Get concurrently, a single one of them will invoke the Updater, and the others will wait for it to finish.

The given context is used to perform the update.

func (*InMemory[V]) Get 

func (m *InMemory[V]) Get(ctx context.Context, refreshStrategy AccessStrategy[V]) (V, error)

Get returns a value from the cache.

The value currently in the cache is passed to the given refresh strategy to determine what should be done.

If the refresh strategy returns `Return`, the value currently in the cache will be returned immediately.

If the refresh strategy returns `TriggerLoadAndReturn`, the cache will trigger asynchronous loading of a new value if this is not already in progress, and the value currently in the cache will be returned immediately.

If the refresh strategy returns `WaitForLoad`, the cache will block waiting for a new value to be loaded; the process then begins again with the new value being passed to the refresh strategy to determine what should be done with it. During the waiting, if the context is cancelled, the context error will be returned and the last considered value will be returned.

func (*InMemory[V]) GetImmediately added in v0.2.1 

func (m *InMemory[V]) GetImmediately(refreshStrategy RefreshStrategy[V]) V

GetImmediately returns the current value without blocking or potentially failing.

The value currently in the cache is passed to the given function to determine whether an update should be triggered before returning. If the refresh strategy returns `true`, a refresh is trigger, otherwise it is not.

func (*InMemory[V]) Peek 

func (m *InMemory[V]) Peek() V

Peek returns the current value without triggering a load.

type Loader 

type Loader[V any] func(context.Context) (V, error)

Loader is a function that loads values into the cache. It is not used directly by the cache, but can be used to create an Updater.

func NewTimestampedLoader added in v0.2.0 

func NewTimestampedLoader[V any](loader Loader[V]) Loader[Timestamped[V]]

NewTimestampedLoader creates a loader that automatically wraps values with a timestamp derived from the time loading started. This is useful for determining whether to trigger or wait for a new value to be loaded. Alternatively, the value you are loading may have its own timestamp, in which case you can use that instead of using this utility.

type RefreshStrategy added in v0.2.1 

type RefreshStrategy[V any] func(currentValue V) bool

RefreshStrategy is a function that determines whether to trigger a refresh of the cached value.

It can either return `true` to trigger a refresh, or `false` to not trigger a refresh.

It is given the currently cached value, in case this decision needs to be value-dependent.

func TriggerIfAged added in v0.2.0 

func TriggerIfAged[V any](threshold time.Duration) RefreshStrategy[Timestamped[V]]

TriggerIfAged returns a AccessStrategy for Timestamped values that never waits but triggers a refresh if the value's age is at least the given duration.

func TriggerUnlessNewerThan added in v0.2.0 

func TriggerUnlessNewerThan[V any](threshold time.Time) RefreshStrategy[Timestamped[V]]

TriggerUnlessNewerThan returns a AccessStrategy for Timestamped values that never waits but triggers a refresh unless the value's timestamp is newer than the given time.

type Timestamped added in v0.2.0 

type Timestamped[V any] struct {
	Value     V
	Timestamp time.Time
}

Timestamped augments a value with a timestamp.

type Updater added in v0.2.0 

type Updater[V any] func(ctx context.Context, previous V) V

Updater is function that loads values into the cache.

The Updater receives the previous timestamped value, and returns a new one.

If the Updater fails to load a new value it should return the previous value. The Updater may be implemented with the assumption that there will only be a single invocation at a time. The Updater can block as necessary while loading the value, and may even choose to sleep e.g. to implement backoff logic to avoid overwhelming a struggling value source.

func NewBackoffUpdater added in v0.2.0 

func NewBackoffUpdater[V any](
	loader Loader[Timestamped[V]],
	backoffStrategy BackoffStrategy,
	errorHandler ErrorHandler[Timestamped[V]],
) Updater[Timestamped[V]]

NewBackoffUpdater returns an updater that uses a backoff strategy to determine how long to wait after an error before retrying.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.