cache

Documentation

Overview

Package cache provides a generic key/value cache with support for background filling.

Index

• Constants
• func NewIncomingRefreshContext(ctx context.Context) context.Context
• func SetCacheRefreshKey(k string)
• func ShouldRefreshFromIncomingContext(ctx context.Context) bool
• func ShouldRefreshFromRequest(r *http.Request) bool
• type Cache
  • func New[Key Keyer, Value any](fillFunc func(ctx context.Context, req Key) (Value, error), cfg Config) *Cache[Key, Value]
  • func (c *Cache[Key, Value]) Add(key Key, val Value, d time.Duration) error
  • func (c *Cache[Key, Value]) Close() error
  • func (c *Cache[Key, Value]) Debug() map[string]any
  • func (c *Cache[Key, Value]) EnqueueBackfill(k Key) bool
  • func (c *Cache[Key, Value]) Get(key Key) (Value, bool)
• type Config
• type Keyer

Constants

const (
	// DefaultExpiration is used to instruct calls to Add to use the default
	// cache expiration.
	// See https://pkg.go.dev/github.com/patrickmn/go-cache@v2.1.0+incompatible#pkg-constants.
	DefaultExpiration = cache.DefaultExpiration
	// NoExpiration is used to create caches that do not expire items by
	// default.
	// See https://pkg.go.dev/github.com/patrickmn/go-cache@v2.1.0+incompatible#pkg-constants.
	NoExpiration = cache.NoExpiration

	// DefaultBackfillEnqueueWaitTime is the default value used for waiting to
	// enqueue backfill requests, if a config is passed with a non-positive
	// BackfillEnqueueWaitTime.
	DefaultBackfillEnqueueWaitTime = time.Millisecond * 50
	// DefaultBackfillRequestTTL is the default value used for how stale of
	// backfill requests to still process, if a config is passed with a
	// non-positive BackfillRequestTTL.
	DefaultBackfillRequestTTL = time.Millisecond * 100
)

Functions

func NewIncomingRefreshContext

func NewIncomingRefreshContext(ctx context.Context) context.Context

NewIncomingRefreshContext returns an incoming gRPC context with metadata set to signal a cache refresh.

func SetCacheRefreshKey

func SetCacheRefreshKey(k string)

SetCacheRefreshKey sets the global HTTP header and gRPC metadata keys for requests to force a cache refresh. Any whitespace characters are replaced with hyphens.

It is not threadsafe, and should be called only at startup or in an init function.

func ShouldRefreshFromIncomingContext

func ShouldRefreshFromIncomingContext(ctx context.Context) bool

ShouldRefreshFromIncomingContext returns true if the gRPC metadata in the incoming context signals a cache refresh was requested.

func ShouldRefreshFromRequest

func ShouldRefreshFromRequest(r *http.Request) bool

ShouldRefreshFromRequest returns true if the HTTP request headers signal a cache refresh was requested.

Types

type Cache

type Cache[Key Keyer, Value any] struct {
	// contains filtered or unexported fields
}

Cache is a generic cache supporting background fills. To add things to the cache, call Add. To enqueue a background fill, call EnqueueBackfill with a Keyer implementation, which will be passed to the fill func provided to New.

For example, to create a schema cache that can backfill full payloads (including size aggregation):

var c *cache.Cache[BackfillSchemaRequest, *vtadminpb.Schema]
c := cache.New(func(ctx context.Context, req BackfillSchemaRequest) (*vtadminpb.Schema, error) {
	// Fetch schema based on fields in `req`.
	// If err is nil, the backfilled schema will be added to the cache.
	return cluster.fetchSchema(ctx, req)
})

func New

func New[Key Keyer, Value any](fillFunc func(ctx context.Context, req Key) (Value, error), cfg Config) *Cache[Key, Value]

New creates a new cache with the given backfill func. When a request is enqueued (via EnqueueBackfill), fillFunc will be called with that request.

func (*Cache[Key, Value]) Add

func (c *Cache[Key, Value]) Add(key Key, val Value, d time.Duration) error

Add adds a (key, value) to the cache directly, following the semantics of (github.com/patrickmn/go-cache).Cache.Add.

func (*Cache[Key, Value]) Close

func (c *Cache[Key, Value]) Close() error

Close closes the backfill goroutine, effectively rendering this cache unusable for further background fills.

func (*Cache[Key, Value]) Debug

func (c *Cache[Key, Value]) Debug() map[string]any

Debug implements debug.Debuggable for Cache.

func (*Cache[Key, Value]) EnqueueBackfill

func (c *Cache[Key, Value]) EnqueueBackfill(k Key) bool

EnqueueBackfill submits a request to the backfill queue.

func (*Cache[Key, Value]) Get

func (c *Cache[Key, Value]) Get(key Key) (Value, bool)

Get returns the Value stored for the key, if present in the cache. If the key is not cached, the zero value for the given type is returned, along with a boolean to indicated presence/absence.

type Config

type Config struct {
	// DefaultExpiration is how long to keep Values in the cache by default (the
	// duration passed to Add takes precedence). Use the sentinel NoExpiration
	// to make Values never expire by default.
	DefaultExpiration time.Duration `json:"default_expiration"`
	// CleanupInterval is how often to remove expired Values from the cache.
	CleanupInterval time.Duration `json:"cleanup_interval"`

	// BackfillRequestTTL is how long a backfill request is considered valid.
	// If the backfill goroutine encounters a request older than this, it is
	// discarded.
	BackfillRequestTTL time.Duration `json:"backfill_request_ttl"`
	// BackfillRequestDuplicateInterval is how much time must pass before the
	// backfill goroutine will re-backfill the same key. It is used to prevent
	// multiple callers queuing up too many requests for the same key, when one
	// backfill would satisfy all of them.
	BackfillRequestDuplicateInterval time.Duration `json:"backfill_request_duplicate_interval"`
	// BackfillQueueSize is how many outstanding backfill requests to permit.
	// If the queue is full, calls to EnqueueBackfill will return false and
	// those requests will be discarded.
	BackfillQueueSize int `json:"backfill_queue_size"`
	// BackfillEnqueueWaitTime is how long to wait when attempting to enqueue a
	// backfill request before giving up.
	BackfillEnqueueWaitTime time.Duration `json:"backfill_enqueue_wait_time"`
}

Config is the configuration for a cache.

type Keyer

type Keyer interface{ Key() string }

Keyer is the interface cache keys implement to turn themselves into string keys.

Note: we define this type rather than using Stringer so users may implement that interface for different string representation needs, for example providing a human-friendly representation.