hypercache

package module
v0.1.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 20, 2025 License: MPL-2.0 Imports: 14 Imported by: 0

README

HyperCache

Go CodeQL

Synopsis

HyperCache is a thread-safe high-performance cache implementation in Go that supports multiple backends with an optional size limit, expiration, and eviction of items with custom algorithms alongside the defaults. It can be used as a standalone cache, distributed environents, or as a cache middleware for a service. It can implement a service interface to intercept and decorate the cache methods with middleware (default or custom). It is optimized for performance and flexibility, allowing to specify of the expiration and eviction intervals and providing and registering new eviction algorithms, stats collectors, and middleware(s). It ships with a default historigram stats collector and several eviction algorithms. However, you can develop and register your own if it implements the Eviction Algorithm interface.:

Features
  • Thread-safe
  • High-performance
  • Supports multiple, custom backends. Default backends are:
    1. In-memory
    2. Redis
  • Store items in the cache with a key and expiration duration
  • Retrieve items from the cache by their key
  • Delete items from the cache by their key
  • Clear the cache of all items
  • Evitc items in the background based on the cache capacity and items access leveraging several custom eviction algorithms
  • Expire items in the background based on their duration
  • Eviction Algorithm interface to implement custom eviction algorithms.
  • Stats collection with a default stats collector or a custom one that implements the StatsCollector interface.
  • Service interface implementation to allow intercepting cache methods and decorate them with custom or default middleware(s).

Installation

Install HyperCache:

go get -u github.com/hyp3rd/hypercache
performance

Running the benchmarks on a 2019 MacBook Pro with a 2.4 GHz 8-Core Intel Core i9 processor and 32 GB 2400 MHz DDR4 memory, the results are as follows on average, using a pretty busy machine:

make bench
cd tests/benchmark && go test -bench=. -benchmem -benchtime=4s . -timeout 30m
goos: darwin
goarch: amd64
pkg: github.com/hyp3rd/hypercache/tests/benchmark
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkHyperCache_Get-16                          37481116           115.7 ns/op         0 B/op          0 allocs/op
BenchmarkHyperCache_Get_ProactiveEviction-16        39486261           116.2 ns/op         0 B/op          0 allocs/op
BenchmarkHyperCache_List-16                         11299632           412.0 ns/op        85 B/op          1 allocs/op
BenchmarkHyperCache_Set-16                           2765406          1556 ns/op         248 B/op          4 allocs/op
BenchmarkHyperCache_Set_Proactive_Eviction-16        2947629          1700 ns/op         162 B/op          3 allocs/op
PASS
ok      github.com/hyp3rd/hypercache/tests/benchmark    30.031s
Examples

To run the examples, use the following command:

make run-example group=eviction  # or any other example

For a complete list of examples, refer to the examples directory.

API

The NewInMemoryWithDefaults function creates a new HyperCache instance with the defaults:

  1. The eviction interval is set to 10 minutes.
  2. The eviction algorithm is set to LRU.
  3. The expiration interval is set to 30 minutes.
  4. The capacity of the in-memory backend is set to 1000 items.

To create a new cache with a given capacity, use the New function as described below:

cache, err := hypercache.NewInMemoryWithDefaults(100)
if err != nil {
    // handle error
}

For fine-grained control over the cache configuration, use the New function, for instance:

config := hypercache.NewConfig[backend.InMemory]()
config.HyperCacheOptions = []hypercache.HyperCacheOption[backend.InMemory]{
    hypercache.WithEvictionInterval[backend.InMemory](time.Minute * 10),
    hypercache.WithEvictionAlgorithm[backend.InMemory]("cawolfu"),
}

config.InMemoryOptions = []backend.Option[backend.InMemory]{
    backend.WithCapacity(10),
}

// Create a new HyperCache with a capacity of 10
cache, err := hypercache.New(config)
if err != nil {
    fmt.Fprintln(os.Stderr, err)
    return
}

Refer to the config.go file for the full configuration options. For a comprehensive API overview, see the documentation.

Usage

Examples can be too broad for a readme, refer to the examples directory for a more comprehensive overview.

License

The code and documentation in this project are released under Mozilla Public License 2.0.

Author

I'm a surfer, and a software architect with 15 years of experience designing highly available distributed production systems and developing cloud-native apps in public and private clouds. Feel free to connect with me on LinkedIn.

LinkedIn

Documentation

Overview

Package hypercache provides a high-performance, generic caching library with configurable backends and eviction algorithms. It supports multiple backend types including in-memory and Redis, with various eviction strategies like LRU, LFU, ARC, and more. The package is designed to be flexible and extensible, allowing users to customize cache behavior through configuration options.

Example usage: 

config := hypercache.NewConfig[string]("inmemory")
cache := hypercache.NewHyperCache[string](config)
cache.Set("key", "value", time.Hour)
value, found := cache.Get("key")

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ApplyHyperCacheOptions added in v0.0.4 

func ApplyHyperCacheOptions[T backend.IBackendConstrain](cache *HyperCache[T], options ...Option[T])

ApplyHyperCacheOptions applies the given options to the given cache.

Types

type BackendManager added in v0.1.3 

type BackendManager struct {
	// contains filtered or unexported fields
}

BackendManager is a factory for creating HyperCache backend instances.

func GetDefaultManager added in v0.1.3 

func GetDefaultManager() *BackendManager

GetDefaultManager returns a new BackendManager with default backends pre-registered. This replaces the previous global instance with a factory function.

func NewBackendManager added in v0.1.3 

func NewBackendManager() *BackendManager

NewBackendManager creates a new BackendManager with default backends pre-registered.

func NewEmptyBackendManager added in v0.1.6 

func NewEmptyBackendManager() *BackendManager

NewEmptyBackendManager creates a new BackendManager without default backends. This is useful for testing or when you want to register only specific backends.

func (*BackendManager) RegisterBackend added in v0.1.3 

func (hcm *BackendManager) RegisterBackend(name string, constructor IBackendConstructor)

RegisterBackend registers a new backend constructor.

type Config added in v0.0.4 

type Config[T backend.IBackendConstrain] struct {
	// BackendType is the type of the backend to use.
	BackendType string
	// InMemoryOptions is a slice of options that can be used to configure the `InMemory`.
	InMemoryOptions []backend.Option[backend.InMemory]
	// RedisOptions is a slice of options that can be used to configure the `Redis`.
	RedisOptions []backend.Option[backend.Redis]
	// HyperCacheOptions is a slice of options that can be used to configure `HyperCache`.
	HyperCacheOptions []Option[T]
}

Config is a struct that wraps all the configuration options to setup `HyperCache` and its backend.

func NewConfig added in v0.0.4 

func NewConfig[T backend.IBackendConstrain](backendType string) *Config[T]

NewConfig returns a new `Config` struct with default values:

  • `InMemoryOptions` is empty
  • `RedisOptions` is empty
  • `HyperCacheOptions` is set to: -- `WithExpirationInterval[T](30 * time.Minute)` -- `WithEvictionAlgorithm[T]("lfu")` -- `WithEvictionInterval[T](10 * time.Minute)`

Each of the above options can be overridden by passing a different option to the `NewConfig` function. It can be used to configure `HyperCache` and its backend and customize the behavior of the cache.

type HyperCache 

type HyperCache[T backend.IBackendConstrain] struct {

	// StatsCollector to collect cache statistics
	StatsCollector stats.ICollector
	// contains filtered or unexported fields
}

HyperCache is a cache that stores items with a key and expiration duration. It supports multiple backends and multiple eviction algorithms. The default in-memory implementation has a custom `ConcurrentMap` to store the items in the cache, The configuration is provided by the `Config` struct and can be customized by using the `With` functions. The cache has two loops that run in the background:

  • The expiration loop runs every `expirationInterval` and checks for expired items.
  • The eviction loop runs every `evictionInterval` and evicts items using the eviction algorithm.

The cache leverages two channels to signal the expiration and eviction loops to start:

  • The expirationTriggerCh channel is used to signal the expiration loop to start.
  • The evictCh channel is used to signal the eviction loop to start.

The cache also has a mutex that is used to protect the eviction algorithm from concurrent access. The stop channel is used to signal the expiration and eviction loops to stop. The evictCh channel is used to signal the eviction loop to start.

func New added in v0.0.5 

func New[T backend.IBackendConstrain](bm *BackendManager, config *Config[T]) (*HyperCache[T], error)

New initializes a new HyperCache with the given configuration. The default configuration is:

  • The eviction interval is set to 5 minutes.
  • The eviction algorithm is set to LRU.
  • The expiration interval is set to 30 minutes.
  • The stats collector is set to the HistogramStatsCollector stats collector.

func NewInMemoryWithDefaults added in v0.0.5 

func NewInMemoryWithDefaults(capacity int) (*HyperCache[backend.InMemory], error)

NewInMemoryWithDefaults initializes a new HyperCache with the default configuration. The default configuration is:

  • The eviction interval is set to 10 minutes.
  • The eviction algorithm is set to LRU.
  • The expiration interval is set to 30 minutes.
  • The capacity of the in-memory backend is set to 0 items (no limitations) unless specified.
  • The maximum cache size in bytes is set to 0 (no limitations).

func (*HyperCache[T]) Allocation added in v0.1.0 

func (hyperCache *HyperCache[T]) Allocation() int64

Allocation returns the size allocation in bytes of the current cache.

func (*HyperCache[T]) Capacity 

func (hyperCache *HyperCache[T]) Capacity() int

Capacity returns the capacity of the cache.

func (*HyperCache[T]) Clear 

func (hyperCache *HyperCache[T]) Clear(ctx context.Context) error

Clear removes all items from the cache.

func (*HyperCache[T]) Count added in v0.1.0 

func (hyperCache *HyperCache[T]) Count(ctx context.Context) int

Count returns the number of items in the cache.

func (*HyperCache[T]) Get 

func (hyperCache *HyperCache[T]) Get(ctx context.Context, key string) (any, bool)

Get retrieves the item with the given key from the cache returning the value and a boolean indicating if the item was found.

func (*HyperCache[T]) GetMultiple 

func (hyperCache *HyperCache[T]) GetMultiple(ctx context.Context, keys ...string) (map[string]any, map[string]error)

GetMultiple retrieves the items with the given keys from the cache.

func (*HyperCache[T]) GetOrSet 

func (hyperCache *HyperCache[T]) GetOrSet(ctx context.Context, key string, value any, expiration time.Duration) (any, error)

GetOrSet retrieves the item with the given key. If the item is not found, it adds the item to the cache with the given value and expiration duration. If the capacity of the cache is reached, leverage the eviction algorithm.

func (*HyperCache[T]) GetStats added in v0.0.4 

func (hyperCache *HyperCache[T]) GetStats() stats.Stats

GetStats returns the stats collected by the cache.

func (*HyperCache[T]) GetWithInfo added in v0.1.0 

func (hyperCache *HyperCache[T]) GetWithInfo(ctx context.Context, key string) (*cache.Item, bool)

GetWithInfo retrieves the item with the given key from the cache returning the `Item` object and a boolean indicating if the item was found.

func (*HyperCache[T]) List 

func (hyperCache *HyperCache[T]) List(ctx context.Context, filters ...backend.IFilter) ([]*cache.Item, error)

List lists the items in the cache that meet the specified criteria. It takes in a variadic number of any type as filters, it then checks the backend type, and calls the corresponding implementation of the List function for that backend, with the filters passed in as arguments.

func (*HyperCache[T]) MaxCacheSize added in v0.1.0 

func (hyperCache *HyperCache[T]) MaxCacheSize() int64

MaxCacheSize returns the maximum size in bytes of the cache.

func (*HyperCache[T]) Remove 

func (hyperCache *HyperCache[T]) Remove(ctx context.Context, keys ...string) error

Remove removes items with the given key from the cache. If an item is not found, it does nothing.

func (*HyperCache[T]) Set 

func (hyperCache *HyperCache[T]) Set(ctx context.Context, key string, value any, expiration time.Duration) error

Set adds an item to the cache with the given key and value. If an item with the same key already exists, it updates the value of the existing item. If the expiration duration is greater than zero, the item will expire after the specified duration. If the capacity of the cache is reached, the cache will leverage the eviction algorithm proactively if the evictionInterval is zero. If not, the background process will take care of the eviction.

func (*HyperCache[T]) SetCapacity 

func (hyperCache *HyperCache[T]) SetCapacity(ctx context.Context, capacity int)

SetCapacity sets the capacity of the cache. If the new capacity is smaller than the current number of items in the cache, it evicts the excess items from the cache.

func (*HyperCache[T]) Stop 

func (hyperCache *HyperCache[T]) Stop()

Stop function stops the expiration and eviction loops and closes the stop channel.

func (*HyperCache[T]) TriggerEviction 

func (hyperCache *HyperCache[T]) TriggerEviction()

TriggerEviction sends a signal to the eviction loop to start.

type IBackendConstructor added in v0.1.3 

type IBackendConstructor interface {
	Create(config any) (any, error)
}

IBackendConstructor is an interface for backend constructors.

type InMemoryBackendConstructor added in v0.1.3 

type InMemoryBackendConstructor struct{}

InMemoryBackendConstructor is a backend constructor for InMemory.

func (InMemoryBackendConstructor) Create added in v0.1.3 

func (ibc InMemoryBackendConstructor) Create(config any) (any, error)

Create creates a new InMemory backend.

type JobFunc added in v0.1.3 

type JobFunc func() error

JobFunc is a function that can be enqueued in a worker pool.

type Middleware added in v0.0.4 

type Middleware func(Service) Service

Middleware describes a service middleware.

type Option 

type Option[T backend.IBackendConstrain] func(*HyperCache[T])

Option is a function type that can be used to configure the `HyperCache` struct.

func WithEvictionAlgorithm added in v0.0.4 

func WithEvictionAlgorithm[T backend.IBackendConstrain](name string) Option[T]

WithEvictionAlgorithm is an option that sets the eviction algorithm name field of the `HyperCache` struct. The eviction algorithm name determines which eviction algorithm will be used to evict items from the cache. The eviction algorithm name must be one of the following:

  • "LRU" (Least Recently Used) - Implemented in the `eviction/lru.go` file
  • "LFU" (Least Frequently Used) - Implemented in the `eviction/lfu.go` file
  • "CAWOLFU" (Cache-Aware Write-Optimized LFU) - Implemented in the `eviction/cawolfu.go` file
  • "FIFO" (First In First Out)
  • "RANDOM" (Random)
  • "CLOCK" (Clock) - Implemented in the `eviction/clock.go` file
  • "ARC" (Adaptive Replacement Cache) - Implemented in the `eviction/arc.go` file
  • "TTL" (Time To Live)
  • "LFUDA" (Least Frequently Used with Dynamic Aging)
  • "SLRU" (Segmented Least Recently Used)

func WithEvictionInterval 

func WithEvictionInterval[T backend.IBackendConstrain](evictionInterval time.Duration) Option[T]

WithEvictionInterval is an option that sets the eviction interval field of the `HyperCache` struct. The eviction interval determines how often the cache will run the eviction process to remove the least recently used items.

func WithExpirationInterval 

func WithExpirationInterval[T backend.IBackendConstrain](expirationInterval time.Duration) Option[T]

WithExpirationInterval is an option that sets the expiration interval field of the `HyperCache` struct. The expiration interval determines how often the cache will check for and remove expired items.

func WithExpirationTriggerBuffer added in v0.1.6 

func WithExpirationTriggerBuffer[T backend.IBackendConstrain](size int) Option[T]

WithExpirationTriggerBuffer sets the buffer size of the expiration trigger channel. If set to <= 0, the default (capacity/2, minimum 1) is used.

func WithExpirationTriggerDebounce added in v0.1.6 

func WithExpirationTriggerDebounce[T backend.IBackendConstrain](interval time.Duration) Option[T]

WithExpirationTriggerDebounce sets an optional debounce interval for coalescing expiration triggers. Triggers arriving within this interval after the last accepted trigger may be dropped.

func WithMaxCacheSize added in v0.1.0 

func WithMaxCacheSize[T backend.IBackendConstrain](maxCacheSize int64) Option[T]

WithMaxCacheSize is an option that sets the maximum size of the cache. The maximum size of the cache is the maximum number of items that can be stored in the cache. If the maximum size of the cache is reached, the least recently used item will be evicted from the cache.

func WithMaxEvictionCount 

func WithMaxEvictionCount[T backend.IBackendConstrain](maxEvictionCount uint) Option[T]

WithMaxEvictionCount is an option that sets the max eviction count field of the `HyperCache` struct. The max eviction count determines the maximum number of items that can be removed during a single eviction run.

func WithStatsCollector 

func WithStatsCollector[T backend.IBackendConstrain](name string) Option[T]

WithStatsCollector is an option that sets the stats collector field of the `HyperCache` struct. The stats collector is used to collect statistics about the cache.

type RedisBackendConstructor added in v0.1.3 

type RedisBackendConstructor struct{}

RedisBackendConstructor is a backend constructor for Redis.

func (RedisBackendConstructor) Create added in v0.1.3 

func (rbc RedisBackendConstructor) Create(config any) (any, error)

Create creates a new Redis backend.

type Service added in v0.0.4 

type Service interface {

	// Capacity returns the capacity of the cache
	Capacity() int
	// Allocation returns the allocation in bytes of the current cache
	Allocation() int64
	// Count returns the number of items in the cache
	Count(ctx context.Context) int
	// TriggerEviction triggers the eviction of the cache
	TriggerEviction()
	// Stop stops the cache
	Stop()
	// GetStats returns the stats of the cache
	GetStats() stats.Stats
	// contains filtered or unexported methods
}

Service is the service interface for the HyperCache. It enables middleware to be added to the service.

func ApplyMiddleware added in v0.0.4 

func ApplyMiddleware(svc Service, mw ...Middleware) Service

ApplyMiddleware applies middlewares to a service.

type WorkerPool added in v0.1.3 

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool is a pool of workers that can execute jobs concurrently.

func NewWorkerPool added in v0.1.3 

func NewWorkerPool(workers int) *WorkerPool

NewWorkerPool creates a new worker pool with the given number of workers.

func (*WorkerPool) Enqueue added in v0.1.3 

func (pool *WorkerPool) Enqueue(job JobFunc)

Enqueue adds a job to the worker pool.

func (*WorkerPool) Errors added in v0.1.3 

func (pool *WorkerPool) Errors() <-chan error

Errors returns a channel that can be used to receive errors from the worker pool.

func (*WorkerPool) Resize added in v0.1.3 

func (pool *WorkerPool) Resize(newSize int)

Resize resizes the worker pool.

func (*WorkerPool) Shutdown added in v0.1.3 

func (pool *WorkerPool) Shutdown()

Shutdown shuts down the worker pool. It waits for all jobs to finish.

Source Files

View all Source files

Directories

Path Synopsis
__examples
clear command
eviction command
get command
list command
redis command
service command
size command
stats command
internal
constants
Package constants defines default configuration values and backend types for the hypercache system.
 Package constants defines default configuration values and backend types for the hypercache system.
introspect
Package introspect provides utilities for runtime inspection and type checking of cache backends.
 Package introspect provides utilities for runtime inspection and type checking of cache backends.
libs/serializer
Package serializer provides serialization interfaces and implementations for converting Go values to and from byte slices.
 Package serializer provides serialization interfaces and implementations for converting Go values to and from byte slices.
sentinel
Package sentinel provides standardized error definitions for the hypercache system.
 Package sentinel provides standardized error definitions for the hypercache system.
pkg
backend
Package backend provides interfaces and types for implementing cache backends.
 Package backend provides interfaces and types for implementing cache backends.
backend/redis
Package redis provides configuration options and utilities for Redis backend implementation.
 Package redis provides configuration options and utilities for Redis backend implementation.
cache
Package cache provides a thread-safe concurrent map implementation with sharding for improved performance in high-concurrency scenarios.
 Package cache provides a thread-safe concurrent map implementation with sharding for improved performance in high-concurrency scenarios.
cache/v2
Package cachev2 provides a high-performance concurrent map implementation optimized for cache operations.
 Package cachev2 provides a high-performance concurrent map implementation optimized for cache operations.
eviction
Package eviction ARC is an in-memory cache that uses the Adaptive Replacement Cache (ARC) algorithm to manage its items.
 Package eviction ARC is an in-memory cache that uses the Adaptive Replacement Cache (ARC) algorithm to manage its items.
middleware
Package middleware provides various middleware implementations for the hypercache service.
 Package middleware provides various middleware implementations for the hypercache service.
stats
Package stats provides a comprehensive statistics collection system for hypercache.
 Package stats provides a comprehensive statistics collection system for hypercache.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.