hypercache

README

HyperCache

Synopsis

HyperCache is a thread-safe high-performance cache implementation in Go that supports multiple backends with optional size limits, item expiration, and pluggable eviction algorithms. It can be used as a standalone cache (single process or distributed via Redis / Redis Cluster) or wrapped by the service interface to decorate operations with middleware (logging, metrics, tracing, etc.).

It is optimized for performance and flexibility:

• Tunable expiration and eviction intervals (or fully proactive eviction when the eviction interval is set to 0).
• Debounced & coalesced expiration trigger channel to avoid thrashing.
• Non-blocking manual TriggerEviction(context.Context) signal.
• Serializer‑aware memory accounting (item size reflects the backend serialization format when available).
• Multiple eviction algorithms with the ability to register custom ones.
• Multiple stats collectors (default histogram) and middleware hooks.

It ships with a default histogram stats collector and several eviction algorithms. You can register new ones by implementing the Eviction Algorithm interface:

• Recently Used (LRU) eviction algorithm
• The Least Frequently Used (LFU) algorithm
• Cache-Aware Write-Optimized LFU (CAWOLFU)
• The Adaptive Replacement Cache (ARC) algorithm — Experimental (not enabled by default)
• The clock eviction algorithm

Features

• Thread-safe & lock‑optimized (sharded map + worker pool)
• High-performance (low allocations on hot paths, pooled items, serializer‑aware sizing)
• Multiple backends (extensible):
  1. In-memory
  2. Redis
  3. Redis Cluster
• Item expiration & proactive expiration triggering (debounced/coalesced)
• Background or proactive (interval = 0) eviction using pluggable algorithms
• Manual, non-blocking eviction triggering (TriggerEviction())
• Maximum cache size (bytes) & capacity (item count) controls
• Serializer‑aware size accounting for consistent memory tracking across backends
• Stats collection (histogram by default) + pluggable collectors
• Middleware-friendly service wrapper (logging, metrics, tracing, custom)
• Zero-cost if an interval is disabled (tickers are only created when > 0)

Optional Management HTTP API

You can enable a lightweight management HTTP server to inspect and control a running cache instance.

Add the option when creating the config:

cfg := hypercache.NewConfig[backend.InMemory](constants.InMemoryBackend)
cfg.HyperCacheOptions = append(cfg.HyperCacheOptions,
    hypercache.WithManagementHTTP[backend.InMemory]("127.0.0.1:9090"),
)
hc, _ := hypercache.New(context.Background(), hypercache.GetDefaultManager(), cfg)

Endpoints (subject to change):

• GET /health – liveness check
• GET /stats – current stats snapshot
• GET /config – sanitized runtime config (now includes replication + virtual node settings when using DistMemory)
• GET /dist/metrics – distributed backend forwarding / replication counters (DistMemory only)
• GET /dist/owners?key=K – current ring owners (IDs) for key K (DistMemory only, debug)
• GET /internal/merkle – Merkle tree snapshot (DistMemory experimental anti-entropy)
• GET /internal/keys – Full key enumeration (debug / anti-entropy fallback; expensive)
• GET /cluster/members – membership snapshot (id, address, state, incarnation, replication factor, virtual nodes)
• GET /cluster/ring – ring vnode hashes (debug / diagnostics)
• POST /evict – trigger eviction cycle
• POST /trigger-expiration – trigger expiration scan
• POST /clear – clear all items

Bind to 127.0.0.1 by default and wrap with an auth function via WithMgmtAuth for production use.

Installation

Install HyperCache:

go get -u github.com/hyp3rd/hypercache

Performance

Sample benchmark output (numbers will vary by hardware / Go version):

goos: darwin
goarch: arm64
pkg: github.com/hyp3rd/hypercache/tests/benchmark
cpu: Apple M2 Pro
BenchmarkHyperCache_Get-12                        72005894          66.71 ns/op         0 B/op           0 allocs/op
BenchmarkHyperCache_Get_ProactiveEviction-12      71068249          67.22 ns/op         0 B/op           0 allocs/op
BenchmarkHyperCache_List-12                       36435114          129.5 ns/op        80 B/op           1 allocs/op
BenchmarkHyperCache_Set-12                        10365289          587.4 ns/op       191 B/op           3 allocs/op
BenchmarkHyperCache_Set_Proactive_Eviction-12      3264818           1521 ns/op       282 B/op           5 allocs/op
PASS
ok   github.com/hyp3rd/hypercache/tests/benchmark 26.853s

Examples

To run the examples, use the following command:

make run-example group=eviction  # or any other example

For a complete list of examples, refer to the examples directory.

Observability (OpenTelemetry)

HyperCache provides optional OpenTelemetry middleware for tracing and metrics.

• Tracing: wrap the service with middleware.NewOTelTracingMiddleware using a trace.Tracer.
• Metrics: wrap with middleware.NewOTelMetricsMiddleware using a metric.Meter.

Example wiring:

svc := hypercache.ApplyMiddleware(svc,
    func(next hypercache.Service) hypercache.Service { return middleware.NewOTelTracingMiddleware(next, tracer) },
    func(next hypercache.Service) hypercache.Service { mw, _ := middleware.NewOTelMetricsMiddleware(next, meter); return mw },
)

Use your preferred OpenTelemetry SDK setup for exporters and processors in production; the snippet uses no-op providers for simplicity.

Eviction algorithms

Available algorithm names you can pass to WithEvictionAlgorithm:

• "lru" — Least Recently Used (default)
• "lfu" — Least Frequently Used (with LRU tie-breaker for equal frequencies)
• "clock" — Second-chance clock
• "cawolfu" — Cache-Aware Write-Optimized LFU
• "arc" — Adaptive Replacement Cache (experimental; not registered by default)

Note: ARC is experimental and isn’t included in the default registry. If you choose to use it, register it manually or enable it explicitly in your build.

API

NewInMemoryWithDefaults(capacity) is the quickest way to start:

cache, err := hypercache.NewInMemoryWithDefaults(100)
if err != nil {
    // handle error
}

For fine‑grained control use NewConfig + New:

config := hypercache.NewConfig[backend.InMemory](constants.InMemoryBackend)
config.HyperCacheOptions = []hypercache.Option[backend.InMemory]{
    hypercache.WithEvictionInterval[backend.InMemory](time.Minute * 5),
    hypercache.WithEvictionAlgorithm[backend.InMemory]("cawolfu"),
    hypercache.WithExpirationTriggerDebounce[backend.InMemory](250 * time.Millisecond),
    hypercache.WithMaxEvictionCount[backend.InMemory](100),
    hypercache.WithMaxCacheSize[backend.InMemory](64 * 1024 * 1024), // 64 MiB logical cap
}
config.InMemoryOptions = []backend.Option[backend.InMemory]{ backend.WithCapacity[backend.InMemory](10) }

cache, err := hypercache.New(context.Background(), hypercache.GetDefaultManager(), config)
if err != nil {
    fmt.Fprintln(os.Stderr, err)
    return
}

Advanced options quick reference

Option	Purpose
WithEvictionInterval	Periodic eviction loop; set to 0 for proactive per-write eviction.
WithExpirationInterval	Periodic scan for expired items.
WithExpirationTriggerBuffer	Buffer size for coalesced expiration trigger channel.
WithExpirationTriggerDebounce	Drop rapid-fire triggers within a window.
WithEvictionAlgorithm	Select eviction algorithm (lru, lfu, clock, cawolfu, arc*).
WithMaxEvictionCount	Cap number of items evicted per cycle.
WithMaxCacheSize	Max cumulative serialized item size (bytes).
WithStatsCollector	Choose stats collector implementation.
WithManagementHTTP	Start optional management HTTP server.
WithDistReplication	(DistMemory) Set replication factor (owners per key).
WithDistVirtualNodes	(DistMemory) Virtual nodes per physical node for consistent hashing.
WithDistMerkleChunkSize	(DistMemory) Keys per Merkle leaf chunk (power-of-two recommended).
WithDistMerkleAutoSync	(DistMemory) Interval for background Merkle sync (<=0 disables).
WithDistMerkleAutoSyncPeers	(DistMemory) Limit peers synced per auto-sync tick (0=all).
WithDistListKeysCap	(DistMemory) Cap number of keys fetched via fallback enumeration.
WithDistNode	(DistMemory) Explicit node identity (id/address).
WithDistSeeds	(DistMemory) Static seed addresses to pre-populate membership.
WithDistTombstoneTTL	(DistMemory) Retain delete tombstones for this duration before compaction (<=0 = infinite).
WithDistTombstoneSweep	(DistMemory) Interval to run tombstone compaction (<=0 disables).

*ARC is experimental (not registered by default).

Redis / Redis Cluster notes

When using Redis or Redis Cluster, item size accounting uses the configured serializer (e.g. msgpack) to align in-memory and remote representations. Provide the serializer via backend options (WithSerializer / WithClusterSerializer).

Refer to config.go for complete option definitions and the GoDoc on pkg.go.dev for an exhaustive API reference.

Usage

Distributed In‑Process Backend (Experimental)

An experimental in‑process distributed backend DistMemory is being developed (feature branch: feat/distributed-backend). It simulates a small cluster inside a single process using a consistent hash ring with virtual nodes, replication, forwarding, replication fan‑out, and read‑repair. This is primarily for experimentation before introducing real network transports and dynamic membership.

Current capabilities:

• Static membership + ring with configurable replication factor & virtual nodes.
• Ownership enforcement (non‑owners forward to primary).
• Replica fan‑out on writes (best‑effort) & replica removals.
• Read‑repair when a local owner misses but another replica has the key.
• Basic delete semantics with tombstones: deletions propagate as versioned tombstones preventing resurrection during anti-entropy (tombstone retention is in‑memory, no persistence yet).
  • Tombstone versioning uses a per-process monotonic counter when no prior item version exists (avoids time-based unsigned casts).
  • Remote pull sync will infer a tombstone when a key present locally is absent remotely and no local tomb exists (anti-resurrection guard).
  • DebugInject intentionally clears any existing tombstone for that key (test helper / simulating authoritative resurrection with higher version).
    • Tombstone TTL + periodic compaction: configure with WithDistTombstoneTTL / WithDistTombstoneSweep; metrics track active & purged counts.
• Metrics exposed via management endpoints (/dist/metrics, /dist/owners, /cluster/members, /cluster/ring).
  • Includes Merkle phase timings (fetch/build/diff nanos) and counters for keys pulled during anti-entropy.
    • Tombstone metrics: TombstonesActive, TombstonesPurged.

Planned next steps (roadmap excerpts): network transport abstraction, quorum reads/writes, versioning (vector clocks or lamport), failure detection / node states, rebalancing & anti‑entropy sync.

Roadmap / PRD Progress Snapshot

Area	Status
Core in-process sharding	Complete (static ring)
Replication fan-out	Implemented (best-effort)
Read-repair	Implemented
Merkle anti-entropy	Implemented (pull-based)
Merkle performance metrics	Implemented (fetch/build/diff nanos)
Remote-only key enumeration fallback	Implemented with optional cap (WithDistListKeysCap)
Delete semantics (tombstones)	Implemented
Tombstone compaction / TTL	Implemented
Quorum read consistency	Implemented
Quorum write consistency	Implemented (acks enforced)
Failure detection / heartbeat	Experimental heartbeat present
Membership changes / dynamic rebalancing	Not yet
Network transport (HTTP partial)	Basic HTTP management + fetch merkle/keys; full RPC TBD
Tracing spans (distributed ops)	Planned
Metrics exposure	Basic + Merkle phase metrics
Persistence	Not in scope yet
Benchmarks & tests	Extensive unit + benchmark coverage

Example minimal setup:

cfg := hypercache.NewConfig[backend.DistMemory](constants.DistMemoryBackend)
cfg.HyperCacheOptions = append(cfg.HyperCacheOptions,
    hypercache.WithManagementHTTP[backend.DistMemory]("127.0.0.1:9090"),
)
cfg.DistMemoryOptions = []backend.Option[backend.DistMemory]{
    backend.WithDistReplication(3),
    backend.WithDistVirtualNodes(64),
    backend.WithDistNode("node-a", "127.0.0.1:7000"),
    backend.WithDistSeeds([]string{"127.0.0.1:7001", "127.0.0.1:7002"}),
}
hc, err := hypercache.New(context.Background(), hypercache.GetDefaultManager(), cfg)
if err != nil { log.Fatal(err) }
defer hc.Stop(context.Background())

Note: DistMemory is not a production distributed cache; it is a stepping stone towards a networked, failure‑aware implementation.

Consistency & Quorum Semantics

DistMemory currently supports three consistency levels configurable independently for reads and writes:

• ONE: Return after the primary (or first reachable owner) succeeds.
• QUORUM: Majority of owners (floor(R/2)+1) must acknowledge.
• ALL: Every owner must acknowledge; any unreachable replica causes failure.

Required acknowledgements are computed at runtime from the ring's current replication factor. For writes, the primary applies locally then synchronously fans out to remaining owners; for reads, it queries owners until the required number of successful responses is achieved (promoting next owner if a primary is unreachable). Read‑repair occurs when a later owner returns a newer version than the local primary copy.

Hinted Handoff

When a replica is unreachable during a write, a hint (deferred write) is enqueued locally keyed by the target node ID. Hints have a TTL (WithDistHintTTL) and are replayed on an interval (WithDistHintReplayInterval). Limits can be applied per node (WithDistHintMaxPerNode) as well as globally across all nodes (WithDistHintMaxTotal total entries, WithDistHintMaxBytes approximate bytes). Expired hints are dropped; delivered hints increment replay counters; globally capped drops increment a separate metric. Metrics exposed via the management endpoint allow monitoring queued, replayed, expired, dropped (transport errors), and globally dropped hints along with current approximate queued bytes.

Test helper methods for forcing a replay cycle (StartHintReplayForTest, ReplayHintsForTest, HintedQueueSize) are compiled only under the test build tag to keep production binaries clean.

To run tests that rely on these helpers:

go test -tags test ./...

Build Tags

The repository uses a //go:build test tag to include auxiliary instrumentation and helpers exclusively in test builds (e.g. hinted handoff queue inspection). Production builds omit these symbols automatically. Heartbeat peer sampling (WithDistHeartbeatSample) and membership state metrics (suspect/dead counters) are part of the experimental failure detection added in Phase 2.

Metrics Snapshot

The /dist/metrics endpoint (and DistMemory.Metrics() API) expose counters for forwarding operations, replica fan‑out, read‑repair, hinted handoff lifecycle, quorum write attempts/acks/failures, Merkle sync timings, tombstone activity, and heartbeat probes. These are reset only on process restart.

Future Evolution

Planned enhancements toward a production‑grade distributed backend include:

• Real network transport (HTTP/JSON → gRPC) for data plane operations.
• Gossip‑based membership & failure detection (alive/suspect/dead) with automatic ring rebuild.
• Rebalancing & key range handoff on join/leave events.
• Incremental & adaptive anti‑entropy (Merkle diff scheduling, deletions reconciliation).
• Advanced versioning (hybrid logical clocks or vector clocks) and conflict resolution strategies.
• Client library for direct owner routing (avoiding extra network hops).
• Optional compression, TLS/mTLS security, auth middleware.

Until these land, DistMemory should be treated as an experimental playground rather than a fault‑tolerant cluster.

Examples can be too broad for a readme, refer to the examples directory for a more comprehensive overview.

License

The code and documentation in this project are released under Mozilla Public License 2.0.

Author

I'm a surfer, and a software architect with 15 years of experience designing highly available distributed production systems and developing cloud-native apps in public and private clouds. Feel free to connect with me on LinkedIn.

Documentation

Overview

Package hypercache provides a high-performance, generic caching library with configurable backends and eviction algorithms. It supports multiple backend types including in-memory and Redis, with various eviction strategies like LRU, LFU, and more. The package is designed to be flexible and extensible, allowing users to customize cache behavior through configuration options.

Example usage:

config := hypercache.NewConfig[string]("inmemory")
cache := hypercache.NewHyperCache[string](config)
cache.Set("key", "value", time.Hour)
value, found := cache.Get("key")

Index

• func ApplyHyperCacheOptions[T backend.IBackendConstrain](cache *HyperCache[T], options ...Option[T])
• type BackendManager
  • func GetDefaultManager() *BackendManager
  • func NewBackendManager() *BackendManager
  • func NewEmptyBackendManager() *BackendManager
  • func (hcm *BackendManager) RegisterBackend(name string, constructor any)
• type Config
  • func NewConfig[T backend.IBackendConstrain](backendType string) *Config[T]
• type DistMemoryBackendConstructor
  • func (DistMemoryBackendConstructor) Create(ctx context.Context, _ *Config[backend.DistMemory]) (backend.IBackend[backend.DistMemory], error)
• type HyperCache
  • func New[T backend.IBackendConstrain](ctx context.Context, bm *BackendManager, config *Config[T]) (*HyperCache[T], error)
  • func NewInMemoryWithDefaults(capacity int) (*HyperCache[backend.InMemory], error)
  • func (hyperCache *HyperCache[T]) Allocation() int64
  • func (hyperCache *HyperCache[T]) Capacity() int
  • func (hyperCache *HyperCache[T]) Clear(ctx context.Context) error
  • func (hyperCache *HyperCache[T]) ClusterOwners(key string) []string
  • func (hyperCache *HyperCache[T]) Count(ctx context.Context) int
  • func (hyperCache *HyperCache[T]) DistHeartbeatMetrics() any
  • func (hyperCache *HyperCache[T]) DistMembershipSnapshot() (members []struct{ ... }, replication int, vnodes int)
  • func (hyperCache *HyperCache[T]) DistMetrics() any
  • func (hyperCache *HyperCache[T]) DistRingHashSpots() []string
  • func (hyperCache *HyperCache[T]) EvictionAlgorithm() string
  • func (hyperCache *HyperCache[T]) EvictionInterval() time.Duration
  • func (hyperCache *HyperCache[T]) ExpirationInterval() time.Duration
  • func (hyperCache *HyperCache[T]) Get(ctx context.Context, key string) (any, bool)
  • func (hyperCache *HyperCache[T]) GetMultiple(ctx context.Context, keys ...string) (map[string]any, map[string]error)
  • func (hyperCache *HyperCache[T]) GetOrSet(ctx context.Context, key string, value any, expiration time.Duration) (any, error)
  • func (hyperCache *HyperCache[T]) GetStats() stats.Stats
  • func (hyperCache *HyperCache[T]) GetWithInfo(ctx context.Context, key string) (*cache.Item, bool)
  • func (hyperCache *HyperCache[T]) List(ctx context.Context, filters ...backend.IFilter) ([]*cache.Item, error)
  • func (hyperCache *HyperCache[T]) ManagementHTTPAddress() string
  • func (hyperCache *HyperCache[T]) MaxCacheSize() int64
  • func (hyperCache *HyperCache[T]) Remove(ctx context.Context, keys ...string) error
  • func (hyperCache *HyperCache[T]) Set(ctx context.Context, key string, value any, expiration time.Duration) error
  • func (hyperCache *HyperCache[T]) SetCapacity(ctx context.Context, capacity int)
  • func (hyperCache *HyperCache[T]) Stop(ctx context.Context) error
  • func (hyperCache *HyperCache[T]) TriggerEviction(_ context.Context)
  • func (hyperCache *HyperCache[T]) TriggerExpiration()
• type IBackendConstructor
• type InMemoryBackendConstructor
  • func (InMemoryBackendConstructor) Create(_ context.Context, cfg *Config[backend.InMemory]) (backend.IBackend[backend.InMemory], error)
• type JobFunc
• type ManagementHTTPOption
  • func WithMgmtAuth(fn func(fiber.Ctx) error) ManagementHTTPOption
  • func WithMgmtReadTimeout(d time.Duration) ManagementHTTPOption
  • func WithMgmtWriteTimeout(d time.Duration) ManagementHTTPOption
• type ManagementHTTPServer
  • func NewManagementHTTPServer(addr string, opts ...ManagementHTTPOption) *ManagementHTTPServer
  • func (s *ManagementHTTPServer) Address() string
  • func (s *ManagementHTTPServer) Shutdown(ctx context.Context) error
  • func (s *ManagementHTTPServer) Start(ctx context.Context, hc managementCache) error
• type Middleware
• type Option
  • func WithEvictionAlgorithm[T backend.IBackendConstrain](name string) Option[T]
  • func WithEvictionInterval[T backend.IBackendConstrain](evictionInterval time.Duration) Option[T]
  • func WithExpirationInterval[T backend.IBackendConstrain](expirationInterval time.Duration) Option[T]
  • func WithExpirationTriggerBuffer[T backend.IBackendConstrain](size int) Option[T]
  • func WithExpirationTriggerDebounce[T backend.IBackendConstrain](interval time.Duration) Option[T]
  • func WithManagementHTTP[T backend.IBackendConstrain](addr string, opts ...ManagementHTTPOption) Option[T]
  • func WithMaxCacheSize[T backend.IBackendConstrain](maxCacheSize int64) Option[T]
  • func WithMaxEvictionCount[T backend.IBackendConstrain](maxEvictionCount uint) Option[T]
  • func WithStatsCollector[T backend.IBackendConstrain](name string) Option[T]
• type RedisBackendConstructor
  • func (RedisBackendConstructor) Create(_ context.Context, cfg *Config[backend.Redis]) (backend.IBackend[backend.Redis], error)
• type RedisClusterBackendConstructor
  • func (RedisClusterBackendConstructor) Create(_ context.Context, cfg *Config[backend.RedisCluster]) (backend.IBackend[backend.RedisCluster], error)
• type Service
  • func ApplyMiddleware(svc Service, mw ...Middleware) Service
• type WorkerPool
  • func NewWorkerPool(workers int) *WorkerPool
  • func (pool *WorkerPool) Enqueue(job JobFunc)
  • func (pool *WorkerPool) Errors() <-chan error
  • func (pool *WorkerPool) Resize(newSize int)
  • func (pool *WorkerPool) Shutdown()

Functions

func ApplyHyperCacheOptions

func ApplyHyperCacheOptions[T backend.IBackendConstrain](cache *HyperCache[T], options ...Option[T])

ApplyHyperCacheOptions applies the given options to the given cache.

Types

type BackendManager

type BackendManager struct {
	// contains filtered or unexported fields
}

BackendManager is a factory for creating HyperCache backend instances. It maintains a registry of backend constructors. We store them as any internally, and cast to the typed constructor at use site based on T.

func GetDefaultManager

func GetDefaultManager() *BackendManager

GetDefaultManager returns a new BackendManager with default backends pre-registered. This replaces the previous global instance with a factory function.

func NewBackendManager

func NewBackendManager() *BackendManager

NewBackendManager creates a new BackendManager with default backends pre-registered.

func NewEmptyBackendManager

func NewEmptyBackendManager() *BackendManager

NewEmptyBackendManager creates a new BackendManager without default backends. This is useful for testing or when you want to register only specific backends.

func (*BackendManager) RegisterBackend

func (hcm *BackendManager) RegisterBackend(name string, constructor any)

RegisterBackend registers a new backend constructor. The constructor should be a value implementing IBackendConstructor[T] for some T; stored as any.

type Config

type Config[T backend.IBackendConstrain] struct {
	// BackendType is the type of the backend to use.
	BackendType string
	// InMemoryOptions is a slice of options that can be used to configure the `InMemory`.
	InMemoryOptions []backend.Option[backend.InMemory]
	// RedisOptions is a slice of options that can be used to configure the `Redis`.
	RedisOptions []backend.Option[backend.Redis]
	// RedisClusterOptions is a slice of options to configure the `RedisCluster` backend.
	RedisClusterOptions []backend.Option[backend.RedisCluster]
	// DistMemoryOptions configure the distributed in-memory (sharded) backend.
	DistMemoryOptions []backend.DistMemoryOption
	// HyperCacheOptions is a slice of options that can be used to configure `HyperCache`.
	HyperCacheOptions []Option[T]
}

Config is a struct that wraps all the configuration options to setup `HyperCache` and its backend.

func NewConfig

func NewConfig[T backend.IBackendConstrain](backendType string) *Config[T]

NewConfig returns a new `Config` struct with default values:

• `InMemoryOptions` is empty
• `RedisOptions` is empty
• `HyperCacheOptions` is set to: -- `WithExpirationInterval[T](30 * time.Minute)` -- `WithEvictionAlgorithm[T]("lfu")` -- `WithEvictionInterval[T](10 * time.Minute)`

Each of the above options can be overridden by passing a different option to the `NewConfig` function. It can be used to configure `HyperCache` and its backend and customize the behavior of the cache.

type DistMemoryBackendConstructor

type DistMemoryBackendConstructor struct{}

DistMemoryBackendConstructor constructs DistMemory backends.

func (DistMemoryBackendConstructor) Create

func (DistMemoryBackendConstructor) Create(
	ctx context.Context,
	_ *Config[backend.DistMemory],
) (backend.IBackend[backend.DistMemory], error)

Create creates a new DistMemory backend.

type HyperCache

type HyperCache[T backend.IBackendConstrain] struct {

	// StatsCollector to collect cache statistics
	StatsCollector stats.ICollector
	// contains filtered or unexported fields
}

HyperCache stores items with a key and optional expiration. It supports multiple backends and eviction algorithms. Configuration is provided via the Config struct using With* options. Background loops:

• expiration loop (interval: expirationInterval) scans for expired items
• eviction loop (interval: evictionInterval) evicts items via the configured algorithm

Channels:

• expirationTriggerCh triggers an on-demand expiration pass (coalesced)
• evictCh triggers an immediate eviction pass when interval is 0 and capacity exceeded

Synchronization:

• mutex protects eviction algorithm state
• stop channel signals background loops to stop

func New

func New[T backend.IBackendConstrain](ctx context.Context, bm *BackendManager, config *Config[T]) (*HyperCache[T], error)

New initializes a new HyperCache with the given configuration. The default configuration is:

• The eviction interval is set to 5 minutes.
• The eviction algorithm is set to LRU.
• The expiration interval is set to 30 minutes.
• The stats collector is set to the HistogramStatsCollector stats collector.

func NewInMemoryWithDefaults

func NewInMemoryWithDefaults(capacity int) (*HyperCache[backend.InMemory], error)

NewInMemoryWithDefaults initializes a new HyperCache with the default configuration. The default configuration is:

• The eviction interval is set to 10 minutes.
• The eviction algorithm is set to LRU.
• The expiration interval is set to 30 minutes.
• The capacity of the in-memory backend is set to 0 items (no limitations) unless specified.
• The maximum cache size in bytes is set to 0 (no limitations).

func (*HyperCache[T]) Allocation

func (hyperCache *HyperCache[T]) Allocation() int64

Allocation returns the size allocation in bytes of the current cache.

func (*HyperCache[T]) Capacity

func (hyperCache *HyperCache[T]) Capacity() int

Capacity returns the capacity of the cache.

func (*HyperCache[T]) Clear

func (hyperCache *HyperCache[T]) Clear(ctx context.Context) error

Clear removes all items from the cache.

func (*HyperCache[T]) ClusterOwners

func (hyperCache *HyperCache[T]) ClusterOwners(key string) []string

ClusterOwners returns the owners for a key if the distributed backend supports it; otherwise empty slice.

func (*HyperCache[T]) Count

func (hyperCache *HyperCache[T]) Count(ctx context.Context) int

Count returns the number of items in the cache.

func (*HyperCache[T]) DistHeartbeatMetrics

func (hyperCache *HyperCache[T]) DistHeartbeatMetrics() any

DistHeartbeatMetrics returns distributed heartbeat metrics if supported.

func (*HyperCache[T]) DistMembershipSnapshot

func (hyperCache *HyperCache[T]) DistMembershipSnapshot() (members []struct {
	ID          string
	Address     string
	State       string
	Incarnation uint64
}, replication int, vnodes int,
)

DistMembershipSnapshot returns a snapshot of membership if distributed backend; otherwise nil slice.

func (*HyperCache[T]) DistMetrics

func (hyperCache *HyperCache[T]) DistMetrics() any

DistMetrics returns distributed backend metrics if the underlying backend is DistMemory. Returns nil if unsupported.

func (*HyperCache[T]) DistRingHashSpots

func (hyperCache *HyperCache[T]) DistRingHashSpots() []string

DistRingHashSpots returns vnode hashes as hex strings if available (debug).

func (*HyperCache[T]) EvictionAlgorithm

func (hyperCache *HyperCache[T]) EvictionAlgorithm() string

EvictionAlgorithm returns eviction algorithm name.

func (*HyperCache[T]) EvictionInterval

func (hyperCache *HyperCache[T]) EvictionInterval() time.Duration

EvictionInterval returns configured eviction interval.

func (*HyperCache[T]) ExpirationInterval

func (hyperCache *HyperCache[T]) ExpirationInterval() time.Duration

ExpirationInterval returns configured expiration interval.

func (*HyperCache[T]) Get

func (hyperCache *HyperCache[T]) Get(ctx context.Context, key string) (any, bool)

Get retrieves the item with the given key from the cache returning the value and a boolean indicating if the item was found.

func (*HyperCache[T]) GetMultiple

func (hyperCache *HyperCache[T]) GetMultiple(ctx context.Context, keys ...string) (map[string]any, map[string]error)

GetMultiple retrieves the items with the given keys from the cache.

func (*HyperCache[T]) GetOrSet

func (hyperCache *HyperCache[T]) GetOrSet(ctx context.Context, key string, value any, expiration time.Duration) (any, error)

GetOrSet retrieves the item with the given key. If the item is not found, it adds the item to the cache with the given value and expiration duration. If the capacity of the cache is reached, leverage the eviction algorithm.

func (*HyperCache[T]) GetStats

func (hyperCache *HyperCache[T]) GetStats() stats.Stats

GetStats returns the stats collected by the cache.

func (*HyperCache[T]) GetWithInfo

func (hyperCache *HyperCache[T]) GetWithInfo(ctx context.Context, key string) (*cache.Item, bool)

GetWithInfo retrieves the item with the given key from the cache returning the `Item` object and a boolean indicating if the item was found.

func (*HyperCache[T]) List

func (hyperCache *HyperCache[T]) List(ctx context.Context, filters ...backend.IFilter) ([]*cache.Item, error)

List lists the items in the cache that meet the specified criteria. It takes in a variadic number of any type as filters, it then checks the backend type, and calls the corresponding implementation of the List function for that backend, with the filters passed in as arguments.

func (*HyperCache[T]) ManagementHTTPAddress

func (hyperCache *HyperCache[T]) ManagementHTTPAddress() string

ManagementHTTPAddress returns the bound address of the optional management HTTP server. Empty string when the server is disabled or failed to start.

func (*HyperCache[T]) MaxCacheSize

func (hyperCache *HyperCache[T]) MaxCacheSize() int64

MaxCacheSize returns the maximum size in bytes of the cache.

func (*HyperCache[T]) Remove

func (hyperCache *HyperCache[T]) Remove(ctx context.Context, keys ...string) error

Remove removes items with the given key from the cache. If an item is not found, it does nothing.

func (*HyperCache[T]) Set

func (hyperCache *HyperCache[T]) Set(ctx context.Context, key string, value any, expiration time.Duration) error

Set adds an item to the cache with the given key and value. If an item with the same key already exists, it updates the value of the existing item. If the expiration duration is greater than zero, the item will expire after the specified duration. If capacity is reached:

• when evictionInterval == 0 we evict immediately
• otherwise the background eviction loop will reclaim space

func (*HyperCache[T]) SetCapacity

func (hyperCache *HyperCache[T]) SetCapacity(ctx context.Context, capacity int)

SetCapacity sets the capacity of the cache. If the new capacity is smaller than the current number of items in the cache, it evicts the excess items from the cache.

func (*HyperCache[T]) Stop

func (hyperCache *HyperCache[T]) Stop(ctx context.Context) error

Stop function stops the expiration and eviction loops and closes the stop channel.

func (*HyperCache[T]) TriggerEviction

func (hyperCache *HyperCache[T]) TriggerEviction(_ context.Context)

TriggerEviction sends a signal to the eviction loop to start.

func (*HyperCache[T]) TriggerExpiration

func (hyperCache *HyperCache[T]) TriggerExpiration()

TriggerExpiration exposes a manual expiration trigger (debounced/coalesced internally).

type IBackendConstructor

type IBackendConstructor[T backend.IBackendConstrain] interface {
	Create(ctx context.Context, cfg *Config[T]) (backend.IBackend[T], error)
}

IBackendConstructor is an interface for backend constructors with type safety. It returns a typed backend.IBackend[T] instead of any.

type InMemoryBackendConstructor

type InMemoryBackendConstructor struct{}

InMemoryBackendConstructor constructs InMemory backends.

func (InMemoryBackendConstructor) Create

func (InMemoryBackendConstructor) Create(_ context.Context, cfg *Config[backend.InMemory]) (backend.IBackend[backend.InMemory], error)

Create creates a new InMemory backend.

type JobFunc

type JobFunc func() error

JobFunc is a function that can be enqueued in a worker pool.

type ManagementHTTPOption

type ManagementHTTPOption func(*ManagementHTTPServer)

ManagementHTTPOption configures the management HTTP server.

func WithMgmtAuth

func WithMgmtAuth(fn func(fiber.Ctx) error) ManagementHTTPOption

WithMgmtAuth sets an auth function (return error to block).

func WithMgmtReadTimeout

func WithMgmtReadTimeout(d time.Duration) ManagementHTTPOption

WithMgmtReadTimeout sets read timeout.

func WithMgmtWriteTimeout

func WithMgmtWriteTimeout(d time.Duration) ManagementHTTPOption

WithMgmtWriteTimeout sets write timeout.

type ManagementHTTPServer

type ManagementHTTPServer struct {
	// contains filtered or unexported fields
}

ManagementHTTPServer holds Fiber app and settings.

func NewManagementHTTPServer

func NewManagementHTTPServer(addr string, opts ...ManagementHTTPOption) *ManagementHTTPServer

NewManagementHTTPServer builds an HTTP server holder (lazy start).

func (*ManagementHTTPServer) Address

func (s *ManagementHTTPServer) Address() string

Address returns the bound address (useful when passing ":0" for ephemeral port). Empty if not started yet.

func (*ManagementHTTPServer) Shutdown

func (s *ManagementHTTPServer) Shutdown(ctx context.Context) error

Shutdown stops the server.

func (*ManagementHTTPServer) Start

func (s *ManagementHTTPServer) Start(ctx context.Context, hc managementCache) error

Start launches listener (idempotent). Caller provides cache for handler wiring.

type Middleware

type Middleware func(Service) Service

Middleware describes a service middleware.

type Option

type Option[T backend.IBackendConstrain] func(*HyperCache[T])

Option is a function type that can be used to configure the `HyperCache` struct.

func WithEvictionAlgorithm

func WithEvictionAlgorithm[T backend.IBackendConstrain](name string) Option[T]

WithEvictionAlgorithm is an option that sets the eviction algorithm name field of the `HyperCache` struct. The eviction algorithm name determines which eviction algorithm will be used to evict items from the cache. The eviction algorithm name must be one of the following:

• "LRU" (Least Recently Used) - Implemented in the `eviction/lru.go` file
• "LFU" (Least Frequently Used) - Implemented in the `eviction/lfu.go` file
• "CAWOLFU" (Cache-Aware Write-Optimized LFU) - Implemented in the `eviction/cawolfu.go` file
• "FIFO" (First In First Out)
• "RANDOM" (Random)
• "CLOCK" (Clock) - Implemented in the `eviction/clock.go` file
• "ARC" (Adaptive Replacement Cache) - Experimental (not enabled by default)
• "TTL" (Time To Live)
• "LFUDA" (Least Frequently Used with Dynamic Aging)
• "SLRU" (Segmented Least Recently Used)

func WithEvictionInterval

func WithEvictionInterval[T backend.IBackendConstrain](evictionInterval time.Duration) Option[T]

WithEvictionInterval is an option that sets the eviction interval field of the `HyperCache` struct. The eviction interval determines how often the cache will run the eviction process to remove the least recently used items.

func WithExpirationInterval

func WithExpirationInterval[T backend.IBackendConstrain](expirationInterval time.Duration) Option[T]

WithExpirationInterval is an option that sets the expiration interval field of the `HyperCache` struct. The expiration interval determines how often the cache will check for and remove expired items.

func WithExpirationTriggerBuffer

func WithExpirationTriggerBuffer[T backend.IBackendConstrain](size int) Option[T]

WithExpirationTriggerBuffer sets the buffer size of the expiration trigger channel. If set to <= 0, the default (capacity/2, minimum 1) is used.

func WithExpirationTriggerDebounce

func WithExpirationTriggerDebounce[T backend.IBackendConstrain](interval time.Duration) Option[T]

WithExpirationTriggerDebounce sets an optional debounce interval for coalescing expiration triggers. Triggers arriving within this interval after the last accepted trigger may be dropped.

func WithManagementHTTP

func WithManagementHTTP[T backend.IBackendConstrain](addr string, opts ...ManagementHTTPOption) Option[T]

WithManagementHTTP enables the optional management HTTP server with the provided address and options. addr format example: ":8080" or "127.0.0.1:9090".

func WithMaxCacheSize

func WithMaxCacheSize[T backend.IBackendConstrain](maxCacheSize int64) Option[T]

WithMaxCacheSize is an option that sets the maximum size of the cache. The maximum size of the cache is the maximum number of items that can be stored in the cache. If the maximum size of the cache is reached, the least recently used item will be evicted from the cache.

func WithMaxEvictionCount

func WithMaxEvictionCount[T backend.IBackendConstrain](maxEvictionCount uint) Option[T]

WithMaxEvictionCount is an option that sets the max eviction count field of the `HyperCache` struct. The max eviction count determines the maximum number of items that can be removed during a single eviction run.

func WithStatsCollector

func WithStatsCollector[T backend.IBackendConstrain](name string) Option[T]

WithStatsCollector is an option that sets the stats collector field of the `HyperCache` struct. The stats collector is used to collect statistics about the cache.

type RedisBackendConstructor

type RedisBackendConstructor struct{}

RedisBackendConstructor constructs Redis backends.

func (RedisBackendConstructor) Create

func (RedisBackendConstructor) Create(_ context.Context, cfg *Config[backend.Redis]) (backend.IBackend[backend.Redis], error)

Create creates a new Redis backend.

type RedisClusterBackendConstructor

type RedisClusterBackendConstructor struct{}

RedisClusterBackendConstructor constructs Redis Cluster backends.

func (RedisClusterBackendConstructor) Create

func (RedisClusterBackendConstructor) Create(
	_ context.Context,
	cfg *Config[backend.RedisCluster],
) (backend.IBackend[backend.RedisCluster], error)

Create creates a new Redis Cluster backend.

type Service

type Service interface {

	// Capacity returns the capacity of the cache
	Capacity() int
	// Allocation returns the allocation in bytes of the current cache
	Allocation() int64
	// Count returns the number of items in the cache
	Count(ctx context.Context) int
	// TriggerEviction triggers the eviction of the cache
	TriggerEviction(ctx context.Context)
	// Stop stops the cache
	Stop(ctx context.Context) error
	// GetStats returns the stats of the cache
	GetStats() stats.Stats
	// contains filtered or unexported methods
}

Service is the service interface for the HyperCache. It enables middleware to be added to the service.

func ApplyMiddleware

func ApplyMiddleware(svc Service, mw ...Middleware) Service

ApplyMiddleware applies middlewares to a service.

type WorkerPool

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool is a pool of workers that can execute jobs concurrently.

func NewWorkerPool

func NewWorkerPool(workers int) *WorkerPool

NewWorkerPool creates a new worker pool with the given number of workers.

func (*WorkerPool) Enqueue

func (pool *WorkerPool) Enqueue(job JobFunc)

Enqueue adds a job to the worker pool.

func (*WorkerPool) Errors

func (pool *WorkerPool) Errors() <-chan error

Errors returns a channel that can be used to receive errors from the worker pool.

func (*WorkerPool) Resize

func (pool *WorkerPool) Resize(newSize int)

Resize resizes the worker pool.

func (*WorkerPool) Shutdown

func (pool *WorkerPool) Shutdown()

Shutdown shuts down the worker pool. It waits for all jobs to finish.