cache

package
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 10, 2026 License: AGPL-3.0 Imports: 8 Imported by: 0

Documentation

Overview

Package cache implements the Stage 1 exact cache for the BubbleFish Nexus 6-stage retrieval cascade. It provides a zero-dependency generic LRU bounded by total byte capacity, scope-isolated cache entries, monotonic watermark invalidation, and Prometheus hit/miss counters.

Reference: Tech Spec Section 3.4 — Stage 1.

Index

Constants

View Source 
const (
	// DefaultMaxBytes is the default maximum byte capacity for the exact cache
	// (50 MiB). Configurable via daemon.toml in later phases.
	DefaultMaxBytes = 50 * 1024 * 1024
)
View Source 
const DefaultSemanticMaxEntries = 1000

DefaultSemanticMaxEntries is the default maximum number of entries in the semantic cache. Configurable via daemon.toml in later phases.

Variables

This section is empty.

Functions

func BuildKey 

func BuildKey(sourceScope, dest, profile, namespace, subject, q string, limit, offset int, policyHash string) [32]byte

BuildKey derives the 32-byte cache key for a query.

Key = SHA256(sourceScope | dest | profile | namespace | subject | q | limit | offset | policyHash)

sourceScope is the source's unique identity (typically source.Name). Including it in the key enforces scope isolation: source A and source B derive different keys for identical query parameters.

policyHash is a caller-provided digest of the policy fields that affect result shape (max_results, field_visibility, etc.). A policy change produces a different key, preventing stale policy-shaped results from being served.

Reference: Tech Spec Section 3.4 — Stage 1 (cache key derivation).

func SemanticScopeKey 

func SemanticScopeKey(sourceName, dest, profile, namespace string) [32]byte

SemanticScopeKey derives the scope hash for semantic cache entries.

Key = SHA256(sourceName | "\x00" | dest | "\x00" | profile | "\x00" | namespace)

Including the source name enforces scope isolation: source A and source B derive different scope keys even for identical query parameters.

Types

type CacheEntry 

type CacheEntry struct {
	Records    []destination.TranslatedPayload
	NextCursor string
	HasMore    bool
	// Watermark is the destination's monotonic write counter at Put time.
	// A value of 0 is valid for destinations that have never been written to.
	Watermark uint64
}

CacheEntry is the value stored in the exact cache for a single query result. Watermark records the destination's write watermark at insertion time; an entry is stale when the destination's current watermark exceeds this value.

type ExactCache 

type ExactCache struct {
	// contains filtered or unexported fields
}

ExactCache is the Stage 1 exact cache. It provides:

  • Scope isolation: cache keys encode the source identity so source A cannot retrieve source B's cached results even for identical queries.
  • Watermark invalidation: entries become stale when the destination watermark advances (i.e., a new write was delivered).
  • LRU eviction: bounded by maxBytes; oldest entries evicted when full.
  • Prometheus counters: hit and miss events are reported via Stats.

All methods are safe for concurrent use by multiple goroutines.

Reference: Tech Spec Section 3.4 — Stage 1.

func NewExactCache 

func NewExactCache(maxBytes int64, stats *Stats) *ExactCache

NewExactCache creates an ExactCache with the given byte capacity and Prometheus registry. Pass DefaultMaxBytes for the 50 MiB default. stats may be nil (counters are skipped) — useful in tests that don't set up a Prometheus registry.

func (*ExactCache) BytesUsed 

func (c *ExactCache) BytesUsed() int64

BytesUsed returns the total estimated byte footprint of cached entries.

func (*ExactCache) Evictions 

func (c *ExactCache) Evictions() int64

Evictions returns the total number of LRU evictions since creation.

func (*ExactCache) Get 

func (c *ExactCache) Get(key [32]byte, dest string) (CacheEntry, bool)

Get retrieves a cached result for key. It returns the entry and true on a hit, or a zero CacheEntry and false on:

  • key not present (miss)
  • entry watermark < current dest watermark (stale miss)

On a miss the miss counter is incremented. On a hit the hit counter is incremented and the entry is promoted to most-recently-used.

Reference: Tech Spec Section 3.4 — Stage 1 (watermark freshness check, semantic short-circuit).

func (*ExactCache) InvalidateDest 

func (c *ExactCache) InvalidateDest(dest string)

InvalidateDest advances the watermark for dest, instantly staling all existing cache entries for that destination. Called by the write path after a payload is successfully delivered.

Reference: Tech Spec Section 3.4 (watermark freshness check), Section 3.7.

func (*ExactCache) Len 

func (c *ExactCache) Len() int

Len returns the number of entries currently in the LRU.

func (*ExactCache) Put 

func (c *ExactCache) Put(key [32]byte, dest string, entry CacheEntry)

Put stores a cache entry for key. The entry's Watermark is set to the current watermark for dest so that future writes invalidate it automatically. Size is estimated from the entry's record fields to enforce the byte cap.

type LRU 

type LRU[K comparable, V any] struct {
	// contains filtered or unexported fields
}

LRU is a generic, thread-safe least-recently-used cache bounded by total byte capacity. Keys must be comparable. Values can be any type.

All exported methods are safe for concurrent use by multiple goroutines. There are no package-level variables — all state lives in struct fields.

Implementation: doubly-linked list (container/list) + map[K]*list.Element. Add is O(1) amortised. Get is O(1). Eviction is O(1) per evicted entry.

Reference: Tech Spec Section 3.4 (zero-dep LRU, Go generics). CRITICAL: Do NOT replace with hashicorp/golang-lru (MPL 2.0 licence).

func NewLRU 

func NewLRU[K comparable, V any](maxBytes int64) *LRU[K, V]

NewLRU creates an LRU bounded to maxBytes of total capacity. maxBytes must be positive; callers that pass ≤0 will get an always-evicting cache.

func (*LRU[K, V]) Add 

func (l *LRU[K, V]) Add(key K, value V, bytes int64)

Add inserts or updates key with value. bytes is the caller's estimate of the entry's memory footprint; it must be ≥1. If the cache is full after the insert the least-recently-used entries are evicted until capacity is satisfied. Updating an existing key moves it to the front (most recent).

func (*LRU[K, V]) BytesUsed 

func (l *LRU[K, V]) BytesUsed() int64

BytesUsed returns the current total byte usage across all entries.

func (*LRU[K, V]) Evictions 

func (l *LRU[K, V]) Evictions() int64

Evictions returns the total number of LRU evictions since creation.

func (*LRU[K, V]) Get 

func (l *LRU[K, V]) Get(key K) (V, bool)

Get retrieves the value for key and promotes it to most-recently-used. Returns the zero value of V and false when the key is absent.

func (*LRU[K, V]) Len 

func (l *LRU[K, V]) Len() int

Len returns the number of entries currently in the cache.

func (*LRU[K, V]) Remove 

func (l *LRU[K, V]) Remove(key K)

Remove deletes key from the cache. No-op if key is absent.

type SemanticCache 

type SemanticCache struct {
	// contains filtered or unexported fields
}

SemanticCache is the Stage 2 semantic cache. It stores recent query results keyed by embedding vector. A new query hits the cache when its embedding is within cosine-similarity threshold of a stored query vector.

Scope isolation: entries carry a scope hash derived from source + dest + profile + namespace. Source A cannot retrieve source B's cached results even for identical query text.

Watermark invalidation: entries become stale when a write is delivered to the destination, matching the ExactCache invalidation strategy.

Eviction: bounded by maxEntries using FIFO eviction (oldest entry removed when the cache is full).

All methods are safe for concurrent use by multiple goroutines.

Reference: Tech Spec Section 3.4 — Stage 2.

func NewSemanticCache 

func NewSemanticCache(maxEntries int, stats *SemanticStats) *SemanticCache

NewSemanticCache creates a SemanticCache with the given maximum entry count. stats may be nil — counters are silently skipped. This is safe for tests that do not set up a Prometheus registry.

func (*SemanticCache) Get 

func (sc *SemanticCache) Get(scope [32]byte, queryVec []float32, dest string, threshold float64) (SemanticCacheEntry, bool)

Get performs a semantic cache lookup.

It scans entries in reverse insertion order (newest first) for entries with a matching scope that are fresh (watermark check) and within cosine-similarity threshold of queryVec. The first matching entry is returned.

Returns the cached entry and true on a hit. Returns a zero-value and false on a miss (no matching entry, stale entry, or no entry within threshold).

Hit increments the hit counter; miss (including after scanning all entries without a match) increments the miss counter.

Reference: Tech Spec Section 3.4 — Stage 2.

func (*SemanticCache) InvalidateDest 

func (sc *SemanticCache) InvalidateDest(dest string)

InvalidateDest advances the watermark for dest, instantly staling all existing semantic cache entries for that destination. Called by the write path after a payload is delivered.

Reference: Tech Spec Section 3.4 (watermark freshness check).

func (*SemanticCache) Len 

func (sc *SemanticCache) Len() int

Len returns the number of entries currently in the cache.

func (*SemanticCache) Put 

func (sc *SemanticCache) Put(scope [32]byte, queryVec []float32, dest string, entry SemanticCacheEntry)

Put stores a semantic cache entry. The watermark is snapshotted at Put time; a subsequent write to the destination will make this entry stale.

When the cache is at capacity, the oldest entry is evicted (FIFO).

type SemanticCacheEntry 

type SemanticCacheEntry struct {
	Records    []destination.TranslatedPayload
	NextCursor string
	HasMore    bool
}

SemanticCacheEntry is the value returned on a Stage 2 cache hit.

type SemanticStats 

type SemanticStats struct {
	// contains filtered or unexported fields
}

SemanticStats holds Prometheus counters for Stage 2 semantic-cache events. Register via NewSemanticStats on the daemon's private registry.

Metric names:

  • bubblefish_cache_semantic_hits_total
  • bubblefish_cache_semantic_misses_total

Reference: Tech Spec Section 11.3.

func NewSemanticStats 

func NewSemanticStats(reg prometheus.Registerer) *SemanticStats

NewSemanticStats creates and registers the two semantic-cache counters on reg. Panics only on programming errors (duplicate names, impossible with a fresh private registry).

func (*SemanticStats) Hit 

func (s *SemanticStats) Hit()

Hit increments the semantic cache hit counter. Safe to call on a nil receiver.

func (*SemanticStats) HitCount 

func (s *SemanticStats) HitCount() int64

HitCount returns the total number of semantic cache hits since startup. Safe to call on a nil receiver (returns 0).

func (*SemanticStats) Miss 

func (s *SemanticStats) Miss()

Miss increments the semantic cache miss counter. Safe to call on a nil receiver.

func (*SemanticStats) MissCount 

func (s *SemanticStats) MissCount() int64

MissCount returns the total number of semantic cache misses since startup. Safe to call on a nil receiver (returns 0).

type Stats 

type Stats struct {
	// contains filtered or unexported fields
}

Stats holds the Prometheus counters for exact-cache hit and miss events. Register counters on the daemon's private registry by passing it to NewStats.

Metric names:

  • bubblefish_cache_exact_hits_total
  • bubblefish_cache_exact_misses_total

Reference: Tech Spec Section 11.3.

func NewStats 

func NewStats(reg prometheus.Registerer) *Stats

NewStats creates and registers the two exact-cache counters on reg. Panics only if reg already contains counters with the same names, which is a programming error that cannot occur with the daemon's private registry.

func (*Stats) Hit 

func (s *Stats) Hit()

Hit increments the cache hit counter.

func (*Stats) HitCount 

func (s *Stats) HitCount() int64

HitCount returns the total number of cache hits since startup.

func (*Stats) Miss 

func (s *Stats) Miss()

Miss increments the cache miss counter.

func (*Stats) MissCount 

func (s *Stats) MissCount() int64

MissCount returns the total number of cache misses since startup.

type WatermarkStore 

type WatermarkStore struct {
	// contains filtered or unexported fields
}

WatermarkStore maintains a monotonically increasing counter per destination. Cache entries store the watermark at insertion time; an entry is stale when its stored watermark is less than the current destination watermark.

Watermarks are advanced by the write path each time a payload is delivered to a destination. This invalidates all exact-cache entries for that destination without requiring an explicit cache scan.

Reference: Tech Spec Section 3.4 — Stage 1 (watermark freshness check).

func NewWatermarkStore 

func NewWatermarkStore() *WatermarkStore

NewWatermarkStore creates an empty WatermarkStore. All destinations start at watermark 0; the first Advance call moves them to 1.

func (*WatermarkStore) Advance 

func (w *WatermarkStore) Advance(dest string) uint64

Advance increments the watermark for dest and returns the new value. After Advance returns, any cache entry that recorded the previous watermark is considered stale and will be rejected by ExactCache.Get.

func (*WatermarkStore) Current 

func (w *WatermarkStore) Current(dest string) uint64

Current returns the current watermark for dest. Returns 0 for unknown destinations (no writes have been delivered yet).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.