store

Documentation

Overview

Package store persists registry-stats snapshots on the local filesystem.

The FS type wraps a data directory plus a (date, mtime)-keyed cache of parsed snapshots so handler paths that walk the full retention window (handlePulls, handlePullsDaily) don't re-parse every file on every request. The on-disk format (/data/YYYY-MM-DD.json, one file per day) and the cache eviction threshold (MaxCachedSnapshots) are part of the inviolate contract — changing either would break the existing Grafana dashboards that depend on the snapshot layout.

Index

• Constants
• type CachedSnap
• type FS
  • func NewFS(dir string, opts ...Option) *FS
  • func (s *FS) CleanupStaleTmp(_ context.Context) error
  • func (s *FS) ListDates(_ context.Context) ([]string, error)
  • func (s *FS) Load(_ context.Context, date string) (*model.Snapshot, error)
  • func (s *FS) Prune(ctx context.Context, retentionDays int) (int, error)
  • func (s *FS) PullSeries(_ context.Context) []model.PullEntry
  • func (s *FS) Save(ctx context.Context, snap *model.Snapshot) error
• type Option
  • func WithDisableWrite() Option
• type PullIndex
  • func NewPullIndex() *PullIndex
  • func (idx *PullIndex) Entries() []model.PullEntry
  • func (idx *PullIndex) PruneOlderThan(cutoff string)
  • func (idx *PullIndex) Rebuild(snapshots map[string]*model.Snapshot)
  • func (idx *PullIndex) Update(date string, snap *model.Snapshot)
• type SnapshotCache
  • func NewCache() *SnapshotCache
  • func (c *SnapshotCache) Get(date string, mtime time.Time) *model.Snapshot
  • func (c *SnapshotCache) Invalidate(date string)
  • func (c *SnapshotCache) Put(date string, mtime time.Time, snap *model.Snapshot)
  • func (c *SnapshotCache) Reset()

Constants

const MaxCachedSnapshots = 120

MaxCachedSnapshots caps the in-memory snapshot cache. 120 entries covers the default 90-day retention with headroom for date-range queries that span slightly beyond retention. With RETENTION_DAYS=0 (keep forever) this prevents unbounded memory growth.

Types

type CachedSnap

type CachedSnap struct {
	Mtime time.Time
	Snap  *model.Snapshot
}

CachedSnap is one entry in the SnapshotCache. Mtime is the modtime of the source file at the moment the cache entry was populated; a later Get with a newer mtime misses the cache and the reader re-parses. Fields are exported so tests can construct zero-value entries directly.

type FS

type FS struct {
	// contains filtered or unexported fields
}

FS persists snapshots as /dir/YYYY-MM-DD.json using an atomic create-temp + rename pattern. Concurrent Save calls for the same date are serialized only through the filesystem's rename semantics; the cache stays consistent because Invalidate runs after rename.

FS satisfies api.Store — the only interface its callers depend on. Direct field access is not part of the contract; use NewFS to construct an instance.

func NewFS

func NewFS(dir string, opts ...Option) *FS

NewFS returns a store rooted at dir with a fresh snapshot cache. The cache is process-local; no on-disk index is kept. The pull index is rebuilt in a background goroutine so the constructor returns immediately without blocking on disk I/O. PullSeries callers block on the idxReady channel until the rebuild completes. When disableWrite is true, Save becomes a no-op (no JSON files written).

func (*FS) CleanupStaleTmp

func (s *FS) CleanupStaleTmp(_ context.Context) error

CleanupStaleTmp removes leftover temp files left by an interrupted Save (a crash between temp-write and rename). Delegates to atomicfile, which matches the temp scheme WriteFile uses (.atomicfile-<digits>.tmp), so the two stay coupled.

func (*FS) ListDates

func (s *FS) ListDates(_ context.Context) ([]string, error)

ListDates returns all snapshot dates in chronological order. Skips non-date filenames, directories, and atomic-write temp files.

func (*FS) Load

func (s *FS) Load(_ context.Context, date string) (*model.Snapshot, error)

Load reads and parses the snapshot for the given date (YYYY-MM-DD). Validates the date format to prevent path traversal, caps file size at 50 MB, and serves from the (date, mtime) cache when possible.

func (*FS) Prune

func (s *FS) Prune(ctx context.Context, retentionDays int) (int, error)

Prune deletes snapshot files older than retentionDays and returns the number pruned. retentionDays <= 0 is a no-op (keep forever). Honors ctx cancellation so shutdown doesn't race the sweep.

func (*FS) PullSeries

func (s *FS) PullSeries(_ context.Context) []model.PullEntry

PullSeries returns the pre-computed pull-count time-series from the in-memory index. Blocks until the background index rebuild (started by NewFS) completes, then returns a copy safe for concurrent use. Design choice: blocking (rather than returning a "not ready" error) keeps the API contract unchanged and is simpler for callers; the wait is sub-second on healthy storage.

func (*FS) Save

func (s *FS) Save(ctx context.Context, snap *model.Snapshot) error

Save atomically writes snap to <dir>/<snap.Timestamp YYYY-MM-DD>.json. Uses CreateTemp + fsync + rename so a power loss between write and rename can't leave a zero-length snapshot on disk (which would corrupt daily-delta calculations). Invalidates the cache entry for the written date so the next reader re-parses from disk.

ctx is threaded into atomicfile.WriteFile, which checks it once the mutex is held; the local filesystem write itself is expected to be sub-second (os.Rename has no Context variant).

type Option

type Option func(*FS)

Option configures a FS store.

func WithDisableWrite

func WithDisableWrite() Option

WithDisableWrite makes Save a no-op (no JSON files written to disk).

type PullIndex

type PullIndex struct {
	// contains filtered or unexported fields
}

PullIndex maintains a pre-computed time-series of per-repo pull counts, updated atomically by Save and Prune. Handlers read from it via Entries() instead of loading every snapshot from disk. The index is process-local (like SnapshotCache) — no on-disk persistence. On startup, NewFS rebuilds it from existing snapshots.

func NewPullIndex

func NewPullIndex() *PullIndex

NewPullIndex returns an empty index ready for population.

func (*PullIndex) Entries

func (idx *PullIndex) Entries() []model.PullEntry

Entries returns a snapshot of all index entries. The returned slice is a copy safe for concurrent iteration by handlers.

func (*PullIndex) PruneOlderThan

func (idx *PullIndex) PruneOlderThan(cutoff string)

PruneOlderThan removes all entries with Date < cutoff.

func (*PullIndex) Rebuild

func (idx *PullIndex) Rebuild(snapshots map[string]*model.Snapshot)

Rebuild replaces the entire index from a set of (date, snapshot) pairs. Used at startup to populate from existing files.

func (*PullIndex) Update

func (idx *PullIndex) Update(date string, snap *model.Snapshot)

Update replaces all entries for the given date with entries derived from the snapshot. Called by Save after a successful write.

type SnapshotCache

type SnapshotCache struct {
	ByDate map[string]CachedSnap
	Mu     sync.Mutex
}

SnapshotCache is a small process-local LRU-less cache for parsed snapshot files. Entries are keyed by (date, mtime); if the on-disk file's mtime changes, the cache miss triggers a fresh read. The cache is shared across handlers and the collect() path via the FS instance owning it.

ByDate and Mu are exported so tests can reset the cache directly (the handler test suite in the main package points dataDir at a fresh t.TempDir() per test and needs to discard stale entries from earlier runs without plumbing a testing hook through every handler).

func NewCache

func NewCache() *SnapshotCache

NewCache returns an empty cache ready for use.

func (*SnapshotCache) Get

func (c *SnapshotCache) Get(date string, mtime time.Time) *model.Snapshot

Get returns the cached snapshot for date if its stored mtime equals the file's current mtime; otherwise returns nil (caller re-reads).

func (*SnapshotCache) Invalidate

func (c *SnapshotCache) Invalidate(date string)

Invalidate removes the entry for date if present. Called on save (so subsequent readers re-parse the fresh file) and on prune.

func (*SnapshotCache) Put

func (c *SnapshotCache) Put(date string, mtime time.Time, snap *model.Snapshot)

Put stores snap under (date, mtime), evicting the chronologically oldest entry if the cache is at capacity. Dates are YYYY-MM-DD strings that sort chronologically, so a linear scan for the minimum key is correct and fast for <= MaxCachedSnapshots entries.

func (*SnapshotCache) Reset

func (c *SnapshotCache) Reset()

Reset empties the cache under lock. Tests that repoint dataDir call this so a previous test's entries for the same date key (under a different temp dir) can't be served from cache.