metrics

README

Hanzo Metrics

High-performance time-series metrics database.

Overview

Hanzo Metrics is a fast, cost-effective time-series database optimized for monitoring AI infrastructure. Store billions of data points with excellent compression and query performance for your observability needs.

Features

• High Performance: Fast ingestion and queries
• Efficient Storage: 10x compression vs alternatives
• PromQL Compatible: Use familiar Prometheus queries
• Long-term Storage: Cost-effective retention
• High Availability: Clustering and replication
• Multi-tenancy: Isolated tenant data

Quick Start

docker run -p 8428:8428 hanzo/metrics

Documentation

See the documentation for detailed guides and API reference.

License

MIT License - see LICENSE for details.

Documentation

Overview

HIP-0106 Mount() entry point for the native metrics subsystem.

import _ "github.com/hanzoai/metrics" // init() registers the subsystem

At startup the cloud binary iterates its registry and calls Mount() for each enabled subsystem, attaching /v1/metrics/* to the shared zip.App. Ingest is luxfi/metric.MetricBatch (the ZAP MsgMetricBatch payload); storage and query are native — there is no prometheus, no scrape endpoint, no /api/ path.

Package metrics is the native, ZAP-native, prometheus-free time-series store for the Hanzo cloud. It replaces the vendored Grafana/Prometheus observability backends with a small in-process store that ingests luxfi/metric.MetricBatch (the same wire shape the ZAP MsgMetricBatch transport carries) and serves range queries under /v1/metrics/*.

The storage API is deliberately tiny (Append + Query + SeriesCount) so a durable per-tenant backend (DataDir-backed, columnar) can replace the in-memory map without changing mount.go or ingest.go. There is ZERO prometheus here.

Index

• Constants
• func Mount(app *zip.App, deps cloud.Deps) error
• type LogRecord
• type LogStore
  • func NewLogStore() *LogStore
  • func (s *LogStore) Append(lr LogRecord)
  • func (s *LogStore) Count() int
  • func (s *LogStore) EnableDurability(path string) error
  • func (s *LogStore) Query(matchers map[string]string, startNs, endNs int64, contains string, limit int) []LogRecord
• type Sample
• type Series
• type Span
• type Store
  • func NewStore() *Store
  • func (s *Store) Append(name string, labels map[string]string, smp Sample)
  • func (s *Store) EnableDurability(path string) error
  • func (s *Store) IngestBatch(b *metric.MetricBatch) int
  • func (s *Store) Query(name string, matchers map[string]string, startNs, endNs int64) []Series
  • func (s *Store) SeriesCount() int
• type TraceStore
  • func NewTraceStore() *TraceStore
  • func (s *TraceStore) Append(sp Span)
  • func (s *TraceStore) ByTrace(traceID string) []Span
  • func (s *TraceStore) Count() int
  • func (s *TraceStore) EnableDurability(path string) error
  • func (s *TraceStore) Recent(startNs, endNs int64, limit int) []Span
• type WAL
  • func OpenWAL(path string) (*WAL, error)
  • func (w *WAL) Append(rec []byte) error
  • func (w *WAL) Close() error
  • func (w *WAL) Replay(fn func([]byte)) error

Constants

const Version = "0.1.0"

Version is surfaced on the /health routes.

Functions

func Mount

func Mount(app *zip.App, deps cloud.Deps) error

Mount registers the native metrics routes on the shared cloud App per HIP-0106.

Types

type LogRecord

type LogRecord struct {
	TsNs   int64             `json:"t"`
	Level  string            `json:"level,omitempty"`
	Body   string            `json:"body"`
	Labels map[string]string `json:"labels,omitempty"`
}

LogRecord is one structured log line — the native, prometheus-free, Loki-free log signal. Bodies are stored verbatim; labels are the indexed dimensions.

type LogStore

type LogStore struct {
	// contains filtered or unexported fields
}

LogStore is the native log store: an append-only bounded ring with label + time-range + case-insensitive substring query. Durable via the shared WAL.

func NewLogStore

func NewLogStore() *LogStore

NewLogStore returns an empty store retaining up to 1Mi records in memory.

func (*LogStore) Append

func (s *LogStore) Append(lr LogRecord)

Append stores one log record (and durably logs it when durability is on).

func (*LogStore) Count

func (s *LogStore) Count() int

Count reports the number of records held.

func (*LogStore) EnableDurability

func (s *LogStore) EnableDurability(path string) error

EnableDurability opens and replays a WAL at path (e.g. <DataDir>/logs/logs.wal).

func (*LogStore) Query

func (s *LogStore) Query(matchers map[string]string, startNs, endNs int64, contains string, limit int) []LogRecord

Query returns up to limit records (newest first) matching labels, the [startNs,endNs] range, and an optional case-insensitive substring of Body.

type Sample

type Sample struct {
	TsNs  int64   `json:"t"`
	Value float64 `json:"v"`
}

Sample is a single timestamped value. Ts is nanoseconds since the Unix epoch to match luxfi/metric.MetricBatch.TimestampNs.

type Series

type Series struct {
	Name    string            `json:"name"`
	Labels  map[string]string `json:"labels,omitempty"`
	Samples []Sample          `json:"samples"`
}

Series is a named, labeled append-only stream of samples.

type Span

type Span struct {
	TraceID string            `json:"traceId"`
	SpanID  string            `json:"spanId"`
	Parent  string            `json:"parentId,omitempty"`
	Name    string            `json:"name"`
	StartNs int64             `json:"startNs"`
	EndNs   int64             `json:"endNs"`
	Attrs   map[string]string `json:"attrs,omitempty"`
}

Span is one unit of a distributed trace — the native, prometheus-free, Tempo-free trace signal. Times are nanoseconds since the Unix epoch.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is the native in-memory time-series store. Each series is bounded to maxPerSeries samples (oldest evicted first). Safe for concurrent use.

func NewStore

func NewStore() *Store

NewStore returns an empty store with a default per-series retention of 64Ki samples (a real deployment sets this from config / per-tenant quota).

func (*Store) Append

func (s *Store) Append(name string, labels map[string]string, smp Sample)

Append adds one sample to the series identified by (name, labels), creating the series on first write and evicting the oldest sample past retention. When durability is enabled the sample is also written to the WAL.

func (*Store) EnableDurability

func (s *Store) EnableDurability(path string) error

EnableDurability opens a write-ahead log at path and replays it into the store so samples survive restart. Replayed samples are not re-logged. Pass a per-deployment path (e.g. <DataDir>/metrics/metrics.wal).

func (*Store) IngestBatch

func (s *Store) IngestBatch(b *metric.MetricBatch) int

IngestBatch writes every sample in a luxfi/metric.MetricBatch into the store. This is the exact wire type the ZAP MsgMetricBatch transport carries, so the same code path serves both the HTTP /v1/metrics/batch endpoint and a future ZAP receiver. Counter/gauge values land directly; histogram/summary families contribute derived <name>_sum and <name>_count series. Returns samples written.

func (*Store) Query

func (s *Store) Query(name string, matchers map[string]string, startNs, endNs int64) []Series

Query returns copies of every series whose Name equals name (or all, if name is "") and whose labels are a superset of matchers, with samples restricted to [startNs, endNs] (a zero bound is treated as unbounded).

func (*Store) SeriesCount

func (s *Store) SeriesCount() int

SeriesCount reports the number of distinct series held (surfaced on /health).

type TraceStore

type TraceStore struct {
	// contains filtered or unexported fields
}

TraceStore is the native span store: an append-only bounded ring with a trace-id index for waterfall lookup and time-range listing. Durable via WAL.

func NewTraceStore

func NewTraceStore() *TraceStore

NewTraceStore returns an empty store retaining up to 1Mi spans in memory.

func (*TraceStore) Append

func (s *TraceStore) Append(sp Span)

Append stores one span (and durably logs it when durability is on).

func (*TraceStore) ByTrace

func (s *TraceStore) ByTrace(traceID string) []Span

ByTrace returns every span belonging to a trace id (the waterfall).

func (*TraceStore) Count

func (s *TraceStore) Count() int

Count reports the number of spans held.

func (*TraceStore) EnableDurability

func (s *TraceStore) EnableDurability(path string) error

EnableDurability opens and replays a WAL at path (e.g. <DataDir>/traces/traces.wal).

func (*TraceStore) Recent

func (s *TraceStore) Recent(startNs, endNs int64, limit int) []Span

Recent returns up to limit spans (newest first) starting within [startNs,endNs].

type WAL

type WAL struct {
	// contains filtered or unexported fields
}

WAL is a simple append-only write-ahead log of length-prefixed records. It gives the in-memory stores durability: every write is appended here and the log is replayed on startup. This is intentionally a single flat segment — a production deployment adds rotation/compaction behind the same Append/Replay API without touching callers.

func OpenWAL

func OpenWAL(path string) (*WAL, error)

OpenWAL opens (creating parent dirs and the file as needed) an append log.

func (*WAL) Append

func (w *WAL) Append(rec []byte) error

Append writes one length-prefixed record and flushes it to disk.

func (*WAL) Close

func (w *WAL) Close() error

Close flushes and closes the log.

func (*WAL) Replay

func (w *WAL) Replay(fn func([]byte)) error

Replay invokes fn for every record from the start, then positions the file at the end so subsequent Appends extend the log. Call once on startup.