observability

package
v1.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 15, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package observability provides shared Prometheus metrics and OpenTelemetry tracing helpers for all nSelf Go services.

profiling.go provides pprof endpoint registration with token-based auth and optional Pyroscope continuous profiling agent integration.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// DurationBuckets for _seconds histograms (HTTP, DB, AI, cache, queue).
	DurationBuckets = []float64{0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10}

	// SizeBuckets for _bytes histograms (request/response body sizes).
	SizeBuckets = []float64{100, 500, 1000, 5000, 10000, 50000, 100000, 500000, 1000000, 5000000}
)

SI-unit histogram buckets per spec §1.1.

Functions

func AuditLog 

func AuditLog(ctx context.Context, event AuditEvent)

AuditLog emits a structured audit event. These events are routed by Promtail to the audit-specific Loki tenant (audit-$ENV) based on the job=audit label.

func BaseLabels 

func BaseLabels(cfg RegistryConfig) prometheus.Labels

BaseLabels returns the standard label set from config plus environment. Services should include these as ConstLabels or via a wrapping registerer.

func InitTracer 

func InitTracer(cfg TracerConfig) (func(context.Context) error, error)

InitTracer initialises the OpenTelemetry trace pipeline with an OTLP/gRPC exporter. It reads standard OTel env vars (OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_TRACES_SAMPLER_ARG, etc.) and falls back to sensible defaults. Returns a shutdown function that flushes remaining spans.

func MetricsHandler 

func MetricsHandler(registry *prometheus.Registry) http.Handler

MetricsHandler returns an http.Handler that serves /metrics for the given registry.

func NewJSONLogger 

func NewJSONLogger(cfg LogConfig) *slog.Logger

NewJSONLogger creates a slog.Logger with JSON output on stdout that includes all 12 standard fields per the observability spec: ts, level, service, env, version, msg, trace_id, span_id, request_id, user_id, tenant, caller.

The returned logger wraps a TraceLogHandler (for trace_id/span_id injection) around a PII-redacting handler around the base slog.JSONHandler.

func NewRegistry 

func NewRegistry(cfg RegistryConfig) *prometheus.Registry

NewRegistry creates a *prometheus.Registry pre-configured with base labels and process/go collectors. It enforces naming conventions (_total for counters, _seconds/_bytes for histograms) and forbids user_id as a label.

func PprofHandler 

func PprofHandler(token string) http.Handler

PprofHandler returns an http.Handler that serves pprof endpoints with token authentication. Useful for registering on an existing mux.

func Redact 

func Redact(s string) string

Redact applies PII pattern matching and replaces matches with [REDACTED].

Coverage (S12.T09): emails, phones, SSNs, IBANs, credit cards, IPv4/IPv6, MAC addresses, JWTs, and API/license/Stripe/AWS/GitHub keys via the apiKeyPrefixes table. Loopback IPs (127.0.0.1, ::1) are preserved.

COPPA + GDPR Article 9 / "special category" data (race, religion, health, minor status, sexual orientation, political opinion, biometric identifiers) is NEVER allowed into telemetry payloads by construction — the doctor check OBS-REDACT-01 enforces that callers do not send such fields at all rather than relying on textual redaction.

func RedactionFailures 

func RedactionFailures() int64

RedactionFailures returns the current count of redaction failures.

func ServeMetrics 

func ServeMetrics(registry *prometheus.Registry, bindAddr string)

ServeMetrics starts an HTTP server on bindAddr serving the /metrics endpoint for the given registry. It blocks, so call in a goroutine.

func ServeProfiling 

func ServeProfiling(cfg ProfilingConfig)

ServeProfiling starts a pprof HTTP server on the configured bind address. It blocks, so call in a goroutine. The server is bound to 127.0.0.1 only and is never exposed publicly via nginx.

func SetDefaultLogger 

func SetDefaultLogger(cfg LogConfig)

SetDefaultLogger creates and sets the global slog default logger.

func TraceMiddleware 

func TraceMiddleware(serviceName string) func(http.Handler) http.Handler

TraceMiddleware creates spans for incoming HTTP requests. Span name: "HTTP <METHOD> <route-template>".

func Tracer 

func Tracer(name string) trace.Tracer

Tracer returns a named tracer from the global provider.

func ValidateLabels 

func ValidateLabels(labels []string) error

ValidateLabels checks that no forbidden labels (e.g. user_id) are used. Returns an error if a forbidden label is found.

Types

type AIMetrics 

type AIMetrics struct {
	CallsTotal        *prometheus.CounterVec
	CallDuration      *prometheus.HistogramVec
	InputTokensTotal  *prometheus.CounterVec
	OutputTokensTotal *prometheus.CounterVec
	CostUSDTotal      *prometheus.CounterVec
	RetriesTotal      *prometheus.CounterVec
	RateLimitHits     *prometheus.CounterVec
	QualityScore      *prometheus.HistogramVec
	ContextTokens     *prometheus.HistogramVec
	ConcurrentCalls   *prometheus.GaugeVec
}

AIMetrics holds the 10 AI call metrics per spec §1.3.

func NewAIMetrics 

func NewAIMetrics(reg *prometheus.Registry) *AIMetrics

NewAIMetrics creates and registers the 10 AI metrics on the given registry.

type AuditEvent 

type AuditEvent struct {
	Event    string            // e.g. "user.login", "admin.delete", "permission.denied"
	Actor    string            // who performed the action (user ID or system identifier)
	Target   string            // what was acted upon (resource type:id)
	Metadata map[string]string // additional context
}

AuditEvent represents a structured audit log entry.

type CacheMetrics 

type CacheMetrics struct {
	HitsTotal      *prometheus.CounterVec
	MissesTotal    *prometheus.CounterVec
	OpDuration     *prometheus.HistogramVec
	EvictionsTotal *prometheus.CounterVec
}

CacheMetrics holds cache metrics per spec §1.5.

func NewCacheMetrics 

func NewCacheMetrics(reg *prometheus.Registry) *CacheMetrics

NewCacheMetrics creates and registers cache metrics on the given registry.

type ClawMetrics 

type ClawMetrics struct {
	MessagesTotal                 *prometheus.CounterVec
	ExtractionsTotal              *prometheus.CounterVec
	ExtractionDuration            *prometheus.HistogramVec
	DedupRatio                    prometheus.Histogram
	TopicClassificationConfidence prometheus.Histogram
	TopicsActive                  prometheus.Gauge
	MemoryBytes                   prometheus.Gauge
	GraphEdgesTotal               prometheus.Counter
	SearchLatency                 *prometheus.HistogramVec
	EmbeddingDim                  prometheus.Gauge
}

ClawMetrics holds the 10 claw-specific metrics per spec §1.7.

func NewClawMetrics 

func NewClawMetrics(reg *prometheus.Registry) *ClawMetrics

NewClawMetrics creates and registers claw domain metrics.

type DBMetrics 

type DBMetrics struct {
	QueryDuration  *prometheus.HistogramVec
	QueriesTotal   *prometheus.CounterVec
	PoolInUse      *prometheus.GaugeVec
	PoolIdle       *prometheus.GaugeVec
	PoolMax        *prometheus.GaugeVec
	ConnWait       *prometheus.HistogramVec
	DeadlocksTotal *prometheus.CounterVec
}

DBMetrics holds the seven database metrics per spec §1.4.

func NewDBMetrics 

func NewDBMetrics(reg *prometheus.Registry) *DBMetrics

NewDBMetrics creates and registers database metrics on the given registry.

type HTTPMetrics 

type HTTPMetrics struct {
	RequestsTotal    *prometheus.CounterVec
	RequestDuration  *prometheus.HistogramVec
	RequestSize      *prometheus.HistogramVec
	ResponseSize     *prometheus.HistogramVec
	RequestsInFlight prometheus.Gauge
	// contains filtered or unexported fields
}

HTTPMetrics holds the five standard HTTP metrics per spec §1.2.

func NewHTTPMetrics 

func NewHTTPMetrics(reg *prometheus.Registry, service string) *HTTPMetrics

NewHTTPMetrics creates and registers the five HTTP metrics on the given registry. service is the service name label value (e.g. "claw", "ai", "mux").

func (*HTTPMetrics) Middleware 

func (m *HTTPMetrics) Middleware(service string) func(http.Handler) http.Handler

Middleware returns a chi-compatible middleware that records all five HTTP metrics. Route labels use chi's route pattern (template), never raw URLs.

type LogConfig 

type LogConfig struct {
	Service   string // e.g. "claw", "ai", "mux"
	Env       string // "dev", "staging", "prod"
	Version   string // e.g. "1.0.3"
	Tenant    string // multi-tenant isolation key
	RequestID string // per-request correlation ID
	UserID    string // optional, PII-gated
}

LogConfig holds configuration for the structured JSON logger.

type MuxMetrics 

type MuxMetrics struct {
	MessagesReceived       *prometheus.CounterVec
	ClassificationTotal    *prometheus.CounterVec
	ClassificationDuration *prometheus.HistogramVec
	ActionSuccess          *prometheus.CounterVec
	FalsePositiveTotal     *prometheus.CounterVec
	ProcessingDuration     *prometheus.HistogramVec
}

MuxMetrics holds the 6 mux-specific metrics per spec §1.8.

func NewMuxMetrics 

func NewMuxMetrics(reg *prometheus.Registry) *MuxMetrics

NewMuxMetrics creates and registers mux domain metrics.

type PingMetrics 

type PingMetrics struct {
	LicenseChecksTotal   *prometheus.CounterVec
	LicenseCheckDuration *prometheus.HistogramVec
	PluginInstallsTotal  *prometheus.CounterVec
	PluginUptimeSeconds  *prometheus.GaugeVec
	PluginVersionInfo    *prometheus.GaugeVec
}

PingMetrics holds the 5 ping_api/license metrics per spec §1.9.

func NewPingMetrics 

func NewPingMetrics(reg *prometheus.Registry) *PingMetrics

NewPingMetrics creates and registers ping_api metrics.

type ProfilingConfig 

type ProfilingConfig struct {
	// BindAddr for pprof HTTP server (default 127.0.0.1:6060).
	BindAddr string
	// Token required in X-Profile-Token header. Empty disables auth.
	Token string
	// PyroscopeEnabled enables continuous profiling push to Pyroscope.
	PyroscopeEnabled bool
	// PyroscopeServerURL is the Pyroscope server address.
	PyroscopeServerURL string
	// ApplicationName identifies this service in Pyroscope.
	ApplicationName string
}

ProfilingConfig configures pprof and Pyroscope integration.

func DefaultProfilingConfig 

func DefaultProfilingConfig() ProfilingConfig

DefaultProfilingConfig reads config from environment variables.

type QueueMetrics 

type QueueMetrics struct {
	Depth          *prometheus.GaugeVec
	EnqueuedTotal  *prometheus.CounterVec
	CompletedTotal *prometheus.CounterVec
	JobDuration    *prometheus.HistogramVec
	JobWait        *prometheus.HistogramVec
	WorkersActive  *prometheus.GaugeVec
	RetriedTotal   *prometheus.CounterVec
	DLQDepth       *prometheus.GaugeVec
}

QueueMetrics holds queue metrics per spec §1.6.

func NewQueueMetrics 

func NewQueueMetrics(reg *prometheus.Registry) *QueueMetrics

NewQueueMetrics creates and registers queue metrics on the given registry.

type RedactHandler 

type RedactHandler struct {
	// contains filtered or unexported fields
}

RedactHandler wraps a slog.Handler and redacts PII from string attribute values.

func NewRedactHandler 

func NewRedactHandler(inner slog.Handler) *RedactHandler

NewRedactHandler creates a handler that redacts PII patterns from log attributes.

func (*RedactHandler) Enabled 

func (h *RedactHandler) Enabled(ctx context.Context, level slog.Level) bool

func (*RedactHandler) Handle 

func (h *RedactHandler) Handle(ctx context.Context, r slog.Record) error

func (*RedactHandler) WithAttrs 

func (h *RedactHandler) WithAttrs(attrs []slog.Attr) slog.Handler

func (*RedactHandler) WithGroup 

func (h *RedactHandler) WithGroup(name string) slog.Handler

type RegistryConfig 

type RegistryConfig struct {
	Service  string // e.g. "claw", "ai", "mux"
	Instance string // hostname or pod name
	Env      string // "dev", "staging", "prod"
	Version  string // e.g. "1.0.3"
}

RegistryConfig configures a new metrics registry.

type StackTraceHandler 

type StackTraceHandler struct {
	// contains filtered or unexported fields
}

StackTraceHandler is a slog.Handler that captures stack traces for warn+ levels. It wraps any inner handler.

func NewStackTraceHandler 

func NewStackTraceHandler(inner slog.Handler) *StackTraceHandler

NewStackTraceHandler wraps a handler to add err.stack for warn and above.

func (*StackTraceHandler) Enabled 

func (h *StackTraceHandler) Enabled(ctx context.Context, level slog.Level) bool

func (*StackTraceHandler) Handle 

func (h *StackTraceHandler) Handle(ctx context.Context, r slog.Record) error

func (*StackTraceHandler) WithAttrs 

func (h *StackTraceHandler) WithAttrs(attrs []slog.Attr) slog.Handler

func (*StackTraceHandler) WithGroup 

func (h *StackTraceHandler) WithGroup(name string) slog.Handler

type TraceLogHandler 

type TraceLogHandler struct {
	// contains filtered or unexported fields
}

TraceLogHandler wraps a slog.Handler to inject trace_id and span_id from the OTel span context into every log record.

func NewTraceLogHandler 

func NewTraceLogHandler(inner slog.Handler) *TraceLogHandler

NewTraceLogHandler creates a slog handler that injects trace context fields.

func (*TraceLogHandler) Enabled 

func (h *TraceLogHandler) Enabled(ctx context.Context, level slog.Level) bool

Enabled delegates to the inner handler.

func (*TraceLogHandler) Handle 

func (h *TraceLogHandler) Handle(ctx context.Context, r slog.Record) error

Handle injects trace_id and span_id from the OTel span context, then delegates to the inner handler.

func (*TraceLogHandler) WithAttrs 

func (h *TraceLogHandler) WithAttrs(attrs []slog.Attr) slog.Handler

WithAttrs delegates to the inner handler.

func (*TraceLogHandler) WithGroup 

func (h *TraceLogHandler) WithGroup(name string) slog.Handler

WithGroup delegates to the inner handler.

type TracerConfig 

type TracerConfig struct {
	ServiceName string
	Version     string
	Env         string
}

TracerConfig holds configuration for the OTel tracer.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.