observability

Documentation

Overview

Package observability wires up tracing, metrics, and logger correlation for services built on the magic library. A single call to Init or New during process bootstrap selects a metrics backend (Prometheus scrape or OTLP push), installs an OTEL TracerProvider (or a no-op when tracing is disabled), registers built-in HTTP and runtime metrics, and publishes the resulting backends to the neutral telemetry package that magic core packages consume.

Typical usage:

cfg := observability.DefaultConfig()
cfg.ServiceName = "my-service"
cfg.MetricsMode = observability.MetricsModePrometheus
obs, err := observability.Init(ctx, cfg)
if err != nil { ... }
defer obs.Shutdown(context.Background())

router := chi.NewRouter()
router.Use(middlewares.ObservabilityWithOptions(obs, middlewares.ObservabilityOptions{
  SkipPaths:        []string{"/metrics"},
  SkipPathPrefixes: []string{"/health/"},
}))
router.Handle("/metrics", obs.MetricsHandler())

Init and New are identical; neither is Go's special func init() hook.

Index

• Constants
• Variables
• func LoggerFromContext(ctx context.Context, l *slog.Logger) *slog.Logger
• type Config
  • func DefaultConfig() Config
• type HTTPMiddlewareState
• type MetricsMode
  • func (m MetricsMode) Valid() bool
• type Observer
  • func Init(ctx context.Context, cfg Config) (*Observer, error)
  • func New(ctx context.Context, cfg Config) (*Observer, error)
  • func (o *Observer) Counter(def telemetry.MetricDefinition) (telemetry.Counter, error)
  • func (o *Observer) Gauge(def telemetry.MetricDefinition) (telemetry.Gauge, error)
  • func (o *Observer) HTTPMiddlewareState() *HTTPMiddlewareState
  • func (o *Observer) Histogram(def telemetry.MetricDefinition) (telemetry.Histogram, error)
  • func (o *Observer) MeterProvider() metric.MeterProvider
  • func (o *Observer) MetricsHandler() http.Handler
  • func (o *Observer) Shutdown(ctx context.Context) error
  • func (o *Observer) Telemetry() *telemetry.Telemetry
  • func (o *Observer) TracerProvider() trace.TracerProvider
  • func (o *Observer) UpDownCounter(def telemetry.MetricDefinition) (telemetry.UpDownCounter, error)

Constants

const (
	// HTTP (emitted by middlewares.Observability).
	HTTPRequestsTotal          = "http_requests_total"
	HTTPRequestDurationSeconds = "http_request_duration_seconds"
	HTTPRequestSizeBytes       = "http_request_size_bytes"
	HTTPResponseSizeBytes      = "http_response_size_bytes"
	HTTPRequestsInFlight       = "http_requests_in_flight"

	// Storage (emitted by instrumented storage adapters in Phase 2).
	StorageOperationsTotal          = "magic_storage_operations_total"
	StorageOperationDurationSeconds = "magic_storage_operation_duration_seconds"
	StorageOperationErrorsTotal     = "magic_storage_operation_errors_total"

	// PubSub (emitted by instrumented publishers in Phase 3).
	PubSubMessagesTotal          = "magic_pubsub_messages_total"
	PubSubPublishDurationSeconds = "magic_pubsub_publish_duration_seconds"
	PubSubErrorsTotal            = "magic_pubsub_errors_total"
)

Names of the built-in metrics emitted by the magic core packages. These are stable public constants so callers can grep dashboards and alerts against canonical identifiers.

They remain plain string constants so values pass through to Prometheus and OTLP registration without conversion. A named string type (e.g. type BuiltInMetricName string) would need to propagate through telemetry APIs and all call sites; that is left for a deliberate future refactor if stronger typing is required.

const (
	LabelHTTPMethod     = "method"
	LabelHTTPRoute      = "route"
	LabelHTTPStatusCode = "status_code"
)

Labels used by the built-in HTTP metrics. Declared here so the middleware and any documentation can reference canonical label keys.

const (
	LabelStorageProvider  = "provider"
	LabelStorageOperation = "operation"
	LabelStorageStatus    = "status"
)

Labels used by the built-in storage metrics. Declared here so the instrumented storage wrapper and dashboards reference canonical label keys.

const (
	StorageStatusOK    = "ok"
	StorageStatusError = "error"
)

Values used for the "status" label on storage metrics. Kept small and stable so operators can alert on a predictable low-cardinality set.

const (
	StorageOpCreate  = "create"
	StorageOpGet     = "get"
	StorageOpUpdate  = "update"
	StorageOpDelete  = "delete"
	StorageOpList    = "list"
	StorageOpSearch  = "search"
	StorageOpCount   = "count"
	StorageOpQuery   = "query"
	StorageOpExecute = "execute"
	StorageOpPing    = "ping"
)

Canonical storage operation names emitted as the "operation" label on storage metrics and as the span-name suffix for storage tracing ("storage.<op>"). Schema/migration methods run once at startup outside any request context and are deliberately not instrumented.

const (
	LabelPubSubProvider    = "provider"
	LabelPubSubDestination = "destination"
	LabelPubSubOperation   = "operation"
	LabelPubSubStatus      = "status"
)

Labels used by the built-in pubsub metrics. Kept small and stable so operators can alert on a predictable low-cardinality set. See docs/observability.md for the cardinality note on `destination` (SNS topic ARNs embed the AWS account ID).

const (
	PubSubStatusOK    = "ok"
	PubSubStatusError = "error"
)

Values used for the "status" label on pubsub metrics.

const (
	PubSubOpPublish = "publish"
)

Canonical pubsub operation names. Only "publish" is supported in v1; consume/ack/nack are deferred until a Consumer interface lands in the pubsub package.

Variables

var ErrUnsupported = errors.New("observability: unsupported on this platform")

ErrUnsupported is returned by hooks that cannot run on the current platform. Kept here so callers can match on it without importing extra error packages.

Functions

func LoggerFromContext

func LoggerFromContext(ctx context.Context, l *slog.Logger) *slog.Logger

LoggerFromContext returns a *slog.Logger derived from l that has trace_id and span_id attributes pre-populated from the OpenTelemetry SpanContext on ctx. It is the escape hatch for call sites that cannot easily use the slog *Context variants — for example, a non-slog logger passed through a third-party library — but still want correlated logs.

Rules:

• If l is nil the process-wide slog.Default() is used so callers can write observability.LoggerFromContext(ctx, nil) without hand-threading a logger.
• If ctx has no valid SpanContext the input logger is returned unchanged. No empty-valued trace fields are attached, to keep non-traced log lines clean.
• The returned logger carries trace_id and span_id as top- level string attributes; additional attributes added via subsequent logger.With calls compose normally.

The handler wired by logger.Init already performs automatic correlation for slog.*Context calls. Prefer those when possible; LoggerFromContext is intentionally a secondary tool.

Types

type Config

type Config struct {
	// ServiceName is the logical service name; becomes the
	// service.name resource attribute on traces and metrics.
	// Required and non-empty.
	ServiceName string

	// ServiceVersion becomes service.version. Optional.
	ServiceVersion string

	// Environment becomes deployment.environment.name. Optional.
	Environment string

	// ResourceAttributes are extra key/value attributes merged
	// into the OTEL resource for both traces and metrics.
	// The built-in service.* and deployment.* keys take
	// precedence on conflict.
	ResourceAttributes map[string]string

	// EnableTracing turns distributed tracing on or off. When
	// false, magic core packages emit no spans even if a tracer
	// is otherwise configured.
	EnableTracing bool

	// TracesOTLPEndpoint is the gRPC endpoint for the OTLP trace
	// exporter (for example "otel-collector:4317"). When empty,
	// Init falls back to OTEL_EXPORTER_OTLP_TRACES_ENDPOINT and
	// then OTEL_EXPORTER_OTLP_ENDPOINT from the environment.
	// If none resolve, Init returns an error when EnableTracing
	// is true.
	TracesOTLPEndpoint string

	// TracesOTLPInsecure sends spans over plaintext gRPC when
	// true; otherwise TLS is used (the gRPC system roots).
	TracesOTLPInsecure bool

	// SamplingRatio controls the parent-based root sampler. A
	// nil pointer (the default) keeps OTEL's default of always
	// sampling root spans. 0.0 disables sampling entirely;
	// 1.0 samples every root span.
	SamplingRatio *float64

	// Sampler, when non-nil, overrides SamplingRatio. Escape
	// hatch for callers that need a fully custom sampler.
	Sampler sdktrace.Sampler

	// Propagator, when non-nil, overrides the default W3C
	// tracecontext+baggage propagator.
	Propagator propagation.TextMapPropagator

	// MetricsMode selects between Prometheus scrape exposition
	// and OTLP push. Required; zero value is rejected.
	MetricsMode MetricsMode

	// MetricsOTLPEndpoint is the gRPC endpoint for the OTLP
	// metric exporter. Used only when MetricsMode is
	// MetricsModeOTLP. Falls back to
	// OTEL_EXPORTER_OTLP_METRICS_ENDPOINT and then
	// OTEL_EXPORTER_OTLP_ENDPOINT like the trace endpoint.
	MetricsOTLPEndpoint string

	// MetricsOTLPInsecure sends metrics over plaintext gRPC
	// when true.
	MetricsOTLPInsecure bool

	// MetricsPushInterval controls how often the OTLP periodic
	// reader pushes. Defaults to 30s when zero. Ignored in
	// Prometheus mode.
	MetricsPushInterval time.Duration

	// AllowUndeclaredLabels relaxes strict-label enforcement on
	// custom metrics. When false (default) observations with
	// label keys not declared in MetricDefinition.Labels are
	// dropped and a one-shot warning is logged.
	AllowUndeclaredLabels bool

	// EnableRuntimeMetrics registers Go runtime metrics
	// (GC stats, goroutines, memstats) on Init. Defaults to
	// true via DefaultConfig.
	EnableRuntimeMetrics bool

	// EnableProcessMetrics registers process metrics (CPU, RSS,
	// open file descriptors) on Init. Defaults to true via
	// DefaultConfig. On platforms where these metrics are
	// unavailable the registration is silently skipped.
	EnableProcessMetrics bool

	// MetricsNamespace, when non-empty, is prefixed to every
	// custom metric name registered through Observer.Counter,
	// Observer.Histogram, and so on. The built-in magic_*
	// metrics are unaffected.
	MetricsNamespace string
}

Config configures the observability stack. Construct it directly or start from DefaultConfig and override the fields you care about. Init validates the config and returns a descriptive error on misuse.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns a Config with sensible defaults. Callers only need to supply ServiceName and MetricsMode (and, when using OTLP, the endpoints).

type HTTPMiddlewareState

type HTTPMiddlewareState struct {
	Tracer trace.Tracer

	RequestsTotal    telemetry.Counter
	RequestDuration  telemetry.Histogram
	RequestSize      telemetry.Histogram
	ResponseSize     telemetry.Histogram
	RequestsInFlight telemetry.UpDownCounter
}

HTTPMiddlewareState is the immutable snapshot of built-in HTTP instruments and tracer wiring required by the observability HTTP middleware implementation in the middlewares package.

type MetricsMode

type MetricsMode string

MetricsMode selects how metrics are exposed to collectors.

const (
	// MetricsModePrometheus exposes metrics via a pull-based
	// /metrics handler returned by Observer.MetricsHandler.
	MetricsModePrometheus MetricsMode = "prometheus"
	// MetricsModeOTLP pushes metrics over OTLP/gRPC to the
	// configured collector endpoint.
	MetricsModeOTLP MetricsMode = "otlp"
)

func (MetricsMode) Valid

func (m MetricsMode) Valid() bool

Valid reports whether m is one of the supported modes.

type Observer

type Observer struct {
	// contains filtered or unexported fields
}

Observer is the live observability handle returned by Init or New. It owns the tracer provider, meter provider (OTLP mode) or Prometheus registry, the built-in HTTP instruments, and any custom instruments registered through the Counter/Histogram/... helpers.

An Observer must be shut down exactly once, after the process stops serving traffic, to flush pending spans and metrics.

func Init

func Init(ctx context.Context, cfg Config) (*Observer, error)

Init configures and installs the observability stack. It validates cfg, builds the metrics backend and tracer, registers built-in instruments, and publishes the result to telemetry.SetGlobal so downstream packages pick it up.

The name Init is unrelated to Go's automatic func init(); this function runs only when you call it.

The returned Observer owns process-wide resources and must be shut down before exit.

func New

func New(ctx context.Context, cfg Config) (*Observer, error)

New is an alias for Init for callers who prefer constructor-style naming. It behaves the same as Init in every way.

func (*Observer) Counter

func (o *Observer) Counter(def telemetry.MetricDefinition) (telemetry.Counter, error)

Counter returns a custom Counter registered on the Observer's metrics backend. The metric name is prefixed with cfg.MetricsNamespace when non-empty. Repeated calls with an equivalent definition return the same instrument; a shape-conflicting re-registration yields an error.

The definition's Kind is always normalized to KindCounter before registration — callers can leave it zero.

func (*Observer) Gauge

func (o *Observer) Gauge(def telemetry.MetricDefinition) (telemetry.Gauge, error)

Gauge returns a custom Gauge. See Counter for registration semantics.

func (*Observer) HTTPMiddlewareState

func (o *Observer) HTTPMiddlewareState() *HTTPMiddlewareState

HTTPMiddlewareState returns the built-in HTTP instrumentation state installed during Init. A nil return means observability has not been initialized or the observer is invalid.

func (*Observer) Histogram

func (o *Observer) Histogram(def telemetry.MetricDefinition) (telemetry.Histogram, error)

Histogram returns a custom Histogram. See Counter for registration semantics. Note that in MetricsModeOTLP the Buckets field is ignored at runtime; declare custom histograms with OTEL Views registered at Init time for precise control over bucket boundaries in that mode.

func (*Observer) MeterProvider

func (o *Observer) MeterProvider() metric.MeterProvider

MeterProvider returns the OTEL MeterProvider when MetricsMode is MetricsModeOTLP. In Prometheus mode it returns nil because the Prometheus backend does not use an OTEL MeterProvider; callers should use MetricsHandler() for exposition instead.

func (*Observer) MetricsHandler

func (o *Observer) MetricsHandler() http.Handler

MetricsHandler returns the HTTP handler that should be mounted on the service's metrics endpoint.

In MetricsModePrometheus the handler produces the standard Prometheus text exposition over the Observer's registry.

In MetricsModeOTLP metrics are pushed to the collector out of band, so the handler returns a 404 with a JSON body explaining the situation. Crucially it does not return nil: chi would otherwise panic during routing. Callers can safely mount it unconditionally and treat the 404 as "not applicable".

func (*Observer) Shutdown

func (o *Observer) Shutdown(ctx context.Context) error

Shutdown flushes pending telemetry and releases resources. Safe to call multiple times; subsequent calls are no-ops. Errors from individual shutdown steps are joined and returned together so callers see the full picture in logs.

func (*Observer) Telemetry

func (o *Observer) Telemetry() *telemetry.Telemetry

Telemetry returns the Telemetry installed globally by Init. Most code should use telemetry.Global() or telemetry.FromContext() instead; this accessor is primarily useful for tests and advanced wiring scenarios.

func (*Observer) TracerProvider

func (o *Observer) TracerProvider() trace.TracerProvider

TracerProvider returns the OTEL TracerProvider used by the installed Telemetry. Callers that need to obtain tracers for non-magic packages (for example, third-party libraries instrumented with OTEL) can use this handle.

func (*Observer) UpDownCounter

func (o *Observer) UpDownCounter(def telemetry.MetricDefinition) (telemetry.UpDownCounter, error)

UpDownCounter returns a custom UpDownCounter. See Counter for registration semantics.