config

Documentation

Index

• type AIConfig
• type AuthConfig
• type BackgroundConfig
• type CHConfig
  • func (c CHConfig) Hosts() []string
• type Config
  • func Load(path string) (*Config, error)
• type ESConfig
• type ESFieldsConfig
• type ExemplarsConfig
• type IngestionConfig
• type ListenConfig
• type LogsConfig
• type OIDCConfig
• type RedisConfig
• type RetentionConfig
• type TrustedHeaderConfig

Types

type AIConfig

type AIConfig struct {
	Provider string `yaml:"provider"` // env: COREMETRY_AI_PROVIDER (anthropic|github|openai)
	APIKey   string `yaml:"api_key"`  // env: COREMETRY_AI_API_KEY
	Model    string `yaml:"model"`    // env: COREMETRY_AI_MODEL
	BaseURL  string `yaml:"base_url"` // env: COREMETRY_AI_BASE_URL (openai provider only)
}

AIConfig wires the optional AI Copilot. Three providers supported:

• "anthropic" (default): APIKey is an `sk-ant-…` key from the Anthropic console.
• "github": APIKey is a GitHub OAuth token (`ghu_…`) with Copilot access; Coremetry exchanges it for a session token and calls api.githubcopilot.com (OpenAI-compatible).
• "openai": Any OpenAI-compatible /v1/chat/completions endpoint. Drives self-hosted local LLMs (Ollama, LM Studio, vLLM, llama.cpp server) and the real OpenAI API. Set BaseURL to your local endpoint (e.g. http://ollama:11434/v1) and APIKey is optional for endpoints that don't gate on it.

Env config is the boot-time default; the admin Settings tab can override at runtime, persisted to system_settings.

type AuthConfig

type AuthConfig struct {
	JWTSecret       string        `yaml:"jwt_secret"`       // HS256 key — set via COREMETRY_JWT_SECRET in prod
	TokenTTL        time.Duration `yaml:"token_ttl"`        // session lifetime (default 24h)
	InitialAdmin    string        `yaml:"initial_admin"`    // email — seeded if users table is empty
	InitialPassword string        `yaml:"initial_password"` // bcrypted on first boot
	// AdminReset (COREMETRY_ADMIN_RESET) makes the env creds authoritative for
	// the bootstrap admin: when true, InitialAdmin's password is reconciled
	// from InitialPassword on EVERY boot, even if the users table already has
	// rows. Set once to recover a locked-out admin (then remove), or leave on
	// for GitOps installs where the secret is the source of truth. Default off
	// preserves the seed-once behaviour (UI password rotation survives restart).
	AdminReset    bool                `yaml:"admin_reset"`
	OIDC          OIDCConfig          `yaml:"oidc"`
	TrustedHeader TrustedHeaderConfig `yaml:"trusted_header"`

	// Demo only — when true, /api/auth/config exposes the initial admin
	// credentials so the login page can pre-fill them. NEVER enable in
	// production: anyone hitting /api/auth/config gets the admin password.
	DemoMode bool `yaml:"demo_mode"`
}

AuthConfig drives JWT + initial-admin bootstrap. The HMAC secret is generated on first run if neither config nor env supply one — but that rotates every restart and invalidates sessions, so production deployments should set it explicitly.

type BackgroundConfig

type BackgroundConfig struct {
	// Interval between anomaly-detector sweeps. Default 2m.
	AnomalyInterval time.Duration `yaml:"anomaly_interval"`
	// Recorder cadence + backfill window. Default 1m / 5m.
	AnomalyRecordInterval time.Duration `yaml:"anomaly_record_interval"`
	AnomalyRecordBackfill time.Duration `yaml:"anomaly_record_backfill"`
	// SMTP settings refresh TTL. Default 30s — short so the
	// operator's Settings change is picked up on the next
	// alert send without a restart.
	SMTPCacheTTL time.Duration `yaml:"smtp_cache_ttl"`
	// Status probe ceiling — hard timeout on /api/status so a
	// stuck CH/Redis client driver doesn't park the goroutine
	// forever. Default 5s, well above the per-probe 2s timeout.
	StatusProbeTimeout time.Duration `yaml:"status_probe_timeout"`
	// LogAnomalyEnabled gates the worker-role log-pattern anomaly
	// recorder + Drain template puller — the jobs that periodically
	// QUERY the logstore (ES at billion-doc scale: curated-pattern
	// _msearch, significant_text, sample pulls). Default true. Set
	// COREMETRY_LOG_ANOMALY_ENABLED=false to silence that ES traffic
	// when the operator doesn't want log-based anomalies. Metric
	// anomaly detection (CH-backed) is unaffected.
	LogAnomalyEnabled bool `yaml:"log_anomaly_enabled"`
}

BackgroundConfig controls the cadence of every internal worker loop — anomaly detector, recorder, status probes, etc. Default values match what was hard-coded in main.go pre-v0.4.95. Tuning these matters at scale: a 2-min detector tick on a busy stack adds up to non-trivial CH load, and operators on slow CH clusters want to back off; operators on demo deployments often want to crank them down so anomalies surface quickly.

Each field is a duration; zero means "use the default". The defaults() function applies them so a config file that names the section but omits a knob still gets the sensible value.

type CHConfig

type CHConfig struct {
	Addr               string `yaml:"addr"` // comma-separated for cluster
	Database           string `yaml:"database"`
	Username           string `yaml:"username"`
	Password           string `yaml:"password"`
	Secure             bool   `yaml:"secure"`               // native-TLS (9440)
	InsecureSkipVerify bool   `yaml:"insecure_skip_verify"` // self-signed CA escape hatch
	MaxOpenConns       int    `yaml:"max_open_conns"`
	DialTimeout        string `yaml:"dial_timeout"`

	// ClusterName turns on Distributed-CH mode. When non-empty:
	//   - All DDL gets `ON CLUSTER <ClusterName>` so schema is
	//     applied across every node in the named ZK-coordinated
	//     cluster (cluster definition lives in CH's remote_servers).
	//   - High-volume tables (spans, logs, metric_points, profiles)
	//     are created as `<name>_local` ReplicatedMergeTree on each
	//     shard plus a `<name>` Distributed wrapper that fans out
	//     inserts and queries.
	//   - Materialized views feed the *_local tables; their
	//     Distributed wrappers re-export.
	// When empty (default), single-node MergeTree behaviour is
	// preserved exactly — existing deployments keep working.
	ClusterName string `yaml:"cluster_name"`
	// ReplicaPath is the ZooKeeper / Keeper prefix used for
	// ReplicatedMergeTree path argument. {shard}/{replica} macros
	// are appended automatically. Default: "/clickhouse/tables".
	ReplicaPath string `yaml:"replica_path"`
	// ShardKey: the SQL expression used as the Distributed shard
	// key. Defaults to "rand()" — even distribution, no locality
	// guarantee. Set to "cityHash64(trace_id)" if you want all
	// spans of a trace to land on the same shard (faster joins,
	// at the cost of slightly less even rows-per-shard).
	ShardKey string `yaml:"shard_key"`
	// AllowUnsetCluster (COREMETRY_CH_ALLOW_UNSET_CLUSTER) is the escape hatch
	// for the genuinely-broken external-Distributed-unset state: `spans` is an
	// external Distributed table but ClusterName is empty, so Coremetry can't
	// own spans_local — MV insert-triggers never fire (empty dashboards) and
	// op_group ALTERs can't reach the shards. By default boot HARD-ERRORS there
	// with the cluster to set; set this true to run in degraded mode anyway
	// (raw-spans reads only). v0.8.213.
	AllowUnsetCluster bool `yaml:"allow_unset_cluster"`

	// Per-query memory limits (v0.9.184) — env-tunable so a large
	// external cluster can raise the conservative built-in defaults
	// (4GB cap / 1GB spill / 1GB sort) without a rebuild. 0 = keep the
	// default. A fleet-wide aggregation (errors-inbox refresh) hit the
	// 4GB cap with CH code 241 "memory limit exceeded" on a big prod
	// cluster whose nodes have far more RAM than 4GB.
	//   COREMETRY_CH_MAX_MEMORY_USAGE
	//   COREMETRY_CH_MAX_BYTES_EXTERNAL_GROUP_BY
	//   COREMETRY_CH_MAX_BYTES_EXTERNAL_SORT
	MaxMemoryUsage          int64 `yaml:"max_memory_usage"`
	MaxBytesExternalGroupBy int64 `yaml:"max_bytes_external_group_by"`
	MaxBytesExternalSort    int64 `yaml:"max_bytes_external_sort"`
}

CHConfig describes the ClickHouse connection. `Addr` accepts either a single endpoint or a comma-separated list of seeds (e.g. "ch1.example:9440,ch2.example:9440,ch3.example:9440,ch4.example:9440") — driver load-balances + fails over across them, so an external N-node cluster can be configured without an upstream LB.

`Secure` toggles native-TLS (port 9440); leave false for plain 9000. `InsecureSkipVerify` is the escape hatch for self-signed internal certs — set false in production once a CA bundle is in place.

func (CHConfig) Hosts

func (c CHConfig) Hosts() []string

Hosts splits Addr on commas and trims surrounding whitespace, so the chart can hand us a single env var with multiple seeds.

type Config

type Config struct {
	Listen     ListenConfig     `yaml:"listen"`
	ClickHouse CHConfig         `yaml:"clickhouse"`
	Retention  RetentionConfig  `yaml:"retention"`
	Ingestion  IngestionConfig  `yaml:"ingestion"`
	Auth       AuthConfig       `yaml:"auth"`
	Redis      RedisConfig      `yaml:"redis"`
	Logs       LogsConfig       `yaml:"logs"`
	AI         AIConfig         `yaml:"ai"`
	Background BackgroundConfig `yaml:"background"`
	Exemplars  ExemplarsConfig  `yaml:"exemplars"`
	// PublicURL is the operator-facing base URL of this
	// Coremetry deployment (e.g. https://coremetry.bank.local).
	// Notification bodies (Slack / Teams / Zoom / email /
	// generic webhook) include deep-links to the relevant
	// problem / anomaly / incident detail when set. Empty
	// disables the linking — back-compat for deployments
	// where the URL isn't reachable from the recipient's
	// network.
	PublicURL string `yaml:"public_url"`
}

func Load

func Load(path string) (*Config, error)

type ESConfig

type ESConfig struct {
	Addresses          []string `yaml:"addresses"`
	Username           string   `yaml:"username"`
	Password           string   `yaml:"password"`
	APIKey             string   `yaml:"api_key"`
	InsecureSkipVerify bool     `yaml:"insecure_skip_verify"`
	Index              string   `yaml:"index"`
	// IndexTemplate (v0.8.231) narrows service-scoped queries to the
	// concrete per-service index instead of fanning out over Index.
	// Placeholders: {service} = the queried service name verbatim,
	// {namespace} = the service's namespace resolved from span resource
	// attributes (k8s.namespace.name / service.namespace); unresolved →
	// "*". Example: "app-{service}.{namespace}". Empty = disabled.
	IndexTemplate string `yaml:"index_template"` // env: COREMETRY_ES_INDEX_TEMPLATE
	// MLEnabled turns on the v0.5.120 read-only ML anomaly job
	// poller — Coremetry calls /_ml/anomaly_detectors and ingests
	// significant records into anomaly_events so existing Elastic
	// ML jobs surface on the /anomalies page alongside the native
	// detectors. Read-only against Elastic.
	MLEnabled bool `yaml:"ml_enabled"`
	// MLMinScore overrides the per-record score threshold (default
	// 75 — Elastic's "critical" band; lower brings more noise).
	MLMinScore float64 `yaml:"ml_min_score"`
	// Fields pins Coremetry's ES query to the operator's document
	// mapping (field paths). Empty members fall back to the ECS-ish
	// defaults in logstore (@timestamp / trace.id / span.id /
	// service.name / message / log.level). Set these when the log
	// pipeline uses different paths so Coremetry queries the right
	// fields WITHOUT re-indexing — e.g. trace_id instead of trace.id.
	Fields ESFieldsConfig `yaml:"fields"`
}

ESConfig mirrors logstore.ESConfig. Kept here so the config package stays the single source of truth for env-var bindings; main.go copies these fields into the logstore.ESConfig at construction time.

type ESFieldsConfig

type ESFieldsConfig struct {
	Timestamp      string `yaml:"timestamp"`       // env: COREMETRY_ES_FIELD_TIMESTAMP   (default @timestamp)
	TraceID        string `yaml:"trace_id"`        // env: COREMETRY_ES_FIELD_TRACE_ID    (default trace.id)
	SpanID         string `yaml:"span_id"`         // env: COREMETRY_ES_FIELD_SPAN_ID     (default span.id)
	Service        string `yaml:"service"`         // env: COREMETRY_ES_FIELD_SERVICE     (default service.name)
	Message        string `yaml:"message"`         // env: COREMETRY_ES_FIELD_MESSAGE     (default message)
	SeverityText   string `yaml:"severity_text"`   // env: COREMETRY_ES_FIELD_SEVERITY_TEXT   (default log.level)
	SeverityNumber string `yaml:"severity_number"` // env: COREMETRY_ES_FIELD_SEVERITY_NUMBER (default "" — skipped)
	// Environment (v0.8.400 — env-separation Phase 4): the document
	// field the ?env= filter targets. Default "" = SELF-DISCOVER via a
	// cached field_caps over the candidate shapes (logstore
	// es_env_field.go); set only when the pipeline uses a custom path.
	Environment string `yaml:"environment"` // env: COREMETRY_ES_FIELD_ENV (default "" — self-discover)
}

ESFieldsConfig is the env/yaml binding for the ES document field map. Mirrors logstore.ESFieldMap; main.go copies it across at construction. Each member maps to a COREMETRY_ES_FIELD_<NAME> env var (see Load). Leaving one empty keeps the logstore default.

type ExemplarsConfig

type ExemplarsConfig struct {
	RequireTraceContext bool `yaml:"require_trace_context"`
	// MaxPerSeriesPerMinute (v0.8.433, exemplar audit Faz C) — ingest-side
	// cap: at most N exemplars per series_fingerprint per wall-clock
	// minute; excess is dropped (intentional, counted separately from
	// data loss). 0 = unlimited (the default and the pre-Faz-C behavior:
	// exemplar volume is producer-bounded — SDKs keep ~1 per series per
	// export — so most installs never need this). Set it when a
	// misbehaving SDK or a custom exporter floods the exemplars table.
	MaxPerSeriesPerMinute int `yaml:"max_per_series_per_minute"` // env: COREMETRY_EXEMPLARS_MAX_PER_SERIES_MIN
}

ExemplarsConfig gates OTLP metric-exemplar ingest (v0.8.328, cross-signal pivot). RequireTraceContext (yaml exemplars.require_trace_context, default TRUE) drops exemplars that carry no trace_id — a stored exemplar exists to be clicked through to its trace, so trace-less ones are dead weight unless the operator explicitly wants them.

Zero-value handling: the default lives in the `defaults` var (true) and Load unmarshals the YAML ON TOP of it — an absent key keeps true while an explicit `require_trace_context: false` is honoured. A post-load zero-fill (the retention-days pattern) would be WRONG for a bool: it can't tell "unset" from "operator said false". Same mechanism as Background.LogAnomalyEnabled.

type IngestionConfig

type IngestionConfig struct {
	BatchSize     int           `yaml:"batch_size"`
	BufferSize    int           `yaml:"buffer_size"`
	FlushInterval time.Duration `yaml:"flush_interval"`
	Workers       int           `yaml:"workers"`
	// ByteBudgetMB caps the APPROXIMATE megabytes EACH signal consumer
	// may hold in memory — channel backlog + accumulating and in-flight
	// batches (v0.8.355, HA audit 🟡#1). BufferSize alone bounds item
	// COUNT; with fat items (15-25KB Java stack-trace log bodies) a
	// 500k-item buffer behind a stalled ClickHouse is multi-GB and the
	// kubelet OOMKill destroys ALL buffered signals — a counted drop
	// loses less. ×5 math: the budget is PER consumer and there are 5
	// (spans / logs / metrics / exemplars / span_links), so worst-case
	// buffered memory ≈ 5 × ByteBudgetMB — the 512 default bounds it at
	// ~2.5GB, sized for typical 4-8GB pods. Explicit 0 disables the
	// byte cap (count-only, pre-v0.8.355 behavior).
	ByteBudgetMB int `yaml:"byte_budget_mb"`
}

type ListenConfig

type ListenConfig struct {
	HTTP string `yaml:"http"`
	GRPC string `yaml:"grpc"`
	// OTLPHTTP is a DEDICATED OTLP/HTTP listener (POST /v1/{traces,logs,metrics}),
	// bound separately from HTTP (which also serves the Web UI + REST API on the
	// same socket). Default ":4318" — the OTel-convention port — so external
	// collectors push OTLP/HTTP to the standard port WITHOUT reaching the
	// login-gated UI that shares :8088. Empty disables it (OTLP/HTTP still
	// answers on HTTP/:8088). Only started in the ingest role. Serves plain
	// HTTP — TLS is terminated at the edge (OpenShift Route edge termination /
	// Ingress / LB), never in-binary. v0.9.168.
	OTLPHTTP string `yaml:"otlp_http"`
}

type LogsConfig

type LogsConfig struct {
	Backend       string   `yaml:"backend"` // "clickhouse" (default) | "elasticsearch"
	Elasticsearch ESConfig `yaml:"elasticsearch"`
}

LogsConfig picks which read backend serves /api/logs. Ingest still always writes to ClickHouse — this only changes the read path so an operator can point Coremetry at an external ES that their existing shipping pipeline already populates, without re-indexing.

backend: clickhouse  → query the local CH `logs` table (default)
backend: elasticsearch → query Elasticsearch via Elastic.Addresses

type OIDCConfig

type OIDCConfig struct {
	Enabled        bool     `yaml:"enabled"`
	IssuerURL      string   `yaml:"issuer_url"` // e.g. https://accounts.google.com
	ClientID       string   `yaml:"client_id"`
	ClientSecret   string   `yaml:"client_secret"`
	RedirectURL    string   `yaml:"redirect_url"`    // public URL of /api/auth/oidc/callback
	Scopes         []string `yaml:"scopes"`          // default: ["openid", "email", "profile"]
	DisplayName    string   `yaml:"display_name"`    // shown on login button (default: "SSO")
	DefaultRole    string   `yaml:"default_role"`    // role for first-time OIDC users (default: viewer)
	AllowedDomains []string `yaml:"allowed_domains"` // optional email-domain whitelist (e.g. ["acme.com"])
}

OIDCConfig is fully optional — when Enabled is false, only local username/password auth is offered. Local auth is never disabled, even when OIDC is on, so admins always have a fallback path.

type RedisConfig

type RedisConfig struct {
	URL string `yaml:"url"` // e.g. "redis://localhost:6379/0"
}

RedisConfig is fully optional. When URL is empty Coremetry runs in single-instance mode: no cache (always misses) and the in-process "always-leader" lock — meaning background workers always run.

type RetentionConfig

type RetentionConfig struct {
	SpansDays   int `yaml:"spans_days"`
	LogsDays    int `yaml:"logs_days"`
	MetricsDays int `yaml:"metrics_days"`
}

type TrustedHeaderConfig

type TrustedHeaderConfig struct {
	Enabled bool `yaml:"enabled"`
	// Header names — defaults match oauth2-proxy (which is the
	// dominant implementation in the OAuth-proxy-fronts-app
	// pattern). Operators using a different proxy override.
	EmailHeader  string `yaml:"email_header"`  // default "X-Auth-Request-Email"
	UserHeader   string `yaml:"user_header"`   // default "X-Auth-Request-User"
	GroupsHeader string `yaml:"groups_header"` // default "X-Auth-Request-Groups"
	// AutoProvision: first-sight user lands in the users table
	// with DefaultRole. Without it, an unmapped email returns
	// 403 — admins pre-create accounts.
	AutoProvision bool   `yaml:"auto_provision"`
	DefaultRole   string `yaml:"default_role"` // default "viewer"
	// TrustedProxies — CIDR blocks the headers are accepted
	// from. Required when Enabled is true; an empty list with
	// Enabled=true is a config error (the validate step refuses
	// to boot to keep operators from accidentally opening a
	// header-spoofing hole).
	TrustedProxies []string `yaml:"trusted_proxies"`
}

TrustedHeaderConfig wires the oauth2-proxy / IAP / Cloudflare Access pattern: an upstream proxy validates SSO and sets identity headers on the request, and Coremetry trusts them without re-doing the OIDC dance itself. Common for banks running oauth2-proxy + Dex / Keycloak in front of every internal service so each app doesn't need its own OIDC client registration.

SECURITY: this mode is only safe behind a proxy that strips + re-injects the headers. Anyone able to send a raw request to Coremetry on a network where the proxy isn't enforced could spoof `X-Auth-Request-Email: admin@bank.com` and become admin. TrustedProxies enforces source-IP gating; we refuse to honour the headers from any other source.