logstore

package
v0.9.679 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 5, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

v0.9.630 — "ELASTICSEARCH UNREACHABLE AT BOOT" çoğu zaman YALAN'dı.

Operatör-bildirimli: boot'ta bu satır çıkıyor, sonra ES kendiliğinden düzeliyor. Sebep ağ değil SIRALAMA: 

main.go  buildLogStore(cfg, …)     ← YALNIZ env/YAML config
         → API key env'de YOK (Settings'te, system_settings'te)
         → ping auth=none ile gidiyor → ES 401 döndürüyor
         → "ELASTICSEARCH UNREACHABLE AT BOOT"
main.go  logsMgr.LoadPersisted(…)  ← API key ANCAK BURADA geliyor
         → ES gerçekten bağlanıyor

Yani küme ayaktaydı, adresler doğruydu, kimlik bilgisi henüz yüklenmemişti. Mesaj operatörü ağa ve adreslere bakmaya yolluyordu — yanlış yere. Bu, kod bir şeyi bilmediği için değil, bildiğini SÖYLEMEDİĞİ için oluşan bir hata: NewES 401'i ayırt ediyor ama çağırana düz bir error olarak veriyor, o da hepsini "unreachable" diye raporluyor.

Package logstore is the read-side abstraction for log queries. The write side stays in chstore (OTLP logs are always batched into the ClickHouse `logs` table on ingest); on the read side an operator can point Coremetry at an external Elasticsearch cluster instead so queries hit the same index their existing logging pipeline ships to.

Two backends ship today:

  • chstore-backed (default) — uses the same `logs` table as ingest
  • elasticsearch-backed — wraps github.com/elastic/go-elasticsearch

Both expose the same `Search` surface so api.go doesn't have to know which is in use.

Index

Constants

View Source 
const LogsTailMax = 500

LogsTailMax caps the per-tick batch the live-tail forward read returns (Filter.SinceNs mode). A bounded read per tick keeps the CH/ES query cheap; when a tick fills this cap the stream emits a `gap` marker so a busy service surfaces "fell behind" instead of silently dropping rows.

View Source 
const PivotTimeout = 3 * time.Second

PivotTimeout is the default per-request budget for pivot log reads. 3s — tighter than the backends' own 10s query knobs on purpose: a pivot is one tab of a trace view already on screen, so past ~3s an empty "backend slow" tab beats a spinner holding the whole view hostage.

Variables

View Source 
var ErrBackendSlow = errors.New("log backend slow/unreachable")

ErrBackendSlow is the sentinel a pivot caller matches with errors.Is to degrade gracefully: the log backend timed out or is unreachable — NOT a query error. The trace view must never block on the log backend (audit §3); callers return a partial result with a degraded flag instead of 5xx.

Functions

func ESBootDiagnosis added in v0.9.630 

func ESBootDiagnosis(err error, expectPersisted bool) (headline, hint string)

ESBootDiagnosis — main.go'nun boot'ta basacağı satır. SAF (tablo testli): sınıflandırma ağdan ve loglamadan ayrı.

expectPersisted: kaydedilmiş (system_settings) bir ES yapılandırması yüklenmek üzere mi? Öyleyse kimliksiz bir 401 gürültü değil, sıradan bir boot adımı.

func FallbackTraceContext added in v0.8.466 

func FallbackTraceContext(attrs map[string]string, body string) (traceID, spanID string)

FallbackTraceContext, boş kalan trace/span kimliklerini attribute path'lerinden ve gövde JSON'ından çıkarır. Bulamadığı için boş dönen değerler çağıranın mevcut boşunu değiştirmez.

func HasPersistedESSettings added in v0.9.630 

func HasPersistedESSettings(ctx context.Context, store ESSettingsStore) bool

HasPersistedESSettings — system_settings'te kimlik BİLGİSİ TAŞIYAN bir ES yapılandırması duruyor mu?

Yalnız "blob var mı" yetmez: kaydedilmiş ama kimliksiz bir blob 401'i açıklamaz. Sorulan şey "birazdan yüklenecek olan şey bu 401'i çözecek mi".

Hata hâlinde false: emin olamadığımızda YUMUŞATMA yapmıyoruz — operatörün gerçek bir yapılandırma hatasını "birazdan düzelir" diye okuması, gereksiz bir uyarı görmesinden kötü.

func MapBackendSlow added in v0.8.350 

func MapBackendSlow(err error, opCtx, parent context.Context) error

MapBackendSlow classifies a failed backend call: slow/unreachable → ErrBackendSlow (wrapped, original cause preserved for logs), genuine query errors (ES 400, bad field, …) pass through unchanged. v0.8.350 (HA 🟡6) — exported so the /logs search, histogram, field-stats and context handlers can extend the v0.8.331 trace-pivot degrade contract (200 {degraded:true} instead of 5xx) to calls that aren't plain Search (Histogram, FieldStats).

opCtx is the context the backend call actually ran under (usually a WithTimeout child); parent is the caller's context. Our own deadline firing can surface as a backend-specific error string (the CH driver wraps ctx errors), so opCtx is checked directly — but only while the PARENT is still live: a client disconnect (context.Canceled all the way up) keeps its honest cancellation error, so healthy backends never get degraded payloads cached on their behalf.

Types

type CHStore 

type CHStore struct {
	// contains filtered or unexported fields
}

CHStore adapts the existing ClickHouse logs table to the LogStore interface. Pure delegation — chstore.GetLogs already takes a similar filter shape.

func NewCH 

func NewCH(store *chstore.Store) *CHStore

func (*CHStore) Backend 

func (s *CHStore) Backend() string

func (*CHStore) CountPatterns added in v0.5.241 

func (s *CHStore) CountPatterns(
	ctx context.Context,
	pats []PatternSpec,
	curStart, baseStart, now time.Time,
) ([]PatternStats, error)

CountPatterns matches each detector pattern against the raw logs table. CH path runs sequentially because each query is already cheap behind the tokenbf_v1 skip index — granules without any of the tokens get pruned before the per-row regex eval. ES batches via _msearch (see elasticsearch.go) because its per-pattern cost dominates the round-trip cost there.

func (*CHStore) EQLSearch added in v0.5.468 

func (s *CHStore) EQLSearch(ctx context.Context, q EQLQuery) ([]EQLSequence, error)

EQLSearch — CH stub (v0.5.468). EQL is ES-native; CH has no equivalent sequence-matching aggregate. Returning a typed error lets the handler surface "not supported on this backend" cleanly and lets the frontend hide the EQL panel.

func (*CHStore) FieldStats added in v0.8.255 

func (s *CHStore) FieldStats(ctx context.Context, f Filter, field string, limit int) (*FieldStatsResult, error)

FieldStats — top-N values of one field in the filtered window (fields-panel accordion, v0.8.255). Well-known ids resolve to their indexed columns; anything else is looked up in the attributes map first, then resource_attributes. Grouping is capped (max_rows_to_group_by + 'any' overflow) so a pathological high-cardinality field (trace_id…) degrades to approximate counts instead of blowing memory at billion-row scale.

func (*CHStore) FieldValues added in v0.5.464 

func (s *CHStore) FieldValues(ctx context.Context, field, prefix string, limit int, from, to time.Time) ([]string, error)

FieldValues — CH stub (v0.5.464). The KQL search box's field-aware autocomplete is Kibana-flavoured and primarily useful on ES installs; CH operators tend to filter via the explicit FilterBuilder UI instead. Returning empty makes the autocomplete simply not surface on CH backends, which degrades gracefully without spurious "no matches" rows. A CH-native implementation (SELECT DISTINCT field WHERE field LIKE prefix% LIMIT) can land as a follow-up if operators report wanting it.

func (*CHStore) Histogram 

func (s *CHStore) Histogram(ctx context.Context, f Filter, bucketSec int, groupBy string) ([]LogSeries, error)

Histogram buckets log volume server-side via the same logs table. Whitelisted groupBy options ("service", "severity", or "" for total) map to indexed LowCardinality columns so the query stays partition-pruned + index-friendly even at billion log/day. Unknown groupBy collapses to a single _total series rather than failing — operator notices empty break-down and can pick a different field.

func (*CHStore) Indices added in v0.5.466 

func (s *CHStore) Indices(ctx context.Context) ([]IndexInfo, error)

Indices — CH stub (v0.5.466). Single physical table on the CH backend; per-shard / part-level surface is on the existing /admin/clickhouse page. Returning nil makes /admin/elastic render its "logs backend isn't Elasticsearch" empty state without touching CH-specific plumbing.

func (*CHStore) Ping 

func (s *CHStore) Ping(ctx context.Context) error

Ping delegates to the wrapped chstore — the same CH connection the rest of the app uses, so no separate liveness contract.

func (*CHStore) RawSearch added in v0.9.193 

func (s *CHStore) RawSearch(ctx context.Context, indices []string, body json.RawMessage, trackTotalCap int) (int64, error)

RawSearch — CH stub (v0.9.x, watcher import Faz-1). An imported watcher's search body is ES query DSL; CH cannot execute it. The typed error follows the EQLSearch precedent so the evaluator logs a clean "not supported on this backend" instead of firing wrong.

func (*CHStore) RawSearchPayload added in v0.9.202 

func (s *CHStore) RawSearchPayload(ctx context.Context, indices []string, body json.RawMessage, trackTotalCap int) (json.RawMessage, int64, error)

RawSearchPayload — CH stub (v0.9.x, watcher Faz-2). Same posture as RawSearch: an ES DSL body (and its aggregations payload) has no CH execution path.

func (*CHStore) RawSearchSamples added in v0.9.202 

func (s *CHStore) RawSearchSamples(ctx context.Context, indices []string, body json.RawMessage, n int) ([]string, error)

RawSearchSamples — CH stub (v0.9.x, watcher Faz-2). The evaluator treats sample errors soft, so on a CH backend fires simply carry no examples.

func (*CHStore) Search 

func (s *CHStore) Search(ctx context.Context, f Filter) (*Page, error)

func (*CHStore) TraceContextDiagnostics added in v0.8.348 

func (s *CHStore) TraceContextDiagnostics(ctx context.Context) (*TraceContextReport, error)

TraceContextDiagnostics implements TraceContextDiagnoser. Same error posture as the ES side: backend failures come back as a typed report (available:false + reason for the overall count; verdict kept + Reason set when only the per-service breakdown failed) — never a raw 5xx.

type Diagnoser added in v0.8.230 

type Diagnoser interface {
	Diagnostics() ESDiagnostics
}

Diagnoser is implemented by backends that track per-query failure diagnostics (currently only ES). The API layer type-asserts — the CH backend doesn't implement it and the endpoint reports an empty set.

type EQLEvent added in v0.5.468 

type EQLEvent struct {
	Timestamp int64  `json:"timestamp"` // unix ns
	Body      string `json:"body"`
	Service   string `json:"service"`
	Severity  string `json:"severity"`
}

EQLEvent — one event row inside a matched sequence. Carries only the columns Coremetry's /logs renderer needs.

type EQLQuery added in v0.5.468 

type EQLQuery struct {
	Query string    // EQL expression: `sequence … [event …] [event …]`
	From  time.Time // window start (zero = no lower bound)
	To    time.Time // window end (zero = now)
	Size  int       // max sequences to return; 0 = backend default (~10)
}

EQLQuery — parameters for an Event Query Language sequence detection (v0.5.468). ES-native; CH backend rejects.

type EQLSequence added in v0.5.468 

type EQLSequence struct {
	JoinKeys []string   `json:"joinKeys"`
	Events   []EQLEvent `json:"events"`
}

EQLSequence — one matched sequence. JoinKeys lifts the `by` columns the operator's EQL expression grouped on.

type ESConfig 

type ESConfig struct {
	Addresses []string
	Username  string
	Password  string
	// APIKey is the base64 "id:api_key" string — the `encoded` field
	// returned by POST /_security/api_key. Takes precedence over basic
	// auth: when it's set, NewES drops Username/Password entirely (see
	// resolveESAuth) so an api-key install sends ONLY an
	// Authorization: ApiKey header — never basic-auth creds alongside it.
	APIKey             string
	InsecureSkipVerify bool

	// Index pattern, e.g. "app-*" — supports glob/data-stream patterns.
	// Queries don't hit the raw pattern: es_indices.go resolves it to
	// the concrete dailies covering the queried window (v0.8.109).
	Index string

	// IndexTemplate (v0.8.231, operator-requested) — when set and a
	// query is pinned to a single service, the store resolves this to
	// the concrete per-service index instead of searching Index.
	// Placeholders: {service} = Filter.Service verbatim; {namespace} =
	// the service's namespace via ESStore.NamespaceResolver (span
	// resource attrs). Unresolved {namespace} substitutes "*" so the
	// query still covers the family. Example operator convention:
	// "app-{service}.{namespace}". Empty = disabled (pattern path).
	IndexTemplate string

	// Field paths inside each ES document. Override per-deployment via
	// config so any shipping pipeline (Filebeat, Logstash, OTel
	// Collector → ES exporter) can be queried without re-indexing.
	Fields ESFieldMap
}

ESConfig is the operator-supplied connection + field-mapping spec for an external Elasticsearch cluster. Field paths default to OTel-spec names — most ECS / OTel-shipped indices already use these.

type ESDiagnostics added in v0.8.230 

type ESDiagnostics struct {
	QueryErrors  int64          `json:"queryErrors"`
	RecentErrors []ESQueryError `json:"recentErrors"` // newest-first, ≤20
}

ESDiagnostics is the ES backend's self-observation snapshot: total failed queries since process start + the most recent failures.

type ESFieldMap 

type ESFieldMap struct {
	Timestamp  string `json:"timestamp,omitempty"`  // default "@timestamp"
	TraceID    string `json:"traceId,omitempty"`    // default "trace.id"
	SpanID     string `json:"spanId,omitempty"`     // default "span.id"
	Service    string `json:"service,omitempty"`    // default "service.name"
	Body       string `json:"body,omitempty"`       // default "message"
	SeverityNo string `json:"severityNo,omitempty"` // numeric, default "" (skip if absent)
	SeverityTx string `json:"severityTx,omitempty"` // text, default "log.level"
	// Env (v0.8.400 — env-separation Phase 4) — the deployment-
	// environment field the ?env= filter targets. Default "" =
	// SELF-DISCOVER via a cached field_caps over the candidate shapes
	// (es_env_field.go); set it only when the pipeline uses a path the
	// candidates don't cover.
	Env string `json:"env,omitempty"`
}

ESFieldMap carries the document field paths. JSON tags (v0.8.232) are the wire shape for the UI-managed settings blob + the Settings tab form — keep them stable.

type ESManager added in v0.8.232 

type ESManager struct {
	// contains filtered or unexported fields
}

ESManager owns the UI-managed logstore config: current effective settings, persistence, and applying changes to the shared Switchable. One instance per process, wired in main().

func NewESManager added in v0.8.232 

func NewESManager(sw *Switchable, chFallback Store, resolver func(ctx context.Context, service string) string, boot ESSettings) *ESManager

NewESManager seeds the manager with the env/YAML-derived effective settings so the Settings tab shows real config before any UI save, and empty-secret-preserves works against env credentials too.

func (*ESManager) CurrentSettings added in v0.8.232 

func (m *ESManager) CurrentSettings() ESSettings

CurrentSettings returns the full config including secrets — only for the PUT handler's empty-secret merge (tempo.CurrentSettings contract). Never echo the return value over the wire.

func (*ESManager) LoadPersisted added in v0.8.232 

func (m *ESManager) LoadPersisted(ctx context.Context, store ESSettingsStore) error

LoadPersisted hydrates from the saved blob: at boot (overlaying the env seed — UI wins), on the peer-pod config-reload signal, and on the 30s StartConfigRefresh tick. No blob = env config stays. An unchanged blob is a no-op — rebuilding the ES client (+ its eager ping) every 30s would churn connections for nothing.

func (*ESManager) SavePersisted added in v0.8.232 

func (m *ESManager) SavePersisted(ctx context.Context, store ESSettingsStore, cfg ESSettings) error

SavePersisted validates + applies cfg to the live Switchable, then persists it. Apply-first: a config that can't connect is rejected with the real error (the operator asked to SEE failures) and nothing is written — the stored blob always describes a config that worked at save time.

func (*ESManager) Snapshot added in v0.8.232 

func (m *ESManager) Snapshot() ESSettingsSnapshot

Snapshot returns the secret-free config view for the GET handler.

func (*ESManager) StartConfigRefresh added in v0.8.232 

func (m *ESManager) StartConfigRefresh(ctx context.Context, store ESSettingsStore, interval time.Duration)

StartConfigRefresh keeps multi-pod deployments converged on the shared persisted blob (worker/ingest pods have no PUT handler or reload signal — they pick up an api-pod save within `interval`). Mirrors tempo.StartConfigRefresh; interval ≤ 0 → 30s.

func (*ESManager) Test added in v0.8.232 

func (m *ESManager) Test(ctx context.Context, cfg ESSettings) error

Test builds a candidate store from cfg and reports the connection error without touching the live backend. This is the Settings tab's "Test" button — the exact place a bad address / credential / index pattern surfaces to the operator instead of a silent empty /logs.

type ESPingError added in v0.9.630 

type ESPingError struct {
	Status    int    // 0 = ağa hiç ulaşılamadı
	AuthMode  string // "api-key" | "basic" | "none"
	Addresses []string
	Body      string // ES'in yanıt gövdesi (varsa)
	Err       error  // taşınan ağ hatası (varsa)
}

ESPingError — boot ping'inin NEDEN başarısız olduğunu taşıyan tipli hata. Çağıran (main.go) mesajı buna göre seçiyor.

Status == 0 → hiç yanıt yok (dial/DNS/timeout) = GERÇEKTEN ulaşılamıyor. Status 401/403 → ulaşıldı, kimlik reddedildi. Diğer HTTP → ulaşıldı, başka bir yapılandırma sorunu.

func (*ESPingError) CredentialsAbsent added in v0.9.630 

func (e *ESPingError) CredentialsAbsent() bool

CredentialsAbsent — kimlik reddedildi VE hiç kimlik bilgisi yapılandırılmamıştı.

Bu, boot sırasında BEKLENEN durum: api-key Settings'te (system_settings) duruyorsa env'de yoktur ve buildLogStore ondan önce koşar. Operatöre "yapılandırma hatası" diye bağırmak yanlış; doğru cümle "kaydedilmiş ayarlar birazdan yüklenecek".

func (*ESPingError) Error added in v0.9.630 

func (e *ESPingError) Error() string

func (*ESPingError) Unauthorized added in v0.9.630 

func (e *ESPingError) Unauthorized() bool

Unauthorized — küme CEVAP VERDİ ama kimliği reddetti.

func (*ESPingError) Unreachable added in v0.9.630 

func (e *ESPingError) Unreachable() bool

Unreachable — ağ katmanında hiç yanıt alınamadı.

func (*ESPingError) Unwrap added in v0.9.630 

func (e *ESPingError) Unwrap() error

type ESQueryError added in v0.8.230 

type ESQueryError struct {
	At     int64  `json:"at"` // unix ms
	Op     string `json:"op"`
	Index  string `json:"index"`
	Query  string `json:"query"`
	Status int    `json:"status"` // HTTP status; 0 = transport error
	Error  string `json:"error"`
}

ESQueryError is one failed ES query, captured for the /admin/elastic "recent query errors" panel (v0.8.230, operator-requested: "when ES has a problem I want to see the queries it sent"). Query is the exact request body sent (truncated at 4 KiB) so the operator can replay it with curl against their cluster.

type ESSettings added in v0.8.232 

type ESSettings struct {
	// Backend selects the live read backend: "clickhouse" (default) or
	// "elasticsearch". The UI toggle maps here.
	Backend            string     `json:"backend"`
	Addresses          []string   `json:"addresses,omitempty"`
	Username           string     `json:"username,omitempty"`
	Password           string     `json:"password,omitempty"`
	APIKey             string     `json:"apiKey,omitempty"`
	InsecureSkipVerify bool       `json:"insecureSkipVerify,omitempty"`
	Index              string     `json:"index,omitempty"`
	IndexTemplate      string     `json:"indexTemplate,omitempty"`
	Fields             ESFieldMap `json:"fields"`
}

ESSettings is the full persisted shape, secrets included. Never serialize this to a client — Snapshot() is the wire view.

type ESSettingsSnapshot added in v0.8.232 

type ESSettingsSnapshot struct {
	Backend            string     `json:"backend"`
	Addresses          []string   `json:"addresses"`
	Username           string     `json:"username"`
	HasPassword        bool       `json:"hasPassword"`
	HasAPIKey          bool       `json:"hasApiKey"`
	InsecureSkipVerify bool       `json:"insecureSkipVerify"`
	Index              string     `json:"index"`
	IndexTemplate      string     `json:"indexTemplate"`
	Fields             ESFieldMap `json:"fields"`
	// Source is "env" until the first UI save is loaded, then "ui" —
	// tells the operator whether the form reflects env/YAML bootstrap
	// or a persisted override.
	Source string `json:"source"`
}

ESSettingsSnapshot is the secret-free GET view. HasPassword / HasAPIKey drive the "stored" indicator; empty input on PUT preserves the stored value (rotate by pasting a new one).

type ESSettingsStore added in v0.8.232 

type ESSettingsStore interface {
	GetLogstoreESSettingsRaw(ctx context.Context) ([]byte, error)
	PutLogstoreESSettingsRaw(ctx context.Context, raw []byte) error
}

ESSettingsStore is the narrow chstore surface the manager persists through (prevents an import cycle; matches tempo.settingsStore).

type ESStore 

type ESStore struct {

	// NamespaceResolver maps a service name to its namespace for the
	// {namespace} placeholder in cfg.IndexTemplate (v0.8.231). Wired by
	// main.go to a TTL-cached chstore.GetServiceNamespaces lookup — the
	// same span-resource-attr derivation the topology grouping uses.
	// Nil resolver or "" result → the placeholder substitutes "*".
	NamespaceResolver func(ctx context.Context, service string) string
	// contains filtered or unexported fields
}

ESStore implements the LogStore Search interface against an external Elasticsearch cluster. Read-only — Coremetry never writes to ES.

func NewES 

func NewES(cfg ESConfig) (*ESStore, error)

func (*ESStore) Backend 

func (s *ESStore) Backend() string

func (*ESStore) CountPatterns added in v0.5.241 

func (s *ESStore) CountPatterns(
	ctx context.Context,
	pats []PatternSpec,
	curStart, baseStart, now time.Time,
) ([]PatternStats, error)

CountPatterns batches all N pattern probes into a SINGLE ES `_msearch` HTTP round-trip. At billion-log scale on an external cluster, the per-call overhead (TLS, request parse, shard coordination) dominates the per-pattern cost; sending all 11 patterns as one request lets ES schedule the shard fan-out internally and drops wall time from ~N×network_rtt to ~1×network_rtt + max(pattern_cost).

Per pattern: tokens become a `query_string` OR clause against the body field (inverted-index lookup), two filter aggs split the cur vs base window counts, top_hits returns a sample + the latest timestamp, terms returns the dominant service. Regex itself is ignored — detector authors must ship tokens that have zero false-negatives vs the regex.

Empty Tokens = drop the probe (returns zero stats). Skips the regex-fallback path because regex queries on a billion- doc index are pathological.

func (*ESStore) Diagnostics added in v0.8.230 

func (s *ESStore) Diagnostics() ESDiagnostics

Diagnostics returns the failure counter + recent failed queries, newest-first. Implements logstore.Diagnoser.

func (*ESStore) EQLSearch added in v0.5.468 

func (s *ESStore) EQLSearch(ctx context.Context, q EQLQuery) ([]EQLSequence, error)

EQLSearch runs ES Event Query Language sequence detection. Powers operator queries like "login then error within 5m" or "deploy then anomaly within 10m" that the plain /logs search can't express. v0.5.468.

The ES side: POST <index>/_eql/search with the EQL string, an optional filter (time bounds), and size. Response is the EQL search result shape — we map it to []EQLSequence so the frontend sees one row per matched sequence with its ordered event list.

Field names — Coremetry's logs schema uses different paths across operators; for the body/service/severity extraction we pick the most common keys (ECS conformant) and fall back to flat aliases. timestamp is normalised to unix-ns.

func (*ESStore) ExecSQL added in v0.5.138 

func (s *ESStore) ExecSQL(ctx context.Context, query string, fetchSize int) (*SQLResult, error)

ExecSQL forwards a query to Elasticsearch's /_sql endpoint and returns the result in the same shape as the CH SQL playground. Read-only by definition — the Elastic SQL plugin doesn't expose DML; we still keep the safety net (first-token check + 30s fetch budget) so a typo can't dump a billion rows.

func (*ESStore) FieldStats added in v0.8.255 

func (s *ESStore) FieldStats(ctx context.Context, f Filter, field string, limit int) (*FieldStatsResult, error)

FieldStats — top-N values of one field in the filtered window. keyword-preferring: a text field needs its .keyword subfield for a terms agg (bare text → 400 fielddata error), while a pure-keyword field has no .keyword subfield and returns EMPTY buckets (unmapped — not an error). So: try <field>.keyword first; an empty result retries the bare field once (covers keyword / numeric / boolean / date mappings). Worst case 2 bounded round trips, and only when the .keyword shape came back empty.

func (*ESStore) FieldValues added in v0.5.464 

func (s *ESStore) FieldValues(ctx context.Context, field, prefix string, limit int, from, to time.Time) ([]string, error)

FieldValues uses ES _terms_enum (since 7.14) to prefix-match indexed term values for a single field. Sub-ms latency on keyword fields, even on billion-doc indices. v0.5.464.

Tries `field.keyword` first if the operator gave a plain name (most ES mappings expose a .keyword subfield alongside text types — _terms_enum only works on keyword/constant_keyword). Falls back to the bare field on failure. Case-insensitive matching is supported by _terms_enum natively (unlike query_string per v0.5.231). v0.9.291 — the window. This request used to go to the BARE index pattern with no index_filter and no time bound at all: a prefix walk over the term dictionary of every index in retention, warm/cold/frozen included. Its cost scaled with index count × field cardinality and NEVER with the operator's window — and the KQL box fires it on a 180ms keystroke debounce, so it was the most frequent unbounded read on the surface. This is the log sibling of the treatment MetricLabelValues got in v0.9.275; it never reached logs.

Two bounds, both needed. queryIndices narrows the index LIST the same way every other read does (and, since v0.9.283, that actually works on a data-stream cluster). index_filter tells ES the window inside each surviving index, so it can skip shards whose range cannot match — _terms_enum honours it natively.

func (*ESStore) Histogram 

func (s *ESStore) Histogram(ctx context.Context, f Filter, bucketSec int, groupBy string) ([]LogSeries, error)

func (*ESStore) Indices added in v0.5.466 

func (s *ESStore) Indices(ctx context.Context) ([]IndexInfo, error)

Indices surfaces per-index health + ILM lifecycle for the configured index pattern (v0.5.466). One round-trip to _cat/indices for size/health/doc-count, one to _ilm/explain for policy + phase. Both calls are scoped to the configured index pattern (e.g. "logs-*") so multi-tenant clusters don't dump every index back. ILM explain may legitimately error (cluster has no ILM module, or operator hasn't attached policies) — we degrade to "" phase rather than failing.

func (*ESStore) ListFields added in v0.5.136 

func (s *ESStore) ListFields(ctx context.Context) ([]string, error)

func (*ESStore) ListFieldsBounded added in v0.9.292 

func (s *ESStore) ListFieldsBounded(ctx context.Context) (ListFieldsResult, error)

ListFieldsBounded is ListFields with the truncation reported.

v0.9.292 — the mapping GET also stops going to the BARE index pattern. On a data-stream cluster that fetched every backing index's mapping in retention to answer a question about what fields exist today; queryIndices narrows it to the recent window the same way every other read narrows. Older indices can carry paths that no longer appear, but this is a discovery hint for a filter box, not an inventory — the same reasoning that bounds field-values (v0.9.291).

func (*ESStore) Ping 

func (s *ESStore) Ping(ctx context.Context) error

Ping checks ES cluster availability via the lightweight Info API (same endpoint NewES uses at startup). 2-second timeout enforced by the caller's context.

func (*ESStore) RawSearch added in v0.9.193 

func (s *ESStore) RawSearch(ctx context.Context, indicesIn []string, body json.RawMessage, trackTotalCap int) (int64, error)

RawSearch runs the guarded body and returns hits.total.value. Empty indices fall back to the store's configured pattern. Cost posture mirrors Search: request cache off (live data), missing indices are 0 hits rather than 404s, failures ride recordQueryError so /admin/elastic sees them.

Silent-zero guards (review F1 + F2 — proven against live ES 8.15):

  • _shards.total == 0 → ERROR, never 0/nil. ES security resolves index patterns against the credential's grants; an unauthorized or nonexistent index answers HTTP 200 with 0 hits under allow_no_indices/ignore_unavailable, which would leave the watcher permanently green with nothing in /admin/elastic.
  • injected 24h window + 0 hits → one field_caps probe on tsField; if the field is unmapped in the watch's indices the range can never match (a range on a nonexistent field is 0 hits, not an error) → ERROR naming the field. Runs at most once per watch interval and only while the count is zero, so no caching.

func (*ESStore) RawSearchPayload added in v0.9.202 

func (s *ESStore) RawSearchPayload(ctx context.Context, indicesIn []string, body json.RawMessage, trackTotalCap int) (json.RawMessage, int64, error)

RawSearchPayload runs the SAME guarded count search as RawSearch (one shared core — identical guards, identical silent-zero checks) but additionally returns the ctx.payload-shaped response subset the Faz-2 agg-path conditions walk: 

{"hits":{"total":{"value":N}},"aggregations":{…}}

Aggregations pass through VERBATIM (json.RawMessage — no float64 round-trip). Watches without agg conditions stay on RawSearch; this path exists only for compare/array_compare over ctx.payload.aggregations.….

func (*ESStore) RawSearchSamples added in v0.9.202 

func (s *ESStore) RawSearchSamples(ctx context.Context, indicesIn []string, body json.RawMessage, n int) ([]string, error)

RawSearchSamples fetches up to n (clamped ≤5) matching documents for the watch's query and returns one single-line ≤200-char summary per hit ("<ts> [<service>] <message>", newest first) — the notification examples embedded at FIRE time (Faz-2, ES Watcher parity: the watcher's own actions interpolate ctx.payload.hits). Failures are recorded for /admin/elastic but the CALLER treats them soft: a broken sample fetch must never block the fire itself.

func (*ESStore) Search 

func (s *ESStore) Search(ctx context.Context, f Filter) (*Page, error)

func (*ESStore) TraceContextDiagnostics added in v0.8.348 

func (s *ESStore) TraceContextDiagnostics(ctx context.Context) (*TraceContextReport, error)

TraceContextDiagnostics implements TraceContextDiagnoser (pivot Phase 1c self-discovery). Errors come back as a typed report — field_caps failure → {available:false, reason}; a coverage failure keeps the field verdict (the operationally critical half) and sets Reason. The error return is reserved for programming errors (marshal), so the handler never turns a slow/misconfigured cluster into a raw 5xx.

type FieldStatsResult added in v0.8.255 

type FieldStatsResult struct {
	Field  string            `json:"field"`
	Total  int64             `json:"total"`
	Values []FieldValueCount `json:"values"`
}

FieldStatsResult — top values of one field plus the total docs they were counted from (top buckets + remainder), so the UI can render percentage bars without a second query.

type FieldValueCount added in v0.8.255 

type FieldValueCount struct {
	Value string `json:"value"`
	Count int64  `json:"count"`
}

FieldValueCount pairs one field value with its doc count in the window. Part of the FieldStats result (fields-panel accordion).

type Filter 

type Filter struct {
	Service string
	Cluster string // v0.5.471 — k8s/openshift cluster name; empty = any
	// Env (v0.8.400 — env-separation Phase 4) — the global ?env=
	// deployment-environment filter. CH backend: bounded res-array
	// lookup over BOTH semconv spellings (deployment.environment.name
	// + legacy deployment.environment). ES backend: term filter on the
	// operator-configured fields.Env or, when that's empty, a
	// SELF-DISCOVERED field (cached field_caps over the candidate
	// shapes — es_env_field.go); when no field resolves, the filter is
	// NOT applied and Page.EnvUnapplied reports it honestly.
	Env         string
	Search      string
	From, To    time.Time
	SeverityMin uint8 // OTel severity number ≥ this; 0 = no filter
	TraceID     string
	// TraceIDs (v0.5.271) — multi-trace filter for the DQL
	// cross-signal join. When non-empty, backends should
	// match ANY trace_id in the list (OR semantics) in
	// addition to / instead of TraceID. Single-string TraceID
	// stays primary for the existing /logs page UX.
	TraceIDs []string
	SpanID   string
	// HasTrace (v0.8.406 — operator ask: "sadece trace'i olan loglar")
	// keeps only records with a non-empty trace correlation, so the
	// operator can filter /logs to pivotable rows. CH: trace_id != ”.
	// ES: exists over the four common trace-field spellings (+ the
	// configured override) — same field fan-out as the TraceID lookup.
	HasTrace bool
	Limit    int
	Offset   int
	// WantCursor (v0.9.286) — the caller DECLARES it intends to page.
	//
	// The ES backend opens a Point-in-Time per uncached search and keeps
	// it alive (2m) whenever it hands back a NextCursor, so the next
	// search_after lands on a stable snapshot. It decided that purely
	// from "was the page full?", which is true for essentially every
	// first page at scale — so every ordinary read-and-abandon search
	// pinned segment readers for two minutes, and the Drain puller did
	// it from a timer every 5 minutes. Leaked PITs are named in
	// elasticsearch.go as an amplifier of the v0.8.3 ES incident; the
	// error paths were fixed then, the never-paged path was not.
	//
	// Zero value = no cursor, no PIT retained. Set it ONLY if you will
	// actually use Page.NextCursor — a caller that discards the cursor
	// and sets this true is asking the cluster to hold segments for
	// nothing. Both backends honour it, so the contract does not differ
	// by backend.
	WantCursor bool
	// Cursor (v0.7.22, SAFE-CORE) — opaque keyset paging token.
	// When non-empty the backend decodes its OWN format and pages
	// AFTER the encoded position instead of using Offset. The API
	// layer treats this string as opaque (it just round-trips the
	// value from the previous Page.NextCursor). Empty = first page,
	// in which case Offset is still honoured for back-compat with
	// callers that page by offset. Each backend defines its own
	// cursor encoding (CH: base64("ch|"+timeNs+"|"+rowKey), where rowKey
	// is a cityHash64 row digest giving a strict total order; ES:
	// base64 of the hit's sort-values JSON array).
	Cursor string
	// Ascending (v0.7.83) — return oldest-first instead of the default
	// newest-first. Used by the /logs Context "after" window so a
	// LIMIT n read yields the n records immediately AFTER the pivot,
	// not the n newest in the forward window. Backends honour it only
	// on a non-cursor read (keyset paging is DESC-only).
	Ascending bool
	// SinceNs (v0.8.x) — FORWARD-TAIL mode for the live-tail SSE stream.
	// When > 0 the backend reads `time >= SinceNs` oldest-first, bounded
	// by Limit, and DELIBERATELY skips the total count() (the per-tick
	// cost the SSE tail exists to remove) AND the keyset Cursor/PIT
	// machinery (a forward tail at the live edge must NOT reuse the
	// DESC keyset cursor — elasticsearch.go's PIT-per-page would re-pin
	// segment readers every tick, the v0.8.3 incident shape). ES uses a
	// plain bounded range query (no PIT, track_total_hits:false, explicit
	// timeout, request_cache off). The caller (streamLogs handler) tracks
	// the newest timestamp it has emitted and passes it back as the next
	// SinceNs; it dedups same-ns boundary rows by LogRecord.ID. `>=` (not
	// `>`) so a log ingested late at the boundary ns is re-read, not
	// silently dropped (the v0.7.15 silent-drop failure).
	SinceNs int64
	// contains filtered or unexported fields
}

Filter is the union of every supported log-query parameter. Backends translate as much as they can; what they can't handle they ignore (with a log line).

type IndexInfo added in v0.5.466 

type IndexInfo struct {
	Name      string `json:"name"`
	DocCount  int64  `json:"docCount"`
	SizeBytes int64  `json:"sizeBytes"`
	Health    string `json:"health"`    // green | yellow | red | "" if unknown
	IlmPolicy string `json:"ilmPolicy"` // policy name attached, empty if none
	IlmPhase  string `json:"ilmPhase"`  // hot | warm | cold | frozen | delete | ""
}

IndexInfo describes one log-backend index (ES) or shard table (CH). Returned by Store.Indices for /admin/elastic to surface per-index health + ILM lifecycle state. v0.5.466.

type ListFieldsResult added in v0.9.292 

type ListFieldsResult struct {
	Fields []string `json:"fields"`
	// Total — how many distinct paths the mapping actually had, before
	// the cap. Equal to len(Fields) when nothing was dropped.
	Total int `json:"total"`
}

ListFieldsResult carries the truncation honestly. A silently clipped list would read as "these are the fields", which is the same wrong-because-unstated class as the ES honesty envelope (v0.9.288).

type LogPoint 

type LogPoint struct {
	T int64 `json:"t"` // unix ns, bucket start
	V int64 `json:"v"` // count
}

type LogRecord 

type LogRecord struct {
	ID                 int64             `json:"id"`
	Timestamp          int64             `json:"timestamp"` // unix ns
	Severity           uint8             `json:"severity"`  // OTel SeverityNumber 0..24
	SeverityText       string            `json:"severityText"`
	Body               string            `json:"body"`
	ServiceName        string            `json:"serviceName"`
	TraceID            string            `json:"traceId"`
	SpanID             string            `json:"spanId"`
	Attributes         map[string]string `json:"attributes"`
	ResourceAttributes map[string]string `json:"resourceAttributes"`
}

LogRecord is the in-memory shape returned by every backend. It mirrors chstore.Log but without the ClickHouse-specific tags so the JSON surface stays stable across backends.

type LogSeries 

type LogSeries struct {
	Name   string     `json:"name"`
	Points []LogPoint `json:"points"`
}

LogSeries is one bucketed timeseries returned by Histogram. Name is the group_value (or "_total" when grouping is off); each Point.T is the bucket-start (unix ns) and V is the count.

type Page 

type Page struct {
	Total int          `json:"total"`
	Logs  []*LogRecord `json:"logs"`
	// NextCursor (v0.7.22, SAFE-CORE) — opaque keyset token the
	// caller passes back as Filter.Cursor to fetch the next page.
	// Empty when this is the last page (fewer than Limit rows
	// returned). Opaque to the API layer; format is backend-owned.
	NextCursor string `json:"nextCursor,omitempty"`
	// EnvUnapplied (v0.8.400 — env-separation Phase 4) — the backend's
	// HONEST signal that Filter.Env was requested but could not be
	// applied (ES: no environment field resolvable in the mapping, and
	// none configured). The results are env-UNFILTERED; the /logs page
	// renders a warning chip instead of silently implying a narrowed
	// view (the v0.8.398 honesty pattern). Never set by the CH backend
	// (the res-array conjunct always applies).
	EnvUnapplied bool `json:"envUnapplied,omitempty"`

	// Partial — ES hit its soft timeout, or shards failed. Counts and
	// rows below are a subset of the true answer.
	Partial bool `json:"partial,omitempty"`
	// ShardsFailed — how many shards did not answer. Non-zero means
	// every number here is missing that shard's contribution.
	ShardsFailed int `json:"shardsFailed,omitempty"`
	// TotalIsLowerBound — Total is "at least this", not "exactly this".
	// ES is asked for track_total_hits: 10000 (counting every matching
	// doc is precisely what you avoid at billion-doc scale), so it
	// answers relation "gte" once the count reaches the cap. Without
	// this flag "10,000" read as an exact figure — while the SAME label
	// on the CH backend really is an exact count(). One string, two
	// backends, two meanings, neither stated.
	TotalIsLowerBound bool `json:"totalIsLowerBound,omitempty"`
}

Page is the result of a Search — total covers the full match count for paging UIs even when len(Logs) < Limit.

func LogsForSpan added in v0.8.331 

func LogsForSpan(ctx context.Context, st Store, traceID, spanID string, from, to time.Time, limit int) (*Page, error)

LogsForSpan narrows LogsForTrace to one span (trace_id AND span_id — both backends translate Filter.SpanID). Same timeout + window semantics.

func LogsForTrace added in v0.8.331 

func LogsForTrace(ctx context.Context, st Store, traceID string, from, to time.Time, limit int) (*Page, error)

LogsForTrace returns the logs carrying one trace id, under the pivot timeout semantics (ErrBackendSlow on a slow/unreachable backend).

Window semantics (the correlate.go finding, kept caller-visible): ES ignores From/To when TraceID is set (a trace link can be older than any default slice); CH AND-applies them. Pass ZERO from/to to make trace_id the sole filter on both backends; pass a bounded window on CH installs to help the partition scan (the logs table has no trace_id skip index yet — audit §1).

func SearchWithTimeout added in v0.8.331 

func SearchWithTimeout(ctx context.Context, st Store, f Filter, timeout time.Duration) (*Page, error)

SearchWithTimeout runs one Search under a per-request deadline (timeout ≤ 0 → PivotTimeout) and maps "the backend is slow or unreachable" — deadline exceeded, transport timeout, dial/connection failures — to ErrBackendSlow (wrapped, so the original cause stays visible in logs). Genuine query errors (bad field, ES 400, …) pass through unchanged: those are bugs to surface, not conditions to degrade on.

type PatternServiceHit added in v0.5.287 

type PatternServiceHit struct {
	Service string `json:"service"`
	Count   uint64 `json:"count"`
}

PatternServiceHit pairs a service name with how many times it produced the pattern in the current detection window.

type PatternSpec added in v0.5.241 

type PatternSpec struct {
	Regex  string
	Tokens []string
}

PatternSpec is the cross-backend description of a "find log lines that look like this" probe. Both backends consume it:

  • ClickHouse evaluates Regex against the body field with the Tokens list driving the tokenbf_v1 prefilter so granules with none of the tokens are pruned before the regex pass.
  • Elasticsearch ignores Regex (regex queries are slow on large indices) and uses Tokens directly as a query_string OR clause against the body field — inverted-index lookup stays sub-second at billion-log scale.

Tokens are lowercase substrings the body must contain when the regex matches. The detector author picks them so the OR clause has zero false-negatives vs the regex.

type PatternStats added in v0.5.241 

type PatternStats struct {
	Cur        uint64
	Base       uint64
	Service    string
	Sample     string
	LastSeenNs int64
	// TopServices — v0.5.287. Per-service breakdown of the
	// current-window hits, sorted by count desc. Up to 5 entries.
	// Populated by both backends when Cur > 0 so the
	// /logs LogPatternStrip can show "fires on these N services"
	// rosette without a follow-up call. Empty when only one
	// service or when the backend doesn't track it.
	TopServices []PatternServiceHit
}

PatternStats is the per-pattern signal a detector consumes: counts in the "current" + "baseline" windows, a representative service + sample, and the most recent occurrence time.

type SQLResult added in v0.5.138 

type SQLResult struct {
	Columns []string `json:"columns"`
	Rows    [][]any  `json:"rows"`
	TookMs  int64    `json:"tookMs"`
}

SQLResult is the wire-shape returned by ExecSQL. Mirrors the AdminSql /api/admin/sql/query shape (columns/rows/tookMs) so the frontend table renderer is single-codepath.

type Store 

type Store interface {
	Search(ctx context.Context, f Filter) (*Page, error)

	// CountPatterns returns per-pattern current-window +
	// baseline-window counts, services, and samples. Plural form
	// so backends can batch — at billion-log scale on ES, an
	// _msearch with all N pattern bodies in a single HTTP
	// round-trip beats N parallel _search calls. CH backend
	// iterates sequentially (queries are cheap behind the
	// tokenbf_v1 skip index). Result slice index matches the
	// input slice index; empty PatternStats indicates "no match
	// in current window" (detector ignores these).
	CountPatterns(ctx context.Context, pats []PatternSpec, curStart, baseStart, now time.Time) ([]PatternStats, error)

	// Histogram returns one bucketed timeseries per group_value for
	// the requested filter. Powers the Logs source in /explore — the
	// caller sets the bucket size (e.g. 30s, 5m) and an optional
	// `groupBy` field name (one of "service", "severity", or any
	// attribute path the backend knows). Empty groupBy → a single
	// "_total" series.
	Histogram(ctx context.Context, f Filter, bucketSec int, groupBy string) ([]LogSeries, error)

	// EQLSearch runs an Elastic Event Query Language sequence
	// detection against the log index — "event A then event B
	// within N minutes" expressions Coremetry can't otherwise
	// express via plain search. Returns the matched sequences,
	// each with its ordered event list. v0.5.468.
	//
	// CH backend returns an "unsupported" error — EQL is ES-
	// specific and the CH frontend hides the panel when the
	// backend reports non-ES.
	EQLSearch(ctx context.Context, q EQLQuery) ([]EQLSequence, error)

	// RawSearch executes an operator-supplied ES search body
	// VERBATIM against the given indices and returns the total hit
	// count — the executable core of the imported ES Watcher path
	// (v0.9.x, Faz-1). The ES backend injects cost guards on top of
	// the untouched query (size:0, per-body soft timeout, capped
	// track_total_hits ≥ trackTotalCap, 24h range fallback when the
	// body has no range clause); callers pass trackTotalCap =
	// threshold*2 so a compare above the ES 10k count saturation is
	// counted correctly. Empty indices fall back to the configured
	// index pattern. CH backend returns an "unsupported" error —
	// an ES DSL body has no CH execution path (EQL precedent).
	RawSearch(ctx context.Context, indices []string, body json.RawMessage, trackTotalCap int) (int64, error)

	// RawSearchPayload — same guarded search as RawSearch (one shared
	// core), returning additionally the ctx.payload-shaped response
	// subset ({"hits":{"total":{"value":N}},"aggregations":{…}},
	// aggregations verbatim) for the Faz-2 agg-path watcher
	// conditions (compare / array_compare on
	// ctx.payload.aggregations.…). Watches without agg conditions
	// stay on RawSearch. CH backend: "unsupported" error.
	RawSearchPayload(ctx context.Context, indices []string, body json.RawMessage, trackTotalCap int) (json.RawMessage, int64, error)

	// RawSearchSamples fetches up to n (clamped ≤5) documents
	// matching the watch's query and returns one single-line
	// ≤200-char summary per hit — the examples embedded into the
	// Problem description when a watcher FIRES (Faz-2, ES Watcher
	// action-payload parity). Guarded like RawSearch (timeout, 24h
	// fallback window) plus restricted _source and newest-first
	// sort; errors are SOFT for the caller — a broken sample fetch
	// must never block the fire. CH backend: "unsupported" error.
	RawSearchSamples(ctx context.Context, indices []string, body json.RawMessage, n int) ([]string, error)

	// Indices lists the log indices the backend currently has,
	// with health + size + ILM lifecycle info per index. Powers
	// /admin/elastic so the operator sees at a glance which
	// indices are hot/warm/cold/frozen, what their ILM policy
	// says, and if any have gone red. CH backend returns nil
	// (no per-index concept). v0.5.466.
	Indices(ctx context.Context) ([]IndexInfo, error)

	// FieldValues returns top values of a single indexed field
	// matching a typed prefix. Backs the /logs search box's
	// field-aware autocomplete (v0.5.464): operator types
	// "service.name:" → dropdown shows suggested service names
	// from the data itself. Empty prefix returns the most common
	// values. ES backend uses _terms_enum for sub-ms prefix
	// lookups; CH backend returns [] for now (CH-native
	// DISTINCT-with-LIKE is straightforward but the KQL search
	// box is Kibana-flavoured already, less urgent on CH).
	// v0.9.291 — from/to bound the lookup. Without them the ES
	// implementation walked the term dictionary of every index in
	// retention on every keystroke; the window is what makes the cost
	// track the operator's question instead of the cluster's age.
	// Zero bounds clamp to the last 10 minutes (clampWindow), same as
	// every other read in this package.
	FieldValues(ctx context.Context, field, prefix string, limit int, from, to time.Time) ([]string, error)

	// FieldStats returns the top-N values of one field within the
	// filtered window, with per-value counts and the total doc count
	// they were drawn from (Discover fields-panel accordion,
	// v0.8.255). Called lazily — only when the operator expands a
	// field — and cached 60s at the API layer, so it must never be
	// polled. ES: single bounded terms agg (keyword-preferring, one
	// bare-field retry when unmapped); CH: GROUP BY over the resolved
	// column/attribute with grouping caps.
	FieldStats(ctx context.Context, f Filter, field string, limit int) (*FieldStatsResult, error)

	// Backend returns a short identifier shown in /api/health so an operator
	// can tell at a glance which log source is wired in.
	Backend() string

	// Ping reports liveness of the underlying backend. Used by /api/status
	// to surface "logs backend is down" before the user runs into an
	// empty-result query.
	Ping(ctx context.Context) error
}

Store is the read interface every backend implements.

func Unwrap added in v0.8.232 

func Unwrap(s Store) Store

Unwrap returns the innermost live Store — the Switchable's current target when s is a Switchable, else s itself. Every call site that type-asserts an optional backend capability (fielder / esSQLRunner / Diagnoser) must go through this so a UI-driven backend swap is respected.

type Switchable added in v0.8.232 

type Switchable struct {
	// contains filtered or unexported fields
}

Switchable wraps the live Store behind a RWMutex so the admin Settings → Elasticsearch tab can swap the logs read backend at runtime without a pod restart (v0.8.232). Every consumer (API handlers, alert evaluator, anomaly recorder, Drain templater) holds the same *Switchable; ESManager.apply calls Swap and the new backend is live for all of them on their next call.

Optional per-backend capabilities (ListFields, ExecSQL, Diagnoser) are NOT forwarded here on purpose — forwarding would make the type assertions at the call sites succeed even when the inner backend lacks the capability, turning clean "not available on this backend" 400s into runtime errors. Call sites assert via Unwrap instead.

func NewSwitchable added in v0.8.232 

func NewSwitchable(inner Store) *Switchable

func (*Switchable) Backend added in v0.8.232 

func (s *Switchable) Backend() string

func (*Switchable) CountPatterns added in v0.8.232 

func (s *Switchable) CountPatterns(ctx context.Context, pats []PatternSpec, curStart, baseStart, now time.Time) ([]PatternStats, error)

func (*Switchable) Current added in v0.8.232 

func (s *Switchable) Current() Store

Current returns the live inner store. Callers must not cache it across requests — that would defeat the swap.

func (*Switchable) EQLSearch added in v0.8.232 

func (s *Switchable) EQLSearch(ctx context.Context, q EQLQuery) ([]EQLSequence, error)

func (*Switchable) FieldStats added in v0.8.255 

func (s *Switchable) FieldStats(ctx context.Context, f Filter, field string, limit int) (*FieldStatsResult, error)

func (*Switchable) FieldValues added in v0.8.232 

func (s *Switchable) FieldValues(ctx context.Context, field, prefix string, limit int, from, to time.Time) ([]string, error)

func (*Switchable) Histogram added in v0.8.232 

func (s *Switchable) Histogram(ctx context.Context, f Filter, bucketSec int, groupBy string) ([]LogSeries, error)

func (*Switchable) Indices added in v0.8.232 

func (s *Switchable) Indices(ctx context.Context) ([]IndexInfo, error)

func (*Switchable) Ping added in v0.8.232 

func (s *Switchable) Ping(ctx context.Context) error

func (*Switchable) RawSearch added in v0.9.193 

func (s *Switchable) RawSearch(ctx context.Context, indices []string, body json.RawMessage, trackTotalCap int) (int64, error)

func (*Switchable) RawSearchPayload added in v0.9.202 

func (s *Switchable) RawSearchPayload(ctx context.Context, indices []string, body json.RawMessage, trackTotalCap int) (json.RawMessage, int64, error)

func (*Switchable) RawSearchSamples added in v0.9.202 

func (s *Switchable) RawSearchSamples(ctx context.Context, indices []string, body json.RawMessage, n int) ([]string, error)

func (*Switchable) Search added in v0.8.232 

func (s *Switchable) Search(ctx context.Context, f Filter) (*Page, error)

func (*Switchable) Swap added in v0.8.232 

func (s *Switchable) Swap(n Store)

Swap atomically replaces the inner store. In-flight queries finish on the store they started with; nil is ignored (a failed rebuild must never leave consumers with a nil backend).

type TraceContextDiagnoser added in v0.8.348 

type TraceContextDiagnoser interface {
	TraceContextDiagnostics(ctx context.Context) (*TraceContextReport, error)
}

TraceContextDiagnoser is the optional per-backend capability behind the report. Both shipped backends implement it (ES via field_caps + one size:0 aggregation, CH via two bounded countIf queries); the API layer still type-asserts through Unwrap — the Switchable deliberately does not forward optional capabilities (see switchable.go).

type TraceContextField added in v0.8.348 

type TraceContextField struct {
	Name         string   `json:"name"`
	Types        []string `json:"types"` // mapping types found (sorted); empty = absent
	Searchable   bool     `json:"searchable"`
	Aggregatable bool     `json:"aggregatable"`
	Configured   bool     `json:"configured"` // the operator-configured TraceID field
}

TraceContextField is one candidate trace-id field shape as the backend maps it. Types empty = the field is absent from the mapping.

type TraceContextReport added in v0.8.348 

type TraceContextReport struct {
	Available      bool   `json:"available"`
	Reason         string `json:"reason,omitempty"`
	EffectiveField string `json:"effectiveField"`
	// EffectiveType: "keyword" | "text" | … | "absent" on ES; "String" on CH.
	EffectiveType string              `json:"effectiveType"`
	PivotReady    bool                `json:"pivotReady"`
	Fields        []TraceContextField `json:"fields"`

	// Coverage over the last WindowHours.
	WindowHours int                           `json:"windowHours"`
	Total       int64                         `json:"total"`
	WithTrace   int64                         `json:"withTrace"`
	Services    []TraceContextServiceCoverage `json:"services"`
}

TraceContextReport is the cross-backend self-discovery result. Available=false + Reason replaces raw errors — the admin surface renders the reason instead of a 5xx. Reason may also be set with Available=true when the field verdict succeeded but the coverage aggregation failed.

type TraceContextServiceCoverage added in v0.8.348 

type TraceContextServiceCoverage struct {
	Service   string `json:"service"`
	Total     int64  `json:"total"`
	WithTrace int64  `json:"withTrace"`
}

TraceContextServiceCoverage is one service's share of logs carrying trace context in the report window.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.