qlcommon

Documentation

Overview

Package qlcommon hosts small, lowering-layer helpers shared across the PromQL / LogQL / TraceQL heads. Functions here translate upstream-QL semantics into shapes the shared chplan IR + the chsql emitter expect, so each language's lowering can stay focused on its own AST.

Index

• Constants
• func EmptyCapturesReplacement(repl string) string
• func OTelDottedFallbackChain(m chplan.Expr, candidates []string) chplan.Expr
• func ReplacementSegments(repl, regex string) ([]chplan.LabelReplaceSegment, string, error)
• type CHReplacement
  • func ReplacementToCH(repl, regex string) (CHReplacement, error)

Constants

const InstantLookback = 5 * time.Minute

InstantLookback is the default Prometheus staleness window: how far back an instant-vector selector looks for the latest sample per series. Prom defaults to 5 minutes; cerberus matches the upstream constant rather than reading a per-deployment override, so the LWR predicate behaves predictably across environments.

One owner, three consumers: PromQL's own instant-selector lowering (internal/promql/modifiers.go), PromQL's subquery staleness lookback (internal/promql/subquery.go, which aliases this value rather than redeclaring it), and Loki's instant-query handler (internal/api/loki/handler.go), which windows its evaluation to `[ts - InstantLookback, ts]` per the same upstream contract. See #1470.

Functions

func EmptyCapturesReplacement

func EmptyCapturesReplacement(repl string) string

EmptyCapturesReplacement returns the result of substituting Go's regex `ExpandString` template `repl` against an EMPTY source string that matched the regex via a match where every capture group binds to "". This matches the semantics of `label_replace(m, dst, repl, src, regex)` when `src` is absent from the input series labels (Prom reads missing labels as the empty string) AND the regex matches that empty string — e.g. `(.*)`, `.*`, `^()$` all match `""` with every group capturing `""`.

Why we need a separate path:

CH ≤ 24.8's `replaceRegexpOne('', '^(.*)$', 'value-\1')` returns
`""` (the empty input is passed through verbatim, regardless of
the replacement template), instead of the spec-correct `"value-"`.
The outer `mapFilter((k, v) -> v != '', …)` then drops the dst
label entirely, diverging from reference Prom which emits
`dst="value-"`. CH ≥ 25.8 honours the replacement on empty inputs,
and the cerberus deployment lane now targets CH 25.8 — but the
short-circuit stays load-bearing: it is forward-safe (collapses to
the same spec-correct value on 25.8) and keeps the emit identical
while the compatibility reference backend moves in lock-step. We
patch the divergence at SQL build time by pre-computing the
empty-captures result and using it as a short-circuit when the
source value is empty at row time.

Substitution rules (the same reference splitter `ReplacementToCH` uses, but every reference resolves to the empty string instead of to CH's `\N` form):

• `$$` → literal `$`
• `$N` / `${N}` / `$name` / `${name}` → empty string. Every reference ExpandString recognises contributes "" here: a group that took part in the empty match expanded to "", and a group that bound to nothing (index past the group count, unknown name) expands to "" as well. No regex is needed to tell the two apart because the answer is the same either way.
• A `$` that starts no reference (end of string, `$-`, `${unclosed`) → preserved verbatim, as ExpandString does.

func OTelDottedFallbackChain

func OTelDottedFallbackChain(m chplan.Expr, candidates []string) chplan.Expr

OTelDottedFallbackChain builds the right-associative if-chain that resolves a Prom label whose OTel form has multiple spellings (see [format.PromLabelToOTelCandidates]) against the label map expression m. The chain reads:

if(mapContains(m, k0), m[k0],
  if(mapContains(m, k1), m[k1],
    ... m[kN-1]))

so the leftmost candidate (the underscored input) wins when present, and the terminal branch is a bare MapAccess against the last candidate — when no candidate's key exists the map's value-type default (empty string) matches "absent label" semantics.

CH's `m['missing']` returns the value-type default rather than NULL, so `coalesce` would short-circuit on the first lookup even when the row's actual key is a later candidate; `mapContains` cleanly distinguishes "present with empty value" from "absent".

candidates must be non-empty. The PromQL and LogQL heads share this so a Grafana dashboard mixing Prom + Loki panels resolves labels symmetrically.

func ReplacementSegments

func ReplacementSegments(repl, regex string) ([]chplan.LabelReplaceSegment, string, error)

ReplacementSegments splits a Go replacement template into the alternating literal runs and capture-group references that make it up, resolving each reference against regex's capture-group metadata.

This is the single walk of the template that both output forms are derived from: [renderCHTemplate] folds it back into a `replaceRegexpOne` substitution string, and the emitter renders it as a `concat` when a referenced group sits above CH's ceiling. Deriving both from one decomposition is what keeps them from drifting apart.

References that bind to nothing — an index past the regex's group count, a name no group carries — contribute nothing at all, exactly as Go's `ExpandString` substitutes the empty string for them.

Types

type CHReplacement

type CHReplacement struct {
	// Template is the `replaceRegexpOne` substitution string.
	Template string
	// ProbedRegex, when non-empty, is the regex the emitter must feed
	// `extractGroups` in place of the one the query named: the same
	// pattern with synthetic capture groups added so that a carrier whose
	// own capture cannot report whether it took part in the match has one
	// that can. It matches the same strings as the original — the added
	// groups only observe — but it NUMBERS groups differently, so every
	// index in Segments is already in its numbering and the two must be
	// read together. See [planCaptureProbes].
	ProbedRegex string
	// Segments is the literal-run / capture-group decomposition the
	// emitter renders as a `concat` over `extractGroups`. A segment's
	// Literal is the DECODED text — `$$` has already collapsed to `$`,
	// and a backslash is a real backslash — so each output form applies
	// whatever escaping its own syntax needs.
	Segments []chplan.LabelReplaceSegment
}

CHReplacement is the ClickHouse-side form of a Go replacement template. Exactly one of its fields describes the substituted value: Template when every reference resolves to a single capture group that fits CH's `\0`–`\9` substitution syntax, Segments when one does not — because it names a group above the ceiling, or because it names a group NAME several groups share and so denotes a selection rather than an index.

func ReplacementToCH

func ReplacementToCH(repl, regex string) (CHReplacement, error)

ReplacementToCH translates a Go-`regexp` replacement template (`$1` / `${1}` / `$$` syntax — used by both PromQL's `label_replace` and LogQL's `label_replace` per their reference implementations) into the equivalent ClickHouse `replaceRegexpOne` replacement (`\1` / `\\` syntax).

PromQL runs the replacement through Go's `regexp.Regexp.ExpandString`; LogQL's `label_replace` does the same. Both treat:

• `$$` → literal `$`
• `$N` / `${N}` → numbered capture group N
• `$name` / `${name}` → named capture group

ClickHouse's `replaceRegexpOne` uses backslash escapes instead:

• `\\` → literal backslash
• `\0` … `\9` → numbered capture group (`\0` = whole match)

Without translation, a replacement like `"svc-$1"` is passed to CH verbatim and emitted as the literal string `svc-$1` — the capture group is never substituted.

Translation rules implemented here — the reference for every one of them is Go's `Regexp.ExpandString`, which is the engine both QLs run their replacement through, so this function reproduces its reference splitter ([extractRef]) rather than approximating it:

• Every literal `\` in the input becomes `\\`, so a literal backslash in the QL template survives as a literal backslash in CH and is not re-read as the start of one of the `\N` backrefs spliced in below.
• `$$` → `$` (literal dollar).
• `$N` / `${N}` for ANY index N — including multi-digit `$10` — resolves to capture group N, provided the regex has at least N capture groups. Groups up to 9 render as `\N`; a higher one takes the Segments form described below.
• `$name` / `${name}` → the index of the capture group that carries that name.
• A reference that binds to nothing — an index past the regex's group count, or a name no group carries — expands to the empty string, exactly as ExpandString does.
• A `$` that starts no reference at all (end of string, `$-`, `${unclosed`) is emitted verbatim, again as ExpandString does.

Two shapes are expressible, just not as a `replaceRegexpOne` template: a reference to a capture group above CH's `\9` substitution ceiling, and a reference to a name several capture groups share. Both carry CHReplacement.Segments instead, which the emitter renders as a `concat` of literal runs and `extractGroups` subscripts — the second selecting among the like-named groups' subscripts with `arrayFirst`. The single shape with no faithful translation at all is that second one when any of the like-named groups can match the EMPTY STRING — see [captureGroups.resolve].

regex is the regex string the replacement is applied against; it is compiled to resolve capture-group names and to count groups so out-of-range backrefs can be rewritten to the empty string. CH validates `replaceRegexpOne`'s substitution string against the regex's capture-group count at SQL-parse time and rejects backrefs that exceed it (Code 36, BAD_ARGUMENTS) — even on rows where match() short-circuits the if-branch that owns the replaceRegexpOne call. Dropping the backref preserves the upstream empty-string semantics on the (unreachable) hot path and unblocks the SQL parser on the (very-much-reachable) cold path where the regex doesn't match anything.