metricsql

package module
v0.81.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 5, 2024 License: Apache-2.0 Imports: 12 Imported by: 32

README

GoDoc Go Report

metricsql

Package metricsql implements MetricsQL and PromQL parser in Go.

Usage
    expr, err := metricsql.Parse(`sum(rate(foo{bar="baz"}[5m])) by (job)`)
    if err != nil {
        // parse error
    }
    // Now expr contains parsed MetricsQL as `*Expr` structs.
    // See Parse examples for more details.

See docs for more details.

Documentation

Overview

Package metricsql implements MetricsQL parser.

This parser can parse PromQL. Additionally it can parse all the MetricsQL extensions. See https://docs.victoriametrics.com/metricsql/ for details about MetricsQL.

Usage: 

expr, err := metricsql.Parse(`sum(rate(foo{bar="baz"}[5m])) by (job)`)
if err != nil {
    // parse error
}
// Now expr contains parsed MetricsQL as `*Expr` structs.
// See Parse examples for more details.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func CompileRegexp 

func CompileRegexp(re string) (*regexp.Regexp, error)

CompileRegexp returns compile regexp re.

func CompileRegexpAnchored 

func CompileRegexpAnchored(re string) (*regexp.Regexp, error)

CompileRegexpAnchored returns compiled regexp `^re$`.

func DurationValue 

func DurationValue(s string, step int64) (int64, error)

DurationValue returns the duration in milliseconds for the given s and the given step.

Duration in s may be combined, i.e. 2h5m, -2h5m or 2h-5m.

The returned duration value can be negative.

func ExpandWithExprs 

func ExpandWithExprs(q string) (string, error)

ExpandWithExprs expands WITH expressions inside q and returns the resulting PromQL without WITH expressions.

Example 
package main

import (
	"fmt"
	"log"

	"github.com/VictoriaMetrics/metricsql"
)

func main() {
	// mql can contain arbitrary MetricsQL extensions - see https://docs.victoriametrics.com/metricsql/
	mql := `WITH (
		commonFilters = {job="$job", instance="$instance"},
		f(a, b) = 100*(a/b),
	)
	f(disk_free_bytes{commonFilters}, disk_total_bytes{commonFilters})`

	// Convert mql to PromQL
	pql, err := metricsql.ExpandWithExprs(mql)
	if err != nil {
		log.Fatalf("cannot expand with expressions: %s", err)
	}

	fmt.Printf("%s\n", pql)

}

Output:

100 * (disk_free_bytes{job="$job",instance="$instance"} / disk_total_bytes{job="$job",instance="$instance"})

func GetRollupArgIdx added in v0.35.0 

func GetRollupArgIdx(fe *FuncExpr) int

GetRollupArgIdx returns the argument index for the given fe, which accepts the rollup argument.

-1 is returned if fe isn't a rollup function.

func IsAggrFunc added in v0.65.0 

func IsAggrFunc(s string) bool

IsAggrFunc returns whether funcName is a known aggregate function.

func IsBinaryOpCmp 

func IsBinaryOpCmp(op string) bool

IsBinaryOpCmp returns true if op is comparison operator such as '==', '!=', etc.

func IsLikelyInvalid added in v0.76.0 

func IsLikelyInvalid(e Expr) bool

IsLikelyInvalid returns true if e contains tricky implicit conversion, which is invalid most of the time.

Examples of invalid expressions: 

rate(sum(foo))
rate(abs(foo))
rate(foo + bar)
rate(foo > 10)

These expressions are implicitly converted into another expressions, which returns unexpected results most of the time: 

rate(default_rollup(sum(foo))[1i:1i])
rate(default_rollup(abs(foo))[1i:1i])
rate(default_rollup(foo + bar)[1i:1i])
rate(default_rollup(foo > 10)[1i:1i])

See https://docs.victoriametrics.com/metricsql/#implicit-query-conversions

Note that rate(foo) is valid expression, since it returns the expected results most of the time, e.g. rate(foo[1i]).

func IsRollupFunc 

func IsRollupFunc(funcName string) bool

IsRollupFunc returns whether funcName is known rollup function.

func IsSupportedFunction added in v0.65.0 

func IsSupportedFunction(funcName string) bool

IsSupportedFunction returns true if funcName contains supported MetricsQL function

func IsTransformFunc 

func IsTransformFunc(funcName string) bool

IsTransformFunc returns whether funcName is known transform function.

func PositiveDurationValue 

func PositiveDurationValue(s string, step int64) (int64, error)

PositiveDurationValue returns positive duration in milliseconds for the given s and the given step.

Duration in s may be combined, i.e. 2h5m or 2h-5m.

Error is returned if the duration in s is negative.

func Prettify added in v0.60.0 

func Prettify(q string) (string, error)

Prettify returns prettified representation of MetricsQL query q.

func VisitAll added in v0.4.0 

func VisitAll(e Expr, f func(expr Expr))

VisitAll recursively calls f for all the Expr children in e.

It visits leaf children at first and then visits parent nodes. It is safe modifying expr in f.

Types

type AggrFuncExpr 

type AggrFuncExpr struct {
	// Name is the function name.
	Name string

	// Args is the function args.
	Args []Expr

	// Modifier is optional modifier such as `by (...)` or `without (...)`.
	Modifier ModifierExpr

	// Optional limit for the number of output time series.
	// This is MetricsQL extension.
	//
	// Example: `sum(...) by (...) limit 10` would return maximum 10 time series.
	Limit int
}

AggrFuncExpr represents aggregate function such as `sum(...) by (...)`

func (*AggrFuncExpr) AppendString 

func (ae *AggrFuncExpr) AppendString(dst []byte) []byte

AppendString appends string representation of ae to dst and returns the result.

type BinaryOpExpr 

type BinaryOpExpr struct {
	// Op is the operation itself, i.e. `+`, `-`, `*`, etc.
	Op string

	// Bool indicates whether `bool` modifier is present.
	// For example, `foo >bool bar`.
	Bool bool

	// GroupModifier contains modifier such as "on" or "ignoring".
	GroupModifier ModifierExpr

	// JoinModifier contains modifier such as "group_left" or "group_right".
	JoinModifier ModifierExpr

	// JoinModifierPrefix is an optional prefix to add to labels specified inside group_left() or group_right() lists.
	//
	// The syntax is `group_left(foo,bar) prefix "abc"`
	JoinModifierPrefix *StringExpr

	// If KeepMetricNames is set to true, then the operation should keep metric names.
	KeepMetricNames bool

	// Left contains left arg for the `left op right` expression.
	Left Expr

	// Right contains right arg for the `left op right` epxression.
	Right Expr
}

BinaryOpExpr represents binary operation.

func (*BinaryOpExpr) AppendString 

func (be *BinaryOpExpr) AppendString(dst []byte) []byte

AppendString appends string representation of be to dst and returns the result.

type DurationExpr added in v0.16.0 

type DurationExpr struct {
	// contains filtered or unexported fields
}

DurationExpr contains the duration

func (*DurationExpr) AppendString added in v0.16.0 

func (de *DurationExpr) AppendString(dst []byte) []byte

AppendString appends string representation of de to dst and returns the result.

func (*DurationExpr) Duration added in v0.16.0 

func (de *DurationExpr) Duration(step int64) int64

Duration returns the duration from de in milliseconds.

func (*DurationExpr) NonNegativeDuration added in v0.63.0 

func (de *DurationExpr) NonNegativeDuration(step int64) (int64, error)

NonNegativeDuration returns non-negative duration for de in milliseconds.

Error is returned if the duration is negative.

type Expr 

type Expr interface {
	// AppendString appends string representation of Expr to dst.
	AppendString(dst []byte) []byte
}

Expr holds any of *Expr types.

func Clone added in v0.39.0 

func Clone(e Expr) Expr

Clone clones the given expression e and returns the cloned copy.

func Optimize added in v0.7.0 

func Optimize(e Expr) Expr

Optimize optimizes e in order to improve its performance.

It performs the following optimizations:

func Parse 

func Parse(s string) (Expr, error)

Parse parses MetricsQL query s.

All the `WITH` expressions are expanded in the returned Expr.

MetricsQL is backwards-compatible with PromQL.

Example 
package main

import (
	"fmt"
	"log"

	"github.com/VictoriaMetrics/metricsql"
)

func main() {
	expr, err := metricsql.Parse(`sum(rate(foo{bar="baz"}[5m])) by (x,y)`)
	if err != nil {
		log.Fatalf("parse error: %s", err)
	}
	fmt.Printf("parsed expr: %s\n", expr.AppendString(nil))

	ae := expr.(*metricsql.AggrFuncExpr)
	fmt.Printf("aggr func: name=%s, arg=%s, modifier=%s\n", ae.Name, ae.Args[0].AppendString(nil), ae.Modifier.AppendString(nil))

	fe := ae.Args[0].(*metricsql.FuncExpr)
	fmt.Printf("func: name=%s, arg=%s\n", fe.Name, fe.Args[0].AppendString(nil))

	re := fe.Args[0].(*metricsql.RollupExpr)
	fmt.Printf("rollup: expr=%s, window=%s\n", re.Expr.AppendString(nil), re.Window.AppendString(nil))

	me := re.Expr.(*metricsql.MetricExpr)
	fmt.Printf("metric: labelFilter1=%s, labelFilter2=%s", me.LabelFilterss[0][0].AppendString(nil), me.LabelFilterss[0][1].AppendString(nil))

}

Output:

parsed expr: sum(rate(foo{bar="baz"}[5m])) by(x,y)
aggr func: name=sum, arg=rate(foo{bar="baz"}[5m]), modifier=by(x,y)
func: name=rate, arg=foo{bar="baz"}[5m]
rollup: expr=foo{bar="baz"}, window=5m
metric: labelFilter1=__name__="foo", labelFilter2=bar="baz"

func PushdownBinaryOpFilters added in v0.39.0 

func PushdownBinaryOpFilters(e Expr, commonFilters []LabelFilter) Expr

PushdownBinaryOpFilters pushes down the given commonFilters to e if possible.

e must be a part of binary operation - either left or right.

For example, if e contains `foo + sum(bar)` and commonFilters={x="y"}, then the returned expression will contain `foo{x="y"} + sum(bar)`. The `{x="y"}` cannot be pusehd down to `sum(bar)`, since this may change binary operation results.

type FuncExpr 

type FuncExpr struct {
	// Name is function name.
	Name string

	// Args contains function args.
	Args []Expr

	// If KeepMetricNames is set to true, then the function should keep metric names.
	KeepMetricNames bool
}

FuncExpr represetns MetricsQL function such as `foo(...)`

func (*FuncExpr) AppendString 

func (fe *FuncExpr) AppendString(dst []byte) []byte

AppendString appends string representation of fe to dst and returns the result.

type LabelFilter 

type LabelFilter struct {
	// Label contains label name for the filter.
	Label string

	// Value contains unquoted value for the filter.
	Value string

	// IsNegative reperesents whether the filter is negative, i.e. '!=' or '!~'.
	IsNegative bool

	// IsRegexp represents whether the filter is regesp, i.e. `=~` or `!~`.
	IsRegexp bool
}

LabelFilter represents MetricsQL label filter like `foo="bar"`.

func TrimFiltersByGroupModifier added in v0.39.0 

func TrimFiltersByGroupModifier(lfs []LabelFilter, be *BinaryOpExpr) []LabelFilter

TrimFiltersByGroupModifier trims lfs by the specified be.GroupModifier.Op (e.g. on() or ignoring()).

The following cases are possible: - It returns lfs as is if be doesn't contain any group modifier - It returns only filters specified in on() - It drops filters specified inside ignoring()

func (*LabelFilter) AppendString 

func (lf *LabelFilter) AppendString(dst []byte) []byte

AppendString appends string representation of me to dst and returns the result.

type MetricExpr 

type MetricExpr struct {
	// LabelFilters contains a list of or-delimited groups of label filters from curly braces.
	// Filter for metric name (aka __name__ label) must go first in every group.
	LabelFilterss [][]LabelFilter
	// contains filtered or unexported fields
}

MetricExpr represents MetricsQL metric with optional filters, i.e. `foo{...}`.

Curly braces may contain or-delimited list of filters. For example: 

x{job="foo",instance="bar" or job="x",instance="baz"}

In this case the filter returns all the series, which match at least one of the following filters: 

x{job="foo",instance="bar"}
x{job="x",instance="baz"}

This allows using or-delimited list of filters inside rollup functions. For example, the following query calculates rate per each matching series for the given or-delimited filters: 

rate(x{job="foo",instance="bar" or job="x",instance="baz"}[5m])

func (*MetricExpr) AppendString 

func (me *MetricExpr) AppendString(dst []byte) []byte

AppendString appends string representation of me to dst and returns the result.

func (*MetricExpr) IsEmpty 

func (me *MetricExpr) IsEmpty() bool

IsEmpty returns true of me equals to `{}`.

type ModifierExpr 

type ModifierExpr struct {
	// Op is modifier operation.
	Op string

	// Args contains modifier args from parens.
	Args []string
}

ModifierExpr represents MetricsQL modifier such as `<op> (...)`

func (*ModifierExpr) AppendString 

func (me *ModifierExpr) AppendString(dst []byte) []byte

AppendString appends string representation of me to dst and returns the result.

type NumberExpr 

type NumberExpr struct {
	// N is the parsed number, i.e. `1.23`, `-234`, etc.
	N float64
	// contains filtered or unexported fields
}

NumberExpr represents number expression.

func (*NumberExpr) AppendString 

func (ne *NumberExpr) AppendString(dst []byte) []byte

AppendString appends string representation of ne to dst and returns the result.

type RollupExpr 

type RollupExpr struct {
	// The expression for the rollup. Usually it is MetricExpr, but may be arbitrary expr
	// if subquery is used. https://prometheus.io/blog/2019/01/28/subquery-support/
	Expr Expr

	// Window contains optional window value from square brackets
	//
	// For example, `http_requests_total[5m]` will have Window value `5m`.
	Window *DurationExpr

	// Offset contains optional value from `offset` part.
	//
	// For example, `foobar{baz="aa"} offset 5m` will have Offset value `5m`.
	Offset *DurationExpr

	// Step contains optional step value from square brackets.
	//
	// For example, `foobar[1h:3m]` will have Step value '3m'.
	Step *DurationExpr

	// If set to true, then `foo[1h:]` would print the same
	// instead of `foo[1h]`.
	InheritStep bool

	// At contains an optional expression after `@` modifier.
	//
	// For example, `foo @ end()` or `bar[5m] @ 12345`
	// See https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier
	At Expr
}

RollupExpr represents MetricsQL expression, which contains at least `offset` or `[...]` part.

func (*RollupExpr) AppendString 

func (re *RollupExpr) AppendString(dst []byte) []byte

AppendString appends string representation of re to dst and returns the result.

func (*RollupExpr) ForSubquery 

func (re *RollupExpr) ForSubquery() bool

ForSubquery returns true if re represents subquery.

type StringExpr 

type StringExpr struct {
	// S contains unquoted value for string expression.
	S string
	// contains filtered or unexported fields
}

StringExpr represents string expression.

func (*StringExpr) AppendString 

func (se *StringExpr) AppendString(dst []byte) []byte

AppendString appends string representation of se to dst and returns the result.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.