metricsql

package module
Version: v0.44.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 30, 2022 License: Apache-2.0 Imports: 12 Imported by: 8

README

GoDoc Go Report

metricsql

Package metricsql implements MetricsQL and PromQL parser in Go.

Usage
    expr, err := metricsql.Parse(`sum(rate(foo{bar="baz"}[5m])) by (job)`)
    if err != nil {
        // parse error
    }
    // Now expr contains parsed MetricsQL as `*Expr` structs.
    // See Parse examples for more details.

See docs for more details.

Documentation

Overview

Package metricsql implements MetricsQL parser.

This parser can parse PromQL. Additionally it can parse all the MetricsQL extensions. See https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/MetricsQL for details about MetricsQL extensions.

Usage:

expr, err := metricsql.Parse(`sum(rate(foo{bar="baz"}[5m])) by (job)`)
if err != nil {
    // parse error
}
// Now expr contains parsed MetricsQL as `*Expr` structs.
// See Parse examples for more details.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func CompileRegexp

func CompileRegexp(re string) (*regexp.Regexp, error)

CompileRegexp returns compile regexp re.

func CompileRegexpAnchored

func CompileRegexpAnchored(re string) (*regexp.Regexp, error)

CompileRegexpAnchored returns compiled regexp `^re$`.

func DurationValue

func DurationValue(s string, step int64) (int64, error)

DurationValue returns the duration in milliseconds for the given s and the given step.

Duration in s may be combined, i.e. 2h5m, -2h5m or 2h-5m.

The returned duration value can be negative.

func ExpandWithExprs

func ExpandWithExprs(q string) (string, error)

ExpandWithExprs expands WITH expressions inside q and returns the resulting PromQL without WITH expressions.

Example
package main

import (
	"fmt"
	"log"

	"github.com/VictoriaMetrics/metricsql"
)

func main() {
	// mql can contain arbitrary MetricsQL extensions - see https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/MetricsQL
	mql := `WITH (
		commonFilters = {job="$job", instance="$instance"},
		f(a, b) = 100*(a/b),
	)
	f(disk_free_bytes{commonFilters}, disk_total_bytes{commonFilters})`

	// Convert mql to PromQL
	pql, err := metricsql.ExpandWithExprs(mql)
	if err != nil {
		log.Fatalf("cannot expand with expressions: %s", err)
	}

	fmt.Printf("%s\n", pql)

}
Output:

100 * (disk_free_bytes{job="$job", instance="$instance"} / disk_total_bytes{job="$job", instance="$instance"})

func GetRollupArgIdx added in v0.35.0

func GetRollupArgIdx(fe *FuncExpr) int

GetRollupArgIdx returns the argument index for the given fe, which accepts the rollup argument.

-1 is returned if fe isn't a rollup function.

func IsBinaryOpCmp

func IsBinaryOpCmp(op string) bool

IsBinaryOpCmp returns true if op is comparison operator such as '==', '!=', etc.

func IsRollupFunc

func IsRollupFunc(funcName string) bool

IsRollupFunc returns whether funcName is known rollup function.

func IsTransformFunc

func IsTransformFunc(funcName string) bool

IsTransformFunc returns whether funcName is known transform function.

func PositiveDurationValue

func PositiveDurationValue(s string, step int64) (int64, error)

PositiveDurationValue returns positive duration in milliseconds for the given s and the given step.

Duration in s may be combined, i.e. 2h5m or 2h-5m.

Error is returned if the duration in s is negative.

func VisitAll added in v0.4.0

func VisitAll(e Expr, f func(expr Expr))

VisitAll recursively calls f for all the Expr children in e.

It visits leaf children at first and then visits parent nodes. It is safe modifying expr in f.

Types

type AggrFuncExpr

type AggrFuncExpr struct {
	// Name is the function name.
	Name string

	// Args is the function args.
	Args []Expr

	// Modifier is optional modifier such as `by (...)` or `without (...)`.
	Modifier ModifierExpr

	// Optional limit for the number of output time series.
	// This is MetricsQL extension.
	//
	// Example: `sum(...) by (...) limit 10` would return maximum 10 time series.
	Limit int
}

AggrFuncExpr represents aggregate function such as `sum(...) by (...)`

func (*AggrFuncExpr) AppendString

func (ae *AggrFuncExpr) AppendString(dst []byte) []byte

AppendString appends string representation of ae to dst and returns the result.

type BinaryOpExpr

type BinaryOpExpr struct {
	// Op is the operation itself, i.e. `+`, `-`, `*`, etc.
	Op string

	// Bool indicates whether `bool` modifier is present.
	// For example, `foo >bool bar`.
	Bool bool

	// GroupModifier contains modifier such as "on" or "ignoring".
	GroupModifier ModifierExpr

	// JoinModifier contains modifier such as "group_left" or "group_right".
	JoinModifier ModifierExpr

	// Left contains left arg for the `left op right` expression.
	Left Expr

	// Right contains right arg for the `left op right` epxression.
	Right Expr
}

BinaryOpExpr represents binary operation.

func (*BinaryOpExpr) AppendString

func (be *BinaryOpExpr) AppendString(dst []byte) []byte

AppendString appends string representation of be to dst and returns the result.

type DurationExpr added in v0.16.0

type DurationExpr struct {
	// contains filtered or unexported fields
}

DurationExpr contains the duration

func (*DurationExpr) AppendString added in v0.16.0

func (de *DurationExpr) AppendString(dst []byte) []byte

AppendString appends string representation of de to dst and returns the result.

func (*DurationExpr) Duration added in v0.16.0

func (de *DurationExpr) Duration(step int64) int64

Duration returns the duration from de in milliseconds.

type Expr

type Expr interface {
	// AppendString appends string representation of Expr to dst.
	AppendString(dst []byte) []byte
}

Expr holds any of *Expr types.

func Clone added in v0.39.0

func Clone(e Expr) Expr

Clone clones the given expression e and returns the cloned copy.

func Optimize added in v0.7.0

func Optimize(e Expr) Expr

Optimize optimizes e in order to improve its performance.

It performs the following optimizations:

func Parse

func Parse(s string) (Expr, error)

Parse parses MetricsQL query s.

All the `WITH` expressions are expanded in the returned Expr.

MetricsQL is backwards-compatible with PromQL.

Example
package main

import (
	"fmt"
	"log"

	"github.com/VictoriaMetrics/metricsql"
)

func main() {
	expr, err := metricsql.Parse(`sum(rate(foo{bar="baz"}[5m])) by (x,y)`)
	if err != nil {
		log.Fatalf("parse error: %s", err)
	}
	fmt.Printf("parsed expr: %s\n", expr.AppendString(nil))

	ae := expr.(*metricsql.AggrFuncExpr)
	fmt.Printf("aggr func: name=%s, arg=%s, modifier=%s\n", ae.Name, ae.Args[0].AppendString(nil), ae.Modifier.AppendString(nil))

	fe := ae.Args[0].(*metricsql.FuncExpr)
	fmt.Printf("func: name=%s, arg=%s\n", fe.Name, fe.Args[0].AppendString(nil))

	re := fe.Args[0].(*metricsql.RollupExpr)
	fmt.Printf("rollup: expr=%s, window=%s\n", re.Expr.AppendString(nil), re.Window.AppendString(nil))

	me := re.Expr.(*metricsql.MetricExpr)
	fmt.Printf("metric: labelFilter1=%s, labelFilter2=%s", me.LabelFilters[0].AppendString(nil), me.LabelFilters[1].AppendString(nil))

}
Output:

parsed expr: sum(rate(foo{bar="baz"}[5m])) by (x, y)
aggr func: name=sum, arg=rate(foo{bar="baz"}[5m]), modifier=by (x, y)
func: name=rate, arg=foo{bar="baz"}[5m]
rollup: expr=foo{bar="baz"}, window=5m
metric: labelFilter1=__name__="foo", labelFilter2=bar="baz"

func PushdownBinaryOpFilters added in v0.39.0

func PushdownBinaryOpFilters(e Expr, commonFilters []LabelFilter) Expr

PushdownBinaryOpFilters pushes down the given commonFilters to e if possible.

e must be a part of binary operation - either left or right.

For example, if e contains `foo + sum(bar)` and commonFilters={x="y"}, then the returned expression will contain `foo{x="y"} + sum(bar)`. The `{x="y"}` cannot be pusehd down to `sum(bar)`, since this may change binary operation results.

type FuncExpr

type FuncExpr struct {
	// Name is function name.
	Name string

	// Args contains function args.
	Args []Expr

	// If KeepMetricNames is set to true, then the function should keep metric names.
	KeepMetricNames bool
}

FuncExpr represetns MetricsQL function such as `foo(...)`

func (*FuncExpr) AppendString

func (fe *FuncExpr) AppendString(dst []byte) []byte

AppendString appends string representation of fe to dst and returns the result.

type LabelFilter

type LabelFilter struct {
	// Label contains label name for the filter.
	Label string

	// Value contains unquoted value for the filter.
	Value string

	// IsNegative reperesents whether the filter is negative, i.e. '!=' or '!~'.
	IsNegative bool

	// IsRegexp represents whether the filter is regesp, i.e. `=~` or `!~`.
	IsRegexp bool
}

LabelFilter represents MetricsQL label filter like `foo="bar"`.

func TrimFiltersByGroupModifier added in v0.39.0

func TrimFiltersByGroupModifier(lfs []LabelFilter, be *BinaryOpExpr) []LabelFilter

TrimFiltersByGroupModifier trims lfs by the specified be.GroupModifier.Op (e.g. on() or ignoring()).

The following cases are possible: - It returns lfs as is if be doesn't contain any group modifier - It returns only filters specified in on() - It drops filters specified inside ignoring()

func (*LabelFilter) AppendString

func (lf *LabelFilter) AppendString(dst []byte) []byte

AppendString appends string representation of me to dst and returns the result.

type MetricExpr

type MetricExpr struct {
	// LabelFilters contains a list of label filters from curly braces.
	// Filter or metric name must be the first if present.
	LabelFilters []LabelFilter
	// contains filtered or unexported fields
}

MetricExpr represents MetricsQL metric with optional filters, i.e. `foo{...}`.

func (*MetricExpr) AppendString

func (me *MetricExpr) AppendString(dst []byte) []byte

AppendString appends string representation of me to dst and returns the result.

func (*MetricExpr) IsEmpty

func (me *MetricExpr) IsEmpty() bool

IsEmpty returns true of me equals to `{}`.

type ModifierExpr

type ModifierExpr struct {
	// Op is modifier operation.
	Op string

	// Args contains modifier args from parens.
	Args []string
}

ModifierExpr represents MetricsQL modifier such as `<op> (...)`

func (*ModifierExpr) AppendString

func (me *ModifierExpr) AppendString(dst []byte) []byte

AppendString appends string representation of me to dst and returns the result.

type NumberExpr

type NumberExpr struct {
	// N is the parsed number, i.e. `1.23`, `-234`, etc.
	N float64
}

NumberExpr represents number expression.

func (*NumberExpr) AppendString

func (ne *NumberExpr) AppendString(dst []byte) []byte

AppendString appends string representation of ne to dst and returns the result.

type RollupExpr

type RollupExpr struct {
	// The expression for the rollup. Usually it is MetricExpr, but may be arbitrary expr
	// if subquery is used. https://prometheus.io/blog/2019/01/28/subquery-support/
	Expr Expr

	// Window contains optional window value from square brackets
	//
	// For example, `http_requests_total[5m]` will have Window value `5m`.
	Window *DurationExpr

	// Offset contains optional value from `offset` part.
	//
	// For example, `foobar{baz="aa"} offset 5m` will have Offset value `5m`.
	Offset *DurationExpr

	// Step contains optional step value from square brackets.
	//
	// For example, `foobar[1h:3m]` will have Step value '3m'.
	Step *DurationExpr

	// If set to true, then `foo[1h:]` would print the same
	// instead of `foo[1h]`.
	InheritStep bool

	// At contains an optional expression after `@` modifier.
	//
	// For example, `foo @ end()` or `bar[5m] @ 12345`
	// See https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier
	At Expr
}

RollupExpr represents MetricsQL expression, which contains at least `offset` or `[...]` part.

func (*RollupExpr) AppendString

func (re *RollupExpr) AppendString(dst []byte) []byte

AppendString appends string representation of re to dst and returns the result.

func (*RollupExpr) ForSubquery

func (re *RollupExpr) ForSubquery() bool

ForSubquery returns true if re represents subquery.

type StringExpr

type StringExpr struct {
	// S contains unquoted value for string expression.
	S string
	// contains filtered or unexported fields
}

StringExpr represents string expression.

func (*StringExpr) AppendString

func (se *StringExpr) AppendString(dst []byte) []byte

AppendString appends string representation of se to dst and returns the result.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL