parser

package
v0.25.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 22, 2025 License: Apache-2.0, BSD-3-Clause Imports: 15 Imported by: 41

Documentation

Overview

Package parser declares an expression parser with support for macro expansion.

Index

Constants

View Source 
const AccumulatorName = "__result__"

AccumulatorName is the traditional variable name assigned to the fold accumulator variable.

View Source 
const HiddenAccumulatorName = "@result"

HiddenAccumulatorName is a proposed update to the default fold accumlator variable. @result is not normally accessible from source, preventing accidental or intentional collisions in user expressions.

Variables

View Source 
var (
	// HasMacro expands "has(m.f)" which tests the presence of a field, avoiding the need to
	// specify the field as a string.
	HasMacro = NewGlobalMacro(operators.Has, 1, MakeHas,
		MacroDocs(
			`check a protocol buffer message for the presence of a field, or check a map`,
			`for the presence of a string key.`,
			`Only map accesses using the select notation are supported.`),
		MacroExamples(
			common.MultilineDescription(
				`// true if the 'address' field exists in the 'user' message`,
				`has(user.address)`),
			common.MultilineDescription(
				`// test whether the 'key_name' is set on the map which defines it`,
				`has({'key_name': 'value'}.key_name) // true`),
			common.MultilineDescription(
				`// test whether the 'id' field is set to a non-default value on the Expr{} message literal`,
				`has(Expr{}.id) // false`),
		))

	// AllMacro expands "range.all(var, predicate)" into a comprehension which ensures that all
	// elements in the range satisfy the predicate.
	AllMacro = NewReceiverMacro(operators.All, 2, MakeAll,
		MacroDocs(`tests whether all elements in the input list or all keys in a map`,
			`satisfy the given predicate. The all macro behaves in a manner consistent with`,
			`the Logical AND operator including in how it absorbs errors and short-circuits.`),
		MacroExamples(
			`[1, 2, 3].all(x, x > 0) // true`,
			`[1, 2, 0].all(x, x > 0) // false`,
			`['apple', 'banana', 'cherry'].all(fruit, fruit.size() > 3) // true`,
			`[3.14, 2.71, 1.61].all(num, num < 3.0) // false`,
			`{'a': 1, 'b': 2, 'c': 3}.all(key, key != 'b') // false`,
			common.MultilineDescription(
				`// an empty list or map as the range will result in a trivially true result`,
				`[].all(x, x > 0) // true`),
		))

	// ExistsMacro expands "range.exists(var, predicate)" into a comprehension which ensures that
	// some element in the range satisfies the predicate.
	ExistsMacro = NewReceiverMacro(operators.Exists, 2, MakeExists,
		MacroDocs(`tests whether any value in the list or any key in the map`,
			`satisfies the predicate expression. The exists macro behaves in a manner`,
			`consistent with the Logical OR operator including in how it absorbs errors and`,
			`short-circuits.`),
		MacroExamples(
			`[1, 2, 3].exists(i, i % 2 != 0) // true`,
			`[0, -1, 5].exists(num, num < 0) // true`,
			`{'x': 'foo', 'y': 'bar'}.exists(key, key.startsWith('z')) // false`,
			common.MultilineDescription(
				`// an empty list or map as the range will result in a trivially false result`,
				`[].exists(i, i > 0) // false`),
			common.MultilineDescription(
				`// test whether a key name equalling 'iss' exists in the map and the`,
				`// value contains the substring 'cel.dev'`,
				`// tokens = {'sub': 'me', 'iss': 'https://issuer.cel.dev'}`,
				`tokens.exists(k, k == 'iss' && tokens[k].contains('cel.dev'))`),
		))

	// ExistsOneMacro expands "range.exists_one(var, predicate)", which is true if for exactly one
	// element in range the predicate holds.
	// Deprecated: Use ExistsOneMacroNew
	ExistsOneMacro = NewReceiverMacro(operators.ExistsOne, 2, MakeExistsOne,
		MacroDocs(`tests whether exactly one list element or map key satisfies`,
			`the predicate expression. This macro does not short-circuit in order to remain`,
			`consistent with logical operators being the only operators which can absorb`,
			`errors within CEL.`),
		MacroExamples(
			`[1, 2, 2].exists_one(i, i < 2) // true`,
			`{'a': 'hello', 'aa': 'hellohello'}.exists_one(k, k.startsWith('a')) // false`,
			`[1, 2, 3, 4].exists_one(num, num % 2 == 0) // false`,
			common.MultilineDescription(
				`// ensure exactly one key in the map ends in @acme.co`,
				`{'wiley@acme.co': 'coyote', 'aa@milne.co': 'bear'}.exists_one(k, k.endsWith('@acme.co')) // true`),
		))

	// ExistsOneMacroNew expands "range.existsOne(var, predicate)", which is true if for exactly one
	// element in range the predicate holds.
	ExistsOneMacroNew = NewReceiverMacro("existsOne", 2, MakeExistsOne,
		MacroDocs(
			`tests whether exactly one list element or map key satisfies the predicate`,
			`expression. This macro does not short-circuit in order to remain consistent`,
			`with logical operators being the only operators which can absorb errors`,
			`within CEL.`),
		MacroExamples(
			`[1, 2, 2].existsOne(i, i < 2) // true`,
			`{'a': 'hello', 'aa': 'hellohello'}.existsOne(k, k.startsWith('a')) // false`,
			`[1, 2, 3, 4].existsOne(num, num % 2 == 0) // false`,
			common.MultilineDescription(
				`// ensure exactly one key in the map ends in @acme.co`,
				`{'wiley@acme.co': 'coyote', 'aa@milne.co': 'bear'}.existsOne(k, k.endsWith('@acme.co')) // true`),
		))

	// MapMacro expands "range.map(var, function)" into a comprehension which applies the function
	// to each element in the range to produce a new list.
	MapMacro = NewReceiverMacro(operators.Map, 2, MakeMap,
		MacroDocs("the three-argument form of map transforms all elements in the input range."),
		MacroExamples(
			`[1, 2, 3].map(x, x * 2) // [2, 4, 6]`,
			`[5, 10, 15].map(x, x / 5) // [1, 2, 3]`,
			`['apple', 'banana'].map(fruit, fruit.upperAscii()) // ['APPLE', 'BANANA']`,
			common.MultilineDescription(
				`// Combine all map key-value pairs into a list`,
				`{'hi': 'you', 'howzit': 'bruv'}.map(k,`,
				`    k + ":" + {'hi': 'you', 'howzit': 'bruv'}[k]) // ['hi:you', 'howzit:bruv']`),
		))

	// MapFilterMacro expands "range.map(var, predicate, function)" into a comprehension which
	// first filters the elements in the range by the predicate, then applies the transform function
	// to produce a new list.
	MapFilterMacro = NewReceiverMacro(operators.Map, 3, MakeMap,
		MacroDocs(`the four-argument form of the map transforms only elements which satisfy`,
			`the predicate which is equivalent to chaining the filter and three-argument`,
			`map macros together.`),
		MacroExamples(
			common.MultilineDescription(
				`// multiply only numbers divisible two, by 2`,
				`[1, 2, 3, 4].map(num, num % 2 == 0, num * 2) // [4, 8]`),
		))

	// FilterMacro expands "range.filter(var, predicate)" into a comprehension which filters
	// elements in the range, producing a new list from the elements that satisfy the predicate.
	FilterMacro = NewReceiverMacro(operators.Filter, 2, MakeFilter,
		MacroDocs(`returns a list containing only the elements from the input list`,
			`that satisfy the given predicate`),
		MacroExamples(
			`[1, 2, 3].filter(x, x > 1) // [2, 3]`,
			`['cat', 'dog', 'bird', 'fish'].filter(pet, pet.size() == 3) // ['cat', 'dog']`,
			`[{'a': 10, 'b': 5, 'c': 20}].map(m, m.filter(key, m[key] > 10)) // [['c']]`,
			common.MultilineDescription(
				`// filter a list to select only emails with the @cel.dev suffix`,
				`['alice@buf.io', 'tristan@cel.dev'].filter(v, v.endsWith('@cel.dev')) // ['tristan@cel.dev']`),
			common.MultilineDescription(
				`// filter a map into a list, selecting only the values for keys that start with 'http-auth'`,
				`{'http-auth-agent': 'secret', 'user-agent': 'mozilla'}.filter(k,`,
				`     k.startsWith('http-auth')) // ['secret']`),
		))

	// AllMacros includes the list of all spec-supported macros.
	AllMacros = []Macro{
		HasMacro,
		AllMacro,
		ExistsMacro,
		ExistsOneMacro,
		ExistsOneMacroNew,
		MapMacro,
		MapFilterMacro,
		FilterMacro,
	}

	// NoMacros list.
	NoMacros = []Macro{}
)

Functions

func MakeAll added in v0.12.0 

func MakeAll(eh ExprHelper, target ast.Expr, args []ast.Expr) (ast.Expr, *common.Error)

MakeAll expands the input call arguments into a comprehension that returns true if all of the elements in the range match the predicate expressions: <iterRange>.all(<iterVar>, <predicate>)

func MakeExists added in v0.12.0 

func MakeExists(eh ExprHelper, target ast.Expr, args []ast.Expr) (ast.Expr, *common.Error)

MakeExists expands the input call arguments into a comprehension that returns true if any of the elements in the range match the predicate expressions: <iterRange>.exists(<iterVar>, <predicate>)

func MakeExistsOne added in v0.12.0 

func MakeExistsOne(eh ExprHelper, target ast.Expr, args []ast.Expr) (ast.Expr, *common.Error)

MakeExistsOne expands the input call arguments into a comprehension that returns true if exactly one of the elements in the range match the predicate expressions: <iterRange>.exists_one(<iterVar>, <predicate>)

func MakeFilter added in v0.12.0 

func MakeFilter(eh ExprHelper, target ast.Expr, args []ast.Expr) (ast.Expr, *common.Error)

MakeFilter expands the input call arguments into a comprehension which produces a list which contains only elements which match the provided predicate expression: <iterRange>.filter(<iterVar>, <predicate>)

func MakeHas added in v0.12.0 

func MakeHas(eh ExprHelper, target ast.Expr, args []ast.Expr) (ast.Expr, *common.Error)

MakeHas expands the input call arguments into a presence test, e.g. has(<operand>.field)

func MakeMap added in v0.12.0 

func MakeMap(eh ExprHelper, target ast.Expr, args []ast.Expr) (ast.Expr, *common.Error)

MakeMap expands the input call arguments into a comprehension that transforms each element in the input to produce an output list.

There are two call patterns supported by map: 

<iterRange>.map(<iterVar>, <transform>)
<iterRange>.map(<iterVar>, <predicate>, <transform>)

In the second form only iterVar values which return true when provided to the predicate expression are transformed.

func Parse deprecated 

func Parse(source common.Source) (*ast.AST, *common.Errors)

Parse converts a source input a parsed expression. This function calls ParseWithMacros with AllMacros.

Deprecated: Use NewParser().Parse() instead.

func Unparse added in v0.3.0 

func Unparse(expr ast.Expr, info *ast.SourceInfo, opts ...UnparserOption) (string, error)

Unparse takes an input expression and source position information and generates a human-readable expression.

Note, unparsing an AST will often generate the same expression as was originally parsed, but some formatting may be lost in translation, notably:

- All quoted literals are doubled quoted. - Byte literals are represented as octal escapes (same as Google SQL). - Floating point values are converted to the small number of digits needed to represent the value. - Spacing around punctuation marks may be lost. - Parentheses will only be applied when they affect operator precedence.

This function optionally takes in one or more UnparserOption to alter the unparsing behavior, such as performing word wrapping on expressions.

Types

type ExprHelper 

type ExprHelper interface {
	// Copy the input expression with a brand new set of identifiers.
	Copy(ast.Expr) ast.Expr

	// Literal creates an Expr value for a scalar literal value.
	NewLiteral(value ref.Val) ast.Expr

	// NewList creates a list literal instruction with an optional set of elements.
	NewList(elems ...ast.Expr) ast.Expr

	// NewMap creates a CreateStruct instruction for a map where the map is comprised of the
	// optional set of key, value entries.
	NewMap(entries ...ast.EntryExpr) ast.Expr

	// NewMapEntry creates a Map Entry for the key, value pair.
	NewMapEntry(key ast.Expr, val ast.Expr, optional bool) ast.EntryExpr

	// NewStruct creates a struct literal expression with an optional set of field initializers.
	NewStruct(typeName string, fieldInits ...ast.EntryExpr) ast.Expr

	// NewStructField creates a new struct field initializer from the field name and value.
	NewStructField(field string, init ast.Expr, optional bool) ast.EntryExpr

	// NewComprehension creates a new one-variable comprehension instruction.
	//
	// - iterRange represents the expression that resolves to a list or map where the elements or
	//   keys (respectively) will be iterated over.
	// - iterVar is the variable name for the list element value, or the map key, depending on the
	//   range type.
	// - accuVar is the accumulation variable name, typically parser.AccumulatorName.
	// - accuInit is the initial expression whose value will be set for the accuVar prior to
	//   folding.
	// - condition is the expression to test to determine whether to continue folding.
	// - step is the expression to evaluation at the conclusion of a single fold iteration.
	// - result is the computation to evaluate at the conclusion of the fold.
	//
	// The accuVar should not shadow variable names that you would like to reference within the
	// environment in the step and condition expressions. Presently, the name __result__ is commonly
	// used by built-in macros but this may change in the future.
	NewComprehension(iterRange ast.Expr,
		iterVar,
		accuVar string,
		accuInit,
		condition,
		step,
		result ast.Expr) ast.Expr

	// NewComprehensionTwoVar creates a new two-variable comprehension instruction.
	//
	// - iterRange represents the expression that resolves to a list or map where the elements or
	//   keys (respectively) will be iterated over.
	// - iterVar is the iteration variable assigned to the list index or the map key.
	// - iterVar2 is the iteration variable assigned to the list element value or the map key value.
	// - accuVar is the accumulation variable name, typically parser.AccumulatorName.
	// - accuInit is the initial expression whose value will be set for the accuVar prior to
	//   folding.
	// - condition is the expression to test to determine whether to continue folding.
	// - step is the expression to evaluation at the conclusion of a single fold iteration.
	// - result is the computation to evaluate at the conclusion of the fold.
	//
	// The accuVar should not shadow variable names that you would like to reference within the
	// environment in the step and condition expressions. Presently, the name __result__ is commonly
	// used by built-in macros but this may change in the future.
	NewComprehensionTwoVar(iterRange ast.Expr,
		iterVar,
		iterVar2,
		accuVar string,
		accuInit,
		condition,
		step,
		result ast.Expr) ast.Expr

	// NewIdent creates an identifier Expr value.
	NewIdent(name string) ast.Expr

	// NewAccuIdent returns an accumulator identifier for use with comprehension results.
	NewAccuIdent() ast.Expr

	// AccuIdentName returns the name of the accumulator identifier.
	AccuIdentName() string

	// NewCall creates a function call Expr value for a global (free) function.
	NewCall(function string, args ...ast.Expr) ast.Expr

	// NewMemberCall creates a function call Expr value for a receiver-style function.
	NewMemberCall(function string, target ast.Expr, args ...ast.Expr) ast.Expr

	// NewPresenceTest creates a Select TestOnly Expr value for modelling has() semantics.
	NewPresenceTest(operand ast.Expr, field string) ast.Expr

	// NewSelect create a field traversal Expr value.
	NewSelect(operand ast.Expr, field string) ast.Expr

	// OffsetLocation returns the Location of the expression identifier.
	OffsetLocation(exprID int64) common.Location

	// NewError associates an error message with a given expression id.
	NewError(exprID int64, message string) *common.Error
}

ExprHelper assists with the creation of Expr values in a manner which is consistent the internal semantics and id generation behaviors of the parser and checker libraries.

type Macro 

type Macro interface {
	// Function name to match.
	Function() string

	// ArgCount for the function call.
	//
	// When the macro is a var-arg style macro, the return value will be zero, but the MacroKey
	// will contain a `*` where the arg count would have been.
	ArgCount() int

	// IsReceiverStyle returns true if the macro matches a receiver style call.
	IsReceiverStyle() bool

	// MacroKey returns the macro signatures accepted by this macro.
	//
	// Format: `<function>:<arg-count>:<is-receiver>`.
	//
	// When the macros is a var-arg style macro, the `arg-count` value is represented as a `*`.
	MacroKey() string

	// Expander returns the MacroExpander to apply when the macro key matches the parsed call
	// signature.
	Expander() MacroExpander
}

Macro interface for describing the function signature to match and the MacroExpander to apply.

Note: when a Macro should apply to multiple overloads (based on arg count) of a given function, a Macro should be created per arg-count.

func NewGlobalMacro 

func NewGlobalMacro(function string, argCount int, expander MacroExpander, opts ...MacroOpt) Macro

NewGlobalMacro creates a Macro for a global function with the specified arg count.

func NewGlobalVarArgMacro 

func NewGlobalVarArgMacro(function string, expander MacroExpander, opts ...MacroOpt) Macro

NewGlobalVarArgMacro creates a Macro for a global function with a variable arg count.

func NewReceiverMacro 

func NewReceiverMacro(function string, argCount int, expander MacroExpander, opts ...MacroOpt) Macro

NewReceiverMacro creates a Macro for a receiver function matching the specified arg count.

func NewReceiverVarArgMacro 

func NewReceiverVarArgMacro(function string, expander MacroExpander, opts ...MacroOpt) Macro

NewReceiverVarArgMacro creates a Macro for a receiver function matching a variable arg count.

type MacroExpander 

type MacroExpander func(eh ExprHelper, target ast.Expr, args []ast.Expr) (ast.Expr, *common.Error)

MacroExpander converts a call and its associated arguments into a new CEL abstract syntax tree.

If the MacroExpander determines within the implementation that an expansion is not needed it may return a nil Expr value to indicate a non-match. However, if an expansion is to be performed, but the arguments are not well-formed, the result of the expansion will be an error.

The MacroExpander accepts as arguments a MacroExprHelper as well as the arguments used in the function call and produces as output an Expr ast node.

Note: when the Macro.IsReceiverStyle() method returns true, the target argument will be nil.

type MacroOpt added in v0.25.0 

type MacroOpt func(*macro) *macro

MacroOpt defines a functional option for configuring macro behavior.

func MacroDocs added in v0.25.0 

func MacroDocs(docs ...string) MacroOpt

MacroDocs configures a list of strings into a multiline description for the macro.

func MacroExamples added in v0.25.0 

func MacroExamples(examples ...string) MacroOpt

MacroExamples configures a list of examples, either as a string or common.MultilineString, into an example set to be provided with the macro Documentation() call.

type Option added in v0.8.0 

type Option func(*options) error

Option configures the behavior of the parser.

func EnableHiddenAccumulatorName added in v0.23.0 

func EnableHiddenAccumulatorName(enabled bool) Option

EnableHiddenAccumulatorName uses an accumulator variable name that is not a normally accessible identifier in source for comprehension macros. Compatibility notes: with this option enabled, a parsed AST would be semantically the same as if disabled, but would have different internal identifiers in any of the built-in comprehension sub-expressions. When disabled, it is possible but almost certainly a logic error to access the accumulator variable.

func EnableIdentEscapeSyntax added in v0.23.0 

func EnableIdentEscapeSyntax(enableIdentEscapeSyntax bool) Option

EnableIdentEscapeSyntax enables backtick (`) escaped field identifiers. This supports extended types of characters in identifiers, e.g. foo.`baz-bar`.

func EnableOptionalSyntax added in v0.13.0 

func EnableOptionalSyntax(optionalSyntax bool) Option

EnableOptionalSyntax enables syntax for optional field and index selection.

func EnableVariadicOperatorASTs added in v0.17.0 

func EnableVariadicOperatorASTs(varArgASTs bool) Option

EnableVariadicOperatorASTs enables a compact representation of chained like-kind commutative operators. e.g. `a || b || c || d` -> `call(op='||', args=[a, b, c, d])`

The benefit of enabling variadic operators ASTs is a more compact representation deeply nested logic graphs.

func ErrorRecoveryLimit added in v0.8.0 

func ErrorRecoveryLimit(limit int) Option

ErrorRecoveryLimit limits the number of attempts the parser will perform to recover from an error.

func ErrorRecoveryLookaheadTokenLimit added in v0.8.0 

func ErrorRecoveryLookaheadTokenLimit(limit int) Option

ErrorRecoveryLookaheadTokenLimit limits the number of lexer tokens that may be considered during error recovery.

Error recovery often involves looking ahead in the input to determine if there's a point at which parsing may successfully resume. In some pathological cases, the parser can look through quite a large set of input which in turn generates a lot of back-tracking and performance degredation.

The limit must be >= 1, and is recommended to be less than the default of 256.

func ErrorReportingLimit added in v0.15.0 

func ErrorReportingLimit(limit int) Option

ErrorReportingLimit limits the number of syntax error reports before terminating parsing.

The limit must be at least 1. If unset, the limit will be 100.

func ExpressionSizeCodePointLimit added in v0.8.0 

func ExpressionSizeCodePointLimit(expressionSizeCodePointLimit int) Option

ExpressionSizeCodePointLimit is an option which limits the maximum code point count of an expression.

func Macros added in v0.8.0 

func Macros(macros ...Macro) Option

Macros adds the given macros to the parser.

func MaxRecursionDepth added in v0.8.0 

func MaxRecursionDepth(limit int) Option

MaxRecursionDepth limits the maximum depth the parser will attempt to parse the expression before giving up.

func PopulateMacroCalls added in v0.9.0 

func PopulateMacroCalls(populateMacroCalls bool) Option

PopulateMacroCalls ensures that the original call signatures replaced by expanded macros are preserved in the `SourceInfo` of parse result.

type Parser added in v0.8.0 

type Parser struct {
	// contains filtered or unexported fields
}

Parser encapsulates the context necessary to perform parsing for different expressions.

func NewParser added in v0.8.0 

func NewParser(opts ...Option) (*Parser, error)

NewParser builds and returns a new Parser using the provided options.

func (*Parser) Parse added in v0.8.0 

func (p *Parser) Parse(source common.Source) (*ast.AST, *common.Errors)

Parse parses the expression represented by source and returns the result.

type UnparserOption added in v0.12.2 

type UnparserOption func(*unparserOption) (*unparserOption, error)

UnparserOption is a functional option for configuring the output formatting of the Unparse function.

func WrapAfterColumnLimit added in v0.12.2 

func WrapAfterColumnLimit(wrapAfter bool) UnparserOption

WrapAfterColumnLimit dictates whether to insert a newline before or after the specified operator when word wrapping is performed.

Example usage: 

Unparse(expr, sourceInfo, WrapOnColumn(40), WrapOnOperators(Operators.LogicalAnd), WrapAfterColumnLimit(false))

This will insert a newline immediately before the logical AND operator for the below example input, ensuring that the length of a line never exceeds the specified column limit:

Input: 'my-principal-group' in request.auth.claims && request.auth.claims.iat > now - duration('5m')

Output: 'my-principal-group' in request.auth.claims && request.auth.claims.iat > now - duration('5m')

func WrapOnColumn added in v0.12.2 

func WrapOnColumn(col int) UnparserOption

WrapOnColumn wraps the output expression when its string length exceeds a specified limit for operators set by WrapOnOperators function or by default, "&&" and "||" will be wrapped.

Example usage: 

Unparse(expr, sourceInfo, WrapOnColumn(40), WrapOnOperators(Operators.LogicalAnd))

This will insert a newline immediately after the logical AND operator for the below example input:

Input: 'my-principal-group' in request.auth.claims && request.auth.claims.iat > now - duration('5m')

Output: 'my-principal-group' in request.auth.claims && request.auth.claims.iat > now - duration('5m')

func WrapOnOperators added in v0.12.2 

func WrapOnOperators(symbols ...string) UnparserOption

WrapOnOperators specifies which operators to perform word wrapping on an output expression when its string length exceeds the column limit set by WrapOnColumn function.

Word wrapping is supported on non-unary symbolic operators. Refer to operators.go for the full list

This will replace any previously supplied operators instead of merging them.

Source Files

View all Source files

Directories

Path Synopsis
Package gen contains all of the ANTLR-generated sources used by the cel-go parser.
 Package gen contains all of the ANTLR-generated sources used by the cel-go parser.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.