bexpr

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 3, 2019 License: MPL-2.0 Imports: 14 Imported by: 87

README

bexpr - Boolean Expression Evaluator GoDoc CircleCI

bexpr is a Go (golang) library to provide generic boolean expression evaluation and filtering for Go data structures.

Limitations

Currently bexpr does not support operating on types with cyclical structures. Attempting to generate the fields of these types will cause a stack overflow. There are however two means of getting around this. First if you do not need the nested type to be available during evaluation then you can simply add the bexpr:"-" struct tag to the fields where that type is referenced and bexpr will not delve further into that type. A second solution is implement the MatchExpressionEvaluator interface and provide the necessary field configurations yourself.

Eventually this lib will support handling these cycles automatically.

Stability

Currently there is a MatchExpressionEvaluator interface that can be used to implement custom behavior. This interface should be considered experimental and is likely to change in the future. One need for the change is to make it easier for custom implementations to re-invoke the main bexpr logic on subfields so that they do not have to implement custom logic for themselves and every sub field they contain. With the current interface its not really possible.

Usage (Reflection)

This example program is available in examples/simple

package main

import (
   "fmt"
   "github.com/hashicorp/go-bexpr"
)

type Example struct {
   X int

   // Can renamed a field with the struct tag
   Y string `bexpr:"y"`

   // Fields can use multiple names for accessing
   Z bool `bexpr:"Z,z,foo"`

   // Tag with "-" to prevent allowing this field from being used
   Hidden string `bexpr:"-"`

   // Unexported fields are not available for evaluation
   unexported string
}

func main() {
   value := map[string]Example{
      "foo": Example{X: 5, Y: "foo", Z: true, Hidden: "yes", unexported: "no"},
      "bar": Example{X: 42, Y: "bar", Z: false, Hidden: "no", unexported: "yes"},
   }

   expressions := []string{
      "foo.X == 5",
      "bar.y == bar",
      "foo.foo != false",
      "foo.z == true",
      "foo.Z == true",

      // will error in evaluator creation
      "bar.Hidden != yes",

      // will error in evaluator creation
      "foo.unexported == no",
   }

   for _, expression := range expressions {
      eval, err := bexpr.CreateEvaluatorForType(expression, nil, (*map[string]Example)(nil))

      if err != nil {
         fmt.Printf("Failed to create evaluator for expression %q: %v\n", expression, err)
         continue
      }

      result, err := eval.Evaluate(value)
      if err != nil {
         fmt.Printf("Failed to run evaluation of expression %q: %v\n", expression, err)
         continue
      }

      fmt.Printf("Result of expression %q evaluation: %t\n", expression, result)
   }
}

This will output:

Result of expression "foo.X == 5" evaluation: true
Result of expression "bar.y == bar" evaluation: true
Result of expression "foo.foo != false" evaluation: true
Result of expression "foo.z == true" evaluation: true
Result of expression "foo.Z == true" evaluation: true
Failed to create evaluator for expression "bar.Hidden != yes": Selector "bar.Hidden" is not valid
Failed to create evaluator for expression "foo.unexported == no": Selector "foo.unexported" is not valid

Testing

The Makefile contains 3 main targets to aid with testing:

  1. make test - runs the standard test suite
  2. make coverage - runs the test suite gathering coverage information
  3. make bench - this will run benchmarks. You can use the benchcmp tool to compare subsequent runs of the tool to compare performance. There are a few arguments you can provide to the make invocation to alter the behavior a bit
    • BENCHFULL=1 - This will enable running all the benchmarks. Some could be fairly redundant but could be useful when modifying specific sections of the code.
    • BENCHTIME=5s - By default the -benchtime paramater used for the go test invocation is 2s. 1s seemed like too little to get results consistent enough for comparison between two runs. For the highest degree of confidence that performance has remained steady increase this value even further. The time it takes to run the bench testing suite grows linearly with this value.
    • BENCHTESTS=BenchmarkEvalute - This is used to run a particular benchmark including all of its sub-benchmarks. This is just an example and "BenchmarkEvaluate" can be replaced with any benchmark functions name.

Documentation

Overview

bexpr is an implementation of a generic boolean expression evaluator. The general goal is to be able to evaluate some expression against some arbitrary data and get back a boolean of whether or not the data was matched by the expression

Index

Constants

This section is empty.

Variables

View Source 
var NilRegistry = (*nilRegistry)(nil)

The pass through registry can be used to prevent using the default registry and thus storing any field configurations

Functions

func CoerceBool 

func CoerceBool(value string) (interface{}, error)

CoerceBool conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into a `bool`

func CoerceFloat32 

func CoerceFloat32(value string) (interface{}, error)

CoerceFloat32 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `float32`

func CoerceFloat64 

func CoerceFloat64(value string) (interface{}, error)

CoerceFloat64 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `float64`

func CoerceInt 

func CoerceInt(value string) (interface{}, error)

CoerceInt conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int`

func CoerceInt16 

func CoerceInt16(value string) (interface{}, error)

CoerceInt16 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int16`

func CoerceInt32 

func CoerceInt32(value string) (interface{}, error)

CoerceInt32 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int32`

func CoerceInt64 

func CoerceInt64(value string) (interface{}, error)

CoerceInt64 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int64`

func CoerceInt8 

func CoerceInt8(value string) (interface{}, error)

CoerceInt8 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int8`

func CoerceString 

func CoerceString(value string) (interface{}, error)

CoerceString conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into a `string`

func CoerceUint 

func CoerceUint(value string) (interface{}, error)

CoerceUint conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int`

func CoerceUint16 

func CoerceUint16(value string) (interface{}, error)

CoerceUint16 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int16`

func CoerceUint32 

func CoerceUint32(value string) (interface{}, error)

CoerceUint32 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int32`

func CoerceUint64 

func CoerceUint64(value string) (interface{}, error)

CoerceUint64 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int64`

func CoerceUint8 

func CoerceUint8(value string) (interface{}, error)

CoerceUint8 conforms to the FieldValueCoercionFn signature and can be used to convert the raw string value of an expression into an `int8`

func Parse 

func Parse(filename string, b []byte, opts ...Option) (interface{}, error)

Parse parses the data from b using filename as information in the error messages.

func ParseFile 

func ParseFile(filename string, opts ...Option) (i interface{}, err error)

ParseFile parses the file identified by filename.

func ParseReader 

func ParseReader(filename string, r io.Reader, opts ...Option) (interface{}, error)

ParseReader parses the data from r using filename as information in the error messages.

Types

type BinaryExpression 

type BinaryExpression struct {
	Left     Expression
	Operator BinaryOperator
	Right    Expression
}

func (*BinaryExpression) ExpressionDump 

func (expr *BinaryExpression) ExpressionDump(w io.Writer, indent string, level int)

type BinaryOperator 

type BinaryOperator int
const (
	BinaryOpAnd BinaryOperator = iota
	BinaryOpOr
)

func (BinaryOperator) String 

func (op BinaryOperator) String() string

type Evaluator 

type Evaluator struct {
	// contains filtered or unexported fields
}

func CreateEvaluator 

func CreateEvaluator(expression string, config *EvaluatorConfig) (*Evaluator, error)

func CreateEvaluatorForType 

func CreateEvaluatorForType(expression string, config *EvaluatorConfig, dataType interface{}) (*Evaluator, error)

func (*Evaluator) Evaluate 

func (eval *Evaluator) Evaluate(datum interface{}) (bool, error)

func (*Evaluator) Validate 

func (eval *Evaluator) Validate(config *EvaluatorConfig, dataType interface{}) error

Validates an existing expression against a possibly different configuration

type EvaluatorConfig 

type EvaluatorConfig struct {
	// Maximum number of matching expressions allowed. 0 means unlimited
	// This does not include and, or and not expressions within the AST
	MaxMatches int
	// Maximum length of raw values. 0 means unlimited
	MaxRawValueLength int
	// The Registry to use for validating expressions for a data type
	// If nil the `DefaultRegistry` will be used. To disable using a
	// registry all together you can set this to `NilRegistry`
	Registry Registry
}

Extra configuration used to perform further validation on a parsed expression and to aid in the evaluation process

type Expression 

type Expression interface {
	ExpressionDump(w io.Writer, indent string, level int)
}

type FieldConfiguration 

type FieldConfiguration struct {
	// Name to use when looking up fields within a struct. This is useful when
	// the name(s) you want to expose to users writing the expressions does not
	// exactly match the Field name of the structure. If this is empty then the
	// user provided name will be used
	StructFieldName string

	// Nested field configurations
	SubFields FieldConfigurations

	// Function to run on the raw string value present in the expression
	// syntax to coerce into whatever form the MatchExpressionEvaluator wants
	// The coercion happens only once and will then be passed as the `value`
	// parameter to all EvaluateMatch invocations on the MatchExpressionEvaluator.
	CoerceFn FieldValueCoercionFn

	// List of MatchOperators supported for this field. This configuration
	// is used to pre-validate an expressions fields before execution.
	SupportedOperations []MatchOperator
}

The FieldConfiguration struct represents how boolean expression validation and preparation should work for the given field. A field in this case is a single element of a selector.

Example: foo.bar.baz has 3 fields separate by '.' characters.

func (*FieldConfiguration) String 

func (config *FieldConfiguration) String() string

type FieldConfigurations 

type FieldConfigurations map[FieldName]*FieldConfiguration

Represents all the valid fields and their corresponding configuration

func GenerateFieldConfigurations 

func GenerateFieldConfigurations(topLevelType interface{}) (FieldConfigurations, error)

`generateFieldConfigurations` can be used to generate the `FieldConfigurations` map It supports generating configurations for either a `map[string]*` or a `struct` as the `topLevelType`

Internally within the top level type the following is supported:

Primitive Types: 

strings
integers (all width types and signedness)
floats (32 and 64 bit)
bool

Compound Types 

`map[*]*`
    - Supports emptiness checking. Does not support further selector nesting.
`map[string]*`
    - Supports in/contains operations on the keys.
`map[string]<supported type>`
    - Will have a single subfield with name `FieldNameAny` (wildcard) and the rest of
      the field configuration will come from the `<supported type>`
`[]*`
    - Supports emptiness checking only. Does not support further selector nesting.
`[]<supported primitive type>`
    - Supports in/contains operations against the primitive values.
`[]<supported compund type>`
    - Will have subfields with the configuration of whatever the supported
      compound type is.
    - Does not support indexing of individual values like a map does currently
      and with the current evaluation logic slices of slices will mostly be
      handled as if they were flattened. One thing that cannot be done is
      to be able to perform emptiness/contains checking against the internal
      slice.
structs
    - No operations are supported on the struct itself
    - Will have subfield configurations generated for the fields of the struct.
    - A struct tag like `bexpr:"<name>"` allows changing the name that allows indexing
      into the subfield.
    - By default unexported fields of a struct are not selectable. If The struct tag is
      present then this behavior is overridden.
    - Exported fields can be made unselectable by adding a tag to the field like `bexpr:"-"`

func (FieldConfigurations) String 

func (configs FieldConfigurations) String() string

type FieldName 

type FieldName string

Strongly typed name of a field 

const FieldNameAny FieldName = ""

Used to represent an arbitrary field name

type FieldValueCoercionFn 

type FieldValueCoercionFn func(value string) (interface{}, error)

Function type for usage with a SelectorConfiguration

type Filter 

type Filter struct {
	// contains filtered or unexported fields
}

func CreateFilter 

func CreateFilter(expression string, config *EvaluatorConfig, dataType interface{}) (*Filter, error)

Creates a filter to operate on the given data type. The data type passed can be either be a container type (map, slice or array) or the element type. For example, if you want to filter a []Foo then the data type to pass here is either []Foo or just Foo. If no expression is provided the nil filter will be returned but is not an error. This is done to allow for executing the nil filter which is just a no-op

func (*Filter) Execute 

func (f *Filter) Execute(data interface{}) (interface{}, error)

Execute the filter. If called on a nil filter this is a no-op and will return the original data

type MatchExpression 

type MatchExpression struct {
	Selector Selector
	Operator MatchOperator
	Value    *MatchValue
}

func (*MatchExpression) ExpressionDump 

func (expr *MatchExpression) ExpressionDump(w io.Writer, indent string, level int)

type MatchExpressionEvaluator 

type MatchExpressionEvaluator interface {
	// FieldConfigurations returns the configuration for this field and any subfields
	// it may have. It must be valid to call this method on nil.
	FieldConfigurations() FieldConfigurations

	// EvaluateMatch returns whether there was a match or not. We are not also
	// expecting any errors because all the validation bits are handled
	// during parsing and cross checking against the output of FieldConfigurations.
	EvaluateMatch(sel Selector, op MatchOperator, value interface{}) (bool, error)
}

MatchExpressionEvaluator is the interface to implement to provide custom evaluation logic for a selector. This could be used to enable synthetic fields or other more complex logic that the default behavior does not support

type MatchOperator 

type MatchOperator int
const (
	MatchEqual MatchOperator = iota
	MatchNotEqual
	MatchIn
	MatchNotIn
	MatchIsEmpty
	MatchIsNotEmpty
)

func (MatchOperator) String 

func (op MatchOperator) String() string

type MatchValue 

type MatchValue struct {
	Raw       string
	Converted interface{}
}

type Option 

type Option func(*parser) Option

Option is a function that can set an option on the parser. It returns the previous setting as an Option.

func AllowInvalidUTF8 

func AllowInvalidUTF8(b bool) Option

AllowInvalidUTF8 creates an Option to allow invalid UTF-8 bytes. Every invalid UTF-8 byte is treated as a utf8.RuneError (U+FFFD) by character class matchers and is matched by the any matcher. The returned matched value, c.text and c.offset are NOT affected.

The default is false.

func Entrypoint 

func Entrypoint(ruleName string) Option

Entrypoint creates an Option to set the rule name to use as entrypoint. The rule name must have been specified in the -alternate-entrypoints if generating the parser with the -optimize-grammar flag, otherwise it may have been optimized out. Passing an empty string sets the entrypoint to the first rule in the grammar.

The default is to start parsing at the first rule in the grammar.

func GlobalStore 

func GlobalStore(key string, value interface{}) Option

GlobalStore creates an Option to set a key to a certain value in the globalStore.

func MaxExpressions 

func MaxExpressions(maxExprCnt uint64) Option

MaxExpressions creates an Option to stop parsing after the provided number of expressions have been parsed, if the value is 0 then the parser will parse for as many steps as needed (possibly an infinite number).

The default for maxExprCnt is 0.

func Recover 

func Recover(b bool) Option

Recover creates an Option to set the recover flag to b. When set to true, this causes the parser to recover from panics and convert it to an error. Setting it to false can be useful while debugging to access the full stack trace.

The default is true.

type Registry 

type Registry interface {
	GetFieldConfigurations(reflect.Type) (FieldConfigurations, error)
}
var DefaultRegistry Registry = NewSyncRegistry()

type Selector 

type Selector []string

func (Selector) String 

func (sel Selector) String() string

type Stats 

type Stats struct {
	// ExprCnt counts the number of expressions processed during parsing
	// This value is compared to the maximum number of expressions allowed
	// (set by the MaxExpressions option).
	ExprCnt uint64

	// ChoiceAltCnt is used to count for each ordered choice expression,
	// which alternative is used how may times.
	// These numbers allow to optimize the order of the ordered choice expression
	// to increase the performance of the parser
	//
	// The outer key of ChoiceAltCnt is composed of the name of the rule as well
	// as the line and the column of the ordered choice.
	// The inner key of ChoiceAltCnt is the number (one-based) of the matching alternative.
	// For each alternative the number of matches are counted. If an ordered choice does not
	// match, a special counter is incremented. The name of this counter is set with
	// the parser option Statistics.
	// For an alternative to be included in ChoiceAltCnt, it has to match at least once.
	ChoiceAltCnt map[string]map[string]int
}

Stats stores some statistics, gathered during parsing

type SyncRegistry 

type SyncRegistry struct {
	// contains filtered or unexported fields
}

func NewSyncRegistry 

func NewSyncRegistry() *SyncRegistry

func (*SyncRegistry) GetFieldConfigurations 

func (r *SyncRegistry) GetFieldConfigurations(rtype reflect.Type) (FieldConfigurations, error)

type UnaryExpression 

type UnaryExpression struct {
	Operator UnaryOperator
	Operand  Expression
}

func (*UnaryExpression) ExpressionDump 

func (expr *UnaryExpression) ExpressionDump(w io.Writer, indent string, level int)

type UnaryOperator 

type UnaryOperator int
const (
	UnaryOpNot UnaryOperator = iota
)

func (UnaryOperator) String 

func (op UnaryOperator) String() string

Source Files

View all Source files

Directories

Path Synopsis
examples
expr-eval command
expr-parse command
filter command
simple command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.