kvql

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 21, 2024 License: Apache-2.0 Imports: 13 Imported by: 2

README

kvql

GoDoc

A SQL-like query language on general Key-Value DB

Syntax

Basic Types:

Number: number such as integer or float

String: string around by ', ", \`,

Boolean: true or false

Select Statement:

SelectStmt ::= "SELECT" Fields "WHERE" WhereConditions ("ORDER" "BY" OrderByFields)? ("GROUP" "BY" GroupByFields)? ("LIMIT" LimitParameter)?

Fields ::= Field (, Field)* |
           "*"

Field ::= Expression ("AS" FieldName)?

FieldName ::= String

OrderByFields ::= OrderByField (, OrderByField)*

OrderByField ::= FieldName ("ASC" | "DESC")*

GroupByFields ::= FieldName (, FieldName)*

LimitParameter ::= Number "," Number |
                   Number

WhereConditions ::= "!"? Expression

Expression ::= "("? BinaryExpression | UnaryExpression ")"?

UnaryExpression ::= KeyValueField | String | Number | Boolean | FunctionCall | FieldName

BinaryExpression ::= Expression Operator Expression |
                     Expression "BETWEEN" Expression "AND" Expression |
                     Expression "IN" "(" Expression (, Expression)* ")" |
                     Expression "IN" FunctionCall

Operator ::= MathOperator | CompareOperator | AndOrOperator

AndOrOperator ::= "&" | "|" | "AND" | "OR"

MathOperator ::= "+" | "-" | "*" | "/"

CompareOperator ::= "=" | "!=" | "^=" | "~=" | ">" | ">=" | "<" | "<="

KeyValueField ::= "KEY" | "VALUE"

FunctionCall ::= FunctionName "(" FunctionArgs ")" |
                 FunctionName "(" FunctionArgs ")" FieldAccessExpression*

FunctionName ::= String

FunctionArgs ::= FunctionArg ("," FunctionArg)*

FunctionArg ::= Expression

FieldAccessExpression ::= "[" String "]" |
                          "[" Number "]"

Put Statement:

PutStmt ::= "PUT" KVPair (, KVPair)*
KVPair ::= "(" Expression, Expression ")"

Delete Statement:

DeleteStmt ::= "DELETE" "WHERE" WhereConditions ("LIMIT" LimitParameter)?

Features:

  1. Scan ranger optimize: EmptyResult, PrefixScan, RangeScan, MultiGet
  2. Plan support Volcano model and Batch model
  3. Expression constant folding
  4. Support scalar function and aggregate function
  5. Support hash aggregate plan
  6. Support JSON and field access expression

Known User

Examples:

# Simple query, get all the key-value pairs with key prefix 'k'
select * where key ^= 'k'

# Projection and complex condition
select key, int(value) + 1 where key in ('k1', 'k2', 'k3') & is_int(value)

# Aggregation query
select count(1), sum(int(value)) as sum, substr(key, 0, 2) as kprefix where key between 'k' and 'l' group by kprefix order by sum desc

# JSON access
select key, json(value)['x']['y'] where key ^= 'k' & int(json(value)['test']) >= 1
select key, json(value)['list'][1] where key ^= 'k'

# Filter by field name defined in select statement
select key, int(value) as f1 where f1 > 10
select key, split(value) as f1 where 'a' in f1
select key, value, l2_distance(list(1,2,3,4), json(value)) as l2_dis where key ^= 'embedding_json' & l2_dis > 0.6 order by l2_dis desc limit 5

# Put data
put ('k1', 'v1'), ('k2', upper('v' + key))

# Delete data by filter and limit delete rows
delete where key ^= 'prefix' and value ~= '^val_' limit 10
delete where key in ('k1', 'k2', 'k3')

How to use this library

A full example:

https://github.com/c4pt0r/kvql/blob/master/examples/memkv/memkv.go

To get better error report, you can conver the error to QueryBinder and set the origin query like below:

...
opt := kvql.NewOptimizer(query)
plan, err := opt.BuildPlan(storage)
if err != nil {
	if qerr, ok := err.(kvql.QueryBinder); ok {
		qerr.BindQuery(query)
	}
	fmt.Printf("Error: %s\n", err.Error())
}
...

After bind the query to error it will output error result like:

padding                    query
v-----vv--------------------------------------------v
Error: select * where key ^= 'asdf' and val ^= 'test'          < query line
                                        ^--                    < error position
       Syntax Error: ^= operator with invalid left expression  < error message

About padding: user can use kvql.DefaultErrorPadding to change the default left padding spaces. Or can use kvql.QueryBinder.SetPadding function to change specify error's padding. The default padding is 7 space characters (length of Error: ).

If you want to display the plan tree, like EXPLAIN statement in SQL, the kvql.FinalPlan.Explain function will return the plan tree in a string list, you can use below code to format the explain output:

...
opt := kvql.NewOptimizer(query)
plan, err := opt.BuildPlan(storage)
if err != nil {
	fatal(err)
}

output := ""
for i, plan := range plan.Explain() {
	padding := ""
	for x := 0; x < i*3; x++ {
		padding += " "
	}
	if i == 0 {
		output += fmt.Sprintf("%s%s\n", padding, plan)
	} else {
		output += fmt.Sprintf("%s`-%s\n", padding, plan)
	}
}
fmt.Println(output)

Operators and Functions

Operators

Conparation operators

  • =: bytes level equals
  • !=: bytes level not equals
  • ^=: prefix match
  • ~=: regexp match
  • >: number or string greater than
  • >=: number or string greater or equals than
  • <: number or string less than
  • <=: number or string less or equals than
  • BETWEEN x AND y: great or equals than x and less or equals than y
  • IN (...): in list followed by in operator

Logical operators

  • &, AND: logical and
  • |, OR: logical or
  • !: logical not

Math operators

  • +: number add or string concate
  • -: number subtraction
  • *: number multiply
  • /: number division
Scalar Functions
Function Description
lower(value: str): str convert value string into lower case
upper(value: str): str convert value string into upper case
int(value: any): int convert value into integer, if cannot convert to integer just return error
float(value: any): float convert value into float, if cannot convert to float just return error
str(value: any): str convert value into string
strlen(value: any): int convert value into string and then calculate string length
is_int(value: any): bool return is value can be converted into integer
is_float(value: any): bool return is value can be converted into float
substr(value: str, start: int, end: int): str return substring of value from start position to end position
split(value: str, spliter: str): list split value into a string list by spliter string
list(elem1: any, elem2: any...): list convert many elements into a list, list elements' type must be same, the list type support int, str, float types
float_list(elem1: float, elem2: float...): list convert many float elements into a list
flist(elem1: float, elem2: float...): list same as float_list
int_list(elem1: int, elem2: int...): list convert many integer elements into a list
ilist(elem1: int, elem2: int...): list same as int_list
len(value: list): int return value list length
l2_distance(left: list, right: list): float calculate l2 distance of two list
cosine_distance(left: list, right: list): float calculate cosine distance of two list
json(value: str): json parse string value into json type
join(seperator: str, val1: any, val2: any...): str join values by seperator
Aggregation Functions
Function Description
count(value: int): int Count value by group
sum(value: int): int Sum value by group
avg(value: int): int Calculate average value by group
min(value: int): int Find the minimum value by group
max(value: int): int Find the maxmum value by group
quantile(value: float, percent: float): float Calculate the Quantile by group
json_arrayagg(value: any): string Aggregate all values into a JSON array
group_concat(value: any, seperator: str): string Join all values into a string by seperator

Documentation

Index

Constants

View Source 
const (
	KeyKW   KVKeyword = 1
	ValueKW KVKeyword = 2

	Unknown     Operator = 0
	And         Operator = 1
	Or          Operator = 2
	Not         Operator = 3
	Eq          Operator = 4
	NotEq       Operator = 5
	PrefixMatch Operator = 6
	RegExpMatch Operator = 7
	Add         Operator = 8
	Sub         Operator = 9
	Mul         Operator = 10
	Div         Operator = 11
	Gt          Operator = 12
	Gte         Operator = 13
	Lt          Operator = 14
	Lte         Operator = 15
	In          Operator = 16
	Between     Operator = 17
	KWAnd       Operator = 18
	KWOr        Operator = 19

	TUNKNOWN Type = 0
	TBOOL    Type = 1
	TSTR     Type = 2
	TNUMBER  Type = 3
	TIDENT   Type = 4
	TLIST    Type = 5
	TJSON    Type = 6
)
View Source 
const (
	EMPTY  byte = 1
	MGET   byte = 2
	PREFIX byte = 3
	RANGE  byte = 4
	FULL   byte = 5
)

* Scan Type is priority of scan operator * Lower value operator means the result set is smaller than higher value operator

View Source 
const (
	LowestPrec  = 0 // non-operators
	UnaryPrec   = 6
	HighestPrec = 7
)
View Source 
const MaxNestLevel = 1e5

Variables

View Source 
var (
	KVKeywordToString = map[KVKeyword]string{
		KeyKW:   "KEY",
		ValueKW: "VALUE",
	}

	OperatorToString = map[Operator]string{
		Eq:          "=",
		NotEq:       "!=",
		And:         "&",
		Or:          "|",
		Not:         "!",
		PrefixMatch: "^=",
		RegExpMatch: "~=",
		Add:         "+",
		Sub:         "-",
		Mul:         "*",
		Div:         "/",
		Gt:          ">",
		Gte:         ">=",
		Lt:          "<",
		Lte:         "<=",
		In:          "in",
		Between:     "between",
		KWAnd:       "and",
		KWOr:        "or",
	}

	StringToOperator = map[string]Operator{
		"=":       Eq,
		"&":       And,
		"|":       Or,
		"!":       Not,
		"^=":      PrefixMatch,
		"~=":      RegExpMatch,
		"!=":      NotEq,
		"+":       Add,
		"-":       Sub,
		"*":       Mul,
		"/":       Div,
		">":       Gt,
		">=":      Gte,
		"<":       Lt,
		"<=":      Lte,
		"in":      In,
		"between": Between,
		"and":     KWAnd,
		"or":      KWOr,
	}
)
View Source 
var (
	PlanBatchSize    = 32
	EnableFieldCache = true
)
View Source 
var (
	DefaultErrorPadding = 7
)
View Source 
var (
	TokenTypeToString = map[TokenType]string{
		SELECT:   "SELECT",
		WHERE:    "WHERE",
		KEY:      "KEY",
		VALUE:    "VALUE",
		OPERATOR: "OP",
		STRING:   "STR",
		LPAREN:   "(",
		RPAREN:   ")",
		NAME:     "NAME",
		SEP:      "SEP",
		NUMBER:   "NUM",
		FLOAT:    "FLOAT",
		LIMIT:    "LIMIT",
		ORDER:    "ORDER",
		BY:       "BY",
		ASC:      "ASC",
		DESC:     "DESC",
		TRUE:     "true",
		FALSE:    "false",
		AS:       "AS",
		GROUP:    "GROUP",
		IN:       "IN",
		BETWEEN:  "BETWEEN",
		AND:      "AND",
		LBRACK:   "[",
		RBRACK:   "]",
		PUT:      "PUT",
		REMOVE:   "REMOVE",
		SEMI:     "SEMI",
		OR:       "OR",
		DELETE:   "DELETE",
	}
)

Functions

func AddAggrFunction 

func AddAggrFunction(f *AggrFunc)

func AddScalarFunction 

func AddScalarFunction(f *Function)

func BuildExecutor 

func BuildExecutor(query string) (*SelectStmt, *FilterExec, error)

func GetFuncNameFromExpr 

func GetFuncNameFromExpr(expr Expression) (string, error)

func IsAggrFunc 

func IsAggrFunc(fname string) bool

func IsAggrFuncExpr 

func IsAggrFuncExpr(expr Expression) bool

func IsScalarFuncExpr 

func IsScalarFuncExpr(expr Expression) bool

func NewExecuteError 

func NewExecuteError(pos int, msg string, args ...any) error

func NewSyntaxError 

func NewSyntaxError(pos int, msg string, args ...any) error

func ScanTypeToString 

func ScanTypeToString(tp byte) string

Types

type AggrFunc 

type AggrFunc struct {
	Name       string
	NumArgs    int
	VarArgs    bool
	ReturnType Type
	Body       AggrFunctor
}

func GetAggrFunctionByName 

func GetAggrFunctionByName(name string) (*AggrFunc, bool)

type AggrFunction 

type AggrFunction interface {
	Update(kv KVPair, args []Expression, ctx *ExecuteCtx) error
	Complete() (any, error)
	Clone() AggrFunction
}

type AggrFunctor 

type AggrFunctor func(args []Expression) (AggrFunction, error)

type AggrPlanField 

type AggrPlanField struct {
	ID        int
	Name      string
	IsKey     bool
	Expr      Expression
	FuncExprs []*FunctionCallExpr
	Funcs     []AggrFunction
	Value     Column
}

type AggregatePlan 

type AggregatePlan struct {
	Storage       Storage
	ChildPlan     Plan
	FieldNames    []string
	FieldTypes    []Type
	Fields        []Expression
	GroupByFields []GroupByField
	AggrAll       bool
	Limit         int
	Start         int
	// contains filtered or unexported fields
}

func (*AggregatePlan) Batch 

func (a *AggregatePlan) Batch(ctx *ExecuteCtx) ([][]Column, error)

func (*AggregatePlan) Explain 

func (a *AggregatePlan) Explain() []string

func (*AggregatePlan) FieldNameList 

func (a *AggregatePlan) FieldNameList() []string

func (*AggregatePlan) FieldTypeList 

func (a *AggregatePlan) FieldTypeList() []Type

func (*AggregatePlan) Init 

func (a *AggregatePlan) Init() error

func (*AggregatePlan) Next 

func (a *AggregatePlan) Next(ctx *ExecuteCtx) ([]Column, error)

func (*AggregatePlan) String 

func (a *AggregatePlan) String() string

type BinaryOpExpr 

type BinaryOpExpr struct {
	Pos   int
	Op    Operator
	Left  Expression
	Right Expression
}

func (*BinaryOpExpr) Check 

func (e *BinaryOpExpr) Check(ctx *CheckCtx) error

func (*BinaryOpExpr) Execute 

func (e *BinaryOpExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*BinaryOpExpr) ExecuteBatch 

func (e *BinaryOpExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*BinaryOpExpr) GetPos 

func (e *BinaryOpExpr) GetPos() int

func (*BinaryOpExpr) ReturnType 

func (e *BinaryOpExpr) ReturnType() Type

func (*BinaryOpExpr) String 

func (e *BinaryOpExpr) String() string

func (*BinaryOpExpr) Walk 

func (e *BinaryOpExpr) Walk(cb WalkCallback)

type BoolExpr 

type BoolExpr struct {
	Pos  int
	Data string
	Bool bool
}

func (*BoolExpr) Check 

func (e *BoolExpr) Check(ctx *CheckCtx) error

func (*BoolExpr) Execute 

func (e *BoolExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*BoolExpr) ExecuteBatch 

func (e *BoolExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*BoolExpr) GetPos 

func (e *BoolExpr) GetPos() int

func (*BoolExpr) ReturnType 

func (e *BoolExpr) ReturnType() Type

func (*BoolExpr) String 

func (e *BoolExpr) String() string

func (*BoolExpr) Walk 

func (e *BoolExpr) Walk(cb WalkCallback)

type CheckCtx 

type CheckCtx struct {
	Fields        []Expression
	FieldNames    []string
	FieldTypes    []Type
	NotAllowKey   bool
	NotAllowValue bool
}

func (*CheckCtx) GetNamedExpr 

func (c *CheckCtx) GetNamedExpr(name string) (Expression, bool)

type Column 

type Column any

type Cursor 

type Cursor interface {
	Seek(prefix []byte) error
	Next() (key []byte, value []byte, err error)
}

type DeletePlan 

type DeletePlan struct {
	Storage   Storage
	ChildPlan Plan
	// contains filtered or unexported fields
}

func (*DeletePlan) Batch 

func (p *DeletePlan) Batch(ctx *ExecuteCtx) ([][]Column, error)

func (*DeletePlan) Explain 

func (p *DeletePlan) Explain() []string

func (*DeletePlan) FieldNameList 

func (p *DeletePlan) FieldNameList() []string

func (*DeletePlan) FieldTypeList 

func (p *DeletePlan) FieldTypeList() []Type

func (*DeletePlan) Init 

func (p *DeletePlan) Init() error

func (*DeletePlan) Next 

func (p *DeletePlan) Next(ctx *ExecuteCtx) ([]Column, error)

func (*DeletePlan) String 

func (p *DeletePlan) String() string

type DeleteStmt 

type DeleteStmt struct {
	Pos   int
	Where *WhereStmt
	Limit *LimitStmt
}

func (*DeleteStmt) Name 

func (s *DeleteStmt) Name() string

func (*DeleteStmt) Validate 

func (s *DeleteStmt) Validate(ctx *CheckCtx) error

type EmptyResultPlan 

type EmptyResultPlan struct {
	Storage Storage
}

func (*EmptyResultPlan) Batch 

func (p *EmptyResultPlan) Batch(ctx *ExecuteCtx) ([]KVPair, error)

func (*EmptyResultPlan) Explain 

func (p *EmptyResultPlan) Explain() []string

func (*EmptyResultPlan) Init 

func (p *EmptyResultPlan) Init() error

func (*EmptyResultPlan) Next 

func (p *EmptyResultPlan) Next(ctx *ExecuteCtx) ([]byte, []byte, error)

func (*EmptyResultPlan) String 

func (p *EmptyResultPlan) String() string

type ExecuteCtx 

type ExecuteCtx struct {
	Hit                 int
	EnableCache         bool
	FieldCaches         map[string]any
	FieldChunkKeyCaches map[string][]any
	FieldChunkCaches    map[string][]any
}

func NewExecuteCtx 

func NewExecuteCtx() *ExecuteCtx

func (*ExecuteCtx) AdjustChunkCache 

func (c *ExecuteCtx) AdjustChunkCache(chooseIdxes []int)

func (*ExecuteCtx) AppendChunkFieldResult 

func (c *ExecuteCtx) AppendChunkFieldResult(name string, chunk []any)

func (*ExecuteCtx) Clear 

func (c *ExecuteCtx) Clear()

func (*ExecuteCtx) GetChunkFieldFinalResult 

func (c *ExecuteCtx) GetChunkFieldFinalResult(name string) ([]any, bool)

func (*ExecuteCtx) GetChunkFieldResult 

func (c *ExecuteCtx) GetChunkFieldResult(name string, key []byte) ([]any, bool)

func (*ExecuteCtx) GetFieldResult 

func (c *ExecuteCtx) GetFieldResult(name string) (any, bool)

func (*ExecuteCtx) SetChunkFieldResult 

func (c *ExecuteCtx) SetChunkFieldResult(name string, key []byte, chunk []any)

func (*ExecuteCtx) SetFieldResult 

func (c *ExecuteCtx) SetFieldResult(name string, value any)

func (*ExecuteCtx) UpdateHit 

func (c *ExecuteCtx) UpdateHit()

type ExecuteError 

type ExecuteError struct {
	Query   string
	Message string
	Pos     int
	Padding int
}

func (*ExecuteError) BindQuery 

func (e *ExecuteError) BindQuery(query string)

func (*ExecuteError) Error 

func (e *ExecuteError) Error() string

func (*ExecuteError) SetPadding 

func (e *ExecuteError) SetPadding(pad int)

type Expression 

type Expression interface {
	Check(ctx *CheckCtx) error
	String() string
	Execute(kv KVPair, ctx *ExecuteCtx) (any, error)
	ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)
	ReturnType() Type
	GetPos() int
	Walk(cb WalkCallback)
}

type ExpressionOptimizer 

type ExpressionOptimizer struct {
	Root Expression
	// contains filtered or unexported fields
}

func (*ExpressionOptimizer) Optimize 

func (o *ExpressionOptimizer) Optimize() Expression

type FieldAccessExpr 

type FieldAccessExpr struct {
	Pos       int
	Left      Expression
	FieldName Expression
}

func (*FieldAccessExpr) Check 

func (e *FieldAccessExpr) Check(ctx *CheckCtx) error

func (*FieldAccessExpr) Execute 

func (e *FieldAccessExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*FieldAccessExpr) ExecuteBatch 

func (e *FieldAccessExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*FieldAccessExpr) GetPos 

func (e *FieldAccessExpr) GetPos() int

func (*FieldAccessExpr) ReturnType 

func (e *FieldAccessExpr) ReturnType() Type

func (*FieldAccessExpr) String 

func (e *FieldAccessExpr) String() string

func (*FieldAccessExpr) Walk 

func (e *FieldAccessExpr) Walk(cb WalkCallback)

type FieldExpr 

type FieldExpr struct {
	Pos   int
	Field KVKeyword
}

func (*FieldExpr) Check 

func (e *FieldExpr) Check(ctx *CheckCtx) error

func (*FieldExpr) Execute 

func (e *FieldExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*FieldExpr) ExecuteBatch 

func (e *FieldExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*FieldExpr) GetPos 

func (e *FieldExpr) GetPos() int

func (*FieldExpr) ReturnType 

func (e *FieldExpr) ReturnType() Type

func (*FieldExpr) String 

func (e *FieldExpr) String() string

func (*FieldExpr) Walk 

func (e *FieldExpr) Walk(cb WalkCallback)

type FieldReferenceExpr 

type FieldReferenceExpr struct {
	Name      *NameExpr
	FieldExpr Expression
}

func (*FieldReferenceExpr) Check 

func (e *FieldReferenceExpr) Check(ctx *CheckCtx) error

func (*FieldReferenceExpr) Execute 

func (e *FieldReferenceExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*FieldReferenceExpr) ExecuteBatch 

func (e *FieldReferenceExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*FieldReferenceExpr) GetPos 

func (e *FieldReferenceExpr) GetPos() int

func (*FieldReferenceExpr) ReturnType 

func (e *FieldReferenceExpr) ReturnType() Type

func (*FieldReferenceExpr) String 

func (e *FieldReferenceExpr) String() string

func (*FieldReferenceExpr) Walk 

func (e *FieldReferenceExpr) Walk(cb WalkCallback)

type FilterExec 

type FilterExec struct {
	Ast *WhereStmt
}

func (*FilterExec) Explain 

func (e *FilterExec) Explain() string

func (*FilterExec) Filter 

func (e *FilterExec) Filter(kvp KVPair, ctx *ExecuteCtx) (bool, error)

func (*FilterExec) FilterBatch 

func (e *FilterExec) FilterBatch(chunk []KVPair, ctx *ExecuteCtx) ([]bool, error)

type FilterOptimizer 

type FilterOptimizer struct {
	// contains filtered or unexported fields
}

func NewFilterOptimizer 

func NewFilterOptimizer(ast *WhereStmt, s Storage, filter *FilterExec) *FilterOptimizer

func (*FilterOptimizer) Optimize 

func (o *FilterOptimizer) Optimize() Plan

type FinalLimitPlan 

type FinalLimitPlan struct {
	Storage Storage
	Start   int
	Count   int

	ChildPlan  FinalPlan
	FieldNames []string
	FieldTypes []Type
	// contains filtered or unexported fields
}

func (*FinalLimitPlan) Batch 

func (p *FinalLimitPlan) Batch(ctx *ExecuteCtx) ([][]Column, error)

func (*FinalLimitPlan) Explain 

func (p *FinalLimitPlan) Explain() []string

func (*FinalLimitPlan) FieldNameList 

func (p *FinalLimitPlan) FieldNameList() []string

func (*FinalLimitPlan) FieldTypeList 

func (p *FinalLimitPlan) FieldTypeList() []Type

func (*FinalLimitPlan) Init 

func (p *FinalLimitPlan) Init() error

func (*FinalLimitPlan) Next 

func (p *FinalLimitPlan) Next(ctx *ExecuteCtx) ([]Column, error)

func (*FinalLimitPlan) String 

func (p *FinalLimitPlan) String() string

type FinalOrderPlan 

type FinalOrderPlan struct {
	Storage    Storage
	Orders     []OrderField
	FieldNames []string
	FieldTypes []Type
	ChildPlan  FinalPlan
	// contains filtered or unexported fields
}

func (*FinalOrderPlan) Batch 

func (p *FinalOrderPlan) Batch(ctx *ExecuteCtx) ([][]Column, error)

func (*FinalOrderPlan) Explain 

func (p *FinalOrderPlan) Explain() []string

func (*FinalOrderPlan) FieldNameList 

func (p *FinalOrderPlan) FieldNameList() []string

func (*FinalOrderPlan) FieldTypeList 

func (p *FinalOrderPlan) FieldTypeList() []Type

func (*FinalOrderPlan) Init 

func (p *FinalOrderPlan) Init() error

func (*FinalOrderPlan) Next 

func (p *FinalOrderPlan) Next(ctx *ExecuteCtx) ([]Column, error)

func (*FinalOrderPlan) String 

func (p *FinalOrderPlan) String() string

type FinalPlan 

type FinalPlan interface {
	String() string
	Explain() []string
	Init() error
	Next(ctx *ExecuteCtx) ([]Column, error)
	Batch(ctx *ExecuteCtx) ([][]Column, error)
	FieldNameList() []string
	FieldTypeList() []Type
}

type FloatExpr 

type FloatExpr struct {
	Pos   int
	Data  string
	Float float64
}

func (*FloatExpr) Check 

func (e *FloatExpr) Check(ctx *CheckCtx) error

func (*FloatExpr) Execute 

func (e *FloatExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*FloatExpr) ExecuteBatch 

func (e *FloatExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*FloatExpr) GetPos 

func (e *FloatExpr) GetPos() int

func (*FloatExpr) ReturnType 

func (e *FloatExpr) ReturnType() Type

func (*FloatExpr) String 

func (e *FloatExpr) String() string

func (*FloatExpr) Walk 

func (e *FloatExpr) Walk(cb WalkCallback)

type FullScanPlan 

type FullScanPlan struct {
	Storage Storage
	Filter  *FilterExec
	// contains filtered or unexported fields
}

func (*FullScanPlan) Batch 

func (p *FullScanPlan) Batch(ctx *ExecuteCtx) ([]KVPair, error)

func (*FullScanPlan) Explain 

func (p *FullScanPlan) Explain() []string

func (*FullScanPlan) Init 

func (p *FullScanPlan) Init() (err error)

func (*FullScanPlan) Next 

func (p *FullScanPlan) Next(ctx *ExecuteCtx) ([]byte, []byte, error)

func (*FullScanPlan) String 

func (p *FullScanPlan) String() string

type Function 

type Function struct {
	Name       string
	NumArgs    int
	VarArgs    bool
	ReturnType Type
	Body       FunctionBody
	BodyVec    VectorFunctionBody
}

func GetScalarFunction 

func GetScalarFunction(expr Expression) (*Function, error)

func GetScalarFunctionByName 

func GetScalarFunctionByName(name string) (*Function, bool)

type FunctionBody 

type FunctionBody func(kv KVPair, args []Expression, ctx *ExecuteCtx) (any, error)

type FunctionCallExpr 

type FunctionCallExpr struct {
	Pos    int
	Name   Expression
	Args   []Expression
	Result any
}

func (*FunctionCallExpr) Check 

func (e *FunctionCallExpr) Check(ctx *CheckCtx) error

func (*FunctionCallExpr) Execute 

func (e *FunctionCallExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*FunctionCallExpr) ExecuteBatch 

func (e *FunctionCallExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*FunctionCallExpr) GetPos 

func (e *FunctionCallExpr) GetPos() int

func (*FunctionCallExpr) ReturnType 

func (e *FunctionCallExpr) ReturnType() Type

func (*FunctionCallExpr) String 

func (e *FunctionCallExpr) String() string

func (*FunctionCallExpr) Walk 

func (e *FunctionCallExpr) Walk(cb WalkCallback)

type GroupByField 

type GroupByField struct {
	Name string
	Expr Expression
}

type GroupByStmt 

type GroupByStmt struct {
	Pos    int
	Fields []GroupByField
}

func (*GroupByStmt) Name 

func (s *GroupByStmt) Name() string

type JSON 

type JSON map[string]any

type KVKeyword 

type KVKeyword byte

type KVPair 

type KVPair struct {
	Key   []byte
	Value []byte
}

func NewKVP 

func NewKVP(key []byte, val []byte) KVPair

func NewKVPStr 

func NewKVPStr(key string, val string) KVPair

type Lexer 

type Lexer struct {
	Query  string
	Length int
}

func NewLexer 

func NewLexer(query string) *Lexer

func (*Lexer) Split 

func (l *Lexer) Split() []*Token

type LimitPlan 

type LimitPlan struct {
	Storage Storage
	Start   int
	Count   int

	ChildPlan Plan
	// contains filtered or unexported fields
}

func (*LimitPlan) Batch 

func (p *LimitPlan) Batch(ctx *ExecuteCtx) ([]KVPair, error)

func (*LimitPlan) Explain 

func (p *LimitPlan) Explain() []string

func (*LimitPlan) Init 

func (p *LimitPlan) Init() error

func (*LimitPlan) Next 

func (p *LimitPlan) Next(ctx *ExecuteCtx) ([]byte, []byte, error)

func (*LimitPlan) String 

func (p *LimitPlan) String() string

type LimitStmt 

type LimitStmt struct {
	Pos   int
	Start int
	Count int
}

func (*LimitStmt) Name 

func (s *LimitStmt) Name() string

type ListExpr 

type ListExpr struct {
	Pos  int
	List []Expression
}

func (*ListExpr) Check 

func (e *ListExpr) Check(ctx *CheckCtx) error

func (*ListExpr) Execute 

func (e *ListExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*ListExpr) ExecuteBatch 

func (e *ListExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*ListExpr) GetPos 

func (e *ListExpr) GetPos() int

func (*ListExpr) ReturnType 

func (e *ListExpr) ReturnType() Type

func (*ListExpr) String 

func (e *ListExpr) String() string

func (*ListExpr) Walk 

func (e *ListExpr) Walk(cb WalkCallback)

type MultiGetPlan 

type MultiGetPlan struct {
	Storage Storage
	Filter  *FilterExec
	Keys    []string
	// contains filtered or unexported fields
}

func (*MultiGetPlan) Batch 

func (p *MultiGetPlan) Batch(ctx *ExecuteCtx) ([]KVPair, error)

func (*MultiGetPlan) Explain 

func (p *MultiGetPlan) Explain() []string

func (*MultiGetPlan) Init 

func (p *MultiGetPlan) Init() error

func (*MultiGetPlan) Next 

func (p *MultiGetPlan) Next(ctx *ExecuteCtx) ([]byte, []byte, error)

func (*MultiGetPlan) String 

func (p *MultiGetPlan) String() string

type NameExpr 

type NameExpr struct {
	Pos  int
	Data string
}

func (*NameExpr) Check 

func (e *NameExpr) Check(ctx *CheckCtx) error

func (*NameExpr) Execute 

func (e *NameExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*NameExpr) ExecuteBatch 

func (e *NameExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*NameExpr) GetPos 

func (e *NameExpr) GetPos() int

func (*NameExpr) ReturnType 

func (e *NameExpr) ReturnType() Type

func (*NameExpr) String 

func (e *NameExpr) String() string

func (*NameExpr) Walk 

func (e *NameExpr) Walk(cb WalkCallback)

type NotExpr 

type NotExpr struct {
	Pos   int
	Right Expression
}

func (*NotExpr) Check 

func (e *NotExpr) Check(ctx *CheckCtx) error

func (*NotExpr) Execute 

func (e *NotExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*NotExpr) ExecuteBatch 

func (e *NotExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*NotExpr) GetPos 

func (e *NotExpr) GetPos() int

func (*NotExpr) ReturnType 

func (e *NotExpr) ReturnType() Type

func (*NotExpr) String 

func (e *NotExpr) String() string

func (*NotExpr) Walk 

func (e *NotExpr) Walk(cb WalkCallback)

type NumberExpr 

type NumberExpr struct {
	Pos  int
	Data string
	Int  int64
}

func (*NumberExpr) Check 

func (e *NumberExpr) Check(ctx *CheckCtx) error

func (*NumberExpr) Execute 

func (e *NumberExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*NumberExpr) ExecuteBatch 

func (e *NumberExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*NumberExpr) GetPos 

func (e *NumberExpr) GetPos() int

func (*NumberExpr) ReturnType 

func (e *NumberExpr) ReturnType() Type

func (*NumberExpr) String 

func (e *NumberExpr) String() string

func (*NumberExpr) Walk 

func (e *NumberExpr) Walk(cb WalkCallback)

type Operator 

type Operator byte

func BuildOp 

func BuildOp(pos int, op string) (Operator, error)

type Optimizer 

type Optimizer struct {
	Query string
	// contains filtered or unexported fields
}

func NewOptimizer 

func NewOptimizer(query string) *Optimizer

func (*Optimizer) BuildPlan 

func (o *Optimizer) BuildPlan(s Storage) (FinalPlan, error)

type OrderField 

type OrderField struct {
	Name  string
	Field Expression
	Order TokenType
}

type OrderStmt 

type OrderStmt struct {
	Pos    int
	Orders []OrderField
}

func (*OrderStmt) Name 

func (s *OrderStmt) Name() string

type Parser 

type Parser struct {
	Query string
	// contains filtered or unexported fields
}

func NewParser 

func NewParser(query string) *Parser

func (*Parser) Parse 

func (p *Parser) Parse() (Statement, error)

type Plan 

type Plan interface {
	String() string
	Explain() []string
	Init() error
	Next(ctx *ExecuteCtx) (key []byte, value []byte, err error)
	Batch(ctx *ExecuteCtx) (rows []KVPair, err error)
}

func NewEmptyResultPlan 

func NewEmptyResultPlan(s Storage, f *FilterExec) Plan

func NewFullScanPlan 

func NewFullScanPlan(s Storage, f *FilterExec) Plan

func NewMultiGetPlan 

func NewMultiGetPlan(s Storage, f *FilterExec, keys []string) Plan

func NewPrefixScanPlan 

func NewPrefixScanPlan(s Storage, f *FilterExec, p string) Plan

func NewRangeScanPlan 

func NewRangeScanPlan(s Storage, f *FilterExec, start []byte, end []byte) Plan

type PrefixScanPlan 

type PrefixScanPlan struct {
	Storage Storage
	Filter  *FilterExec
	Prefix  string
	// contains filtered or unexported fields
}

func (*PrefixScanPlan) Batch 

func (p *PrefixScanPlan) Batch(ctx *ExecuteCtx) ([]KVPair, error)

func (*PrefixScanPlan) Explain 

func (p *PrefixScanPlan) Explain() []string

func (*PrefixScanPlan) Init 

func (p *PrefixScanPlan) Init() (err error)

func (*PrefixScanPlan) Next 

func (p *PrefixScanPlan) Next(ctx *ExecuteCtx) ([]byte, []byte, error)

func (*PrefixScanPlan) String 

func (p *PrefixScanPlan) String() string

type ProjectionPlan 

type ProjectionPlan struct {
	Storage    Storage
	ChildPlan  Plan
	AllFields  bool
	FieldNames []string
	FieldTypes []Type
	Fields     []Expression
}

func (*ProjectionPlan) Batch 

func (p *ProjectionPlan) Batch(ctx *ExecuteCtx) ([][]Column, error)

func (*ProjectionPlan) Explain 

func (p *ProjectionPlan) Explain() []string

func (*ProjectionPlan) FieldNameList 

func (p *ProjectionPlan) FieldNameList() []string

func (*ProjectionPlan) FieldTypeList 

func (p *ProjectionPlan) FieldTypeList() []Type

func (*ProjectionPlan) Init 

func (p *ProjectionPlan) Init() error

func (*ProjectionPlan) Next 

func (p *ProjectionPlan) Next(ctx *ExecuteCtx) ([]Column, error)

func (*ProjectionPlan) String 

func (p *ProjectionPlan) String() string

type PutKVPair 

type PutKVPair struct {
	Key   Expression
	Value Expression
}

func (*PutKVPair) String 

func (p *PutKVPair) String() string

type PutPlan 

type PutPlan struct {
	Storage Storage
	KVPairs []*PutKVPair
	// contains filtered or unexported fields
}

func (*PutPlan) Batch 

func (p *PutPlan) Batch(ctx *ExecuteCtx) ([][]Column, error)

func (*PutPlan) Explain 

func (p *PutPlan) Explain() []string

func (*PutPlan) FieldNameList 

func (p *PutPlan) FieldNameList() []string

func (*PutPlan) FieldTypeList 

func (p *PutPlan) FieldTypeList() []Type

func (*PutPlan) Init 

func (p *PutPlan) Init() error

func (*PutPlan) Next 

func (p *PutPlan) Next(ctx *ExecuteCtx) ([]Column, error)

func (*PutPlan) String 

func (p *PutPlan) String() string

type PutStmt 

type PutStmt struct {
	Pos     int
	KVPairs []*PutKVPair
}

func (*PutStmt) Name 

func (s *PutStmt) Name() string

func (*PutStmt) Validate 

func (s *PutStmt) Validate(ctx *CheckCtx) error

type QueryBinder 

type QueryBinder interface {
	BindQuery(query string)
	SetPadding(pad int)
}

type RangeScanPlan 

type RangeScanPlan struct {
	Storage Storage
	Filter  *FilterExec
	Start   []byte
	End     []byte
	// contains filtered or unexported fields
}

func (*RangeScanPlan) Batch 

func (p *RangeScanPlan) Batch(ctx *ExecuteCtx) ([]KVPair, error)

func (*RangeScanPlan) Explain 

func (p *RangeScanPlan) Explain() []string

func (*RangeScanPlan) Init 

func (p *RangeScanPlan) Init() (err error)

func (*RangeScanPlan) Next 

func (p *RangeScanPlan) Next(ctx *ExecuteCtx) ([]byte, []byte, error)

func (*RangeScanPlan) String 

func (p *RangeScanPlan) String() string

type RemovePlan 

type RemovePlan struct {
	Storage Storage
	Keys    []Expression
	// contains filtered or unexported fields
}

func (*RemovePlan) Batch 

func (p *RemovePlan) Batch(ctx *ExecuteCtx) ([][]Column, error)

func (*RemovePlan) Explain 

func (p *RemovePlan) Explain() []string

func (*RemovePlan) FieldNameList 

func (p *RemovePlan) FieldNameList() []string

func (*RemovePlan) FieldTypeList 

func (p *RemovePlan) FieldTypeList() []Type

func (*RemovePlan) Init 

func (p *RemovePlan) Init() error

func (*RemovePlan) Next 

func (p *RemovePlan) Next(ctx *ExecuteCtx) ([]Column, error)

func (*RemovePlan) String 

func (p *RemovePlan) String() string

type RemoveStmt 

type RemoveStmt struct {
	Pos  int
	Keys []Expression
}

func (*RemoveStmt) Name 

func (s *RemoveStmt) Name() string

func (*RemoveStmt) Validate 

func (s *RemoveStmt) Validate(ctx *CheckCtx) error

type ScanType 

type ScanType struct {
	// contains filtered or unexported fields
}

func (*ScanType) String 

func (st *ScanType) String() string

type SelectStmt 

type SelectStmt struct {
	Pos        int
	AllFields  bool
	FieldNames []string
	FieldTypes []Type
	Fields     []Expression
	Where      *WhereStmt
	Order      *OrderStmt
	Limit      *LimitStmt
	GroupBy    *GroupByStmt
}

func (*SelectStmt) Name 

func (s *SelectStmt) Name() string

func (*SelectStmt) ValidateFields 

func (s *SelectStmt) ValidateFields(ctx *CheckCtx) error

type Statement 

type Statement interface {
	Name() string
}

type Storage 

type Storage interface {
	Get(key []byte) (value []byte, err error)
	Put(key []byte, value []byte) error
	BatchPut(kvs []KVPair) error
	Delete(key []byte) error
	BatchDelete(keys [][]byte) error
	Cursor() (cursor Cursor, err error)
}

type StringExpr 

type StringExpr struct {
	Pos  int
	Data string
}

func (*StringExpr) Check 

func (e *StringExpr) Check(ctx *CheckCtx) error

func (*StringExpr) Execute 

func (e *StringExpr) Execute(kv KVPair, ctx *ExecuteCtx) (any, error)

func (*StringExpr) ExecuteBatch 

func (e *StringExpr) ExecuteBatch(chunk []KVPair, ctx *ExecuteCtx) ([]any, error)

func (*StringExpr) GetPos 

func (e *StringExpr) GetPos() int

func (*StringExpr) ReturnType 

func (e *StringExpr) ReturnType() Type

func (*StringExpr) String 

func (e *StringExpr) String() string

func (*StringExpr) Walk 

func (e *StringExpr) Walk(cb WalkCallback)

type SyntaxError 

type SyntaxError struct {
	Query   string
	Message string
	Pos     int
	Padding int
}

func (*SyntaxError) BindQuery 

func (e *SyntaxError) BindQuery(query string)

func (*SyntaxError) Error 

func (e *SyntaxError) Error() string

func (*SyntaxError) SetPadding 

func (e *SyntaxError) SetPadding(pad int)

type Token 

type Token struct {
	Tp   TokenType
	Data string
	Pos  int
}

func (*Token) Precedence 

func (t *Token) Precedence() int

func (*Token) String 

func (t *Token) String() string

type TokenType 

type TokenType byte
const (
	SELECT   TokenType = 1
	WHERE    TokenType = 2
	KEY      TokenType = 3
	VALUE    TokenType = 4
	OPERATOR TokenType = 5
	STRING   TokenType = 6
	LPAREN   TokenType = 7
	RPAREN   TokenType = 8
	NAME     TokenType = 9
	SEP      TokenType = 10
	NUMBER   TokenType = 11
	FLOAT    TokenType = 12
	LIMIT    TokenType = 13
	ORDER    TokenType = 14
	BY       TokenType = 15
	ASC      TokenType = 16
	DESC     TokenType = 17
	TRUE     TokenType = 18
	FALSE    TokenType = 19
	AS       TokenType = 20
	GROUP    TokenType = 21
	IN       TokenType = 22
	BETWEEN  TokenType = 23
	AND      TokenType = 24
	LBRACK   TokenType = 25
	RBRACK   TokenType = 26
	PUT      TokenType = 27
	REMOVE   TokenType = 28
	SEMI     TokenType = 29
	OR       TokenType = 30
	DELETE   TokenType = 31
)

type Type 

type Type byte

type VectorFunctionBody 

type VectorFunctionBody func(chunk []KVPair, args []Expression, ctx *ExecuteCtx) ([]any, error)

type WalkCallback 

type WalkCallback func(e Expression) bool

type WhereStmt 

type WhereStmt struct {
	Pos  int
	Expr Expression
}

func (*WhereStmt) Name 

func (s *WhereStmt) Name() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.