ntql

package module
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 3, 2025 License: MIT Imports: 7 Imported by: 0

README

NTQL

NTQL (NolaTask Query Language) is a domain specific language designed to filter todo list items by certain criteria.

The project consists of

  • a compiler / transpiler
    • breaks down the syntax into tokens
    • detects validity of the form of a query
    • creates a syntax tree
    • evaluates the syntax tree into SQL
  • an autocompletion engine.
    • predicts the next possible token

Syntax

Multiple queries can be connected using connectors such as AND or OR. Queries can also be negated using an exclamation mark !. You can even influence the order in which each query is evaluated by using parentheses

Examples:

  • query1 OR query2
  • query1 AND query2
  • query1 AND !(query2)
  • query1 AND !(query2 OR query3)

Each individual query is in the following form: subject.verb(<expression>).

Examples:

  • description.contains(<expression>)
  • title.startswith(<expression>)
  • tag.equals(<expression>)

Depending on the subject, the expression can have different data types.

Examples:

  • description: string,
  • tag: tag (evaluates to the numeric id of the entered tag after parsing),
  • completedAt: datetime or date,

Similarly to queries, expressions can also be compounded using connectors. Precedence can be defined using parentheses. Expressions can be negated.

Examples:

  • description.contains(<expression1> OR <expression2>) is equal to description.contains(<expression1>) OR description.contains(<expression2>)
  • description.contains((<expression1> OR <expression2>) AND <expression3>) is equal to (description.contains(<expression1>) OR description.contains(<expression2>)) AND description.contains(<expression3>)
  • description.contains((<expression1> OR <expression2>) AND !(<expression3>)) is equal to (description.contains(<expression1>) OR description.contains(<expression2>)) AND !(description.contains(<expression3>))

The combined query gets parsed into an Abstract Syntax Tree (AST), which is compiled to SQL queries which can be evaluated in the database. These queries can span different database tables and the input needs to be sanitized to prevent SQL injections.

The grammar can be more technically defined in Backus Naur Form:

query = expr
expr = or_expr
or_expr = and_expr ("OR" and_expr)*
and_expr = not_expr ("AND" not_expr)*
not_expr = ["!"] term
term = func_call | "(" expr ")"
func_call = subject "." verb "(" value_expr ")" # subject from list of subjects, verb from subject verbs
value_expr = value_or
value_or = value_and ("OR" value_and)*
value_and = value_not ("AND" value_not)*
value_not = ["!"] value_term
value_term = "(" value_expr ")" | value
value = object # type belonging to current verb
object = NUMBER | STRING | DATE | TAG

Design

The design of the language was created to make autocompletion highly deterministic. It does so by limiting the number of possibilities for the next token class. There can only be a limited number of data types for each expected token.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetValidConnectors 

func GetValidConnectors() []string

func GetValidSubjects 

func GetValidSubjects() []string

func NewConnectorTrie 

func NewConnectorTrie() *trie.Trie

func NewTagTrie 

func NewTagTrie(tags []string) *trie.Trie

Types

type AutocompleteError 

type AutocompleteError struct {
	Code     AutocompleteErrorCode
	Position int
	Message  string
}

func NewAutocompleteError 

func NewAutocompleteError(code AutocompleteErrorCode, pos int, msg string) *AutocompleteError

func (*AutocompleteError) Error 

func (e *AutocompleteError) Error() string

type AutocompleteErrorCode 

type AutocompleteErrorCode int
const (
	InvalidCharacter AutocompleteErrorCode = iota
)

type CompletionEngine 

type CompletionEngine struct {
	// contains filtered or unexported fields
}

CompletionEngine is a trie-based autocompletion engine for NTQL

func NewCompletionEngine added in v0.1.4 

func NewCompletionEngine(tags []string) *CompletionEngine

func (*CompletionEngine) Suggest 

func (e *CompletionEngine) Suggest(s string) ([]string, error)

func (*CompletionEngine) SuggestSubject 

func (e *CompletionEngine) SuggestSubject(s string) ([]string, error)

type DType 

type DType int

Data types for any objects passed as an argument to a function in the NTQL query 

const (
	DTypeString DType = iota
	DTypeInt
	DTypeDate
	DTypeTag
	DTypeDateTime
)

type EngineState 

type EngineState int

type ErrEndOfInput 

type ErrEndOfInput struct{}

func (ErrEndOfInput) Error 

func (e ErrEndOfInput) Error() string

type ErrInvalidLexeme 

type ErrInvalidLexeme struct {
	Input byte
}

func (ErrInvalidLexeme) Error 

func (e ErrInvalidLexeme) Error() string

type ErrInvalidSubject 

type ErrInvalidSubject struct {
	Position int
	Lexeme   Lexeme
}

func (ErrInvalidSubject) Error 

func (e ErrInvalidSubject) Error() string

type ErrInvalidToken 

type ErrInvalidToken struct {
	Expected []TokenType
	Position int
	Lexeme   Lexeme
}

func (ErrInvalidToken) Error 

func (e ErrInvalidToken) Error() string

type Lexeme 

type Lexeme string

type Lexer 

type Lexer struct {
	Tokens         []Token
	Scanner        *Scanner
	InnerDepth     int
	ExpectedTokens []TokenType

	ExpectedDataTypes []DType
	// contains filtered or unexported fields
}

func NewLexer 

func NewLexer(s string) *Lexer

func (*Lexer) GetPosition 

func (t *Lexer) GetPosition() int

func (*Lexer) LastLexeme 

func (t *Lexer) LastLexeme() (Lexeme, error)

func (*Lexer) Lex 

func (t *Lexer) Lex() ([]Token, error)

Lex takes a string and returns a slice of tokens Example: tag.equals(hello OR goodbye) OR (date.before(2024-01-08) AND date.after(2024-01-09)) tag.equals(hello) AND date.before(2021-01-01) AND title.startswith(("bar" OR "c\"\\runch") AND "foo")

func (*Lexer) ScanToken 

func (t *Lexer) ScanToken() error

type MegaTrie 

type MegaTrie struct {
	// contains filtered or unexported fields
}

func NewMegaTrie 

func NewMegaTrie() *MegaTrie

type Operator 

type Operator string // I chose string over iota to increase resilience to changes, especially for communication between the frontend and backend

const (
	// Binary operators
	OperatorAnd Operator = "AND"
	OperatorOr  Operator = "OR"
	OperatorXor Operator = "XOR"

	// Unary operators
	OperatorNot Operator = "NOT"

	// Comparison operators
	OperatorEq  Operator = "equals"
	OperatorNeq Operator = "notEquals"
	OperatorGt  Operator = "greaterThan"
	OperatorLT  Operator = "lessThan"
	OperatorGte Operator = "greaterThanOrEquals"
	OperatorLte Operator = "lessThanOrEquals"

	// String operators
	OperatorCnt Operator = "contains"
	OperatorSW  Operator = "startsWith"
	OperatorEw  Operator = "endsWith"
)

func NewOperator 

func NewOperator(s string) (Operator, error)

func (Operator) ToStr 

func (o Operator) ToStr() string

type Parser 

type Parser struct {
	Tokens []Token
	Pos    int
}

TODO: Sanitize input BNF Grammar: query = expr expr = or_expr or_expr = and_expr ("OR" and_expr)* and_expr = not_expr ("AND" not_expr)* not_expr = ["!"] term term = func_call | "(" expr ")" func_call = subject "." verb "(" value_expr ")" # subject from list of subjects, verb from subject verbs value_expr = value_or value_or = value_and ("OR" value_and)* value_and = value_not ("AND" value_not)* value_not = ["!"] value_term value_term = "(" value_expr ")" | value value = object # type belonging to current verb object = NUMBER | STRING | DATE | TAG

func NewParser 

func NewParser(tokens []Token) *Parser

func (*Parser) AndExpr 

func (p *Parser) AndExpr() (QueryExpr, error)

func (*Parser) Expression 

func (p *Parser) Expression() (QueryExpr, error)

func (*Parser) FunctionCall 

func (p *Parser) FunctionCall() (QueryExpr, error)

func (*Parser) NotExpr 

func (p *Parser) NotExpr() (QueryExpr, error)

func (*Parser) OrExpr 

func (p *Parser) OrExpr() (QueryExpr, error)

func (*Parser) Parse 

func (p *Parser) Parse() (QueryExpr, error)

func (*Parser) Query 

func (p *Parser) Query() (QueryExpr, error)

func (*Parser) Subject 

func (p *Parser) Subject() (string, error)

func (*Parser) Term 

func (p *Parser) Term() (QueryExpr, error)

func (*Parser) ValueAnd 

func (p *Parser) ValueAnd() (ValueExpr, error)

func (*Parser) ValueExpr 

func (p *Parser) ValueExpr() (ValueExpr, error)

func (*Parser) ValueNot 

func (p *Parser) ValueNot() (ValueExpr, error)

func (*Parser) ValueObject 

func (p *Parser) ValueObject() (ValueExpr, error)

func (*Parser) ValueOr 

func (p *Parser) ValueOr() (ValueExpr, error)

func (*Parser) ValueTerm 

func (p *Parser) ValueTerm() (ValueExpr, error)

func (*Parser) Verb 

func (p *Parser) Verb(subject string) (string, error)

type ParserError 

type ParserError struct {
	Message string
	Token   Token
}

func NewParserError 

func NewParserError(message string, t Token) *ParserError

func (*ParserError) Error 

func (e *ParserError) Error() string

type QueryBinaryOp 

type QueryBinaryOp struct {
	Left     QueryExpr `json:"left"`
	Right    QueryExpr `json:"right"`
	Operator Operator  `json:"op"`
}

QueryBinaryOp represents a binary operation in a query.

func NewQueryAnd 

func NewQueryAnd(left QueryExpr, right QueryExpr) *QueryBinaryOp

func NewQueryOr 

func NewQueryOr(left QueryExpr, right QueryExpr) *QueryBinaryOp

func (*QueryBinaryOp) String 

func (q *QueryBinaryOp) String() string

func (*QueryBinaryOp) ToSQL 

func (q *QueryBinaryOp) ToSQL() (string, error)

type QueryCondition 

type QueryCondition struct {
	Field    string   `json:"field"`
	Operator Operator `json:"operator"`
	Value    string   `json:"value"`
}

QueryCondition represents a condition in a query.

func NewQueryCondition 

func NewQueryCondition(subject string, verb string, value string) (*QueryCondition, error)

func (*QueryCondition) String 

func (q *QueryCondition) String() string

func (*QueryCondition) ToSQL 

func (c *QueryCondition) ToSQL() (string, error)

TODO: Input sanitization Convert the query condition to a SQL string.

type QueryExpr 

type QueryExpr interface {
	// ToSQL converts the query expression to a SQL string.
	ToSQL() (string, error)
	String() string
}

func BuildQueryExprFromMap 

func BuildQueryExprFromMap(m map[string]interface{}) (QueryExpr, error)

type QueryUnaryOp 

type QueryUnaryOp struct {
	Operand  QueryExpr `json:"operand"`
	Operator Operator  `json:"operator"`
}

QueryUnaryOp represents a unary operation in a query.

func NewQueryNot 

func NewQueryNot(expr QueryExpr) *QueryUnaryOp

func (*QueryUnaryOp) String 

func (q *QueryUnaryOp) String() string

func (*QueryUnaryOp) ToSQL 

func (q *QueryUnaryOp) ToSQL() (string, error)

type Scanner 

type Scanner struct {
	Lexemes []Lexeme
	Pos     int
	S       string
}

func NewScanner 

func NewScanner(s string) *Scanner

func (*Scanner) GetPosition 

func (s *Scanner) GetPosition() int

func (*Scanner) HasNext 

func (s *Scanner) HasNext() bool

func (*Scanner) LastLexeme 

func (s *Scanner) LastLexeme() (Lexeme, error)

func (*Scanner) ScanLexeme 

func (s *Scanner) ScanLexeme() (Lexeme, error)

type ScannerError 

type ScannerError struct {
	Message error
}

func (*ScannerError) Error 

func (e *ScannerError) Error() string

type Subject 

type Subject struct {
	Name       string
	Aliases    []string
	ValidVerbs []Verb
	ValidTypes []DType
}

type Token 

type Token struct {
	Kind     TokenType
	Literal  string
	Position int
}

Example: tag.equals(hello OR goodbye) date.before(2024-01-08) AND date.after(2024-01-09)

func (Token) String 

func (t Token) String() string

type TokenType 

type TokenType int
const (
	TokenSubject TokenType = iota
	TokenBang
	TokenVerb
	TokenString
	TokenTag
	TokenDate
	TokenDateTime
	TokenInt
	TokenBool
	TokenDot
	TokenLParen
	TokenRParen
	TokenAnd
	TokenOr
)

func (TokenType) String 

func (t TokenType) String() string

type Value 

type Value struct {
	Value string
}

func (*Value) Transform 

func (v *Value) Transform(subject string, verb string) (QueryExpr, error)

type ValueBinaryOp 

type ValueBinaryOp struct {
	Operator Operator
	Left     ValueExpr
	Right    ValueExpr
}

func (*ValueBinaryOp) Transform 

func (v *ValueBinaryOp) Transform(subject string, verb string) (QueryExpr, error)

type ValueExpr 

type ValueExpr interface {
	Transform(subject string, verb string) (QueryExpr, error)
}

type ValueUnaryOp 

type ValueUnaryOp struct {
	Operator Operator
	Operand  ValueExpr
}

func (*ValueUnaryOp) Transform 

func (v *ValueUnaryOp) Transform(subject string, verb string) (QueryExpr, error)

type Verb 

type Verb struct {
	Name    string
	Aliases []string
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.