parser

README

cdpparser

基于tidb实现的解析器工具,主要是抽离了tidb中的解析器代码,对其进行了二次开发

实现功能

1. 正解反解sql

正解析sql语句并且修改ast树中某个元素后,还能将sql语句反解析回sql字符串;

1. 方便获取/设置元素

为sql语句的ast树,添加了库表的get set方法,方便获取sql中的库表信息,不需要再用正则扣;

但不支持有多个库表的sql语句

Documentation

Overview

Example (ParseSQL)

This example show how to parse a text sql into ast.

// 0. make sure import parser_driver implemented by TiDB(user also can implement own driver by self).
// and add `import _ "github.com/pingcap/tidb/types/parser_driver"` in the head of file.

// 1. Create a parser. The parser is NOT goroutine safe and should
// not be shared among multiple goroutines. However, parser is also
// heavy, so each goroutine should reuse its own local instance if
// possible.
p := parser.New()

// 2. Parse a text SQL into AST([]ast.StmtNode).
stmtNodes, err := p.Parse("select * from tbl where id = 1", "", "")

// 3. Use AST to do cool things.
fmt.Println(stmtNodes[0], err)

Index

• Variables
• func ParseErrorWith(errstr string, lineno int) *terror.Error
• func TrimComment(txt string) string
• type Parser
  • func New() *Parser
  • func (parser *Parser) EnableWindowFunc(val bool)
  • func (parser *Parser) Parse(sql, charset, collation string) ([]ast.StmtNode, error)
  • func (parser *Parser) ParseOneStmt(sql, charset, collation string) (ast.StmtNode, error)
  • func (parser *Parser) SetSQLMode(mode mysql.SQLMode)
• type Pos
• type Scanner
  • func NewScanner(s string) *Scanner
  • func (s *Scanner) EnableWindowFunc(val bool)
  • func (s *Scanner) Errorf(format string, a ...interface{})
  • func (s *Scanner) Errors() []error
  • func (s *Scanner) GetSQLMode() mysql.SQLMode
  • func (s *Scanner) Lex(v *yySymType) int
  • func (s *Scanner) SetSQLMode(mode mysql.SQLMode)

Examples

• Package (ParseSQL)

Variables

var (
	// ErrSyntax returns for sql syntax error.
	ErrSyntax = terror.ClassParser.New(codeErrSyntax, mysql.MySQLErrName[mysql.ErrSyntax])
	// ErrParse returns for sql parse error.
	ErrParse = terror.ClassParser.New(codeErrParse, mysql.MySQLErrName[mysql.ErrParse])
	// SpecFieldPattern special result field pattern
	SpecFieldPattern = regexp.MustCompile(`(\/\*!(M?[0-9]{5,6})?|\*\/)`)
)

Functions

func ParseErrorWith

func ParseErrorWith(errstr string, lineno int) *terror.Error

ParseErrorWith returns "You have a syntax error near..." error message compatible with mysql.

func TrimComment

func TrimComment(txt string) string

TrimComment trim comment for special comment code of MySQL.

Types

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser represents a parser instance. Some temporary objects are stored in it to reduce object allocation during Parse function.

func New

func New() *Parser

New returns a Parser object.

func (*Parser) EnableWindowFunc

func (parser *Parser) EnableWindowFunc(val bool)

EnableWindowFunc controls whether the parser to parse syntax related with window function.

func (*Parser) Parse

func (parser *Parser) Parse(sql, charset, collation string) ([]ast.StmtNode, error)

Parse parses a query string to raw ast.StmtNode. If charset or collation is "", default charset and collation will be used.

func (*Parser) ParseOneStmt

func (parser *Parser) ParseOneStmt(sql, charset, collation string) (ast.StmtNode, error)

ParseOneStmt parses a query and returns an ast.StmtNode. The query must have one statement, otherwise ErrSyntax is returned.

func (*Parser) SetSQLMode

func (parser *Parser) SetSQLMode(mode mysql.SQLMode)

SetSQLMode sets the SQL mode for parser.

type Pos

type Pos struct {
	Line   int
	Col    int
	Offset int
}

Pos represents the position of a token.

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner implements the yyLexer interface.

func NewScanner

func NewScanner(s string) *Scanner

NewScanner returns a new scanner object.

func (*Scanner) EnableWindowFunc

func (s *Scanner) EnableWindowFunc(val bool)

EnableWindowFunc controls whether the scanner recognize the keywords of window function.

func (*Scanner) Errorf

func (s *Scanner) Errorf(format string, a ...interface{})

Errorf tells scanner something is wrong. Scanner satisfies yyLexer interface which need this function.

func (*Scanner) Errors

func (s *Scanner) Errors() []error

Errors returns the errors during a scan.

func (*Scanner) GetSQLMode

func (s *Scanner) GetSQLMode() mysql.SQLMode

GetSQLMode return the SQL mode of scanner.

func (*Scanner) Lex

func (s *Scanner) Lex(v *yySymType) int

Lex returns a token and store the token value in v. Scanner satisfies yyLexer interface. 0 and invalid are special token id this function would return: return 0 tells parser that scanner meets EOF, return invalid tells parser that scanner meets illegal character.

func (*Scanner) SetSQLMode

func (s *Scanner) SetSQLMode(mode mysql.SQLMode)

SetSQLMode sets the SQL mode for scanner.