memefish

package module
v0.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 24, 2025 License: MIT Imports: 9 Imported by: 18

README

méméfish

méméfish is the Spanner SQL parser for Go.

GoDoc Reference

News

ℹ️

Since 2023/4/1, this repository has been moved from MakeNowJust to cloudspannerecosystem. You may need to migrate import paths from github.com/MakeNowJust/memefish to github.com/cloudspannerecosystem/memefish like:

 import (
-	"github.com/MakeNowJust/memefish/pkg/parser"
+	"github.com/cloudspannerecosystem/memefish/pkg/parser"
 )
ℹ️

Since 2023/4/12, the layout of this repository has been changed. Now, the old parser package has been moved to the top of the repository as the new memefish package, and sub-packages in the pkg directory are placed under the top. You may need to migrate import paths like:

 import (
-	"github.com/cloudspannerecosystem/memefish/pkg/token"
-	"github.com/cloudspannerecosystem/memefish/pkg/parser"
+	parser "github.com/cloudspannerecosystem/memefish"
+	"github.com/cloudspannerecosystem/memefish/token"
 )

In addition, we removed the analyzer package from the repository. Thus, if you want to use the old analyzer package, please pin the dependency by the commit hash before this change.

Features

  • Parse Spanner SQL to AST
  • Generate Spanner SQL from AST (unparse)

Notice

This project is originally developed under "Expert team Go Engineer (Backend)" of Mercari Summer Internship for Engineer 2019.

License

This project is licensed under MIT license.

Documentation

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func ParseDDL 

func ParseDDL(filepath, s string) (ast.DDL, error)

ParseDDL parses an input string containing a DDL statement. filepath can be empty, it is only used in error message.

Example 
sql := heredoc.Doc(`
		CREATE TABLE foo (
			x int64,
			y int64,
		) PRIMARY KEY (x)
	`)

ddl, err := memefish.ParseDDL("path/to/file.sql", sql)
if err != nil {
	panic(err)
}

fmt.Println(ddl.SQL())

Output:

CREATE TABLE foo (
  x INT64,
  y INT64
) PRIMARY KEY (x)

func ParseDDLs 

func ParseDDLs(filepath, s string) ([]ast.DDL, error)

ParseDDLs parses an input string containing DDL statements. filepath can be empty, it is only used in error message.

Example 
sql := heredoc.Doc(`
		CREATE TABLE foo (x int64, y int64) PRIMARY KEY (x);

		CREATE TABLE bar (
			x int64, z int64,
		)
		PRIMARY KEY (x, z),
		INTERLEAVE IN PARENT foo;
	`)

ddls, err := memefish.ParseDDLs("path/to/file.sql", sql)
if err != nil {
	panic(err)
}

for _, ddl := range ddls {
	fmt.Printf("%s;\n", ddl.SQL())
}

Output:

CREATE TABLE foo (
  x INT64,
  y INT64
) PRIMARY KEY (x);
CREATE TABLE bar (
  x INT64,
  z INT64
) PRIMARY KEY (x, z),
  INTERLEAVE IN PARENT foo;

func ParseDML 

func ParseDML(filepath, s string) (ast.DML, error)

ParseDML parses an input string containing a DML statement. filepath can be empty, it is only used in error message.

Example 
sql := heredoc.Doc(`
		INSERT INTO foo (x, y)
		VALUES (1, 2),
		       (3, 4)
	`)

dml, err := memefish.ParseDML("path/to/file.sql", sql)
if err != nil {
	panic(err)
}

fmt.Println(dml.SQL())

Output:

INSERT INTO foo (x, y) VALUES (1, 2), (3, 4)

func ParseDMLs 

func ParseDMLs(filepath, s string) ([]ast.DML, error)

ParseDMLs parses an input string containing DML statements. filepath can be empty, it is only used in error message.

Example 
sql := heredoc.Doc(`
		INSERT INTO foo (x, y) VALUES (1, 2), (3, 4);
		DELETE FROM foo WHERE foo.x = 1 AND foo.y = 2;
	`)

dmls, err := memefish.ParseDMLs("path/to/file.sql", sql)
if err != nil {
	panic(err)
}

for _, dml := range dmls {
	fmt.Printf("%s;\n", dml.SQL())
}

Output:

INSERT INTO foo (x, y) VALUES (1, 2), (3, 4);
DELETE FROM foo WHERE foo.x = 1 AND foo.y = 2;

func ParseExpr 

func ParseExpr(filepath, s string) (ast.Expr, error)

ParseExpr parses an input string containing an expression. filepath can be empty, it is only used in error message.

Example 
package main

import (
	"fmt"

	"github.com/cloudspannerecosystem/memefish"
)

func main() {
	stmt, err := memefish.ParseExpr("", "a * b")
	if err != nil {
		panic(err)
	}

	fmt.Println(stmt.SQL())

}

Output:

a * b

func ParseQuery 

func ParseQuery(filepath, s string) (*ast.QueryStatement, error)

ParseQuery parses an input string containing a query statement. filepath can be empty, it is only used in error message.

Example 
package main

import (
	"fmt"

	"github.com/cloudspannerecosystem/memefish"
)

func main() {
	stmt, err := memefish.ParseQuery("path/to/file.sql", "SELECT * FROM foo")
	if err != nil {
		panic(err)
	}

	fmt.Println(stmt.SQL())

}

Output:

SELECT * FROM foo

func ParseStatement 

func ParseStatement(filepath, s string) (ast.Statement, error)

ParseStatement parses an input string containing a statement. filepath can be empty, it is only used in error message.

Example 
package main

import (
	"fmt"

	"github.com/cloudspannerecosystem/memefish"
)

func main() {
	stmt, err := memefish.ParseStatement("path/to/file.sql", "SELECT * FROM foo")
	if err != nil {
		panic(err)
	}

	fmt.Println(stmt.SQL())

}

Output:

SELECT * FROM foo

func ParseStatements 

func ParseStatements(filepath, s string) ([]ast.Statement, error)

ParseStatements parses an input string containing statements. filepath can be empty, it is only used in error message.

Example 
package main

import (
	"fmt"

	"github.com/cloudspannerecosystem/memefish"
)

func main() {
	stmts, err := memefish.ParseStatements("path/to/file.sql", "SELECT 1; INSERT foo (x, y) VALUES (1, 2)")
	if err != nil {
		panic(err)
	}

	for _, stmt := range stmts {
		fmt.Printf("%s;\n", stmt.SQL())
	}

}

Output:

SELECT 1;
INSERT INTO foo (x, y) VALUES (1, 2);

func ParseType 

func ParseType(filepath, s string) (ast.Type, error)

ParseType parses an input string containing a type. filepath can be empty, it is only used in error message.

Example 
package main

import (
	"fmt"

	"github.com/cloudspannerecosystem/memefish"
)

func main() {
	stmt, err := memefish.ParseType("", "ARRAY<STRUCT<n INT64, s STRING>>")
	if err != nil {
		panic(err)
	}

	fmt.Println(stmt.SQL())

}

Output:

ARRAY<STRUCT<n INT64, s STRING>>

Types

type Error 

type Error struct {
	Message  string
	Position *token.Position
}

Error is an error occured on parsing.

func (*Error) Error 

func (e *Error) Error() string

Error returns an error message.

func (*Error) FullError added in v0.2.0 

func (e *Error) FullError() string

func (*Error) String 

func (e *Error) String() string

type Lexer 

type Lexer struct {
	*token.File
	Token token.Token
	// contains filtered or unexported fields
}

func (*Lexer) Clone 

func (l *Lexer) Clone() *Lexer

func (*Lexer) NextToken 

func (l *Lexer) NextToken() (err error)

NextToken reads a next token from source, then updates its Token field.

type MultiError 

type MultiError []*Error

MultiError is a list of errors occured on parsing.

Note that Parse* methods returns this wrapped error even if the error is just one.

func (MultiError) Error 

func (list MultiError) Error() string

Error returns an error message.

This message only shows the first error's message and other errors' messages are omitted. If you want to obtain all messages of errors at once, you can use FullError instead.

func (MultiError) FullError 

func (list MultiError) FullError() string

FullError returns a full error message.

func (MultiError) String 

func (list MultiError) String() string

type Parser 

type Parser struct {
	*Lexer
	// contains filtered or unexported fields
}

func (*Parser) ParseDDL 

func (p *Parser) ParseDDL() (ast.DDL, error)

ParseDDL parses a CREATE/ALTER/DROP statement.

func (*Parser) ParseDDLs 

func (p *Parser) ParseDDLs() ([]ast.DDL, error)

ParseDDLs parses CREATE/ALTER/DROP statements list separated by semi-colon.

func (*Parser) ParseDML 

func (p *Parser) ParseDML() (ast.DML, error)

ParseDML parses a INSERT/DELETE/UPDATE statement.

func (*Parser) ParseDMLs 

func (p *Parser) ParseDMLs() ([]ast.DML, error)

ParseDMLs parses INSERT/DELETE/UPDATE statements list separated by semi-colon.

func (*Parser) ParseExpr 

func (p *Parser) ParseExpr() (ast.Expr, error)

ParseExpr parses a SQL expression.

func (*Parser) ParseQuery 

func (p *Parser) ParseQuery() (*ast.QueryStatement, error)

ParseQuery parses a query statement.

func (*Parser) ParseStatement 

func (p *Parser) ParseStatement() (ast.Statement, error)

ParseStatement parses a SQL statement.

func (*Parser) ParseStatements 

func (p *Parser) ParseStatements() ([]ast.Statement, error)

ParseStatements parses SQL statements list separated by semi-colon.

func (*Parser) ParseType 

func (p *Parser) ParseType() (ast.Type, error)

ParseType parses a type name.

type RawStatement 

type RawStatement struct {
	Pos, End  token.Pos
	Statement string
}

func SplitRawStatements 

func SplitRawStatements(filepath, s string) ([]*RawStatement, error)

SplitRawStatements splits an input string to statement strings at terminating semicolons without parsing. It preserves all comments. Statements are terminated by `;`, `<eof>` or `;<eof>` and the minimum output will be []string{""}. See terminating semicolons. This function won't panic but return error if lexer become error state. filepath can be empty, it is only used in error message.

Source Files

View all Source files

Directories

Path Synopsis
Package ast provides AST nodes definitions.
 Package ast provides AST nodes definitions.
examples
parse-unparse
tools
astcatalog
gen-ast-pos
gen-ast-walk
parse
util/astcatalog
util/poslang
Package poslang provides an implementation of the POS expression.
 Package poslang provides an implementation of the POS expression.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.