machparse

package module

v0.1.0 Latest Latest Go to latest Published: Feb 1, 2026 License: MIT Imports: 4 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/freeeve/machparse

Links

Open Source Insights

README ¶

machparse

A high-performance SQL parser for Go. Parses MySQL, PostgreSQL, and SQLite syntax.

Features

Fast: 3-6x faster than vitess-sqlparser
Low memory: Up to 300x fewer allocations with pooling
Multi-dialect: Supports MySQL, PostgreSQL, and SQLite syntax
Complete: SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, DROP, and more
Round-trip safe: Parse → Format → Parse produces identical AST

Installation

go get github.com/freeeve/machparse

Quick Start

package main

import (
    "fmt"
    "github.com/freeeve/machparse"
)

func main() {
    stmt, err := machparse.Parse("SELECT id, name FROM users WHERE active = true")
    if err != nil {
        panic(err)
    }

    // Format back to SQL
    fmt.Println(machparse.String(stmt))
    // Output: SELECT id, name FROM users WHERE active = true
}

Performance

Benchmarked against vitess-sqlparser:

Parser	Time	Memory	Allocations
machparse	6,019 ns	3,301 B	41
machparse + Repool	3,001 ns	113 B	6
vitess-sqlparser	17,877 ns	34,072 B	123

machparse is 3x faster out of the box, and 6x faster with Repool().

High-Performance Mode

For maximum performance when parsing many queries, call Repool() when done with a statement:

stmt, err := machparse.Parse(sql)
if err != nil {
    return err
}
defer machparse.Repool(stmt)

// ... use stmt ...

This returns AST nodes to internal pools for reuse, reducing allocations by ~85%.

Note: Repool() is optional. If not called, nodes are garbage collected normally. Use it in high-throughput scenarios like SQL proxies or query analyzers.

API

Parsing

// Parse a single statement
stmt, err := machparse.Parse("SELECT * FROM users")

// Parse multiple statements
stmts, err := machparse.ParseAll("SELECT 1; SELECT 2")

Formatting

// Format AST back to SQL
sql := machparse.String(stmt)

Walking the AST

machparse.Walk(stmt, func(node machparse.Node) bool {
    if col, ok := node.(*machparse.ColName); ok {
        fmt.Printf("Found column: %s\n", col.Name)
    }
    return true // continue walking
})

Rewriting the AST

// Rewrite is called post-order (children first, then parent)
rewritten := machparse.Rewrite(stmt, func(node machparse.Node) machparse.Node {
    // Replace table names
    if tn, ok := node.(*machparse.TableName); ok {
        tn.Name = "new_" + tn.Name
        return tn
    }
    return node
})

Pooling (Optional)

// Return AST nodes to pool when done (optional, for max performance)
machparse.Repool(stmt)

Supported SQL

Statements

SELECT (with JOINs, subqueries, CTEs, window functions, UNION/INTERSECT/EXCEPT)
INSERT (with ON CONFLICT, RETURNING)
UPDATE
DELETE
CREATE TABLE/INDEX/VIEW
ALTER TABLE
DROP TABLE/INDEX/VIEW
TRUNCATE
EXPLAIN

Expressions

Binary operators (+, -, *, /, %, AND, OR, etc.)
Comparison operators (=, !=, <, >, <=, >=, LIKE, IN, BETWEEN, etc.)
Functions (COUNT, SUM, AVG, COALESCE, etc.)
CASE expressions
CAST/type conversion
Subqueries
Window functions (ROW_NUMBER, RANK, LAG, LEAD, etc.)
Array expressions and subscripts
JSON operators (PostgreSQL ->, ->>, etc.)

Dialect Features

MySQL: backtick quotes, AUTO_INCREMENT, ON DUPLICATE KEY
PostgreSQL: double-colon casts, RETURNING, ON CONFLICT, dollar-quoted strings
SQLite: AUTOINCREMENT, WITHOUT ROWID

Examples

Extract Table Names

func extractTables(stmt machparse.Statement) []string {
    var tables []string
    machparse.Walk(stmt, func(node machparse.Node) bool {
        if tn, ok := node.(*machparse.TableName); ok {
            tables = append(tables, tn.Name)
        }
        return true
    })
    return tables
}

Rewrite Column References

func prefixColumns(stmt machparse.Statement, prefix string) machparse.Statement {
    return machparse.Rewrite(stmt, func(node machparse.Node) machparse.Node {
        if col, ok := node.(*machparse.ColName); ok {
            col.Name = prefix + col.Name
        }
        return node
    }).(machparse.Statement)
}

High-Throughput Parsing

func parseQueries(queries []string) ([]*machparse.SelectStmt, error) {
    results := make([]*machparse.SelectStmt, 0, len(queries))

    for _, sql := range queries {
        stmt, err := machparse.Parse(sql)
        if err != nil {
            return nil, err
        }

        if sel, ok := stmt.(*machparse.SelectStmt); ok {
            results = append(results, sel)
        }

        // Optional: return nodes to pool if you're done processing
        // and don't need to keep the AST around
        // machparse.Repool(stmt)
    }

    return results, nil
}

License

MIT

Documentation ¶

Overview ¶

Package machparse provides a high-performance SQL parser.

machparse is a dialect-agnostic SQL parser that supports MySQL, PostgreSQL, and SQLite query syntax. It provides Parse, Walk, and Rewrite functionality similar to vitess-sqlparser.

Basic usage:

stmt, err := machparse.Parse("SELECT * FROM users WHERE id = 1")
if err != nil {
    log.Fatal(err)
}
fmt.Println(machparse.String(stmt))

Walking the AST:

machparse.Walk(stmt, func(node ast.Node) bool {
    if col, ok := node.(*ast.ColName); ok {
        fmt.Printf("Found column: %s\n", col.Name)
    }
    return true
})

Rewriting nodes:

rewritten := machparse.Rewrite(stmt, func(n ast.Node) ast.Node {
    // Transform nodes as needed
    return n
})

Index ¶

Constants
func Parse(sql string) (ast.Statement, error)
func ParseAll(sql string) ([]ast.Statement, error)
func Repool(stmt Statement)
func Rewrite(node ast.Node, fn func(ast.Node) ast.Node) ast.Node
func String(node ast.Node) string
func Walk(node ast.Node, fn func(ast.Node) bool)
type AliasedExpr
type AliasedTableExpr
type AlterTableStmt
type BetweenExpr
type BinaryExpr
type CTE
type CaseExpr
type CastExpr
type ColName
type CreateIndexStmt
type CreateTableStmt
type DeleteStmt
type DropIndexStmt
type DropTableStmt
type ExistsExpr
type ExplainStmt
type Expr
type FuncExpr
type InExpr
type InsertStmt
type IsExpr
type JoinExpr
type LikeExpr
type Limit
type Literal
type Node
type OrderByExpr
type ParenExpr
type SelectStmt
type StarExpr
type Statement
type Subquery
type TableName
type TruncateStmt
type UnaryExpr
type UpdateStmt
type WithClause

Constants ¶

View Source

const (
	JoinInner = ast.JoinInner
	JoinLeft  = ast.JoinLeft
	JoinRight = ast.JoinRight
	JoinFull  = ast.JoinFull
	JoinCross = ast.JoinCross
)

Join types

View Source

const (
	LiteralNull   = ast.LiteralNull
	LiteralInt    = ast.LiteralInt
	LiteralFloat  = ast.LiteralFloat
	LiteralString = ast.LiteralString
	LiteralBool   = ast.LiteralBool
)

Literal types

Variables ¶

This section is empty.

Functions ¶

func Parse ¶

func Parse(sql string) (ast.Statement, error)

Parse parses a single SQL statement. The parser uses internal pooling for efficiency. For maximum performance when parsing many queries, call Repool(stmt) when done with the statement (optional, see Repool).

func ParseAll ¶

func ParseAll(sql string) ([]ast.Statement, error)

ParseAll parses all statements in the input. For maximum performance, call Repool on each statement when done (optional).

func Repool ¶

func Repool(stmt Statement)

Repool returns AST nodes to internal pools for reuse. This is optional - if not called, nodes are garbage collected normally. Calling Repool after you're done with a statement improves performance when parsing many queries by reducing allocations.

Example:

stmt, err := machparse.Parse(sql)
if err != nil {
    return err
}
defer machparse.Repool(stmt)
// ... use stmt ...

func Rewrite ¶

func Rewrite(node ast.Node, fn func(ast.Node) ast.Node) ast.Node

Rewrite traverses the AST allowing node replacement. The function is called in post-order (children first, then parent). Return the replacement node or the original to keep it.

func String ¶

func String(node ast.Node) string

String formats an AST node back to SQL.

func Walk ¶

func Walk(node ast.Node, fn func(ast.Node) bool)

Walk traverses the AST calling the function for each node. If the function returns false, children are not visited.