syntax

Documentation

Overview

Package syntax holds the scanner and recursive-descent parser that turn an Onigmo/Ruby regular-expression pattern into an abstract syntax tree (the node types live in the sibling ast package).

Phase 0 covers the common subset: literal and escaped characters, the dot metacharacter, character classes (with ranges, negation, and the Perl class escapes), the anchors \A \z \Z ^ $, the greedy quantifiers * + ? and {m,n}, capturing and non-capturing groups, and alternation. Phase 1 adds named groups (?<name>...) and backreferences (\1, \k<name>). Phase 2 adds the lookaround assertions (?=...) (?!...) (?<=...) (?<!...) and the \G anchor. Phase 3 begins with POSIX bracket expressions [[:name:]] (and negated [[:^name:]]) inside character classes, for the standard classes alpha, digit, alnum, upper, lower, space, blank, cntrl, graph, print, punct, xdigit, and word; their byte ranges match Onigmo's defaults for the ASCII byte space. It also adds the inline options (?flags) (a set directive scoped to the rest of the enclosing group), (?flags:...) (a scoped group), and the (?-flags) / (?f-f:...) turn-off forms, exactly as in Onigmo/Ruby. Three option letters are recognised: i (case-insensitive matching — a folded letter literal is lowered to a rune-aware FoldLiteral via unicode.SimpleFold, character classes fold rune-aware, and backreferences fold ASCII-only), m (dot-all: the dot also matches a newline), and x (extended/free-spacing: unescaped whitespace and # comments in the pattern are ignored, except inside a character class). Any other flag letter is reported as a syntax error. Phase 3 also adds the Unicode property escapes \p{name} / \P{name} (with the in-brace negation \p{^name}), both as a standalone atom and as a character-class member; the recognised names are validated by the sibling charset package.

Lookbehind, as in Onigmo/Ruby, requires each alternative of its body to have a constant byte width (different alternatives may differ, e.g. (?<=ab|c)); bodies whose width can vary — unbounded or {m,n} (m != n) quantifiers, and backreferences — are rejected at parse time.

Index

• Variables
• type Encoding
• type Result
  • func Parse(pattern string) (Result, error)
  • func ParseEnc(pattern string, enc Encoding) (Result, error)

Variables

var ErrSyntax = errors.New("syntax error")

ErrSyntax is the base error returned for malformed patterns. All parse failures wrap it so callers can test with errors.Is.

Types

type Encoding

type Encoding int

Encoding selects how the byte-oriented input-advancing atoms traverse the input. It is fixed at parse time because it governs the byte-width validation of a fixed-width lookbehind: in UTF8 mode the dot and a byte-oriented class match a whole code point (a variable byte width, 1..utf8.UTFMax), whereas in ASCII8BIT (binary, Ruby /n) mode every atom is exactly one byte.

const (
	// UTF8 is the default encoding: the dot and byte-oriented classes span a
	// whole UTF-8 code point.
	UTF8 Encoding = iota
	// ASCII8BIT is Ruby's binary (/n) encoding: every atom is one byte.
	ASCII8BIT
)

type Result

type Result struct {
	Root       ast.Node
	NumCapture int
	Names      map[string]int
}

Result is the outcome of parsing a pattern: the AST root, the number of capturing groups, and the name→index map for named captures.

func Parse

func Parse(pattern string) (Result, error)

Parse compiles a pattern string into an AST in the default UTF-8 encoding. It returns an error wrapping ErrSyntax when the pattern is malformed.

func ParseEnc

func ParseEnc(pattern string, enc Encoding) (Result, error)

ParseEnc is Parse with an explicit input encoding (see Encoding). The encoding only affects the byte-width validation of a fixed-width lookbehind body, where the dot and a byte-oriented class span a whole code point under UTF8 but one byte under ASCII8BIT.