sre

Documentation

Overview

Package sre is the bytecode matcher behind the pure-Python re package.

re parses and compiles a pattern into the SRE bytecode in re._parser and re._compiler, then hands the finished bytecode to _sre.compile, which builds the compiled pattern (see pkg/objects/sre.go). This package walks that bytecode. It is a faithful port of CPython's Modules/_sre engine and depends on nothing but the standard library, so the object-model glue that turns a run into a Match object lives elsewhere.

Index

• Constants
• type Result
  • func Match(input []int32, code []uint32, groups, pos, endpos int, ...) (Result, error)
  • func Search(input []int32, code []uint32, groups, pos, endpos int, mustAdvance bool) (Result, error)

Constants

const (
	OpFailure uint32 = iota
	OpSuccess
	OpAny
	OpAnyAll
	OpAssert
	OpAssertNot
	OpAt
	OpBranch
	OpCategory
	OpCharset
	OpBigcharset
	OpGroupref
	OpGrouprefExists
	OpIn
	OpInfo
	OpJump
	OpLiteral
	OpMark
	OpMaxUntil
	OpMinUntil
	OpNotLiteral
	OpNegate
	OpRange
	OpRepeat
	OpRepeatOne
	OpSubpattern
	OpMinRepeatOne
	OpAtomicGroup
	OpPossessiveRepeat
	OpPossessiveRepeatOne
	OpGrouprefIgnore
	OpInIgnore
	OpLiteralIgnore
	OpNotLiteralIgnore
	OpGrouprefLocIgnore
	OpInLocIgnore
	OpLiteralLocIgnore
	OpNotLiteralLocIgnore
	OpGrouprefUniIgnore
	OpInUniIgnore
	OpLiteralUniIgnore
	OpNotLiteralUniIgnore
	OpRangeUniIgnore
)

Opcode values for the SRE bytecode. The ordering and numeric values match the OPCODES list re._constants builds, after the two parser-only entries at the tail are stripped.

const (
	AtBeginning uint32 = iota
	AtBeginningLine
	AtBeginningString
	AtBoundary
	AtNonBoundary
	AtEnd
	AtEndLine
	AtEndString
	AtLocBoundary
	AtLocNonBoundary
	AtUniBoundary
	AtUniNonBoundary
)

Position codes used by OpAt for anchors and word-boundary checks.

const (
	CategoryDigit uint32 = iota
	CategoryNotDigit
	CategorySpace
	CategoryNotSpace
	CategoryWord
	CategoryNotWord
	CategoryLinebreak
	CategoryNotLinebreak
	CategoryLocWord
	CategoryLocNotWord
	CategoryUniDigit
	CategoryUniNotDigit
	CategoryUniSpace
	CategoryUniNotSpace
	CategoryUniWord
	CategoryUniNotWord
	CategoryUniLinebreak
	CategoryUniNotLinebreak
)

Character-category codes used by OpCategory for \d, \D, \s, \S, \w, \W, the linebreak class, and their locale and Unicode variants.

const (
	SreInfoPrefix  uint32 = 1
	SreInfoLiteral uint32 = 2
	SreInfoCharset uint32 = 4
)

INFO-block flags appearing in the operand of OpInfo, which re._compiler emits at the head of a program to carry the prefix and charset search hints.

const (
	MagicNumber        = 20230612
	CodeSize           = 4
	MaxRepeat   uint32 = 0xFFFFFFFF
	MaxGroups          = 1073741823
)

Engine identifiers the Python layer reads. MagicNumber stamps the bytecode version re._compiler emits, CodeSize is the byte width of one bytecode word in CPython's UCS4 packing, MaxRepeat is the unbounded-count sentinel, and MaxGroups caps the group count.

Types

type Result

type Result struct {
	Matched   bool
	Locs      []int
	LastIndex int
}

Result is the outcome of a match or search attempt. Locs is the group-span vector in [group0_start, group0_end, group1_start, group1_end, ...] order, length 2 + 2*groups, with -1 for a group that never matched. LastIndex is the 1-based number of the last group that matched, or 0 when none did.

func Match

func Match(input []int32, code []uint32, groups, pos, endpos int, matchAll, mustAdvance bool) (Result, error)

Match runs the anchored matcher at pos. matchAll requires the match to reach endpos, the fullmatch behaviour; mustAdvance forbids an empty match at the start position, which the finditer and sub loops set to make progress.

func Search

func Search(input []int32, code []uint32, groups, pos, endpos int, mustAdvance bool) (Result, error)

Search walks the start positions from pos looking for the first match. mustAdvance forbids an empty match at the start position.