compile

Documentation

Index

• func CmdCompile(cfg config.BuildConfig, output string) error
• func CmdWriteDiagJSON(cfg config.BuildConfig, output, diagPath string) error
• func Compile(patterns []config.RegexEntry, tableBase int64, standalone bool, ...) ([]byte, int64, error)
• func CompileFile(cfg config.BuildConfig, output string) ([]byte, int64, error)
• func CompileSet(spec SetSpec, prefixPool, suffixPool *dfaPool, opts CompileSetOptions) (*compiledSet, error)
• func HasMandatoryLit(pattern string) bool
• type AcceptKind
• type BucketDiag
• type CompileOptions
• type CompileSetOptions
• type ConflictDiag
• type Diagnostics
• type EngineType
  • func SelectEngine(pattern string, opts CompileOptions) (EngineType, error)
  • func (e EngineType) String() string
• type PatternInfo
• type PatternRef
  • func (p PatternRef) String() string
• type SetDiag
• type SetSpec

Functions

func CmdCompile

func CmdCompile(cfg config.BuildConfig, output string) error

CmdCompile compiles all regexp patterns (and optional sets) from cfg to a single WASM module. When cfg.Sets is non-empty, CompileFile is used instead of the bare Compile path, so set-match functions are included in the output. output is the output path (absolute, relative to cwd, or "-" for stdout). Mode is auto-selected from cfg.Output: empty → standalone; non-empty → embedded.

func CmdWriteDiagJSON

func CmdWriteDiagJSON(cfg config.BuildConfig, output, diagPath string) error

CmdWriteDiagJSON re-runs CompileFile, collects set diagnostics, and writes the Diagnostics structure as JSON to diagPath (or stdout if diagPath == "-"). Independent of slog level — the JSON is always complete.

func Compile

func Compile(patterns []config.RegexEntry, tableBase int64, standalone bool, userOpts ...CompileOptions) ([]byte, int64, error)

Compile compiles multiple regexp patterns to a single WASM module.

• standalone=false: module imports "main" memory as memory[0] (input) and declares its own memory for DFA tables (becomes memory[1] after wasm-merge)
• standalone=true: module declares its own memory and exports it as "memory" (for JS/TS/browser direct use)
• tableBase: starting address for DFA/capture tables within the module's memory; use 0 for embedded modules (tables start at address 0 of own memory). Callers like re2test/perftest pass a non-zero value to reserve low pages for their own test input buffers.

All patterns must compile successfully; any error stops compilation immediately.

func CompileFile

func CompileFile(cfg config.BuildConfig, output string) ([]byte, int64, error)

CompileFile compiles all regexp patterns and sets from cfg into a single WASM module. When cfg.Sets is empty, it is byte-identical to the existing Compile() path.

func CompileSet

func CompileSet(spec SetSpec, prefixPool, suffixPool *dfaPool, opts CompileSetOptions) (*compiledSet, error)

CompileSet compiles one set specification into a compiledSet. prefixPool and suffixPool are shared dedup pools across all sets in the file.

func HasMandatoryLit

func HasMandatoryLit(pattern string) bool

HasMandatoryLit reports whether pattern contains a non-empty mandatory literal that the analyser can locate. This is a necessary but NOT sufficient condition for using the pattern as an anchor in set composition: the AST path to the literal must also be splittable (see splitAtPath), which excludes literals reached only through OpPlus, OpRepeat, or OpAlternate. The set router applies that additional check itself; callers that need a yes/no answer on "is this pattern usable as a set anchor?" must do the same.

Types

type AcceptKind

type AcceptKind int

AcceptKind describes how accept bits are encoded in the merged suffix DFA. Phase 6 will add AcceptSparseSet for WAF-scale patterns.

const (
	AcceptBitmask AcceptKind = iota + 1 // one bit per pattern in a u64 per DFA state
)

type BucketDiag

type BucketDiag struct {
	ID           int          `json:"id"`
	Type         string       `json:"type"`        // "merged" | "singleton" | "fallback"
	AcceptKind   string       `json:"accept_kind"` // "bitmask" (Phases 2–5)
	Literal      string       `json:"literal"`
	Patterns     []PatternRef `json:"patterns"`
	SuffixStates int          `json:"suffix_states"`
	TableBytes   int          `json:"table_bytes"`
}

BucketDiag describes one merged bucket.

type CompileOptions

type CompileOptions struct {
	// MaxDFAStates is the maximum number of states allowed when building a DFA
	// (match/find) or TDFA (capture groups). If the DFA/TDFA exceeds this limit
	// the engine falls back to Backtracking. 0 means use the default (1024).
	// Exposed as max_dfa_states in the YAML config.
	MaxDFAStates int
	// MaxTDFARegs is the maximum number of WASM capture registers a TDFA may
	// use before falling back to Backtracking. 0 means use the default (32).
	// Exposed as max_tdfa_regs in the YAML config.
	MaxTDFARegs   int
	MaxDFAMemory  int        // Maximum DFA memory in bytes (default: 102400)
	Unicode       bool       // Enable Unicode support
	ForceEngine   EngineType // If non-zero, skip engine selection and use this engine type
	LeftmostFirst bool       // Use leftmost-first (RE2/Perl) semantics for alternations
	// CompiledDFAThreshold is the maximum minimised WASM state count for which the
	// compiled dispatch path (EngineCompiledDFA) is used instead of the table-driven
	// interpreter. 0 means use the default (256). Capped at 256 (u8 state index
	// constraint). Negative value disables the compiled path entirely.
	// NOT exposed in the YAML config schema — internal/programmatic use only.
	CompiledDFAThreshold int
	// MemoBudget is the maximum bytes allocated for the BitState memoization
	// buffer. Only used when the pattern requires BitState (needsBitState == true).
	// Defaults to 128*1024 (128 KB) when zero.
	MemoBudget int
	// contains filtered or unexported fields
}

CompileOptions contains optional parameters for engine selection.

type CompileSetOptions

type CompileSetOptions struct {
	BitmaskWidth          int   // max patterns per bucket using AcceptBitmask; default 32
	MaxPatternsPerBucket  int   // hard cap for AcceptSparseSet (Phase 6); default 4096
	BudgetBytes           int   // max merged DFA table bytes per bucket; default 65536
	BudgetStates          int   // max DFA states per merged bucket; default 512
	BudgetStatesPreFilter int   // pre-filter: suffixStates * combinedClassCount; default 65536
	MaxFallbackStates     int   // max DFA states for a single-pattern fallback bucket; default 1024
	TableBase             int32 // byte offset where this set's DFA tables start in memory; default 0
	TableMemIdx           int   // 0 = standalone (single memory), 1 = embedded (multi-memory after merge)
}

CompileSetOptions holds tunable parameters for set composition. Zero value uses defaults.

type ConflictDiag

type ConflictDiag struct {
	Pattern         PatternRef             `json:"pattern"`
	CandidateBucket int                    `json:"candidate_bucket"`
	Reason          string                 `json:"reason"`
	Detail          map[string]interface{} `json:"detail,omitempty"`
}

ConflictDiag records a bin-packing rejection.

type Diagnostics

type Diagnostics struct {
	PatternsTotal       int       `json:"patterns_total"`
	CaptureBearing      int       `json:"capture_bearing"`
	InSet               int       `json:"in_set"`
	PrefixDedupPoolSize int       `json:"prefix_dedup_pool_size"`
	Sets                []SetDiag `json:"sets"`
}

Diagnostics is the top-level diagnostic structure produced by CompileFile.

type EngineType

type EngineType byte

EngineType represents the type of regexp engine implementation.

const (
	EngineDFA EngineType = iota + 1
	EngineBacktrack
	EngineCompiledDFA // DFA with compiled (br_table) dispatch; no transition table at runtime
	EngineTDFA        // Tagged DFA: O(n) matching with full capture support
)

func SelectEngine

func SelectEngine(pattern string, opts CompileOptions) (EngineType, error)

SelectEngine returns the EngineType that would be chosen for the given pattern, without actually compiling it. Returns an error if the pattern cannot be parsed or compiled to NFA bytecode.

func (EngineType) String

func (e EngineType) String() string

String returns the human-readable name of the engine type.

type PatternInfo

type PatternInfo struct {
	// contains filtered or unexported fields
}

PatternInfo holds the analysis result for a single pattern in a set. Populated by analyzePattern; consumed by set composition (Phase 2+).

type PatternRef

type PatternRef struct {
	ID   int    `json:"id"`
	Name string `json:"name"` // empty when entry has no `name:` field
}

PatternRef is the canonical pattern reference used in all log events and JSON diagnostic output.

func (PatternRef) String

func (p PatternRef) String() string

type SetDiag

type SetDiag struct {
	Name                  string         `json:"name"`
	Frontend              string         `json:"frontend"` // "teddy", "ac", "scalar"
	Buckets               []BucketDiag   `json:"buckets"`
	Conflicts             []ConflictDiag `json:"conflicts"`
	CaptureBearingDropped []PatternRef   `json:"capture_bearing_dropped"`
	StateLimitDropped     []PatternRef   `json:"state_limit_dropped,omitempty"`
}

SetDiag holds diagnostics for one set.

type SetSpec

type SetSpec struct {
	Name       string
	FindAny    string
	FindAll    string
	Match      string         // anchored match export name, or ""
	Patterns   []*PatternInfo // resolved, capture-bearing dropped
	PatternIDs []int          // global indices into the regexps list
}

SetSpec is the resolved specification for one set, ready for compilation.