automaton

Documentation

Index

• type Node
• type Path
• type PreNode
• type Token

Types

type Node

type Node struct {
	Final bool // final state
	Paths []Path
	Token Token
}

Node is a state of the Tokenizer automaton

type Path

type Path struct {
	Exp    re.Regexp // regular expression used as transition condition
	Target *Node     // target node
}

Path connects two nodes over a regular expression

type PreNode

type PreNode struct {
	Final bool              `json:"final"`
	Paths map[string]string `json:"paths"`
	Token Token             `json:"token,omitempty"`
}

PreNode is used by the Tokenizer constructor to store unprocessed values that will form a complete Node

type Token

type Token string

Token takes values defined in the rules