Documentation
¶
Index ¶
Constants ¶
const ( EBNFLexerTypeIdentifier tokens.TokenType = "identifier" EBNFLexerTypeString tokens.TokenType = "string" EBNFLexerTypeAssign tokens.TokenType = "::=" EBNFLexerTypeOr tokens.TokenType = "|" EBNFLexerTypeLParen tokens.TokenType = "(" EBNFLexerTypeRParen tokens.TokenType = ")" EBNFLexerTypeLBracket tokens.TokenType = "[" EBNFLexerTypeRBracket tokens.TokenType = "]" EBNFLexerTypeLBrace tokens.TokenType = "{" EBNFLexerTypeRBrace tokens.TokenType = "}" EBNFLexerTypeSemicolon tokens.TokenType = ";" EBNFLexerTypeDash tokens.TokenType = "-" EBNFLexerTypeDot tokens.TokenType = "." EBNFLexerTypeArrow tokens.TokenType = "->" EBNFLexerTypeColon tokens.TokenType = ":" EBNFLexerTypeComma tokens.TokenType = "," EBNFLexerTypeInteger tokens.TokenType = "integer" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AbstractLexer ¶
type AbstractLexer interface {
// On EOF, the token-type will be EOF.
// On error, the token-type will be Error, with Lexeme slot having the errortext.
Scan() (token *tokens.Token)
}
func NewEBNFLexer ¶
func NewEBNFLexer(r io.Reader) AbstractLexer
NewEBNFLexer returns a lexer that reads from r (streaming). For string input use NewEBNFLexerFromString.
func NewEBNFLexerFromString ¶ added in v0.2.0
func NewEBNFLexerFromString(s string) AbstractLexer
NewEBNFLexerFromString returns a lexer over s (convenience for tests and -e mode).
func NewEBNFLexerFromStringWithSourceName ¶ added in v0.2.0
func NewEBNFLexerFromStringWithSourceName(s string, sourceName string) AbstractLexer
NewEBNFLexerFromStringWithSourceName is like NewEBNFLexerFromString with a source name.
func NewEBNFLexerWithSourceName ¶
func NewEBNFLexerWithSourceName(r io.Reader, sourceName string) AbstractLexer
NewEBNFLexerWithSourceName returns a lexer that reads from r with a source name for error messages.
type EBNFLexer ¶
type EBNFLexer struct {
// contains filtered or unexported fields
}
EBNFLexer tokenizes a common EBNF dialect with identifiers, string literals, ::= assignments (or =), alternation, and grouping operators. It reads from an io.Reader (streaming); use NewEBNFLexerFromString for string input.
type LookaheadLexer ¶
type LookaheadLexer struct {
// contains filtered or unexported fields
}
LookaheadLexer wraps an AbstractLexer so that there is a LookAhead() token (one lookahead level) and an Advance().
func NewLookaheadLexer ¶
func NewLookaheadLexer(underlying AbstractLexer) *LookaheadLexer
func (*LookaheadLexer) Advance ¶
func (lal *LookaheadLexer) Advance() (token *tokens.Token)
Advance scans the next token. Nominally the error will have already accepted the current token. It returns the current token as a convenience; the same can be gotten from calling LookAhead before Advance.
func (*LookaheadLexer) LookAhead ¶
func (lal *LookaheadLexer) LookAhead() (token *tokens.Token)
LookAhead returns the current lookahead token without advancing the underlying scanner. On EOF, the token-type will be EOF. On error, the token-type will be Error, with Lexeme slot having the errortext.
type RunePredicateFunc ¶
RunePredicateFunc is used by lexers (e.g. EBNFLexer) for predicates like unicode.IsSpace. Exported so that app-side lexers in apps/go/manual can use the same type.
Source Files