Documentation
¶
Index ¶
- type Enclosure
- type Parser
- func (parser *Parser) Advance()
- func (parser *Parser) Cursor() Token
- func (parser *Parser) ExpectPeek(t TokenKind) bool
- func (parser *Parser) IsCursor(t TokenKind) bool
- func (parser *Parser) IsPeek(t TokenKind) bool
- func (parser *Parser) Peek() Token
- func (parser *Parser) Split(delimiter TokenKind) (splits []string)
- func (parser *Parser) Unparsed() string
- func (parser *Parser) Unwrap(enc Enclosure) (string, error)
- type ParserOption
- type Token
- type TokenKind
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Enclosure ¶
type Enclosure struct {
// contains filtered or unexported fields
}
Enclosure is a tuple of unicode code points that indicate start and stop pairs. They cannot be the same.
func EnclosureAngle ¶
func EnclosureAngle() Enclosure
EnclosureAngle returns an Enclosure set for Angle Brackets '<>'
func EnclosureCurly ¶
func EnclosureCurly() Enclosure
EnclosureCurly returns an Enclosure set for Curly Brackets '{}'
func EnclosureParens ¶
func EnclosureParens() Enclosure
EnclosureParens returns an Enclosure set for Parenthesis '()'
func EnclosureSquare ¶
func EnclosureSquare() Enclosure
EnclosureSquare returns an Enclosure set for Square Brackets '[]'
func NewEnclosure ¶
NewEnclosure generates a new Enclosure set and returns it. Throws an error if the start and stop code points are identical
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
Parser is a symbol parser that parse a given string input and handle operations like unwrapping enclosed data or splitting by a given delimiter
func NewParser ¶
func NewParser(input string, opts ...ParserOption) *Parser
NewParser generates a new Parser for a given input string and some options that modify the parser behaviour such as ignoring whitespaces or using custom keywords
func (*Parser) Advance ¶
func (parser *Parser) Advance()
Advance moves the parser's cursor and peek tokens
func (*Parser) ExpectPeek ¶
ExpectPeek advances the cursor if the next token is of the specified TokenKind. If it is not the same type, the parser does not advance. The returned boolean indicates if the parser was advanced.
func (*Parser) IsPeek ¶
IsPeek checks if the next token is of the specified TokenKind. This look ahead is performed without moving the parser's cursor
func (*Parser) Split ¶
Split attempts to split the remaining contents of the parser into a set of strings separated by the given delimiting TokenKind. This process exhausts the parser consuming all the tokens within it.
func (*Parser) Unwrap ¶
Unwrap attempts to unravel a substring enclosed between to characters described with an Enclosure. When calling Unwrap, the parse cursor must be the opening character of the given Enclosure. Returns an error if the opening character is not found or if the symbol terminates before the closing character.
Note: Unwrap will resolve nested enclosures attempting to match one opening character with one closing character until it fully resolves.
type ParserOption ¶
type ParserOption func(config *parseConfig)
ParserOption represents an option to modify the Parser behaviour. It must be provided with the constructor for Parser.
func IgnoreWhitespaces ¶
func IgnoreWhitespaces() ParserOption
IgnoreWhitespaces returns a ParserOption that specifies the Parser to ignore unicode characters with the whitespace property (' ', '\t', '\n', '\r', etc). They are consumed instead of generating Tokens for them.
func Keywords ¶
func Keywords(keywords map[string]TokenKind) ParserOption
Keywords returns a ParserOption that can be used to provide the Parser with a set of special keywords mapped to some custom TokenKind value. If the Parser encounters identifiers that match any of the given keywords, it returns a Token with the given kind and the actual literal encountered. Any default keywords are overwritten if specified in the custom set.
Note: Use TokenKind values less than -10 for custom Token classes. -10 to -1 are reserved for standard token classes while 0 and above correspond the unicode code points.
type Token ¶
Token represents a lexical Token. It may be either a lone unicode character or some literal value
func UnicodeToken ¶
UnicodeToken returns a Token for a given rune character. The TokenKind of the returned Token has the same value as it's unicode code point.
func (Token) Value ¶ added in v0.2.0
Value returns an object value for the Token. If the Token is kind TokenString -> string (literal is returned as is) If the Token is kind TokenBoolean -> bool (parsed with strconv.ParseBool) If the Token is kind TokenNumber -> uint64/int64 (parsed with strconv depending on if a negative sign is present) If the Token is kind TokenHexNumber -> []byte (decoded with hex.DecodeString after trimming the 0x) All other Token kinds will return an error if attempted to convert to values
type TokenKind ¶
type TokenKind int32
TokenKind is an enum for representing token grouping/values. For unicode tokens, the TokenKind is equal to its code point value. For literal such identifiers and numerics, the TokenKind values descend from 0. Note: Custom TokenKind values can be used by external packages for keyword detection for special literals, but these values should be below -10 to prevent collisions