lex

Documentation

Overview

Contains the lexical analysis package for the interpreter.

Index

• Constants
• func IsAlphaNumeric(r rune) bool
• func IsAlphabet(r rune) bool
• func IsNewLine(r rune) bool
• func IsNumber(r rune) bool
• func IsWhitespace(r rune) bool
• type Command
• type Lexer
  • func NewLexer(manager *manager.ErrorManager) *Lexer
  • func (l *Lexer) Reset() *Lexer
  • func (l *Lexer) SetInitialState(state StateFn)
  • func (l *Lexer) SetInput(input string) *Lexer
  • func (l *Lexer) Tokenize() ([]token.Token, []manager.ErrorTranspiler)
• type StateFn

Constants

const EOF = -1

Used to represent the end of the input

const MAX_DEPTH_DEFAULT = 1000

Maximum depth of the lexer, if reached, the lexer will stop processing

Functions

func IsAlphaNumeric

func IsAlphaNumeric(r rune) bool

Returns true if the rune is a letter or a number

func IsAlphabet

func IsAlphabet(r rune) bool

Returns true if the rune is a letter

func IsNewLine

func IsNewLine(r rune) bool

Returns true if the rune is a new line character

func IsNumber

func IsNumber(r rune) bool

Returns true if the rune is a number

func IsWhitespace

func IsWhitespace(r rune) bool

Returns true if the rune is a whitespace character or a new line character

Types

type Command

type Command string

Represent different types of commands e.g add

const (
	CommandAdd     Command = "add"     // Add a new task
	CommandDepends Command = "depends" // Add a dependency
	CommandNext    Command = "next"    // Mark this task as the next task to be executed

)

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

A lexer is used to tokenize a string into a series of tokens

func NewLexer

func NewLexer(manager *manager.ErrorManager) *Lexer

Create a new lexer that will tokenize the given input

func (*Lexer) Reset

func (l *Lexer) Reset() *Lexer

func (*Lexer) SetInitialState

func (l *Lexer) SetInitialState(state StateFn)

Useful for parsing partial input

func (*Lexer) SetInput

func (l *Lexer) SetInput(input string) *Lexer

func (*Lexer) Tokenize

func (l *Lexer) Tokenize() ([]token.Token, []manager.ErrorTranspiler)

Tokenize the input string and return the tokens

type StateFn

type StateFn func(*Lexer) StateFn

A lexer is a function that processes the input and returns the next state