token

Documentation

Overview

Package token contains the tokens we understand when it comes to parsing our BASIC input.

Index

• Constants
• type Token
  • func (t Token) String() string
• type Type
  • func LookupIdentifier(identifier string) Type

Constants

const (
	// Core
	EOF     = "EOF"     // End of file
	NEWLINE = "NEWLINE" // Newlines are kept in our lexer-stream
	LINENO  = "LINENO"  // Line-number of each input.

	// Types
	IDENT   = "IDENT"   // Identifier (i.e. variable name)
	INT     = "INT"     // integer literal
	STRING  = "STRING"  // string literal
	BUILTIN = "BUILTIN" // builtin-function

	// Implemented keywords.
	END    = "END"
	GOSUB  = "GOSUB"
	GOTO   = "GOTO"
	INPUT  = "INPUT"
	LET    = "LET"
	REM    = "REM"
	RETURN = "RETURN"

	// Did I mention that for-loops work?  :D
	FOR  = "FOR"
	NEXT = "NEXT"
	STEP = "STEP"
	TO   = "TO"

	// And conditionals?
	IF   = "IF"
	THEN = "THEN"
	ELSE = "ELSE"

	// Binary operators
	AND = "AND"
	OR  = "OR"
	XOR = "XOR"

	// Misc
	DEF  = "DEF"
	DIM  = "DIM"
	FN   = "FN"
	READ = "READ"
	SWAP = "SWAP"
	DATA = "DATA"

	// Woo-operators
	ASSIGN   = "=" // LET x = 3
	ASTERISK = "*" // integer multiplication
	COMMA    = "," // PRINT 3, 54
	MINUS    = "-" // integer subtraction
	MOD      = "%" // integer modulus
	PLUS     = "+" // integer addition
	SLASH    = "/" // integer division
	POW      = "^" // power

	COLON     = ":"
	SEMICOLON = ";"
	LBRACKET  = "("
	RBRACKET  = ")"
	LINDEX    = "["
	RINDEX    = "]"

	// Comparison functions.
	GT        = ">"
	GTEQUALS  = ">="
	LT        = "<"
	LTEQUALS  = "<="
	NOTEQUALS = "<>"
)

pre-defined token-types

Types

type Token

type Token struct {
	// Type holds the type of the token
	Type Type

	// Literal holds the literal value.
	Literal string
}

Token contains a single token.

func (Token) String

func (t Token) String() string

String creates a string-representation of a token

type Type

type Type string

Type is a string

func LookupIdentifier

func LookupIdentifier(identifier string) Type

LookupIdentifier used to determine whether identifier is keyword nor not. We handle both upper-case and lower-cased keywords, for example both "print" and "PRINT" are considered identical.