token

Documentation

Overview

Package token defines constants representing the lexical tokens for pg.

Index

• type Pos
  • func (p Pos) String() string
• type Type
  • func (t Type) IsLiteral() bool
  • func (t Type) IsOperator() bool
  • func (t Type) String() string

Types

type Pos

type Pos struct {
	Filename string // filename, if any
	Offset   int    // offset, starting at 0
	Line     int    // line number, starting at 1
	Column   int    // column number, starting at 1 (character count)
}

Pos describes an arbitrary source position including the file, line, and column location. A position is valid if the line number is > 0.

func (Pos) String

func (p Pos) String() string

String returns a string in one of several forms:

file:line:column    valid position with filename
line:column         valid position without filename
file                invalid position with filename
-                   invalid position without filename

type Type

type Type int

Type is the set of lexical tokens of pg.

const (
	ILLEGAL Type = iota
	EOF

	IDENT  // Foo
	STRING // "abc"

	ARROW  // -> or →
	PERIOD // .
	PIPE   // |

	EPSILON // e or ε
)

Token constants

func (Type) IsLiteral

func (t Type) IsLiteral() bool

IsLiteral returns true for tokens corresponding to literals; it returns false otherwise.

func (Type) IsOperator

func (t Type) IsOperator() bool

IsOperator returns true for tokens corresponding to operators and delimiters; it returns false otherwise.

func (Type) String

func (t Type) String() string

String returns the string corresponding to the token.