token

Documentation

Overview

Package token defines the lexical elements of an Alloy config and utilities surrounding their position.

Index

• Constants
• Variables
• type File
  • func NewFile(filename string) *File
  • func (f *File) AddLine(offset int)
  • func (f *File) Name() string
  • func (f *File) Pos(off int) Pos
  • func (f *File) PositionFor(p Pos) Position
• type Pos
  • func (p Pos) Add(n int) Pos
  • func (p Pos) File() *File
  • func (p Pos) Offset() int
  • func (p Pos) Position() Position
  • func (p Pos) String() string
  • func (p Pos) Valid() bool
• type Position
  • func (pos Position) String() string
  • func (pos *Position) Valid() bool
• type Token
  • func Lookup(ident string) Token
  • func (t Token) BinaryPrecedence() int
  • func (t Token) GoString() string
  • func (t Token) IsKeyword() bool
  • func (t Token) IsLiteral() bool
  • func (t Token) IsOperator() bool
  • func (t Token) String() string

Constants

const (
	LowestPrecedence  = 0 // non-operators
	UnaryPrecedence   = 7
	HighestPrecedence = 8
)

Levels of precedence for operator tokens.

Variables

var NoPos = Pos{}

NoPos is the zero value for Pos. It has no file or line information associated with it, and NoPos.Valid is false.

Types

type File

type File struct {
	// contains filtered or unexported fields
}

File holds position information for a specific file.

func NewFile

func NewFile(filename string) *File

NewFile creates a new File for storing position information.

func (*File) AddLine

func (f *File) AddLine(offset int)

AddLine tracks a new line from a byte offset. The line offset must be larger than the offset for the previous line, otherwise the line offset is ignored.

func (*File) Name

func (f *File) Name() string

Name returns the name of the file.

func (*File) Pos

func (f *File) Pos(off int) Pos

Pos returns a Pos given a byte offset. Pos panics if off is < 0.

func (*File) PositionFor

func (f *File) PositionFor(p Pos) Position

PositionFor returns a Position from an offset.

type Pos

type Pos struct {
	// contains filtered or unexported fields
}

Pos is a compact representation of a position within a file. It can be converted into a Position for a more convenient, but larger, representation.

func (Pos) Add

func (p Pos) Add(n int) Pos

Add creates a new Pos relative to p.

func (Pos) File

func (p Pos) File() *File

File returns the file used by the Pos. This will be nil for invalid positions.

func (Pos) Offset

func (p Pos) Offset() int

Offset returns the byte offset associated with Pos.

func (Pos) Position

func (p Pos) Position() Position

Position converts the Pos into a Position.

func (Pos) String

func (p Pos) String() string

String returns the string form of the Pos (the offset).

func (Pos) Valid

func (p Pos) Valid() bool

Valid reports whether the Pos is valid.

type Position

type Position struct {
	Filename string // Filename (if any)
	Offset   int    // Byte offset (starting at 0)
	Line     int    // Line number (starting at 1)
	Column   int    // Offset from start of line (starting at 1)
}

Position holds full position information for a location within an individual file.

func (Position) String

func (pos Position) String() string

String returns a string in one of the following forms:

file:line:column   Valid position with file name
file:line          Valid position with file name but no column
line:column        Valid position with no file name
line               Valid position with no file name or column
file               Invalid position with file name
-                  Invalid position with no file name

func (*Position) Valid

func (pos *Position) Valid() bool

Valid reports whether the position is valid. Valid positions must have a Line value greater than 0.

type Token

type Token int

Token is an individual Alloy lexical token.

const (
	ILLEGAL Token = iota // Invalid token.
	LITERAL              // Literal text.
	EOF                  // End-of-file.
	COMMENT              // // Hello, world!

	IDENT  // foobar
	NUMBER // 1234
	FLOAT  // 1234.0
	STRING // "foobar"

	BOOL // true
	NULL // null

	OR  // ||
	AND // &&
	NOT // !

	ASSIGN // =

	EQ  // ==
	NEQ // !=
	LT  // <
	LTE // <=
	GT  // >
	GTE // >=

	ADD // +
	SUB // -
	MUL // *
	DIV // /
	MOD // %
	POW // ^

	LCURLY // {
	RCURLY // }
	LPAREN // (
	RPAREN // )
	LBRACK // [
	RBRACK // ]
	COMMA  // ,
	DOT    // .

	TERMINATOR // \n
)

List of all lexical tokens and examples that represent them.

LITERAL is used by token/builder to represent literal strings for writing tokens, but never used for reading (so scanner never returns a token.LITERAL).

func Lookup

func Lookup(ident string) Token

Lookup maps a string to its keyword token or IDENT if it's not a keyword.

func (Token) BinaryPrecedence

func (t Token) BinaryPrecedence() int

BinaryPrecedence returns the operator precedence of the binary operator t. If t is not a binary operator, the result is LowestPrecedence.

func (Token) GoString

func (t Token) GoString() string

GoString returns the %#v format of t.

func (Token) IsKeyword

func (t Token) IsKeyword() bool

IsKeyword returns true if the token corresponds to a keyword.

func (Token) IsLiteral

func (t Token) IsLiteral() bool

IsLiteral returns true if the token corresponds to a literal token or identifier.

func (Token) IsOperator

func (t Token) IsOperator() bool

IsOperator returns true if the token corresponds to an operator or delimiter.

func (Token) String

func (t Token) String() string

String returns the string representation corresponding to the token.