scanner

Documentation

Overview

Package scanner implements a scanner for Extract tokens.

Index

• type Atom
• type Dot
  • func (t Dot) String() string
• type Float
• type Ident
• type Int
• type Lparen
  • func (t Lparen) String() string
• type Pin
  • func (t Pin) String() string
• type Rparen
  • func (t Rparen) String() string
• type Scanner
  • func New(r io.Reader) *Scanner
  • func (s *Scanner) All() iter.Seq[Token]
  • func (s *Scanner) Err() error
  • func (s *Scanner) Scan() bool
  • func (s *Scanner) Token() Token
• type String
• type Token
• type TokenError
  • func (err *TokenError) Error() string
  • func (err *TokenError) Unwrap() error
• type UnexpectedRuneError
  • func (err *UnexpectedRuneError) Error() string

Types

type Atom

type Atom string

Token value type.

type Dot

type Dot struct{}

Token value type.

func (Dot) String

func (t Dot) String() string

type Float

type Float float64

Token value type.

type Ident

type Ident string

Token value type.

type Int

type Int int64

Token value type.

type Lparen

type Lparen struct{}

Token value type.

func (Lparen) String

func (t Lparen) String() string

type Pin

type Pin struct{}

Token value type.

func (Pin) String

func (t Pin) String() string

type Rparen

type Rparen struct{}

Token value type.

func (Rparen) String

func (t Rparen) String() string

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner produces Extract parser tokens from an io.Reader.

func New

func New(r io.Reader) *Scanner

New returns a new Scanner which reads from r. The Scanner starts before the first token, so the user must call [Scan] at least once before accessing tokens.

func (*Scanner) All

func (s *Scanner) All() iter.Seq[Token]

All returns a single-use iterator which yields all of the tokens from the scanner in turn. If an error is encountered during the iteration, [Err] will return it.

func (*Scanner) Err

func (s *Scanner) Err() error

Err returns whatever error caused the scanner to stop, or nil if the scanner has not yet stopped or if the scanner stopped because it completely drained the underlying io.Reader without any errors.

func (*Scanner) Scan

func (s *Scanner) Scan() bool

Scan advances the scanner to the next token. The current token can be retrieved using Token. If there are no more tokens, possibly because of an error, Scan returns false.

func (*Scanner) Token

func (s *Scanner) Token() Token

Token returns the current token. See [Scan].

type String

type String string

Token value type.

type Token

type Token struct {
	Line, Col int
	Val       any
}

Token is an Extract language parser token. If the token is valid, Val will be one of the token types defined in this package.

type TokenError

type TokenError struct {
	Line, Col int
	Err       error
}

TokenError is yielded when an unexpected error occurs during the scanning of a token. Line and Col are for the beginning of the token, not the exact location of the error.

func (*TokenError) Error

func (err *TokenError) Error() string

func (*TokenError) Unwrap

func (err *TokenError) Unwrap() error

type UnexpectedRuneError

type UnexpectedRuneError struct {
	Line, Col int
	Rune      rune
}

UnexpectedRuneError is yielded when an unexpected rune is found during the course of scanning.

func (*UnexpectedRuneError) Error

func (err *UnexpectedRuneError) Error() string