scanner

Documentation

Overview

Package scanner implements a scanner for pg.

Index

• type Scanner
  • func New(src []byte, filename string) *Scanner
  • func (s *Scanner) Scan() (pos token.Pos, typ token.Type, lit string)

Examples

• Scanner.Scan

Types

type Scanner

type Scanner struct {
	Err      func(token.Pos, string) // error reporting; or nil
	ErrCount int                     // number of errors encountered
	// contains filtered or unexported fields
}

Scanner represents a lexical scanner for pg.

func New

func New(src []byte, filename string) *Scanner

New creates and initializes a new instance of Scanner using src as its source content and filename as the current filename.

func (*Scanner) Scan

func (s *Scanner) Scan() (pos token.Pos, typ token.Type, lit string)

Scan scans the next token and returns its position, type and literal.

Example

package main

import (
	"fmt"

	"github.com/davidrjenni/pg/scanner"
	"github.com/davidrjenni/pg/token"
)

func main() {
	src := []byte(`E -> T "+" T | T | ε .`)
	s := scanner.New(src, "example")

	for {
		pos, tok, lit := s.Scan()
		if tok == token.EOF {
			break
		}
		fmt.Printf("%s\t%s\t%q\n", pos, tok, lit)
	}

}

Output:

example:1:1	IDENT	"E"
example:1:3	ARROW	"->"
example:1:6	IDENT	"T"
example:1:8	STRING	"\"+\""
example:1:12	IDENT	"T"
example:1:14	PIPE	""
example:1:16	IDENT	"T"
example:1:18	PIPE	""
example:1:20	EPSILON	"ε"
example:1:23	PERIOD	""