lexer

Documentation

Overview

Package lexer scans SystemVerilog/Verilog source text into a token stream (see the sibling token package for the vocabulary). It is error-tolerant by design: unrecognized input never stops scanning or panics, it's tagged token.KindInvalid and lexing continues, since a language server has to make sense of source that's broken mid-edit far more often than a batch tool does.

Index

• type Error
  • func Lex(src string) ([]token.Token, []Error)

Types

type Error

type Error struct {
	Line      int
	Character int
	Message   string
}

Error is a non-fatal problem noticed while lexing (an unrecognized character, an unterminated comment or string). Lex never stops or panics because of one -- Errors accumulates them for a caller to surface as diagnostics if it wants to.

func Lex

func Lex(src string) ([]token.Token, []Error)

Lex tokenizes src in full. The returned token slice always ends with exactly one token.KindEOF token, so a caller (a future parser, chiefly) never needs a separate end-of-input check.