preprocessor

Documentation

Overview

Package preprocessor implements SV compiler-directive handling on top of svparse/lexer's token stream: macro definition and expansion, conditional compilation, and “ `include “ resolution, all with source maps so an expanded or included token still resolves back to a real, editor-true position rather than "somewhere inside a macro definition" or "the position “ `include “ itself sat at". Like the lexer, it's error-tolerant: malformed directives are recorded and skipped, never fatal.

This package is purely syntactic, same as the rest of svparse: it has no notion of a workspace, multiple files beyond what “ `include “ itself pulls in, or which macros "should" already be defined coming in. A caller wanting company-workspace-wide `+define+`/`+incdir+` semantics (as sigils's filelist convention allows) builds that on top, via IncludeResolver and whatever it seeds into a fresh call to Preprocess.

Index

• func Preprocess(path, text string, resolver IncludeResolver) ([]Token, []Error)
• func PreprocessWithOptions(path, text string, resolver IncludeResolver, opts Options) ([]Token, []Error)
• func SplitBalancedArgs(toks []Token, pos int) (groups [][]Token, newPos int, closed bool)
• type Error
• type IncludeResolver
• type Options
• type Token

Functions

func Preprocess

func Preprocess(path, text string, resolver IncludeResolver) ([]Token, []Error)

Preprocess preprocesses text (path's contents, already read by the caller -- e.g. an open editor buffer or a disk read, same "open buffer wins" pattern a caller likely already applies before calling this) and every file it transitively includes via resolver.

func PreprocessWithOptions

func PreprocessWithOptions(path, text string, resolver IncludeResolver, opts Options) ([]Token, []Error)

PreprocessWithOptions is Preprocess with additional configuration -- see Options.

func SplitBalancedArgs

func SplitBalancedArgs(toks []Token, pos int) (groups [][]Token, newPos int, closed bool)

SplitBalancedArgs consumes toks starting at pos, which must be positioned right after an already-consumed opening '(', up to and including its matching ')', splitting the tokens in between into comma-separated groups. Nesting inside another ( { [ '{ pair -- a nested call, an array/bit-select, an assignment-pattern literal -- isn't split at its own, deeper comma. newPos is the index just past the matching ')' (or, if none was found, wherever scanning stopped).

Exported because both this package's own macro-argument/parameter- list parsing and svparse/parser's instantiation port-connection/ parameter-override parsing need the identical splitting shape and would otherwise duplicate it.

Returns closed=false if input ran out before a balancing ')' was found (malformed/truncated input) -- the caller decides how to report that.

Types

type Error

type Error struct {
	File      string
	Line      int
	Character int
	Message   string
}

Error is a non-fatal problem noticed while preprocessing (a malformed directive, an unbalanced “ `ifdef“/“ `endif“, ...). Preprocess never stops or panics because of one.

type IncludeResolver

type IncludeResolver interface {
	Resolve(includedPath, fromFile string) (text, resolvedPath string, err error)
}

IncludeResolver resolves an “ `include “d path (as written in source) against the file that referenced it, returning the included file's text and a canonical path used both for source-map attribution and include-cycle detection. Preprocess never touches a filesystem directly -- this is the only seam a workspace-specific caller implements, mirroring how sigils's own workspace.FilelistDiscoverer already resolves and dedupes paths in its own domain.

type Options

type Options struct {
	// InitialMacros seeds object-like macros as if each had been
	// “ `define “d before the first token of path -- the preprocessing
	// equivalent of a compiler's "-D" / "+define+" command-line flags. A
	// map value of "" defines the name with an empty body (“ `define FOO “,
	// no replacement text); a non-empty value is lexed into the macro's
	// body tokens (“ `define FOO <value> “). There is no way to seed a
	// function-like macro this way, matching +define+'s own CLI-level
	// semantics -- it only ever defines a name to a flat token sequence.
	InitialMacros map[string]string
}

Options carries preprocessing configuration beyond the always-required path/text/resolver, kept separate so adding a new option never breaks Preprocess's existing callers.

type Token

type Token struct {
	token.Token
	File        string
	MacroName   string
	InvokedFrom *Token
}

Token is one preprocessed token. Kind/Text/Line/Character (embedded from token.Token) always describe its true position in File -- they are never rewritten by expansion. MacroName/InvokedFrom are set iff this token is the result of expanding a macro, letting a caller walk back to the real invocation site (or, if that invocation was itself inside another expansion, further back still) instead of stopping at "somewhere inside a macro definition".