fastbelt

package module
v0.0.0-...-d8696ed Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 10, 2026 License: MIT Imports: 13 Imported by: 0

README

Fastbelt

Fastbelt is a high-performance DSL toolkit for Go with a parser generator and Language Server Protocol (LSP) support.

It is designed for language tooling that needs low latency and good throughput on large workspaces.

For background and benchmarks, see the Fastbelt introduction blog post.

Installation

Adding to a Module

Fastbelt ships as a Go module:

go get typefox.dev/fastbelt@latest
go get -tool typefox.dev/fastbelt/cmd/fastbelt@latest

Global Install

You can also globally install the fastbelt CLI:

go install typefox.dev/fastbelt/cmd/fastbelt@latest

Quick start

The first step is to write a grammar definition file, e.g. grammar.fb, which has a similar format as the grammar language of Langium.

To run the code generator for your grammar definition:

# globally installed
fastbelt generate ./grammar.fb -o ./

# on demand install
go run typefox.dev/fastbelt/cmd/fastbelt@latest generate ./grammar.fb -o ./

This writes generated Go files for services such as lexer, parser, linker, and type definitions.

Typically you will want to run generation using go generate. Add a directive to some file in your module (assumes install with go tool):

//go:generate go tool typefox.dev/fastbelt/cmd/fastbelt generate ./grammar.fb -o ./

Scaffolding

To bootstrap a new Go module for a language (minimal .fb grammar, go:generate using go tool on this CLI, LSP command, and VS Code extension layout), run:

fastbelt scaffold --module example.com/you/mylang --language "MyLanguage" --vscode

That creates a directory named after the last segment of --module (here ./mylang) in the current working directory, runs go mod init, pulls in fastbelt as a library and tool dependency, lays down the files, and runs go generate. Use fastbelt scaffold -h for full usage.

Examples

A minimal state machine example is available in examples/statemachine.

For editor integration, see the VS Code extension in internal/vscode-extensions/statemachine.

Contributing

See CONTRIBUTING.md for contribution guidelines.

License

Fastbelt is licensed under the MIT License.

Documentation

Overview

Package fastbelt is a language engineering toolkit for Go.

Overview

Fastbelt covers lexing, parsing, AST creation, cross-reference linking, workspace handling, and Language Server Protocol (LSP) support.

Fastbelt is primarily inspired by Xtext (Java) and Langium (TypeScript).

Like these frameworks, Fastbelt uses a grammar definition file as the entry point, based on a Grammar language that is itself implemented with Fastbelt. See typefox.dev/fastbelt/grammar for details.

The typefox.dev/fastbelt/cmd/fastbelt command generates Go code from a grammar definition, which you can integrate and customize with additional code.

Scaffolding

The typefox.dev/fastbelt/cmd/fastbelt command can scaffold a new language project with: 

fastbelt scaffold -module example.com/you/mylang -language "MyLanguage"

The scaffold creates these root files in the generated package directory:

  • gen.go with go:generate directives for grammar-based code generation.
  • services.go with customization points for generated services.
  • <language-id>.fb as the initial grammar definition file.
  • cmd/<language-id>-lsp/main.go as the LSP server entrypoint.

By default, scaffolding also creates VS Code extension files in a `vscode-extension` subfolder.

After writing templates, scaffolding runs go generate and go mod tidy so the generated parser, lexer, linker, and supporting files are ready to use.

Defining the Grammar

Defining a language in Fastbelt is an iterative design process. You co-design the concrete syntax that users write and the abstract syntax tree (AST) that your tooling consumes.

In practice, you write interface declarations for AST node types and parser rules that describe how instances of those node types are created from input text. Each assignment in a parser rule maps matched input to a field on the node that the rule builds.

Parser rules always consume contiguous regions of text. They match keywords and tokens directly and delegate to other rules for nested regions. This keeps grammar structure aligned with language structure and makes it clear how parsed text becomes AST subtrees.

Cross-references provide built-in linking from symbol usage to symbol definition. Instead of creating a new node, a cross-reference records an identifier that the linker resolves to an existing AST node in scope.

Because Go interfaces are generated directly from grammar interfaces, these interfaces should reflect how you plan to implement semantic checks, interpreters, or code generators. At the same time, there's a close correspondence between interfaces and parser rules so the parse model and runtime model stay easy to reason about.

The grammar language itself is documented in typefox.dev/fastbelt/grammar, while the code generation workflow is documented in typefox.dev/fastbelt/cmd/fastbelt. A common workflow is to evolve the grammar in small steps and run code generation after each change.

Customization

There are two ways to add custom behavior to your language:

  • Attach behavior directly to specific generated AST node `Impl` types.
  • Implement custom services and wire them through the service container.

Validations are implemented with the first approach. To add validation checks for a node type, implement Validator on that node's generated `Impl` struct: 

type PersonImpl struct {
    PersonData
}

func (p *PersonImpl) Validate(_ context.Context, _ string, accept fastbelt.ValidationAcceptor) {
    if name := p.Name(); name != "" && !unicode.IsUpper([]rune(name)[0]) {
        accept(fastbelt.NewDiagnostic(
            fastbelt.SeverityError,
            "Name must start with an uppercase letter.",
            p,
            fastbelt.WithToken(p.NameToken()),
        ))
    }
}

This keeps checks close to the node they validate and works naturally with generated AST types.

For service-container based customization, define a language-specific `SetupServices` function and a `CreateServices` helper. The service container API is documented at typefox.dev/fastbelt/util/service.

By convention, `SetupServices`:

  • registers custom values and service implementations,
  • calls needed `SetupDefaultServices` functions from framework packages, and
  • calls `SetupGeneratedServices` from generated code.

When you scaffold a new language project, Fastbelt generates a `services.go` file that follows this pattern and can be used as a starting point for your own service-container customization.

To Go Further

For implementing the exact behavior of your language, these subpackages are the most relevant:

Index

Constants

View Source 
const CommentGroup = -2

CommentGroup marks token types that the lexer should collect in Document.Comments.

View Source 
const FileScheme = "file"

FileScheme is the URI scheme used for local file system documents.

View Source 
const SkippedGroup = -1

SkippedGroup marks token types that the lexer should drop from all output streams.

Variables

View Source 
var EOF = NewTokenType(
	0,
	"EOF",
	"EOF",
	0,
	TokenKindToken,
	0,
	false,
	nil,
	nil,
)

EOF is the sentinel token type used to represent end of input.

View Source 
var EOFToken = NewToken(EOF, "", 0, 0, 0, 0, 0, 0)

EOFToken is the reusable sentinel token value for end of input.

View Source 
var EmptySymbolDescriptions = extiter.Empty[*SymbolDescription]()

EmptySymbolDescriptions is an empty SymbolSeq sentinel.

It can be reused by implementations that have no symbols to return.

Functions

func AllChildren 

func AllChildren(node AstNode) iter.Seq[AstNode]

AllChildren creates an iterator over all descendant nodes of the given node, excluding the node itself.

Early loop exit is honored correctly, but does not short-circuit the traversal.

func AllNodes 

func AllNodes(node AstNode) iter.Seq[AstNode]

AllNodes creates an iterator over the given node and all its descendant nodes.

Early loop exit is honored correctly, but does not short-circuit the traversal.

func AssignContainers 

func AssignContainers(doc *Document, root AstNode)

AssignContainers recursively assigns document and parent pointers for root and its subtree.

It also assigns document and container on composite reference units reachable via references.

func AssignToken 

func AssignToken(node AstNode, token *Token, kind int)

AssignToken appends token to node and records node and kind on the token.

It is primarily used by generated parsers while constructing nodes incrementally.

func AssignTokens 

func AssignTokens(node AstNode, tokens []*Token)

AssignTokens replaces node tokens and records node as owner for each token.

It is primarily used by generated parsers while constructing nodes incrementally.

func ChildNodes 

func ChildNodes(node AstNode) iter.Seq[AstNode]

ChildNodes creates an iterator over the direct child nodes of the given node.

This function wraps [AstNode.ForEachNode] in an iter.Seq. Early loop exit is honored correctly, but does not short-circuit the traversal.

func ContainerOfType 

func ContainerOfType[T AstNode](node AstNode) T

ContainerOfType walks up node's container chain and returns the first ancestor assignable to T. 

func DefaultLink(scope Scope, text string) (*SymbolDescription, *ReferenceError)

DefaultLink resolves text in scope and returns the matching symbol description.

DefaultLink is the standard implementation used by generated linker code. It returns the first match from [Scope.ElementByName], or a default ReferenceError if no symbol is found.

DefaultLink panics if scope is nil. Use EmptyScope to represent "no visible symbols".

func MergeTokens 

func MergeTokens(newNode AstNode, oldTokens []*Token)

MergeTokens prepends oldTokens to newNode's existing token list.

It is used when parser actions replace the current node while preserving already consumed text.

func References 

func References(node AstNode) iter.Seq[UntypedReference]

References creates an iterator over all references of the given node.

This function wraps [AstNode.ForEachReference] in an iter.Seq. Early loop exit is honored correctly, but does not short-circuit the traversal.

Types

type AstNode 

type AstNode interface {
	// Document returns the owning document of the node.
	Document() *Document
	// SetDocument sets the owning document of the node.
	//
	// When constructing an AST programmatically, use [AssignContainers] to link the node in the AST.
	SetDocument(document *Document)
	// Container returns the direct parent node of the node in the AST.
	// It returns nil if this is the root node.
	Container() AstNode
	// SetContainer sets the direct parent node of the node.
	//
	// When constructing an AST programmatically, use [AssignContainers] to link the node in the AST.
	SetContainer(container AstNode)
	// Tokens returns the tokens associated with the node.
	Tokens() []*Token
	// SetToken appends token to the node's token list.
	SetToken(token *Token)
	// SetTokens replaces the node's token list with tokens.
	SetTokens(tokens []*Token)
	// Segment returns the text segment metadata of the node.
	Segment() *TextSegment
	// SetSegment sets the full text segment metadata of the node.
	//
	// It is primarily used by generated parsers while constructing nodes incrementally.
	SetSegment(segment *TextSegment)
	// SetSegmentStartToken sets the start of the node's segment from token.
	//
	// It is primarily used by generated parsers while constructing nodes incrementally.
	SetSegmentStartToken(token *Token)
	// SetSegmentEndToken sets the end of the node's segment from token.
	//
	// It is primarily used by generated parsers while constructing nodes incrementally.
	SetSegmentEndToken(token *Token)
	// Text returns the source substring covered by the node's segment.
	Text() string
	// ForEachNode calls fn for each direct child node of node.
	//
	// Note that this does not traverse the entire subtree. Use [AllNodes] or [AllChildren] for that.
	//
	// Calling this method directly is not recommended. Use [ChildNodes] instead for better readability.
	ForEachNode(fn func(AstNode))
	// ForEachReference calls fn for each reference field of node.
	//
	// Calling this method directly is not recommended. Use [References] instead for better readability.
	ForEachReference(fn func(UntypedReference))
}

AstNode is the base interface for all AST nodes.

Every language-specific AST node type which is generated from a grammar definition embeds this interface.

type AstNodeBase 

type AstNodeBase struct {
	// contains filtered or unexported fields
}

AstNodeBase provides the default AstNode implementation used by generated AST node types.

func NewAstNode 

func NewAstNode() AstNodeBase

NewAstNode creates an AstNodeBase with initialized token storage.

It is intended for generated node implementations that embed AstNodeBase. AstNodeBase carries framework metadata only and has no language-specific semantic fields on its own.

func (*AstNodeBase) Container 

func (node *AstNodeBase) Container() AstNode

Container returns the direct parent node of the node in the AST. It returns nil if this is the root node.

func (*AstNodeBase) Document 

func (node *AstNodeBase) Document() *Document

Document returns the owning document of the node.

func (*AstNodeBase) ForEachNode 

func (node *AstNodeBase) ForEachNode(fn func(AstNode))

ForEachNode calls fn for each direct child node of node.

ForEachNode on AstNodeBase is a no-op because the base type has no child fields.

func (*AstNodeBase) ForEachReference 

func (node *AstNodeBase) ForEachReference(fn func(UntypedReference))

ForEachReference calls fn for each reference field of node.

ForEachReference on AstNodeBase is a no-op because the base type has no reference fields.

func (*AstNodeBase) Segment 

func (node *AstNodeBase) Segment() *TextSegment

Segment returns the text segment metadata of the node.

func (*AstNodeBase) SetContainer 

func (node *AstNodeBase) SetContainer(container AstNode)

SetContainer sets the direct parent node of the node.

func (*AstNodeBase) SetDocument 

func (node *AstNodeBase) SetDocument(document *Document)

SetDocument sets the owning document of the node.

func (*AstNodeBase) SetSegment 

func (node *AstNodeBase) SetSegment(segment *TextSegment)

SetSegment sets the full text segment metadata of the node.

func (*AstNodeBase) SetSegmentEndToken 

func (node *AstNodeBase) SetSegmentEndToken(token *Token)

SetSegmentEndToken sets the end of the node's segment from token.

func (*AstNodeBase) SetSegmentStartToken 

func (node *AstNodeBase) SetSegmentStartToken(token *Token)

SetSegmentStartToken sets the start of the node's segment from token.

func (*AstNodeBase) SetToken 

func (node *AstNodeBase) SetToken(token *Token)

SetToken appends token to the node's token list.

func (*AstNodeBase) SetTokens 

func (node *AstNodeBase) SetTokens(tokens []*Token)

SetTokens replaces the node's token list with tokens.

func (*AstNodeBase) Text 

func (node *AstNodeBase) Text() string

Text returns the source substring covered by the node's segment.

func (*AstNodeBase) Tokens 

func (node *AstNodeBase) Tokens() []*Token

Tokens returns the tokens associated with the node.

type CompositeNode 

type CompositeNode interface {
	AstNode
	StringUnit
	// IsCompositeNode marks a type as implementing [CompositeNode].
	IsCompositeNode()
}

CompositeNode represents a composed string value that is made up of multiple tokens.

A common example for this is a fully qualified name that consists of multiple identifiers and dots, e.g. "a.b.c". Every "composite" rule of a grammar will be represented as a CompositeNode in the AST, even if it only consists of a single token.

func NewCompositeNode 

func NewCompositeNode() CompositeNode

NewCompositeNode creates a CompositeNode backed by CompositeNodeBase.

type CompositeNodeBase 

type CompositeNodeBase struct {
	AstNodeBase
	// contains filtered or unexported fields
}

CompositeNodeBase provides the default CompositeNode implementation for generated composite rules.

func (*CompositeNodeBase) IsCompositeNode 

func (node *CompositeNodeBase) IsCompositeNode()

IsCompositeNode marks CompositeNodeBase as implementing CompositeNode.

func (*CompositeNodeBase) Owner 

func (node *CompositeNodeBase) Owner() AstNode

Owner returns the AST node that owns this string unit.

func (*CompositeNodeBase) String 

func (node *CompositeNodeBase) String() string

String returns the concatenated token images of node, caching the computed value.

type Diagnostic 

type Diagnostic struct {
	// Range is the source range where the diagnostic applies.
	Range TextRange
	// Severity is the diagnostic severity level; when unset, clients may use a default severity.
	Severity DiagnosticSeverity
	// Message is the primary human-readable diagnostic message.
	Message string
	// Source identifies the diagnostic source, such as a tool or check name.
	Source string
	// Code is an optional diagnostic identifier shown to users and used for filtering.
	Code string
	// CodeDescription provides additional metadata for Code, typically a documentation link.
	CodeDescription *DiagnosticCodeDescription
	// Tags carry additional semantic classification, such as unnecessary or deprecated code.
	Tags []DiagnosticTag
	// Data is preserved between publishDiagnostics and codeAction requests.
	Data any
}

Diagnostic represents a diagnostic message such as an error or warning. The struct mirrors lsp.Diagnostic so the core package stays free of that dependency.

func NewDiagnostic 

func NewDiagnostic(severity DiagnosticSeverity, message string, node AstNode, opts ...DiagnosticOption) *Diagnostic

NewDiagnostic creates a Diagnostic anchored to the given node's text range.

type DiagnosticCodeDescription 

type DiagnosticCodeDescription struct {
	Href string
}

A DiagnosticCodeDescription provides additional metadata for a diagnostic code.

It mirrors lsp.CodeDescription and currently only carries a documentation URL.

type DiagnosticOption 

type DiagnosticOption func(d *Diagnostic)

DiagnosticOption configures optional fields of a Diagnostic created by NewDiagnostic.

func WithCode 

func WithCode(code string) DiagnosticOption

WithCode sets the diagnostic code.

func WithCodeDescription 

func WithCodeDescription(codeDescription *DiagnosticCodeDescription) DiagnosticOption

WithCodeDescription sets additional metadata for the diagnostic code.

The current metadata format mirrors LSP and supports a documentation URL.

func WithCodeDescriptionHref 

func WithCodeDescriptionHref(href string) DiagnosticOption

WithCodeDescriptionHref sets a hyperlink for the error code description.

func WithData 

func WithData(data any) DiagnosticOption

WithData attaches arbitrary data to the diagnostic.

func WithRange 

func WithRange(r TextRange) DiagnosticOption

WithRange sets an explicit range on the diagnostic, overriding any node or token range.

func WithReference 

func WithReference(ref UntypedReference) DiagnosticOption

WithReference narrows the diagnostic range to the given reference text, if available.

func WithStringUnit 

func WithStringUnit(unit StringUnit) DiagnosticOption

WithStringUnit narrows the diagnostic range to the given StringUnit, if available.

func WithTags 

func WithTags(tags ...DiagnosticTag) DiagnosticOption

WithTags sets diagnostic tags (e.g. TagUnnecessary, TagDeprecated).

func WithToken 

func WithToken(token *Token) DiagnosticOption

WithToken narrows the diagnostic range to the given token's text segment. NOTE: These options might clash with other options in this package. If that happens, we can either rename them to DiagnosticToken etc. or move them to a separate package.

type DiagnosticSeverity 

type DiagnosticSeverity int

DiagnosticSeverity mirrors LSP DiagnosticSeverity values. 

const (
	SeverityError   DiagnosticSeverity = 1
	SeverityWarning DiagnosticSeverity = 2
	SeverityInfo    DiagnosticSeverity = 3
	SeverityHint    DiagnosticSeverity = 4
)

Diagnostic severity values used by Diagnostic.

func (DiagnosticSeverity) String 

func (s DiagnosticSeverity) String() string

String returns the human-readable label of s.

type DiagnosticTag 

type DiagnosticTag int

DiagnosticTag mirrors LSP DiagnosticTag values. 

const (
	TagUnnecessary DiagnosticTag = 1
	TagDeprecated  DiagnosticTag = 2
)

Diagnostic tag values used by Diagnostic.

type Document 

type Document struct {
	// URI identifies the document in the workspace.
	URI URI
	// State tracks which build phases already ran for this document.
	State DocumentState
	// Root is the AST root produced by parsing.
	// It is nil until parsing succeeds.
	Root AstNode
	// Tokens is the token slice produced by lexing.
	// It is empty until lexing succeeds.
	Tokens TokenSlice
	// Comments contains comment tokens extracted by the lexer.
	// It is empty until lexing succeeds.
	Comments TokenSlice
	// LocalSymbols stores symbols declared in this document.
	// It is nil until the local symbols build phase succeeds.
	LocalSymbols LocalSymbols
	// ExportedSymbols stores symbols this document contributes to global linking.
	// It is nil until the exported symbols build phase succeeds.
	ExportedSymbols SymbolContainer
	// ImportedSymbols stores symbols imported from other workspace documents.
	// It is nil until the imported symbols build phase succeeds.
	ImportedSymbols SymbolContainer
	// ParserErrors contains syntax and parser-recovery errors.
	// It is empty until parsing succeeds.
	ParserErrors []*ParserError
	// LexerErrors contains lexical errors found while tokenizing the document text.
	// It is empty until lexing succeeds.
	LexerErrors []*LexerError
	// References contains unresolved and resolved cross-references found during linking.
	// It is empty until the linking build phase succeeds.
	References []UntypedReference
	// ReferenceDescriptions describes references for features such as "find references".
	// It is nil until the reference descriptions build phase succeeds.
	ReferenceDescriptions ReferenceDescriptions
	// TextDoc is the backing text document handle.
	TextDoc textdoc.Handle
	// Diagnostics contains validation and analysis diagnostics for editor clients.
	// It is empty until the validation build phase succeeds.
	Diagnostics []*Diagnostic
	// Data can be used to store arbitrary additional information related to the document.
	// This is not used by the framework itself, but adopters may find it useful.
	// The document builder does not clear this data during the build process.
	// It is the responsibility of the caller to manage it appropriately (e.g. clearing or
	// updating it when the document changes).
	//
	// The map is concurrent to allow storing data from different goroutines without
	// additional synchronization.
	Data sync.Map
}

Document represents a document in the workspace. The data stored does not have to be complete during the whole lifecycle of the document. For example, the Root node may be nil if the document has not been parsed yet.

Access to the fields of Document should be synchronized using a typefox.dev/fastbelt/workspace Lock. The document struct should never be copied after creation.

func NewDocument 

func NewDocument(textDoc textdoc.Handle) *Document

NewDocument creates a Document from a text document handle.

It initializes all build-related collections to empty values and sets [Document.TextDoc] and [Document.URI], but leaves semantic data (such as [Document.Root]) unset until the corresponding build phases run.

func NewDocumentFromString 

func NewDocumentFromString(uri, languageId, content string) (*Document, error)

NewDocumentFromString creates a Document backed by an in-memory text document.

It is useful in tests, benchmarks, and tooling code that needs a document instance without going through the workspace file-loading pipeline.

type DocumentState 

type DocumentState uint32

DocumentState is a bitmask capturing the already completed build phases of a document. 

const (
	// DocStateParsed marks that lexing and parsing completed and produced AST and token data.
	DocStateParsed DocumentState = 1 << iota // 0x0001
	// DocStateExportedSymbols marks that exported symbols were collected for cross-document linking.
	DocStateExportedSymbols // 0x0002
	// DocStateImportedSymbols marks that symbols from other documents were imported.
	DocStateImportedSymbols // 0x0004
	// DocStateLocalSymbols marks that local symbols for this document were collected.
	DocStateLocalSymbols // 0x0008
	// DocStateLinked marks that cross-references were linked.
	DocStateLinked // 0x0010
	// DocStateReferences marks that reference descriptions were collected.
	DocStateReferences // 0x0020
	// DocStateValidated marks that validators were executed and diagnostics were collected.
	DocStateValidated // 0x0040
)

func (DocumentState) Has 

func (s DocumentState) Has(flag DocumentState) bool

Has reports whether flag is set in s.

func (DocumentState) String 

func (s DocumentState) String() string

String returns a readable representation of the set state flags.

func (DocumentState) With 

func (s DocumentState) With(flag DocumentState) DocumentState

With returns s with flag set.

func (DocumentState) Without 

func (s DocumentState) Without(flag DocumentState) DocumentState

Without returns s with flag cleared.

type LexerError 

type LexerError struct {
	// Msg is the lexer diagnostic message.
	Msg string
	// StartOffset is the byte offset where the invalid segment starts.
	StartOffset int
	// EndOffset is the exclusive byte offset where the invalid segment ends.
	EndOffset int
	// StartLine is the zero-based line where the invalid segment starts.
	StartLine int
	// EndLine is the zero-based line where the invalid segment ends.
	EndLine int
	// StartColumn is the zero-based column where the invalid segment starts.
	StartColumn int
	// EndColumn is the zero-based column where the invalid segment ends.
	EndColumn int
}

A LexerError describes invalid input encountered during tokenization.

func NewLexerError 

func NewLexerError(msg string, startOffset, endOffset, startLine, endLine, startColumn, endColumn int) *LexerError

NewLexerError returns a LexerError for msg and the offending text range.

type LocalSymbols 

type LocalSymbols interface {
	// For returns the [SymbolContainer] for the given AST node, which contains all symbols
	// that are locally visible in that node.
	For(node AstNode) SymbolContainer
}

LocalSymbols are used for lexical scoping within a document.

type MapScope 

type MapScope struct {
	// contains filtered or unexported fields
}

MapScope is a scope backed by a name-indexed multimap.

MapScope is optimized for repeated name lookups by grouping local symbols by their string name and optionally chaining to an outer scope.

func NewMapScope 

func NewMapScope(elements collections.MultiMap[string, *SymbolDescription], outer Scope) *MapScope

NewMapScope creates a MapScope from a name-indexed symbol multimap and an outer scope.

func NewMapScopeFromSeq 

func NewMapScopeFromSeq(elements iter.Seq[*SymbolDescription], outer Scope) *MapScope

NewMapScopeFromSeq builds a MapScope from an iterator of symbols.

Symbols are consumed once and indexed by their [SymbolDescription.Name] string.

func NewMapScopeFromSlice 

func NewMapScopeFromSlice(elements []*SymbolDescription, outer Scope) *MapScope

NewMapScopeFromSlice builds a MapScope from a slice of symbols.

Symbols are indexed by their [SymbolDescription.Name] string.

func (*MapScope) AllElements 

func (s *MapScope) AllElements() iter.Seq[*SymbolDescription]

AllElements returns all local elements and, when present, all outer elements.

Local iteration order follows the underlying multimap and is not guaranteed.

func (*MapScope) ElementByName 

func (s *MapScope) ElementByName(name string) *SymbolDescription

ElementByName returns one local symbol named name, then checks outer.

If multiple local symbols have the same name, the first inserted symbol is returned.

func (*MapScope) ElementsByName 

func (s *MapScope) ElementsByName(name string) iter.Seq[*SymbolDescription]

ElementsByName returns all local symbols named name followed by outer matches.

type NamedCompositeNode 

type NamedCompositeNode interface {
	NamedNode
	// NameNode returns the composite node stored in the node's "Name" field.
	NameNode() CompositeNode
}

NamedCompositeNode represents a NamedNode whose name is represented by a CompositeNode, stored in the "Name" field of the node.

type NamedNode 

type NamedNode interface {
	AstNode
	// Name returns the name of this node as a string.
	Name() string
}

NamedNode represents an AstNode whose name is accessible as a string in the Name field.

type NamedTokenNode 

type NamedTokenNode interface {
	NamedNode
	// NameToken returns the token stored in the node's "Name" field.
	NameToken() *Token
}

NamedTokenNode represents a NamedNode whose name is represented by a Token, stored in the "Name" field of the node.

type ParserError 

type ParserError struct {
	// Msg is the parser diagnostic message.
	Msg string
	// Token is the token this error is associated with.
	// It is nil when the parser error refers to unexpected end of input.
	Token *Token
}

A ParserError describes a syntax error detected by the parser.

func NewParserError 

func NewParserError(msg string, token *Token) *ParserError

NewParserError returns a ParserError for msg at token.

type Reference 

type Reference[T AstNode] struct {
	// contains filtered or unexported fields
}

Reference represents a reference to another AST node of type T.

Resolving a reference is thread safe. The resolution is triggered when Reference.Ref, Reference.RefNode or Reference.Resolve are called for the first time.

func NewReference 

func NewReference[T AstNode](owner AstNode, unit StringUnit, getter ReferenceGetter[T]) *Reference[T]

NewReference creates a lazy, typed reference owned by owner and backed by getter.

func (*Reference[T]) Description 

func (r *Reference[T]) Description() *SymbolDescription

Description returns the resolved symbol description, or nil when unresolved.

func (*Reference[T]) Error 

func (r *Reference[T]) Error() *ReferenceError

Error returns the resolution error for this reference, if any.

func (*Reference[T]) Owner 

func (r *Reference[T]) Owner() AstNode

Owner returns the AST node that owns this reference.

func (*Reference[T]) Ref 

func (r *Reference[T]) Ref(ctx context.Context) T

Ref resolves the reference and returns the typed target node. It returns the zero value of T when resolution fails.

func (*Reference[T]) RefNode 

func (r *Reference[T]) RefNode(ctx context.Context) AstNode

RefNode returns the resolved target node as AstNode.

func (*Reference[T]) Reset 

func (r *Reference[T]) Reset()

Reset clears all cached resolution results so the reference can be resolved again.

func (*Reference[T]) Resolve 

func (r *Reference[T]) Resolve(ctx context.Context)

Resolve resolves the reference exactly once for this instance. It is safe for concurrent use.

func (*Reference[T]) Segment 

func (r *Reference[T]) Segment() *TextSegment

Segment returns the text segment of the reference in the source document.

func (*Reference[T]) Text 

func (r *Reference[T]) Text() string

Text returns the textual value of the reference, or an empty string if unavailable.

func (*Reference[T]) Unit 

func (r *Reference[T]) Unit() StringUnit

Unit returns the StringUnit that contains the textual reference.

type ReferenceDescription 

type ReferenceDescription struct {
	// SourceNode is the node that contains the reference.
	SourceNode AstNode
	// TargetNode is the node that is being referenced.
	TargetNode AstNode
	// Segment is the text segment of the reference in the source document (usually the symbol name).
	Segment *TextSegment
}

ReferenceDescription describes one concrete use of a symbol in source text.

func NewReferenceDescription 

func NewReferenceDescription(source, target AstNode, segment *TextSegment) *ReferenceDescription

NewReferenceDescription creates a ReferenceDescription for a source-to-target link.

func (*ReferenceDescription) SourceURI 

func (d *ReferenceDescription) SourceURI() URI

SourceURI returns the URI of the document containing the source node, or nil if not available.

func (*ReferenceDescription) TargetURI 

func (d *ReferenceDescription) TargetURI() URI

TargetURI returns the URI of the document containing the target node, or nil if not available.

type ReferenceDescriptions 

type ReferenceDescriptions interface {
	// All returns an iterator over all reference descriptions in the document.
	All() iter.Seq[*ReferenceDescription]
	// ForTarget returns an iterator over all reference descriptions that point to the given target node.
	ForTarget(target AstNode) iter.Seq[*ReferenceDescription]
}

ReferenceDescriptions is a collection of reference descriptions for a document, indexed by target node. It allows efficient retrieval of all references to a given target node.

func NewReferenceDescriptionsFromMap 

func NewReferenceDescriptionsFromMap(descriptions collections.MultiMap[AstNode, *ReferenceDescription]) ReferenceDescriptions

NewReferenceDescriptionsFromMap wraps precomputed descriptions keyed by target node.

type ReferenceError 

type ReferenceError struct {
	// Msg is the human-readable error message shown in diagnostics.
	Msg string
	// Severity is an LSP-compatible [DiagnosticSeverity] value.
	// Use [SeverityError], [SeverityWarning], [SeverityInfo], or [SeverityHint].
	Severity int
}

A ReferenceError reports a failure while resolving a cross-reference. It is converted to diagnostics during linking.

func NewReferenceError 

func NewReferenceError(msg string) *ReferenceError

NewReferenceError returns a ReferenceError with SeverityError.

type ReferenceGetter 

type ReferenceGetter[T AstNode] func(context.Context, *Reference[T]) (*SymbolDescription, *ReferenceError)

ReferenceGetter resolves a Reference and returns its symbol description. The returned ReferenceError is stored on the reference when not nil.

type Scope 

type Scope interface {
	// ElementByName returns one symbol for name, or nil when no symbol matches.
	//
	// When multiple symbols with the same name are visible, the returned element
	// is implementation-defined.
	ElementByName(name string) *SymbolDescription
	// ElementsByName returns all visible symbols named name.
	//
	// Implementations can return multiple symbols for overload-like scenarios or
	// duplicate declarations.
	ElementsByName(name string) iter.Seq[*SymbolDescription]
	// AllElements returns all symbols visible in this scope chain.
	AllElements() iter.Seq[*SymbolDescription]
}

Scope provides name-based lookup for symbol descriptions visible at a reference site.

Implementations usually represent one lexical scope and optionally chain to an outer scope. 

var EmptyScope Scope = &emptyScope{}

EmptyScope is a scope with no symbols.

It is used as a non-nil sentinel for "no visible symbols", for example when a scope provider cannot compute applicable candidates.

type SeqScope 

type SeqScope struct {
	// contains filtered or unexported fields
}

SeqScope is a scope backed by an iterator over symbol descriptions.

SeqScope performs linear lookup over its local elements and optionally delegates misses to an outer scope.

func NewSeqScope 

func NewSeqScope(elements iter.Seq[*SymbolDescription], outer Scope) *SeqScope

NewSeqScope creates a SeqScope from local elements and an optional outer scope.

Local elements are searched before the outer scope.

func (*SeqScope) AllElements 

func (s *SeqScope) AllElements() iter.Seq[*SymbolDescription]

AllElements returns local elements followed by all outer elements.

func (*SeqScope) ElementByName 

func (s *SeqScope) ElementByName(name string) *SymbolDescription

ElementByName returns the first local symbol named name, then checks outer.

func (*SeqScope) ElementsByName 

func (s *SeqScope) ElementsByName(name string) iter.Seq[*SymbolDescription]

ElementsByName returns all local symbols named name followed by outer matches.

type StringUnit 

type StringUnit interface {
	// Owner returns the AST node that owns this string unit.
	Owner() AstNode
	// Segment returns the text segment metadata of this string unit.
	Segment() *TextSegment
	// String returns the string representation of this string unit.
	String() string
}

StringUnit is a common interface for both Token and CompositeNode.

type SymbolContainer 

type SymbolContainer interface {
	// Put attempts to put the given description into the container.
	// Returns true if the description was added.
	// The container is allowed to reject descriptions that it does not want to hold, for example
	// because they are of the wrong type.
	Put(desc *SymbolDescription) bool
	// All returns an iterator over all descriptions in the container.
	All() SymbolSeq
	// ForType returns an iterator over all descriptions in the container whose node is of the given type.
	ForType(targetType reflect.Type) SymbolSeq
}

A SymbolContainer is an efficient data structure for storing and querying symbol descriptions. References usually need to query symbols for specific AST types. Language specific implementations optimize for this by indexing descriptions by the type of their AST node. 

var EmptySymbolContainer SymbolContainer = &emptySymbolContainer{}

EmptySymbolContainer is an immutable SymbolContainer with no symbols.

func MergeSymbolContainers 

func MergeSymbolContainers(containers iter.Seq[SymbolContainer]) SymbolContainer

MergeSymbolContainers merges multiple symbol containers into one. The resulting container is immutable and reflects the combined contents of all input containers.

type SymbolContainers 

type SymbolContainers interface {
	// New returns a container instance for storing symbol descriptions.
	New() SymbolContainer
}

SymbolContainers is a service that is able to generate new SymbolContainer items for the current language. It is used for the [Document.LocalSymbols], [Document.ExportedSymbols], and [Document.ImportedSymbols] fields.

type SymbolDescription 

type SymbolDescription struct {
	// URI is the document URI where the symbol is declared.
	URI URI
	// Node is the AST node that declares the symbol.
	Node AstNode
	// Name is the source unit that provides the symbol's textual name.
	Name StringUnit
}

SymbolDescription describes a named AST declaration that references can resolve to.

func NewCompositeNodeSymbolDescription 

func NewCompositeNodeSymbolDescription(node NamedCompositeNode) *SymbolDescription

NewCompositeNodeSymbolDescription returns a SymbolDescription for a NamedCompositeNode.

It uses [NamedCompositeNode.NameNode] as the symbol name.

func NewSymbolDescription 

func NewSymbolDescription(node AstNode, name StringUnit) *SymbolDescription

NewSymbolDescription returns a SymbolDescription for node and name.

The description URI is derived from node's document.

func NewTokenSymbolDescription 

func NewTokenSymbolDescription(node NamedTokenNode) *SymbolDescription

NewTokenSymbolDescription returns a SymbolDescription for a NamedTokenNode.

It uses [NamedTokenNode.NameToken] as the symbol name.

type SymbolSeq 

type SymbolSeq = iter.Seq[*SymbolDescription]

SymbolSeq is a sequence of symbol descriptions.

type TextColumn 

type TextColumn int32

TextColumn is a zero-based byte column within a line.

type TextIndex 

type TextIndex int32

TextIndex is a zero-based byte offset in source text.

type TextIndexRange 

type TextIndexRange struct {
	// Start is the inclusive start byte offset.
	Start TextIndex
	// End is the exclusive end byte offset.
	End TextIndex
}

A TextIndexRange describes a half-open byte range [Start, End).

type TextLine 

type TextLine int32

TextLine is a zero-based line number in source text.

type TextLocation 

type TextLocation struct {
	// Line is the zero-based line number.
	Line TextLine
	// Column is the zero-based byte column in Line.
	Column TextColumn
}

A TextLocation identifies a position in source text.

func (TextLocation) LspPosition 

func (l TextLocation) LspPosition() lsp.Position

LspPosition returns l as an lsp.Position using the same coordinates.

type TextRange 

type TextRange struct {
	// Start is the inclusive start location.
	Start TextLocation
	// End is the exclusive end location.
	End TextLocation
}

A TextRange describes a half-open range from Start to End.

func (TextRange) LspRange 

func (r TextRange) LspRange() lsp.Range

LspRange returns r as an lsp.Range using the same boundaries.

type TextSegment 

type TextSegment struct {
	// Indices stores the span as byte offsets.
	Indices TextIndexRange
	// Range stores the same span as line and column locations.
	Range TextRange
}

A TextSegment combines byte offsets and line/column locations for one span.

type Token 

type Token struct {
	// Type points to the matched token type metadata.
	Type *TokenType
	// Image is the exact text matched for this token.
	Image string
	// TypeId caches Type.Id for parser hot paths.
	TypeId int
	// TextSegment stores byte offsets and line/column ranges for this token.
	TextSegment TextSegment
	// Element points to the AST node this token was assigned to during parsing.
	Element AstNode
	// Kind stores the generated assignment slot identifier within Element.
	Kind int
}

Token represents one lexed source slice with positional metadata.

func NewToken 

func NewToken(tokenType *TokenType, image string, startOffset, endOffset, startLine, endLine, startColumn, endColumn int) Token

NewToken creates a token with image text and half-open source coordinates.

func (*Token) Assign 

func (t *Token) Assign(element AstNode, kind int)

Assign records the owning AST element and assignment slot kind for t.

func (*Token) IsEOF 

func (t *Token) IsEOF() bool

IsEOF reports whether t uses the end-of-input token type.

func (*Token) IsSkipped 

func (t *Token) IsSkipped() bool

IsSkipped reports whether t belongs to the skipped lexer group.

func (*Token) Owner 

func (t *Token) Owner() AstNode

Owner returns the AST node that owns t as a string unit.

For tokens attached to a CompositeNode, Owner returns the composite's container.

func (*Token) Segment 

func (t *Token) Segment() *TextSegment

Segment returns the token text segment.

func (*Token) String 

func (t *Token) String() string

String returns the token image.

type TokenKind 

type TokenKind int

TokenKind names the grammar construct that produced a TokenType. It is distinct from Group: Group controls lexer-stream behaviour (skipped / comment), while Kind describes the grammar origin and is consumed by downstream features such as the completion engine, which by default only surfaces keyword-kind tokens as completion candidates. 

const (
	// TokenKindToken is the default - a TokenType produced by a named
	// `token` rule in the .fb grammar (regex-matched). Hidden and comment
	// tokens are also TokenKindToken; their stream behaviour is encoded
	// separately in Group.
	TokenKindToken TokenKind = 0
	// TokenKindKeyword is a TokenType produced by a literal string in a
	// parser rule (e.g. `"statemachine"`). Matched by a string prefix.
	TokenKindKeyword TokenKind = 1
	// TokenKindGroup is a TokenType that represents a named group of other
	// TokenTypes. It is not directly matched against the input, but instead
	// serves as a convenient (and fast!) way to refer to multiple TokenTypes
	// in the grammar and downstream features.
	TokenKindGroup TokenKind = 2
)

type TokenMatcher 

type TokenMatcher func(input string, offset int) int

TokenMatcher is a function that attempts to match a token at the given offset in the input string. Returns the byte-length of the match if successful, or 0 if no match is found.

type TokenSlice 

type TokenSlice []Token

TokenSlice is a token sequence sorted by source offsets.

func (TokenSlice) SearchOffset 

func (ts TokenSlice) SearchOffset(offset int) *Token

SearchOffset returns the token that contains offset.

It expects ts to be sorted by token offsets and uses binary search.

type TokenType 

type TokenType struct {
	// Id is the stable numeric token identifier used in parser tables and token bitsets.
	Id int
	// Name is the token name from generated code.
	Name string
	// Label is the user-facing token label, for example in completion output.
	Label string
	// StartChars contains candidate start runes used for lexer preselection.
	StartChars []rune
	// Group controls lexer output routing (default stream, skipped, comments, or custom groups).
	Group int
	// Kind records whether the token comes from a keyword literal or a token rule.
	Kind TokenKind
	// PushMode selects the next lexer mode after this token is matched.
	PushMode int
	// PopMode reports whether matching this token pops one lexer mode.
	PopMode bool
	// Match performs the actual token match at a given input offset.
	Match TokenMatcher
	// Matches returns whether the token type matches another, given type
	Matches TokenTypeMatcher
	// All TokenTypes that are matched by this TokenType.
	MatchingTokens []*TokenType
	// contains filtered or unexported fields
}

TokenType describes one lexer token category generated from a grammar.

func NewTokenGroup 

func NewTokenGroup(id int, name, label string, matchingTypes []*TokenType) *TokenType

NewTokenGroup creates a token type that represents a named group of other token types.

func NewTokenType 

func NewTokenType(id int, name, label string, group int, kind TokenKind, pushMode int, popMode bool, match TokenMatcher, startChars []rune) *TokenType

NewTokenType creates a token type descriptor used by generated lexers and parsers.

func (*TokenType) Bitset 

func (t *TokenType) Bitset() *collections.BitSet

Bitset returns a bitset that contains all token type IDs matched by t.

func (*TokenType) IsComment 

func (t *TokenType) IsComment() bool

IsComment reports whether t is routed to the comment-token group.

func (*TokenType) IsKeyword 

func (t *TokenType) IsKeyword() bool

IsKeyword reports whether t originates from a grammar keyword literal.

func (*TokenType) IsSkipped 

func (t *TokenType) IsSkipped() bool

IsSkipped reports whether t is routed to the skipped-token group.

type TokenTypeMatcher 

type TokenTypeMatcher func(other *TokenType) bool

TokenTypeMatcher is a function that checks if the specified other TokenType can be matched by this TokenType. Used for optimizations in the lookahead and other parts of the parser.

type URI 

type URI interface {
	// Scheme returns the URI scheme.
	Scheme() string
	// Authority returns the URI authority.
	Authority() string
	// Path returns the URI path.
	Path() string
	// Query returns the URI query component without a leading '?'.
	Query() string
	// Fragment returns the URI fragment component without a leading '#'.
	Fragment() string
	// String returns the URI as a string with percent-encoding applied to the components as needed.
	// This is the standard way to serialize URIs and should be used when interoperability with other tools is required.
	String() string
	// StringUnencoded returns the URI as a string without percent-encoding.
	// This is useful for debugging and logging purposes.
	StringUnencoded() string
	// DocumentURI converts the URI to a lsp.DocumentURI, which is the format used by the LSP library.
	DocumentURI() lsp.DocumentURI
	// WithScheme returns a new URI with the specified scheme, keeping other components unchanged.
	WithScheme(scheme string) URI
	// WithAuthority returns a new URI with the specified authority, keeping other components unchanged.
	WithAuthority(authority string) URI
	// WithPath returns a new URI with the specified path, keeping other components unchanged.
	WithPath(path string) URI
	// WithQuery returns a new URI with the specified query, keeping other components unchanged.
	WithQuery(query string) URI
	// WithFragment returns a new URI with the specified fragment, keeping other components unchanged.
	WithFragment(fragment string) URI
	// With returns a new URI with the specified components, keeping other components unchanged.
	// The components are pointers, so you can pass nil for components that should remain unchanged.
	//
	// Note that this method will not validate or normalize the components,
	// so it's the caller's responsibility to ensure that the resulting URI is valid.
	With(scheme, authority, path, query, fragment *string) URI
	// Equal checks if this [URI] is equal to another [URI], based on their components.
	// Note that this comparison is case-sensitive for all components.
	// Use [FileURI] and [ParseURI] to ensure consistent normalization for reliable comparisons.
	Equal(other URI) bool
}

A URI represents a parsed URI split into scheme, authority, path, query, and fragment components.

URI values are treated as immutable: methods prefixed with With return a new URI instead of mutating the receiver.

func FileURI 

func FileURI(path string) URI

FileURI returns a file URI for path.

FileURI replaces Windows backslashes with forward slashes, ensures the URI path starts with a slash, and normalizes Windows drive letters to uppercase. This is used when creating document URIs from workspace file paths.

func NewURI 

func NewURI(scheme, authority, path, query, fragment string) URI

NewURI returns a URI from the given components. The components are not validated or normalized, so callers must ensure they are valid for their use case.

Instead of creating a new URI from scratch, it is recommended to parse URIs from a string using ParseURI.

func ParseURI 

func ParseURI(value string) URI

ParseURI parses value into URI components.

ParseURI is lenient: it extracts components best-effort instead of failing on malformed input.

ParseURI normalizes scheme and authority to lowercase and normalizes Windows drive letters in paths to uppercase for stable comparisons across platforms.

type UntypedReference 

type UntypedReference interface {
	Owner() AstNode
	Description() *SymbolDescription
	RefNode(ctx context.Context) AstNode
	Resolve(ctx context.Context)
	Reset()
	Error() *ReferenceError
	Unit() StringUnit
	Segment() *TextSegment
	Text() string
}

UntypedReference is an untyped representation of a Reference that can be used when the target type is not known at compile time.

Used throughout the fastbelt codebase to generically deal with different references.

func ReferenceOfToken 

func ReferenceOfToken(token *Token) UntypedReference

Computes the reference that this token represents. Returns nil if the token does not represent a reference.

type ValidationAcceptor 

type ValidationAcceptor func(diagnostic *Diagnostic)

ValidationAcceptor is a callback that collects diagnostics reported during validation.

type Validator 

type Validator interface {
	// Validate performs validation on the receiver node.
	// The level parameter identifies when validation runs (e.g. "on-type", "on-save").
	// The accept callback is used to collect diagnostics.
	Validate(ctx context.Context, level string, accept ValidationAcceptor)
}

Validator can be implemented by AST node Impl structs to provide custom validation checks.

Source Files

View all Source files

Directories

Path Synopsis
cmd
fastbelt command
examples
statemachine
statemachine/server command
Package grammar documents the Fastbelt grammar language.
 Package grammar documents the Fastbelt grammar language.
internal
atn
automatons
bench command
generator
grammar
grammar/server command
languages/completion
languages/token_groups
regexp
regexp/benchmark
regexp/benchmark/generated
scaffold
Package test provides utilities for testing Fastbelt language implementations.
 Package test provides utilities for testing Fastbelt language implementations.
util
codegen
collections
extiter
service

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.