treesitter

package module
v0.0.0-...-abc3e95 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2026 License: MIT Imports: 10 Imported by: 0

README

go-tree-sitter

A tree-sitter runtime written in pure Go. No cgo, no WebAssembly runtime, and no external binary at run time.

Every other Go binding to tree-sitter is a cgo wrapper around the upstream C library. That rules them all out for a consumer such as slopfix. It ships as a fat APE, cross-compiled for linux and darwin on amd64 and arm64 with CGO_ENABLED=0. This runtime builds under that setting. It therefore cross-compiles to any target Go supports.

The three parts, because this is the thing readers get wrong

A grammar.js file is data. It describes a language and it parses nothing.

tree-sitter generate reads that data and emits a parse table, as C source. The table is the language.

A runtime walks that table against real bytes and builds a tree. This repository is the runtime. grammars/ holds tables cmd/ts-translate reads out of that C.

Install

go get github.com/wow-look-at-my/go-tree-sitter

Parse a file

package main

import (
	"fmt"

	ts "github.com/wow-look-at-my/go-tree-sitter"
	"github.com/wow-look-at-my/go-tree-sitter/grammars/golang"
)

func main() {
	parser := ts.NewParser()
	parser.SetLanguage(golang.Language())

	tree := parser.ParseString(nil, []byte("package main\n"))
	root := tree.RootNode()

	fmt.Println(root.Type())       // source_file
	fmt.Println(root.StartPoint()) // {0 0}
	fmt.Println(root.String())     // (source_file (package_clause (package_identifier)))
}

Tree.Walk returns a cursor for a full traversal.

Status

Each grammar runs the upstream grammar repository's own test/corpus suite, and every case in it passes. The real tree-sitter test command reports the same case counts on the same commits. That is what proves the reader is not dropping cases.

The corpus reader is a port of the tree-sitter CLI's own reader. A case is therefore read and compared the way upstream reads and compares it.

Grammars

Go, C, C++, Rust, Bash, JavaScript, TypeScript and TSX.

A grammar that ships a hand-written scanner.c needs that scanner ported by hand. No machine translation reads C well enough to do it. grammars/*/scanner.go is that port. The upstream corpus says whether it is right. TSX shares TypeScript's scanner, the way upstream compiles both from one header.

ts-translate writes tables.zst per grammar, and the parser.go that embeds it. Neither is committed. Both are derived. The repository keeps the input, and the generate step produces the rest at build time, into the binary. grammar.go beside them is hand written and compiles without them, which is what lets a consumer import a grammar for its Scanner(). Language() panics until the generate step ran.

The blob holds the lexer too. A tree-sitter lexer arrives as a C function of a few thousand gotos. Translating that into Go source put about a megabyte of it in each grammar. It is a state machine, so it travels as data instead. The generated Go is down to the loader that embeds the blob. lexprog.go defines multiple instructions and runs them.

A whole parse costs about 12% more than the compiled form did, which BenchmarkParse measures.

Queries

The query engine is a port of lib/src/query.c from tree-sitter v0.27.2, commit 3fb7be49145acfb07e33b61ed5689e2b6771b2e9, plus the predicate layer the Rust binding adds in lib/binding_rust/lib.rs. The query*.go files hold it, one concern per file, and every function names the upstream function it ports.

query, err := ts.NewQuery(golang.Language(), `(function_declaration name: (identifier) @name)`)
if err != nil {
	log.Fatal(err) // a *ts.QueryError with Kind, Offset, Row and Column
}
cursor := ts.NewQueryCursor()
for match := range cursor.Matches(query, tree.RootNode(), source) {
	for _, capture := range match.Captures {
		fmt.Println(query.CaptureNameForID(capture.Index), capture.Node.Type())
	}
}

Matches and Captures are iter.Seq and iter.Seq2 wrappers that apply the text predicates. Byte and point ranges, containing ranges, the match limit, the maximum start depth and the progress callback all carry their upstream names. Any other predicate comes back as data.

Oracles check the port against upstream:

  • The upstream query suite, crates/cli/src/tests/query_test.rs, is ported in query_upstream*_test.go. Every test is ported or sits in the skip table with its reason. It generates a grammar from source, uses a test fixture grammar, or draws random patterns. A test written for a grammar this repository lacks is rewritten against one it has, and the pinned CLI confirmed each rewrite.
  • Goldens from tree-sitter query 0.27.2, in match and captures mode, for every bundled .scm file over each grammar's samples, small examples and corpus inputs. Every golden file matches byte for byte. Examples over 16 KB are left out to keep the goldens small.
  • Compile parity: every bundled query file compiles in both. Hand-written invalid queries under grammars/golang/testdata/invalid-queries cover every error kind, with the same kind, offset, row and column as the CLI.
Check Count
Upstream query tests 131
Ported and passing 124
Skipped, with a reason 7
Query golden files, byte for byte 416
Bundled .scm files, compile parity 22
Hand-written invalid queries 9

FuzzNewQuery and FuzzQueryExec fuzz compilation and execution. BenchmarkCompileHighlights and BenchmarkHighlightCaptures in the Rust and TypeScript grammars report allocations per operation.

Known divergences from upstream:

  • #match? uses Go's regexp, which is RE2 syntax, where upstream uses the Rust regex crate. The bundled queries use no construct the two read differently. query_regex_test.go lists the constructs that differ and refuses a bundled regex that uses one.
  • The pattern reader classifies letters and spaces as the C locale does, with ASCII only, because that is the locale the CLI runs under.
  • A query with CRLF line endings reports the column the CLI reports. The Rust binding drops each \r while it counts lines, so the column drifts by one per earlier line. The byte offset is exact in both.
  • A TextProvider returns one byte slice per node. The Rust one yields chunks.
  • Executing over a null node reports no matches. Upstream reads through the null pointer.

The limit worth knowing

A grammar reaches this runtime by way of cmd/ts-translate, which refuses an ABI outside the range parser_core.go accepts. A grammar outside that range fails the build rather than becoming a package that loads and parses nothing.

Tests

The corpora are git submodules. A clone needs them:

git clone --recurse-submodules https://github.com/wow-look-at-my/go-tree-sitter.git
cd go-tree-sitter
go-toolchain

A checkout without the submodules fails the corpus tests loudly rather than reporting a green over zero cases.

Each grammar's bundled queries also run against its samples, its small examples and its corpus inputs. The output must match tree-sitter query 0.27.2 byte for byte. The goldens live in grammars/*/testdata/queries, and internal/querygoldens describes their layout. To record them again:

TS_GOLDENS_RECORD=1 TS_GOLDENS_CLI=/path/to/tree-sitter go-toolchain

ts-goldens query -lang all -cli /path/to/tree-sitter does the same from a built binary.

Documentation

Index

Constants

View Source
const (
	ParseActionTypeShift uint8 = iota
	ParseActionTypeReduce
	ParseActionTypeAccept
	ParseActionTypeRecover
)

Parse action types.

View Source
const (
	MinABIVersion = 13
	MaxABIVersion = 15
)

A generator must refuse an ABI outside this range: the tables would parse nothing. See checkABI in cmd/ts-translate.

View Source
const (
	BinOr int32 = iota
	BinAnd
	BinBitOr
	BinXor
	BinBitAnd
	BinEq
	BinNe
	BinLt
	BinGt
	BinLe
	BinGe
	BinShl
	BinShr
	BinAdd
	BinSub
	BinMul
	BinDiv
	BinMod
)

The binary operators a lex condition uses, as OpBinary's Arg.

Variables

This section is empty.

Functions

func DecodeUTF8

func DecodeUTF8(s []byte) (int32, uint32)

DecodeUTF8 mirrors the reference implementation's byte sequence handling, including how far it advances over an ill formed sequence.

func DecodeUTF16BE

func DecodeUTF16BE(s []byte) (int32, uint32)

DecodeUTF16BE decodes a single code point from big endian input.

func DecodeUTF16LE

func DecodeUTF16LE(s []byte) (int32, uint32)

DecodeUTF16LE decodes a single code point from little endian input.

func EncodeTables

func EncodeTables(t *Tables) ([]byte, error)

EncodeTables writes a grammar's data as a compressed blob.

func SetContains

func SetContains(ranges []CharacterRange, lookahead int32) bool

SetContains reports whether a code point falls in any of the sorted ranges.

Types

type CharacterRange

type CharacterRange struct {
	Start int32
	End   int32
}

CharacterRange is an inclusive range of code points.

type ExternalScanner

type ExternalScanner interface {
	Create() any
	Destroy(payload any)
	Scan(payload any, lexer *Lexer, validSymbols []bool) bool
	Serialize(payload any, buffer []byte) uint32
	Deserialize(payload any, buffer []byte)
}

ExternalScanner is the set of callbacks a hand written scanner provides.

type FieldID

type FieldID = uint16

FieldID identifies a field name in a grammar.

type FieldMapEntry

type FieldMapEntry struct {
	FieldID    FieldID
	ChildIndex uint8
	Inherited  bool
}

FieldMapEntry associates a field name with a child position.

type Input

type Input struct {
	Read     func(byteOffset uint32, position Point) []byte
	Encoding InputEncoding
	Decode   func([]byte) (int32, uint32)
}

Input supplies source text to the parser in chunks.

type InputEdit

type InputEdit struct {
	StartByte   uint32
	OldEndByte  uint32
	NewEndByte  uint32
	StartPoint  Point
	OldEndPoint Point
	NewEndPoint Point
}

InputEdit describes a change to a document.

type InputEncoding

type InputEncoding int

InputEncoding selects how the source bytes are decoded.

const (
	EncodingUTF8 InputEncoding = iota
	EncodingUTF16LE
	EncodingUTF16BE
	EncodingCustom
)

Supported source encodings.

type Language

type Language struct {
	ABIVersion            uint32
	SymbolCount           uint32
	AliasCount            uint32
	TokenCount            uint32
	ExternalTokenCount    uint32
	StateCount            uint32
	LargeStateCount       uint32
	ProductionIDCount     uint32
	FieldCount            uint32
	MaxAliasSequenceLen   uint16
	ParseTable            []uint16
	SmallParseTable       []uint16
	SmallParseTableMap    []uint32
	ParseActions          []ParseActionEntry
	SymbolNames           []string
	FieldNames            []string
	FieldMapSlices        []MapSlice
	FieldMapEntries       []FieldMapEntry
	SymbolMetadataTable   []SymbolMetadata
	PublicSymbolMap       []Symbol
	AliasMap              []uint16
	AliasSequences        []Symbol
	LexModes              []LexerMode
	LexFn                 func(*Lexer, StateID) bool
	KeywordLexFn          func(*Lexer, StateID) bool
	KeywordCaptureToken   Symbol
	ExternalScannerStates []bool
	ExternalScannerSymbol []Symbol
	Scanner               ExternalScanner
	PrimaryStateIDs       []StateID
	Name                  string
	ReservedWords         []Symbol
	MaxReservedWordSetLen uint16
	SupertypeCount        uint32
	SupertypeSymbols      []Symbol
	SupertypeMapSlices    []MapSlice
	SupertypeMapEntries   []Symbol
	Metadata              LanguageMetadata
}

Language holds a grammar's parse tables and lexer functions.

func LoadGrammar

func LoadGrammar(name string, blob []byte, scanner ExternalScanner) *Language

LoadGrammar decodes a grammar's table blob and wires its lexer. Pass a nil scanner for a grammar with no external tokens.

It panics on a blob this runtime cannot read. A grammar that parses nothing is worse to hand back than a stop at the point the fault is visible.

func (*Language) AliasAt

func (l *Language) AliasAt(productionID uint32, childIndex uint32) Symbol

AliasAt reports the alias applied to a single child of a production.

func (*Language) AliasesForSymbol

func (l *Language) AliasesForSymbol(originalSymbol Symbol) []Symbol

AliasesForSymbol reports every public name a symbol can take.

func (*Language) FieldIDForName

func (l *Language) FieldIDForName(name string) FieldID

FieldIDForName returns the identifier of a named field.

func (*Language) FieldNameForID

func (l *Language) FieldNameForID(id FieldID) string

FieldNameForID returns a field's name.

func (*Language) LanguageName

func (l *Language) LanguageName() string

LanguageName reports the grammar's own name.

func (*Language) StateIsPrimary

func (l *Language) StateIsPrimary(state StateID) bool

StateIsPrimary reports whether a state is the canonical any of its class.

func (*Language) Subtypes

func (l *Language) Subtypes(supertype Symbol) []Symbol

Subtypes reports the symbols that a supertype covers.

func (*Language) Supertypes

func (l *Language) Supertypes() []Symbol

Supertypes reports the grammar's supertype symbols.

func (*Language) SymbolCountTotal

func (l *Language) SymbolCountTotal() uint32

SymbolCountTotal reports the number of symbols including aliases.

func (*Language) SymbolForName

func (l *Language) SymbolForName(name string, isNamed bool) Symbol

SymbolForName finds a symbol by its display name.

func (*Language) SymbolName

func (l *Language) SymbolName(symbol Symbol) string

SymbolName returns the display name for a symbol.

type LanguageMetadata

type LanguageMetadata struct {
	MajorVersion uint8
	MinorVersion uint8
	PatchVersion uint8
}

LanguageMetadata is a grammar's semantic version.

type LexInstr

type LexInstr struct {
	Op  LexOp
	Arg int32
}

LexInstr is a single instruction and its single argument.

type LexOp

type LexOp uint8

LexOp is a single instruction of a lexer program.

const (
	// OpPushLit pushes Arg.
	OpPushLit LexOp = iota
	// OpPushLookahead pushes the code point at the current position.
	OpPushLookahead
	// OpPushEOF pushes the end-of-input flag this iteration read.
	OpPushEOF
	// OpPushSet pushes whether character set Arg holds the lookahead.
	OpPushSet
	// OpNot and OpNeg replace the top of the stack.
	OpNot
	OpNeg
	// OpBinary combines the top values with binary operator Arg.
	OpBinary
	// OpJumpIfFalse pops, and jumps to Arg when the value is empty.
	OpJumpIfFalse
	// OpDispatch jumps to the body of the case for the current state.
	OpDispatch
	// OpAdvance and OpSkip move to state Arg. Skip drops what it consumed.
	OpAdvance
	OpSkip
	// OpAdvanceMap reads map Arg, and advances on a lookahead the map names.
	OpAdvanceMap
	// OpAccept marks the token Arg, ending here.
	OpAccept
	// OpEnd answers with what the run accepted. OpReturn answers Arg.
	OpEnd
	OpReturn
	// OpReadEOF reads the end-of-input flag.
	OpReadEOF
)

type LexProgram

type LexProgram struct {
	Code   []LexInstr
	States []int32
	Maps   [][]int32
}

LexProgram is a lexer state machine. Code opens with the prologue and an OpDispatch, and the case bodies follow it.

func (*LexProgram) Run

func (p *LexProgram) Run(sets [][]CharacterRange, lexer *Lexer, state StateID) bool

Run walks the program for a single token.

type Lexer

type Lexer struct {
	// Lookahead is the code point at the current position.
	Lookahead int32
	// ResultSymbol is set by a lex function when it accepts a token.
	ResultSymbol Symbol
	// contains filtered or unexported fields
}

Lexer scans the source text on behalf of a grammar's lex functions.

func (*Lexer) Advance

func (l *Lexer) Advance(skip bool)

Advance consumes the current code point.

func (*Lexer) EOF

func (l *Lexer) EOF() bool

EOF reports whether the lexer consumed all included ranges.

func (*Lexer) GetColumn

func (l *Lexer) GetColumn() uint32

GetColumn reports the column of the current position.

func (*Lexer) IsAtIncludedRangeStart

func (l *Lexer) IsAtIncludedRangeStart() bool

IsAtIncludedRangeStart reports whether the lexer sits on a range boundary.

func (*Lexer) MarkEnd

func (l *Lexer) MarkEnd()

MarkEnd records the end of the token being scanned.

type LexerMode

type LexerMode struct {
	LexState          uint16
	ExternalLexState  uint16
	ReservedWordSetID uint16
}

LexerMode selects the lex state used when scanning in a parse state.

type MapSlice

type MapSlice struct {
	Index  uint16
	Length uint16
}

MapSlice indexes into the field and supertype maps.

type Node

type Node struct {
	// contains filtered or unexported fields
}

Node is a single node in a syntax tree.

func (Node) Child

func (n Node) Child(index uint32) Node

Child returns the visible child at the given index.

func (Node) ChildByFieldID

func (n Node) ChildByFieldID(fieldID FieldID) Node

ChildByFieldID returns the child registered under the given field.

func (Node) ChildByFieldName

func (n Node) ChildByFieldName(name string) Node

ChildByFieldName returns the child registered under the given field name.

func (Node) ChildCount

func (n Node) ChildCount() uint32

ChildCount reports the number of visible children.

func (Node) ChildWithDescendant

func (n Node) ChildWithDescendant(descendant Node) Node

ChildWithDescendant returns the child that contains the given descendant.

func (Node) Children

func (n Node) Children() []Node

Children returns every visible child, as Child returns each. It reads the children once.

func (Node) DescendantCount

func (n Node) DescendantCount() uint32

DescendantCount reports how many visible nodes the subtree holds.

func (Node) DescendantForByteRange

func (n Node) DescendantForByteRange(start, end uint32) Node

DescendantForByteRange returns the smallest node spanning the byte range.

func (Node) DescendantForPointRange

func (n Node) DescendantForPointRange(start, end Point) Node

DescendantForPointRange returns the smallest node spanning the point range.

func (*Node) Edit

func (n *Node) Edit(edit InputEdit)

Edit shifts the node's recorded position to account for a change.

func (Node) EndByte

func (n Node) EndByte() uint32

EndByte reports the offset past the node.

func (Node) EndPoint

func (n Node) EndPoint() Point

EndPoint reports the row and column past the node.

func (Node) Equal

func (n Node) Equal(other Node) bool

Equal reports whether nodes refer to the same place in the same tree.

func (Node) FieldNameForChild

func (n Node) FieldNameForChild(index uint32) string

FieldNameForChild reports the field name of the visible child at an index.

func (Node) FieldNameForNamedChild

func (n Node) FieldNameForNamedChild(index uint32) string

FieldNameForNamedChild reports the field name of the named child at an index.

func (Node) GrammarSymbol

func (n Node) GrammarSymbol() Symbol

GrammarSymbol reports the node's symbol before alias resolution.

func (Node) GrammarType

func (n Node) GrammarType() string

GrammarType reports the node's rule name before alias resolution.

func (Node) HasChanges

func (n Node) HasChanges() bool

HasChanges reports whether an edit touched the node.

func (Node) HasError

func (n Node) HasError() bool

HasError reports whether the node or a descendant is an error.

func (Node) IsError

func (n Node) IsError() bool

IsError reports whether the node itself is an error node.

func (Node) IsExtra

func (n Node) IsExtra() bool

IsExtra reports whether the node sits outside the grammar's main rules.

func (Node) IsMissing

func (n Node) IsMissing() bool

IsMissing reports whether the parser inserted the node during recovery.

func (Node) IsNamed

func (n Node) IsNamed() bool

IsNamed reports whether the node has a name in the grammar.

func (Node) IsNull

func (n Node) IsNull() bool

IsNull reports whether the node refers to nothing.

func (Node) NamedChild

func (n Node) NamedChild(index uint32) Node

NamedChild returns the named child at the given index.

func (Node) NamedChildCount

func (n Node) NamedChildCount() uint32

NamedChildCount reports the number of named children.

func (Node) NamedChildren

func (n Node) NamedChildren() []Node

NamedChildren returns every named child, as NamedChild returns each.

func (Node) NamedDescendantForByteRange

func (n Node) NamedDescendantForByteRange(start, end uint32) Node

NamedDescendantForByteRange returns the smallest named node spanning the range.

func (Node) NamedDescendantForPointRange

func (n Node) NamedDescendantForPointRange(start, end Point) Node

NamedDescendantForPointRange returns the smallest named node spanning the range.

func (Node) NextNamedSibling

func (n Node) NextNamedSibling() Node

NextNamedSibling returns the next named node with the same parent.

func (Node) NextParseState

func (n Node) NextParseState() StateID

NextParseState reports the state reached after the node.

func (Node) NextSibling

func (n Node) NextSibling() Node

NextSibling returns the next visible node with the same parent.

func (Node) Parent

func (n Node) Parent() Node

Parent returns the node that contains this.

func (Node) ParseState

func (n Node) ParseState() StateID

ParseState reports the parse state the node was created in.

func (Node) PrevNamedSibling

func (n Node) PrevNamedSibling() Node

PrevNamedSibling returns the named node with the same parent.

func (Node) PrevSibling

func (n Node) PrevSibling() Node

PrevSibling returns the visible node with the same parent.

func (Node) StartByte

func (n Node) StartByte() uint32

StartByte reports the node's earliest byte offset.

func (Node) StartPoint

func (n Node) StartPoint() Point

StartPoint reports the node's earliest row and column.

func (Node) String

func (n Node) String() string

String renders the node as an S expression.

func (Node) Symbol

func (n Node) Symbol() Symbol

Symbol reports the node's public grammar symbol. A null node has none.

func (Node) Tree

func (n Node) Tree() *Tree

Tree reports the tree that owns the node.

func (Node) Type

func (n Node) Type() string

Type reports the node's grammar rule name. A null node has none.

func (Node) Walk

func (n Node) Walk() *TreeCursor

Walk returns a cursor positioned on the node.

type ParseAction

type ParseAction struct {
	Type              uint8
	State             StateID
	Extra             bool
	Repetition        bool
	ChildCount        uint8
	Symbol            Symbol
	DynamicPrecedence int16
	ProductionID      uint16
}

ParseAction is a single entry of a parse table cell.

type ParseActionEntry

type ParseActionEntry struct {
	Action   ParseAction
	Count    uint8
	Reusable bool
}

ParseActionEntry is either a group header or a single action.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser turns source text into a syntax tree.

func NewParser

func NewParser() *Parser

NewParser creates a parser with no language assigned.

func (*Parser) IncludedRanges

func (p *Parser) IncludedRanges() []Range

IncludedRanges reports the regions the parser reads.

func (*Parser) Language

func (p *Parser) Language() *Language

Language reports the grammar this parser uses.

func (*Parser) Parse

func (p *Parser) Parse(oldTree *Tree, input Input) *Tree

Parse builds a syntax tree from the given input.

func (*Parser) ParseString

func (p *Parser) ParseString(oldTree *Tree, source []byte) *Tree

ParseString builds a syntax tree from a source string.

func (*Parser) Reset

func (p *Parser) Reset()

Reset clears any parse state left over from a previous call.

func (*Parser) SetIncludedRanges

func (p *Parser) SetIncludedRanges(ranges []Range) bool

SetIncludedRanges restricts parsing to the given regions.

func (*Parser) SetLanguage

func (p *Parser) SetLanguage(language *Language) bool

SetLanguage assigns the grammar this parser uses.

type Point

type Point struct {
	Row    uint32
	Column uint32
}

Point is a row/column position in a document.

type PredicateStep

type PredicateStep struct {
	Type    PredicateStepType
	ValueID uint32
}

PredicateStep is one token of a predicate, as TSQueryPredicateStep. A Capture step's ValueID is a capture id.

type PredicateStepType

type PredicateStepType uint8

PredicateStepType says what a predicate step holds. The values follow. TSQueryPredicateStepType.

const (
	PredicateStepDone PredicateStepType = iota
	PredicateStepCapture
	PredicateStepString
)

The step types. A Done step ends one predicate.

type PropertyPredicate

type PropertyPredicate struct {
	Property QueryProperty
	Positive bool
}

PropertyPredicate is a property paired with whether it was `#is?` (true) or `#is-not?` (false).

type Quantifier

type Quantifier uint8

Quantifier says how many times a capture can appear in a pattern. The values follow TSQuantifier.

const (
	QuantifierZero Quantifier = iota
	QuantifierZeroOrOne
	QuantifierZeroOrMore
	QuantifierOne
	QuantifierOneOrMore
)

The quantifiers a capture can carry.

func (Quantifier) String

func (q Quantifier) String() string

String names the quantifier the way the Rust binding's CaptureQuantifier does.

type Query

type Query struct {
	// contains filtered or unexported fields
}

Query is a TSQuery: patterns compiled from S-expressions. A Query is immutable during execution, so cursors can share it.

func NewQuery

func NewQuery(language *Language, source string) (*Query, error)

NewQuery is ts_query_new, plus the predicate parsing the Rust binding's Query::new performs. The error is a *QueryError.

func (*Query) CaptureCount

func (q *Query) CaptureCount() int

CaptureCount is ts_query_capture_count.

func (*Query) CaptureIndexForName

func (q *Query) CaptureIndexForName(name string) (uint32, bool)

CaptureIndexForName reports the capture id of a name, as the Rust binding's capture_index_for_name.

func (*Query) CaptureNameForID

func (q *Query) CaptureNameForID(id uint32) string

CaptureNameForID is ts_query_capture_name_for_id.

func (*Query) CaptureNames

func (q *Query) CaptureNames() []string

CaptureNames reports every capture name, indexed by capture id.

func (*Query) CaptureQuantifierForID

func (q *Query) CaptureQuantifierForID(patternIndex, captureIndex uint32) Quantifier

CaptureQuantifierForID is ts_query_capture_quantifier_for_id.

func (*Query) CaptureQuantifiers

func (q *Query) CaptureQuantifiers(patternIndex int) []Quantifier

CaptureQuantifiers reports a pattern's quantifier for every capture id, as the Rust binding's capture_quantifiers.

func (*Query) Copy

func (q *Query) Copy() *Query

Copy is ts_query_copy: an independent query to disable captures or patterns on.

func (*Query) DisableCapture

func (q *Query) DisableCapture(name string)

DisableCapture is ts_query_disable_capture.

func (*Query) DisablePattern

func (q *Query) DisablePattern(patternIndex int)

DisablePattern is ts_query_disable_pattern.

func (*Query) EndByteForPattern

func (q *Query) EndByteForPattern(patternIndex int) uint32

EndByteForPattern is ts_query_end_byte_for_pattern.

func (*Query) GeneralPredicates

func (q *Query) GeneralPredicates(patternIndex int) []QueryPredicate

GeneralPredicates reports the predicates of a pattern that the library leaves to the caller, as the Rust binding's general_predicates.

func (*Query) IsPatternGuaranteedAtStep

func (q *Query) IsPatternGuaranteedAtStep(byteOffset uint32) bool

IsPatternGuaranteedAtStep is ts_query_is_pattern_guaranteed_at_step.

func (*Query) IsPatternNonLocal

func (q *Query) IsPatternNonLocal(patternIndex int) bool

IsPatternNonLocal is ts_query_is_pattern_non_local.

func (*Query) IsPatternRooted

func (q *Query) IsPatternRooted(patternIndex int) bool

IsPatternRooted is ts_query_is_pattern_rooted.

func (*Query) Language

func (q *Query) Language() *Language

Language reports the grammar the query was compiled for.

func (*Query) PatternCount

func (q *Query) PatternCount() int

PatternCount is ts_query_pattern_count.

func (*Query) PredicatesForPattern

func (q *Query) PredicatesForPattern(patternIndex int) []PredicateStep

PredicatesForPattern is ts_query_predicates_for_pattern. Each predicate ends with a Done step.

func (*Query) PropertyPredicates

func (q *Query) PropertyPredicates(patternIndex int) []PropertyPredicate

PropertyPredicates reports a pattern's `#is?` and `#is-not?` properties, as the Rust binding's property_predicates.

func (*Query) PropertySettings

func (q *Query) PropertySettings(patternIndex int) []QueryProperty

PropertySettings reports a pattern's `#set!` properties, as the Rust binding's property_settings.

func (*Query) StartByteForPattern

func (q *Query) StartByteForPattern(patternIndex int) uint32

StartByteForPattern is ts_query_start_byte_for_pattern.

func (*Query) StringCount

func (q *Query) StringCount() int

StringCount is ts_query_string_count.

func (*Query) StringValueForID

func (q *Query) StringValueForID(id uint32) string

StringValueForID is ts_query_string_value_for_id.

type QueryCapture

type QueryCapture struct {
	Node  Node
	Index uint32
}

QueryCapture is a node a pattern captured, as TSQueryCapture. Index is the capture id, for CaptureNameForID.

type QueryCursor

type QueryCursor struct {
	// contains filtered or unexported fields
}

QueryCursor is a TSQueryCursor: it executes one query over one tree at a time, and can be reused across executions.

func NewQueryCursor

func NewQueryCursor() *QueryCursor

NewQueryCursor is ts_query_cursor_new.

func (*QueryCursor) Captures

func (c *QueryCursor) Captures(query *Query, node Node, source []byte) iter.Seq2[*QueryMatch, uint32]

Captures runs the query over the node and yields each capture in document order. Those Captures are with its match and its index in the match's captures, as QueryCursor::captures.

func (*QueryCursor) CapturesWithOptions

func (c *QueryCursor) CapturesWithOptions(query *Query, node Node, text TextProvider, options *QueryCursorOptions) iter.Seq2[*QueryMatch, uint32]

CapturesWithOptions is QueryCursor::captures_with_options.

func (*QueryCursor) DidExceedMatchLimit

func (c *QueryCursor) DidExceedMatchLimit() bool

DidExceedMatchLimit is ts_query_cursor_did_exceed_match_limit.

func (*QueryCursor) Exec

func (c *QueryCursor) Exec(query *Query, node Node)

Exec is ts_query_cursor_exec: it starts the query over the node's subtree.

func (*QueryCursor) ExecWithOptions

func (c *QueryCursor) ExecWithOptions(query *Query, node Node, options *QueryCursorOptions)

ExecWithOptions is ts_query_cursor_exec_with_options.

func (*QueryCursor) MatchLimit

func (c *QueryCursor) MatchLimit() uint32

MatchLimit is ts_query_cursor_match_limit.

func (*QueryCursor) Matches

func (c *QueryCursor) Matches(query *Query, node Node, source []byte) iter.Seq[*QueryMatch]

Matches runs the query over the node and yields each match whose text predicates hold, as QueryCursor::matches. The yielded match is valid until the next one.

func (*QueryCursor) MatchesWithOptions

func (c *QueryCursor) MatchesWithOptions(query *Query, node Node, text TextProvider, options *QueryCursorOptions) iter.Seq[*QueryMatch]

MatchesWithOptions is QueryCursor::matches_with_options.

func (*QueryCursor) NextCapture

func (c *QueryCursor) NextCapture(match *QueryMatch) (uint32, bool)

NextCapture is ts_query_cursor_next_capture. It returns captures in document order, which can come from a match that has not finished yet. The index names the capture in the match's list.

func (*QueryCursor) NextMatch

func (c *QueryCursor) NextMatch(match *QueryMatch) bool

NextMatch is ts_query_cursor_next_match. The match's captures alias the cursor's storage and are valid until the next call.

func (*QueryCursor) RemoveMatch

func (c *QueryCursor) RemoveMatch(matchID uint32)

RemoveMatch is ts_query_cursor_remove_match: no more captures of the match are returned.

func (*QueryCursor) SetByteRange

func (c *QueryCursor) SetByteRange(startByte, endByte uint32) bool

SetByteRange is ts_query_cursor_set_byte_range. An end of zero means the end of the document. It reports false for a start past the end.

func (*QueryCursor) SetContainingByteRange

func (c *QueryCursor) SetContainingByteRange(startByte, endByte uint32) bool

SetContainingByteRange is ts_query_cursor_set_containing_byte_range: only a match whose nodes lie within the range is returned.

func (*QueryCursor) SetContainingPointRange

func (c *QueryCursor) SetContainingPointRange(startPoint, endPoint Point) bool

SetContainingPointRange is ts_query_cursor_set_containing_point_range.

func (*QueryCursor) SetMatchLimit

func (c *QueryCursor) SetMatchLimit(limit uint32)

SetMatchLimit is ts_query_cursor_set_match_limit.

func (*QueryCursor) SetMaxStartDepth

func (c *QueryCursor) SetMaxStartDepth(maxStartDepth uint32)

SetMaxStartDepth is ts_query_cursor_set_max_start_depth.

func (*QueryCursor) SetPointRange

func (c *QueryCursor) SetPointRange(startPoint, endPoint Point) bool

SetPointRange is ts_query_cursor_set_point_range.

type QueryCursorOptions

type QueryCursorOptions struct {
	Payload          any
	ProgressCallback func(state *QueryCursorState) bool
}

QueryCursorOptions is a TSQueryCursorOptions. ProgressCallback runs after a bounded amount of work.

type QueryCursorState

type QueryCursorState struct {
	Payload           any
	CurrentByteOffset uint32
}

QueryCursorState is a TSQueryCursorState: what a progress callback sees.

type QueryError

type QueryError struct {
	Kind    QueryErrorKind
	Offset  int
	Row     int
	Column  int
	Message string
}

QueryError reports why a query did not compile. Offset is the byte offset ts_query_new reports.

func (*QueryError) Error

func (e *QueryError) Error() string

Error renders the error the way the Rust binding's Display does.

type QueryErrorKind

type QueryErrorKind int

QueryErrorKind says which rule a query broke. The values follow TSQueryError.

const (
	QueryErrorNone QueryErrorKind = iota
	QueryErrorSyntax
	QueryErrorNodeType
	QueryErrorField
	QueryErrorCapture
	QueryErrorStructure
	QueryErrorLanguage
	QueryErrorPredicate
)

The error kinds a query compile can report. QueryErrorPredicate comes from the predicate layer the Rust binding adds; the others come from ts_query_new.

func (QueryErrorKind) String

func (k QueryErrorKind) String() string

String names the kind the way the Rust binding's QueryErrorKind does.

type QueryMatch

type QueryMatch struct {
	ID           uint32
	PatternIndex uint16
	Captures     []QueryCapture
}

QueryMatch is one match of a pattern, as TSQueryMatch. Captures aliases a list the cursor owns: it is valid until the cursor's next call.

func (*QueryMatch) NodesForCaptureIndex

func (m *QueryMatch) NodesForCaptureIndex(captureIndex uint32) iter.Seq[Node]

NodesForCaptureIndex reports the nodes a match captured under one capture id, as QueryMatch::nodes_for_capture_index.

func (*QueryMatch) SatisfiesTextPredicates

func (m *QueryMatch) SatisfiesTextPredicates(query *Query, text TextProvider) bool

SatisfiesTextPredicates is QueryMatch::satisfies_text_predicates: whether the text predicates of the match's pattern hold.

type QueryPredicate

type QueryPredicate struct {
	Operator string
	Args     []QueryPredicateArg
}

QueryPredicate is a QueryPredicate: a predicate the library does not interpret, handed to the caller as data.

type QueryPredicateArg

type QueryPredicateArg struct {
	Kind         QueryPredicateArgKind
	CaptureIndex uint32
	Value        string
}

QueryPredicateArg is a QueryPredicateArg.

type QueryPredicateArgKind

type QueryPredicateArgKind uint8

QueryPredicateArgKind says what a general predicate's argument is.

const (
	QueryPredicateArgCapture QueryPredicateArgKind = iota
	QueryPredicateArgString
)

The argument kinds, as QueryPredicateArg.

type QueryProperty

type QueryProperty struct {
	Key          string
	Value        string
	HasValue     bool
	CaptureIndex int
}

QueryProperty is a QueryProperty. A property with no value has HasValue false.

type Range

type Range struct {
	StartPoint Point
	EndPoint   Point
	StartByte  uint32
	EndByte    uint32
}

Range is a contiguous region of a document.

type StateID

type StateID = uint16

StateID identifies a parse state.

type Symbol

type Symbol = uint16

Symbol identifies a terminal or non-terminal in a grammar.

const BuiltinSymEnd Symbol = 0

BuiltinSymEnd is the symbol for the end of input.

const BuiltinSymError Symbol = 0xFFFF

BuiltinSymError is the symbol for an error node.

type SymbolMetadata

type SymbolMetadata struct {
	Visible   bool
	Named     bool
	Supertype bool
}

SymbolMetadata records how a symbol appears in a tree.

type Tables

type Tables struct {
	Language
	// CharacterSets holds the sets the lexer searches.
	CharacterSets [][]CharacterRange
	// Lex and KeywordLex are the grammar's lexer state machines.
	Lex        *LexProgram
	KeywordLex *LexProgram
}

The embedded Language's function fields and its scanner are not encoded.

func DecodeTables

func DecodeTables(blob []byte) (*Tables, error)

DecodeTables reads back what EncodeTables wrote.

type TextProvider

type TextProvider func(node Node) []byte

TextProvider returns the text of a node, as the Rust binding's TextProvider.

func SourceText

func SourceText(source []byte) TextProvider

SourceText is the TextProvider over a whole document.

type Tree

type Tree struct {
	// contains filtered or unexported fields
}

Tree is a syntax tree produced by a parser.

func (*Tree) Copy

func (t *Tree) Copy() *Tree

Copy returns a tree that shares the receiver's nodes.

func (*Tree) Edit

func (t *Tree) Edit(edit InputEdit)

Edit adjusts the tree's node positions to account for a change.

func (*Tree) IncludedRanges

func (t *Tree) IncludedRanges() []Range

IncludedRanges reports the regions the parser read.

func (*Tree) Language

func (t *Tree) Language() *Language

Language reports the grammar used to build the tree.

func (*Tree) RootNode

func (t *Tree) RootNode() Node

RootNode returns the tree's root.

func (*Tree) RootNodeWithOffset

func (t *Tree) RootNodeWithOffset(offsetBytes uint32, offsetExtent Point) Node

RootNodeWithOffset returns the root shifted by the given offset.

func (*Tree) String

func (t *Tree) String() string

String renders the tree as an S expression.

func (*Tree) Walk

func (t *Tree) Walk() *TreeCursor

Walk returns a cursor positioned on the tree's root.

type TreeCursor

type TreeCursor struct {
	// contains filtered or unexported fields
}

TreeCursor walks a syntax tree. A step to a sibling costs the same at any index, because each level reads its children once.

func (*TreeCursor) Copy

func (c *TreeCursor) Copy() *TreeCursor

Copy returns an independent cursor at the same position.

func (*TreeCursor) Depth

func (c *TreeCursor) Depth() uint32

Depth reports how far the cursor has descended from its starting node.

func (*TreeCursor) FieldName

func (c *TreeCursor) FieldName() string

FieldName reports the field the current node fills in its parent.

func (*TreeCursor) GotoFirstChild

func (c *TreeCursor) GotoFirstChild() bool

GotoFirstChild moves to the earliest visible child, reporting whether a single exists.

func (*TreeCursor) GotoNextSibling

func (c *TreeCursor) GotoNextSibling() bool

GotoNextSibling moves to the next visible sibling, reporting whether a single exists.

func (*TreeCursor) GotoParent

func (c *TreeCursor) GotoParent() bool

GotoParent moves to the parent node, reporting whether a single exists.

func (*TreeCursor) GotoPreviousSibling

func (c *TreeCursor) GotoPreviousSibling() bool

GotoPreviousSibling moves to the visible sibling.

func (*TreeCursor) Node

func (c *TreeCursor) Node() Node

Node reports the node the cursor sits on.

func (*TreeCursor) Reset

func (c *TreeCursor) Reset(n Node)

Reset moves the cursor to the given node and forgets its history.

Directories

Path Synopsis
cmd
ts-fetch command
Command ts-fetch puts a grammar's upstream sources where ts-translate can read them.
Command ts-fetch puts a grammar's upstream sources where ts-translate can read them.
ts-goldens command
Command ts-goldens records reference results with the upstream tree-sitter command line tool.
Command ts-goldens records reference results with the upstream tree-sitter command line tool.
ts-translate command
Command ts-translate turns a tree-sitter generated parser.c into the table blob a grammar package embeds, and the loader that embeds it.
Command ts-translate turns a tree-sitter generated parser.c into the table blob a grammar package embeds, and the loader that embeds it.
grammars
clang
Package clang holds the tree-sitter grammar for the C language.
Package clang holds the tree-sitter grammar for the C language.
cpp
golang
Package golang holds the tree-sitter grammar for the Go language.
Package golang holds the tree-sitter grammar for the Go language.
tsx
Package tsx is TypeScript with JSX.
Package tsx is TypeScript with JSX.
internal
corpus
Package corpus reads the test corpus format that every tree-sitter grammar repository ships under test/corpus.
Package corpus reads the test corpus format that every tree-sitter grammar repository ships under test/corpus.
goldens
Package goldens compares parse results against files produced by the reference tree-sitter command line tool.
Package goldens compares parse results against files produced by the reference tree-sitter command line tool.
grammarsrc
Package grammarsrc puts a grammar's upstream sources where a translator can read them.
Package grammarsrc puts a grammar's upstream sources where a translator can read them.
querygoldens
Package querygoldens compares query results against files the reference tree-sitter command line tool wrote with `tree-sitter query`.
Package querygoldens compares query results against files the reference tree-sitter command line tool wrote with `tree-sitter query`.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL