highlight

README

go-tree-sitter-highlight

This highlighter is based on the rust tree-sitter-highlight crate. It relies on the go-tree-sitter bindings for the tree-sitter library.

Documentation

Overview

Package highlight provides a simple way to highlight text via [tree-sitter](https://github.com/tree-sitter/tree-sitter).

The package is a wrapper around the [tree-sitter-go](https://github.com/tree-sitter/tree-sitter-go) module, and provides a simple API to highlight text.

Usage

To highlight your text you first need to create a highlight.Configuration. This struct holds the configuration for the highlighter, including the language, the queries to use. Next call highlight.Configuration.Configure to configure the capture names used by your theme. After that create a new highlight.Highlighter and call the highlight.Highlighter.Highlight method to highlight your text. This returns a [iter.Seq2[Event, error]] that you can iterate over to get the highlighted text areas & languages in your text.

source := []byte("package main\n\n import \"fmt\"\n\nfunc main() {\n\tfmt.Println(\"Hello, World!\")\n}")

captureNames := []string{
	"variable",
	"function",
	"string",
	"keyword",
	"comment",
}

language := tree_sitter.NewLanguage(tree_sitter_go.Language())

cfg, err := NewConfiguration(language, "go", highlightsQuery, injectionQuery, localsQuery)
if err != nil {
	log.Fatal(err)
}

cfg.Configure(captureNames)

highlighter := New()
events := highlighter.Highlight(context.Background(), cfg, source, func(name string) *Configuration {
	return nil
})

for event, err := range events {
	if err != nil {
		log.Fatal(err)
	}

	switch e := event.(type) {
		case EventLayerStart:
			log.Printf("Layer start: %s", e.LanguageName)
		case EventLayerEnd:
			log.Printf("Layer end")
		case EventCaptureStart:
			log.Printf("Capture start: %d", e.Highlight)
		case EventCaptureEnd:
			log.Printf("Capture end")
		case EventSource:
			log.Printf("Source: %d-%d", e.StartByte, e.EndByte)
		}
	}
}

Index

• Constants
• Variables
• type AttributeCallback
• type Configuration
  • func NewConfiguration(language *tree_sitter.Language, languageName string, highlightsQuery []byte, ...) (*Configuration, error)
  • func (c *Configuration) Configure(recognizedNames []string)
  • func (c *Configuration) Names() []string
  • func (c *Configuration) NonconformantCaptureNames(captureNames []string) []string
• type Event
• type EventCaptureEnd
• type EventCaptureStart
• type EventLayerEnd
• type EventLayerStart
• type EventSource
• type HTMLRender
  • func NewHTMLRender() *HTMLRender
  • func (r *HTMLRender) Render(w io.Writer, events iter.Seq2[Event, error], source []byte, ...) error
  • func (r *HTMLRender) RenderCSS(w io.Writer, theme map[string]string) error
  • func (r *HTMLRender) RenderDocument(w io.Writer, events iter.Seq2[Event, error], title string, source []byte, ...) error
• type Highlight
• type Highlighter
  • func New() *Highlighter
  • func (h *Highlighter) Highlight(ctx context.Context, cfg Configuration, source []byte, ...) iter.Seq2[Event, error]
• type InjectionCallback

Constants

const DefaultHighlight = Highlight(^uint(0))

Variables

var StandardCaptureNames = []string{
	"attribute",
	"boolean",
	"carriage-return",
	"comment",
	"comment.documentation",
	"constant",
	"constant.builtin",
	"constructor",
	"constructor.builtin",
	"embedded",
	"error",
	"escape",
	"function",
	"function.builtin",
	"keyword",
	"markup",
	"markup.bold",
	"markup.heading",
	"markup.italic",
	"markup.link",
	"markup.link.url",
	"markup.list",
	"markup.list.checked",
	"markup.list.numbered",
	"markup.list.unchecked",
	"markup.list.unnumbered",
	"markup.quote",
	"markup.raw",
	"markup.raw.block",
	"markup.raw.inline",
	"markup.strikethrough",
	"module",
	"number",
	"operator",
	"property",
	"property.builtin",
	"punctuation",
	"punctuation.bracket",
	"punctuation.delimiter",
	"punctuation.special",
	"string",
	"string.escape",
	"string.regexp",
	"string.special",
	"string.special.symbol",
	"tag",
	"type",
	"type.builtin",
	"variable",
	"variable.builtin",
	"variable.member",
	"variable.parameter",
}

StandardCaptureNames is a list of common capture names used in tree-sitter queries. This list is opinionated and may not align with the capture names used in a particular tree-sitter grammar.

Types

type AttributeCallback

type AttributeCallback func(h Highlight, languageName string) []byte

AttributeCallback is a callback function that returns the html element attributes for a highlight span. This can be anything from classes, ids, or inline styles.

type Configuration

type Configuration struct {
	Language                      *tree_sitter.Language
	LanguageName                  string
	Query                         *tree_sitter.Query
	CombinedInjectionsQuery       *tree_sitter.Query
	LocalsPatternIndex            uint
	HighlightsPatternIndex        uint
	HighlightIndices              []*Highlight
	NonLocalVariablePatterns      []bool
	InjectionContentCaptureIndex  *uint
	InjectionLanguageCaptureIndex *uint
	LocalScopeCaptureIndex        *uint
	LocalDefCaptureIndex          *uint
	LocalDefValueCaptureIndex     *uint
	LocalRefCaptureIndex          *uint
}

func NewConfiguration

func NewConfiguration(language *tree_sitter.Language, languageName string, highlightsQuery []byte, injectionQuery []byte, localsQuery []byte) (*Configuration, error)

NewConfiguration creates a new highlight configuration from a [tree_sitter.Language] and a set of queries.

func (*Configuration) Configure

func (c *Configuration) Configure(recognizedNames []string)

Configure sets the list of recognized highlight names.

Tree-sitter syntax-highlighting queries specify highlights in the form of dot-separated highlight names like `punctuation.bracket` and `function.method.builtin`. Consumers of these queries can choose to recognize highlights with different levels of specificity. For example, the string `function.builtin` will match against `function` and `function.builtin.constructor`, but will not match `function.method`.

When highlighting, results are returned as `Highlight` values, which contain the index of the matched highlight this list of highlight names.

func (*Configuration) Names

func (c *Configuration) Names() []string

Names gets a slice containing all the highlight names used in the configuration.

func (*Configuration) NonconformantCaptureNames

func (c *Configuration) NonconformantCaptureNames(captureNames []string) []string

NonconformantCaptureNames returns the list of this configuration's capture names that are neither present in the list of predefined 'canonical' names nor start with an underscore (denoting 'private' captures used as part of capture internals).

type Event

type Event interface {
	// contains filtered or unexported methods
}

Event is an interface that represents a highlight event. Possible implementations are: - EventLayerStart - EventLayerEnd - EventCaptureStart - EventCaptureEnd - EventSource

type EventCaptureEnd

type EventCaptureEnd struct{}

EventCaptureEnd is emitted when a highlight region ends.

type EventCaptureStart

type EventCaptureStart struct {
	// Highlight is the capture name of the highlight.
	Highlight Highlight
}

EventCaptureStart is emitted when a highlight region starts.

type EventLayerEnd

type EventLayerEnd struct{}

EventLayerEnd is emitted when a language injection ends.

type EventLayerStart

type EventLayerStart struct {
	// LanguageName is the name of the language that is being injected.
	LanguageName string
}

EventLayerStart is emitted when a language injection starts.

type EventSource

type EventSource struct {
	StartByte uint
	EndByte   uint
}

EventSource is emitted when a source code range is highlighted.

type HTMLRender

type HTMLRender struct {
	ClassNamePrefix string
}

HTMLRender is a renderer that outputs HTML.

func NewHTMLRender

func NewHTMLRender() *HTMLRender

NewHTMLRender returns a new HTMLRender.

func (*HTMLRender) Render

func (r *HTMLRender) Render(w io.Writer, events iter.Seq2[Event, error], source []byte, callback AttributeCallback) error

Render renders the code code to the writer with spans for each highlight capture. The AttributeCallback is used to generate the classes or inline styles for each span.

func (*HTMLRender) RenderCSS

func (r *HTMLRender) RenderCSS(w io.Writer, theme map[string]string) error

RenderCSS renders the css classes for a theme to the writer.

func (*HTMLRender) RenderDocument

func (r *HTMLRender) RenderDocument(w io.Writer, events iter.Seq2[Event, error], title string, source []byte, captureNames []string, theme map[string]string) error

RenderDocument renders a full HTML document with the code and theme embedded.

type Highlight

type Highlight uint

Highlight represents the index of a capture name.

type Highlighter

type Highlighter struct {
	Parser *tree_sitter.Parser
	// contains filtered or unexported fields
}

Highlighter is a syntax highlighter that uses tree-sitter to parse source code and apply syntax highlighting. It is not thread-safe.

func New

func New() *Highlighter

New returns a new highlighter. The highlighter is not thread-safe and should not be shared between goroutines, but it can be reused to highlight multiple source code snippets.

func (*Highlighter) Highlight

func (h *Highlighter) Highlight(ctx context.Context, cfg Configuration, source []byte, injectionCallback InjectionCallback) iter.Seq2[Event, error]

Highlight highlights the given source code using the given configuration. The source code is expected to be UTF-8 encoded. The function returns an [iter.Seq2[Event, error]] that yields the highlight events or an error.

type InjectionCallback

type InjectionCallback func(languageName string) *Configuration

InjectionCallback is called when a language injection is found to load the configuration for the injected language.