stream

Documentation

Overview

Package stream highlights a document with a CodeMirror 5-style stream mode, as CodeMirror's StreamLanguage does for the languages @codemirror/legacy-modes covers and Lezer does not.

Index

• Constants
• func Highlight(text []uint16, mode Mode, sets *lezer.TagSets, ...)
• func IsDigit(u uint16) bool
• func IsSpace(u uint16) bool
• func IsWord(u uint16) bool
• type Mode
• type Stream
  • func (s *Stream) BackUp(n int)
  • func (s *Stream) Current() []uint16
  • func (s *Stream) EOL() bool
  • func (s *Stream) Eat(c uint16) bool
  • func (s *Stream) EatFunc(match func(uint16) bool) bool
  • func (s *Stream) EatSpace() bool
  • func (s *Stream) EatWhile(match func(uint16) bool) bool
  • func (s *Stream) Line() []uint16
  • func (s *Stream) LookingAt(text string) bool
  • func (s *Stream) Match(text string) bool
  • func (s *Stream) MatchFunc(match func(rest []uint16) int) bool
  • func (s *Stream) Next() int
  • func (s *Stream) Peek() int
  • func (s *Stream) SOL() bool
  • func (s *Stream) SkipTo(c uint16) bool
  • func (s *Stream) SkipToEnd()

Constants

const EOL = -1

EOL is what Peek and Next return at the end of the line, where JavaScript's StringStream returns undefined.

Functions

func Highlight

func Highlight(text []uint16, mode Mode, sets *lezer.TagSets, span func(from, to int, set *lezer.TagSet))

Highlight calls span, in order, for every token of text (UTF-16 code units, split into lines at "\n" alone) whose style names tags, as highlightTree does with a highlighter that gives every tag its own class over the tree a StreamLanguage parses. Positions are code units.

A mode that returns a token without reading anything ten times in a row is a bug, and Highlight panics, where CodeMirror's parse throws.

func IsDigit

func IsDigit(u uint16) bool

IsDigit is JavaScript's \d: an ASCII digit.

func IsSpace

func IsSpace(u uint16) bool

IsSpace is JavaScript's \s, which StringStream's eatSpace extends with U+00A0, already part of it.

func IsWord

func IsWord(u uint16) bool

IsWord is JavaScript's \w: an ASCII letter, digit or underscore.

Types

type Mode

type Mode interface {
	// Token reads one token from s and returns its style, "" for none: a
	// legacy name ("variable", "def") or tag names joined by "." and " ",
	// as @codemirror/language's legacy token table reads them.
	Token(s *Stream) string
}

Mode is a stream mode with its state: a fresh one for each document, since a mode carries its state from line to line.

type Stream

type Stream struct {
	// contains filtered or unexported fields
}

Stream is one line of a document, read by a mode one token at a time. Like JavaScript's StringStream it counts in UTF-16 code units, so a character outside the BMP is two units and a mode sees them one by one.

func (*Stream) BackUp

func (s *Stream) BackUp(n int)

BackUp moves back n code units.

func (*Stream) Current

func (s *Stream) Current() []uint16

Current is the text of the token read so far.

func (*Stream) EOL

func (s *Stream) EOL() bool

EOL reports whether the stream is at the end of the line.

func (*Stream) Eat

func (s *Stream) Eat(c uint16) bool

Eat reads the next code unit if it is c.

func (*Stream) EatFunc

func (s *Stream) EatFunc(match func(uint16) bool) bool

EatFunc reads the next code unit if match accepts it.

func (*Stream) EatSpace

func (s *Stream) EatSpace() bool

EatSpace reads whitespace, and reports whether it read any.

func (*Stream) EatWhile

func (s *Stream) EatWhile(match func(uint16) bool) bool

EatWhile reads code units as long as match accepts them, and reports whether it read any.

func (*Stream) Line

func (s *Stream) Line() []uint16

Line is the whole line.

func (*Stream) LookingAt

func (s *Stream) LookingAt(text string) bool

LookingAt reports whether the line continues with text, an ASCII string, without reading it: match(text, false).

func (*Stream) Match

func (s *Stream) Match(text string) bool

Match reads text if the line continues with it, as StringStream's match does with a string.

func (*Stream) MatchFunc

func (s *Stream) MatchFunc(match func(rest []uint16) int) bool

MatchFunc reads what match finds at the start of the rest of the line, the job of a regular expression starting with ^ in StringStream's match: match returns the length it matched, or -1 for none.

func (*Stream) Next

func (s *Stream) Next() int

Next reads the next code unit, EOL (and no move) at the end of the line.

func (*Stream) Peek

func (s *Stream) Peek() int

Peek is the next code unit, EOL at the end of the line.

func (*Stream) SOL

func (s *Stream) SOL() bool

SOL reports whether the stream is at the start of the line.

func (*Stream) SkipTo

func (s *Stream) SkipTo(c uint16) bool

SkipTo moves to just before the next c on the line, and reports whether there was one; without one the stream stays where it is.

func (*Stream) SkipToEnd

func (s *Stream) SkipToEnd()

SkipToEnd moves to the end of the line.