README
¶
internal/textcell
Measures and manipulates already-rendered terminal text in cells — the unit a terminal actually charges for — rather than in bytes or runes.
Why it's its own package
Three callers have to agree, exactly, on how wide a string is:
internal/renderlays content out and decides where a box's edges fall.tuipaints that layout onto a real screen and positions the hardware cursor on top of it.documentturns a click's column back into an offset into a value.
Any disagreement between them is visible: a cursor sitting inside a glyph, a
border landing a column past its box, a repaint that smears. Before this
package existed, tui reached into internal/render for two measurement
functions and kept a byte-for-byte copy of a third, and the width rules lived
in a 979-line file next to roman numerals and CSS text-transform. Putting
the measurement in one leaf package that all three depend on makes their
agreement structural instead of a convention someone has to remember.
It also fixes the dependency shape: tui depends on a leaf, not on the
layout engine.
Three units, and which one to use
Mixing these up is the single most common bug this package exists to prevent:
| Unit | What it is | Where it belongs |
|---|---|---|
| bytes | Go's native string index | never a screen position |
| runes | Unicode code points | caret/selection offsets, []rune indexing |
| cells (columns) | what a terminal charges | every width, every screen position |
Runes and cells coincide for ASCII and diverge everywhere else: a CJK
ideograph is one rune and two cells; a combining accent is a second rune and
zero extra cells; an emoji ZWJ sequence is five runes, one grapheme cluster,
and two cells. Arithmetic that adds a rune count to a column is wrong even
when its tests pass, because an all-ASCII corpus cannot tell the two apart.
That's what widecolumn_test.go in the repository root is for — it renders
awkward-width payloads through every box-producing construct and checks the
result is the width the layout claimed.
What must NOT be folded in here
A text entry's caret and selection are rune offsets, and they stay that way. This is not an inconsistency waiting to be tidied up:
- The DOM's
selectionStart/selectionEnd/setSelectionRangeare offsets into the value's character sequence.documentmirrors them so a caller who knows the web platform gets the answer they expect (seeCOMPATIBILITY.md's text-selection deviation — rune offsets, not UTF-16 code units, and not columns either). - Columns are a property of how a value is rendered. They change with
East Asian width settings, and they don't exist at all for a value that
was never rendered — a
Documentthat has never hadRendercalled still has a perfectly well-defined selection.
So the conversion is a boundary, not a representation. ColumnForRuneIndex
and RuneIndexForColumn are the only two places the units meet, and they're
called at the edges: placing a terminal cursor (tui's focusCursorPos), or
turning a click into an offset (document's caretIndexFromClick).
Everything on the document side stays in runes; everything on the painting
side stays in cells.
"These are all just character counts, unify them" is the tempting wrong move. It would silently break every non-ASCII selection.
API shape
- Measure —
Width,VisibleLen(ANSI-aware),NextGrapheme - Convert —
ColumnForRuneIndex,RuneIndexForColumn(the boundary above) - Slice by column —
SplitAtWidth,VisibleWindow,SpliceColumns,VisiblePrefix,TruncateToWidth,TrimTrailingSpace - ANSI plumbing —
Carry(style/hyperlink state carried across an inserted line break),ConsumeANSI,Strip,SplitTokens,CollapseDeadSpans
Everything is a pure function of its arguments. There is deliberately no HTML, no CSS, and no layout here: callers that need to know what a string means keep that knowledge. This package only answers how much room it takes and where its boundaries fall.
Documentation
¶
Overview ¶
Package textcell measures and manipulates already-rendered terminal text in *cells*, the unit a terminal charges for, rather than in bytes or runes.
It exists because three different callers have to agree, exactly, on how wide a string is: internal/render, which lays content out; tui, which paints that layout onto a real screen and positions the hardware cursor; and document, which turns a click's column back into an offset in a value. Any disagreement between them shows up as a cursor sitting inside a glyph, a border landing a column past its box, or a repaint that smears. Keeping the measurement in one leaf package, depended on by all three, is what makes agreement structural instead of a convention.
Everything here is a pure function of its arguments. There is deliberately no HTML, no CSS, and no layout in this package. Callers that need to know what a string *means* keep that knowledge; this package only answers how much room it takes and where its boundaries fall.
Three units, and which one to use ¶
Mixing these up is the single most common bug this package exists to prevent, so they're named apart:
- bytes: Go's native string index. Never a screen position.
- runes: Unicode code points. What a caret offset is measured in (see the caution below), and what []rune indexes.
- cells, or columns: what a terminal charges. What Width, VisibleLen, SpliceColumns and everything else here return and accept.
Runes and cells coincide for ASCII and diverge everywhere else. A CJK ideograph is one rune and two cells; a combining accent is a second rune and zero extra cells; an emoji ZWJ sequence is five runes, one grapheme cluster, and two cells. Any arithmetic that adds a rune count to a column is wrong even when its tests pass, because an all-ASCII corpus can't tell the two apart. The repository's column-invariant suite, in widecolumn_test.go, exists to catch exactly that.
What must NOT be folded into this package ¶
A text entry's caret and selection are rune offsets into its value, and they must stay that way. That is not an oversight to be tidied up. The DOM's selectionStart, selectionEnd, and setSelectionRange are offsets into the value's character sequence, and document mirrors them so a caller that knows the web platform gets the answer it expects. Screen columns are a property of how that value happens to be *rendered*: they change with font width tables and East Asian width settings, and they don't exist at all for a value that was never rendered.
So the conversion is a boundary, not a representation. ColumnForRuneIndex and RuneIndexForColumn are the only two places the units meet, and they are called at the edges, when placing a terminal cursor or turning a click into an offset. Everything on the document side of that boundary stays in runes; everything on the painting side stays in cells. "These are all just character counts, unify them" is the tempting wrong move, and it would silently break every non-ASCII selection.
Index ¶
- func CollapseDeadSpans(s string) string
- func ColumnForRuneIndex(s string, idx int) int
- func ConsumeANSI(runes []rune, i int) int
- func NextGrapheme(runes []rune, i int) (width int, next int)
- func RuneIndexForColumn(s string, col int) int
- func SpliceColumns(line string, col, width int, replacement string) string
- func SplitTokens(text string) []string
- func Strip(s string) string
- func TrimTrailingSpace(s string) (trimmed string, ok bool)
- func TruncateToWidth(s string, width int, suffix string) string
- func VisibleLen(s string) int
- func VisiblePrefix(s string, width int) string
- func VisibleWindow(s string, offset, width int) string
- func Width(s string) int
- type Carry
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CollapseDeadSpans ¶
CollapseDeadSpans removes SGR and OSC8 "open" escape sequences that are immediately superseded by a full reset of the same kind, with no visible character in between: spans that never got the chance to style or link anything. Dropping such an open and keeping only the reset is always safe regardless of any earlier context. A full SGR reset, "\x1b[m" or "\x1b[0m", or an OSC8 reset, the empty-URI "\x1b]8;;...", unconditionally clears that kind of state no matter what, if anything, was open beforehand, so whether the dead open ran first makes no observable difference.
This is deliberately one-directional. It does NOT collapse "open A, open B", with no reset between, down to just B, even though B logically supersedes A. This codebase's SGR sequences are additive rather than full-replace (see Carry's doc comment), so "\x1b[3;4m", italic plus underline, directly followed by "\x1b[3m", italic, with no reset in between would leave underline stuck on, since there's no explicit "turn off underline" code involved. Only a real reset ever fully clears state, so only a real reset is a safe deletion trigger for what precedes it.
wordWrapTokens routinely produces exactly the pattern this removes. coalesceTextRuns concatenates several already-independently-styled spans, each self-contained from inlineStyle.render, into one string before SplitTokens re-tokenizes it purely on literal space characters, with no awareness of where one span's styling ends and the next begins. A closing sequence that immediately follows a space, closing the word before it, routinely ends up re-attached to the front of the *next* word's token instead, and the line-wrap carry mechanism, ensureOpen and closeAndPush, can then reopen a span right before that misplaced close cancels it out again. One case is an <a>'s own trailing single-space text node, styled only with underline, sitting right at a word-wrap boundary next to a differently-styled neighbor. The result is well-formed but pointless escape sequences like "\x1b[3m\x1b[m" wrapping zero characters. This is a final cleanup pass, not a fix for the token misattribution itself. It only guarantees the misattribution can never surface as a dangling or empty span in output.
func ColumnForRuneIndex ¶
ColumnForRuneIndex returns the terminal column, relative to the start of plain, escape-free text s, that the caret sits at when it's idx runes into s. Rune offsets and columns are not interchangeable, since a CJK or emoji cluster is one offset but two columns, so every caret-placement site has to convert rather than using an offset as a column directly.
An idx landing *inside* a multi-rune grapheme cluster resolves to that cluster's own starting column. A terminal cursor can't be placed halfway through a cluster, and rounding down keeps this the exact inverse of RuneIndexForColumn, so a click and the cursor it produces agree on a cell.
Exported for htmlterm/tui's use, in focusCursorPos, for the same reason NextGrapheme is: the terminal's own hardware cursor has to land on the column this package's layout put that rune at.
func ConsumeANSI ¶
ConsumeANSI returns the index just past the escape sequence starting at runes[i], which must be '\x1b'. It is the same recognizer every ANSI-aware helper in this package tokenizes escape sequences with, including wordWrapTokens, VisibleLen, and the Carry state machine. It delegates to x/ansi's decoder rather than hand-rolling CSI and OSC recognition a second time. runes[i:] is re-encoded to a string because the decoder works in bytes, and the consumed byte count is converted back to a rune count since every caller here indexes into a []rune.
func NextGrapheme ¶
NextGrapheme returns the terminal column width of the grapheme cluster starting at runes[i], and the index of the rune immediately following that cluster. A cluster is usually one rune, but can span several: VS16 pairs, ZWJ sequences, regional-indicator flag pairs, skin-tone modifiers. displaywidth's grapheme segmenter finds the boundary and scores the whole cluster as one unit, so callers no longer need to track a "previous rune's width" the way the old VS16-only correction did.
The caller is responsible for not calling this with runes[i] == '\x1b'. ANSI escape recognition is handled separately (see ConsumeANSI) by every caller of this function, so a cluster can never span into an escape sequence.
Exported for htmlterm/tui's use. cellbridge.go's writeANSILine must walk an already-rendered ANSI line and paint it onto a real tcell.Screen one cluster at a time, not one rune at a time, advancing column position by exactly what this function returns. Otherwise the painted frame desyncs from the frame this package's own layout and wrapping already measured, which is what previously caused a wide emoji to shift every subsequent character, including the scrollbar gutter, one column left of where htmlterm's own layout placed it.
func RuneIndexForColumn ¶
RuneIndexForColumn is ColumnForRuneIndex's inverse: the rune offset into plain, escape-free text s that terminal column col falls in. A column landing on the trailing half of a double-width cluster resolves to that cluster's own offset rather than the following one, so clicking either cell of a wide character puts the caret before it, and clicking a cell then drawing the cursor for the resulting caret lands back on the same cluster. A column past the end of s returns len([]rune(s)).
Exported for document's use, in caretIndexFromClick, the click-to-caret counterpart of tui's cursor placement.
func SpliceColumns ¶
SpliceColumns overwrites visible columns [col, col+width) of line with replacement, preserving line's ANSI styling for the untouched prefix and suffix and re-carrying any open span across the splice boundary. It is the primitive popup and overlay rendering needs (see docs/RENDERING.md's popup section) to punch replacement content into an already-composed line without disturbing anything outside the overwritten range, unlike splitAtVisualWidth and SplitAtWidth, which only chop a string into sequential chunks from column 0 and have no notion of an interior overwrite. replacement is inserted verbatim, and the caller is responsible for it being exactly width visible columns wide, a popup's own already-rendered box.width for instance, so the result's total visible width matches line's. SpliceColumns does not itself pad or truncate it.
If line is narrower than col, it's padded with spaces so replacement lands at the requested column. If line is narrower than col+width, there's nothing past its own end to preserve, so the result simply ends after replacement.
func SplitTokens ¶
SplitTokens splits text on whitespace but keeps ANSI sequences attached.
func TrimTrailingSpace ¶
TrimTrailingSpace removes s's last visible, non-ANSI-escape rune if it's a space, preserving every escape sequence exactly where it was, including ones immediately before or after the removed rune, so a styled span's opening and closing codes stay correctly paired around the shortened content. ok is false, and s is returned unchanged, if s has no visible content or its last visible rune isn't a space.
func TruncateToWidth ¶
TruncateToWidth truncates s to at most width terminal *columns*, not runes, which is what its name and this comment used to say. It measures with VisibleLen, so ANSI escapes are free and a double-width cluster costs two. suffix is appended when content is clipped — use "" for a clip with no indicator, "…" for an ellipsis — and is itself measured the same way.
func VisibleLen ¶
VisibleLen returns the number of visible (non-ANSI-escape) runes in s.
func VisiblePrefix ¶
func VisibleWindow ¶
VisibleWindow returns the [offset, offset+width) visible-column window of s. It is wide-rune and ANSI aware, and, like SplitAtWidth's internal chunk boundaries, self-contained: an SGR or OSC8 span still open at offset is reopened at the window's start and closed at its end. It is used for overflow-x: scroll|auto's horizontal scroll-offset slicing (see docs/SCROLLING.md), where offset can land anywhere within a line, unlike SplitAtWidth, which only ever chops a string into sequential chunks from column 0. A wide rune straddling either boundary is dropped entirely rather than split, the same rule VisiblePrefix already applies at its own single right-hand boundary.
func Width ¶
Width is the terminal column width of an entire plain, escape-free string: the sum of each grapheme cluster's display width, with VS16 pairs, ZWJ sequences, flag pairs, and the rest already scored as one unit. There is no ANSI-escape interleaving to preserve here, unlike this file's other width-scanning functions, so this calls displaywidth directly instead of looping via NextGrapheme.
Types ¶
type Carry ¶
type Carry struct {
// contains filtered or unexported fields
}
Carry tracks the single currently-open SGR style span, the single currently-open OSC8 hyperlink span, or both, while scanning already-styled or hyperlinked text left to right. A line break inserted mid-span, by word-wrap or a hard break, can then close the span before the break and reopen the identical span right after it, making every emitted line independently self-contained.
It is deliberately flat rather than a stack. htmlterm's styling pipeline, style.go's inlineStyle.render plus block.go's wrapHyperlink, only ever emits one combined SGR open and one full reset per span, and one OSC8 open and one OSC8 reset per hyperlink. There are never partial resets and never two concurrently-open spans of the same kind. If that invariant changes, this needs to become a stack.
func SplitAtWidth ¶
SplitAtWidth is splitAtVisualWidth's carry-aware core. start is whatever SGR or OSC8 span is already open as of the start of s, from preceding text this call doesn't see, and the returned carry is whatever span is left open at the end of s. Every internal width-driven break closes the currently-open span before the break and reopens the identical span immediately after, so every returned chunk is self-contained.
func (*Carry) Apply ¶
Apply classifies one fully-extracted escape sequence, as produced by ConsumeANSI, and updates the carry. An SGR sequence, CSI ... 'm', with empty or "0" params clears the active SGR span, and any other SGR sequence replaces it. An OSC8 sequence with an empty URI clears the active hyperlink span, and any other OSC8 sequence replaces it. Anything else is left untouched.
Source Files