displaywidth

README

displaywidth

A high-performance Go package for measuring the monospace display width of strings, UTF-8 bytes, and runes.

Install

go get github.com/clipperhouse/displaywidth

Usage

package main

import (
    "fmt"
    "github.com/clipperhouse/displaywidth"
)

func main() {
    width := displaywidth.String("Hello, 世界!")
    fmt.Println(width)

    width = displaywidth.Bytes([]byte("🌍"))
    fmt.Println(width)

    width = displaywidth.Rune('🌍')
    fmt.Println(width)
}

For most purposes, you should use the String or Bytes methods. They sum the widths of grapheme clusters in the string or byte slice.

Note: in your application, iterating over runes to measure width is likely incorrect; the smallest unit of display is a grapheme, not a rune.

Iterating over graphemes

If you need the individual graphemes:

import (
    "fmt"
    "github.com/clipperhouse/displaywidth"
)

func main() {
    g := displaywidth.StringGraphemes("Hello, 世界!")
    for g.Next() {
        width := g.Width()
        value := g.Value()
        // do something with the width or value
    }
}

Options

Create the options you need, and then use methods on the options struct.

var myOptions = displaywidth.Options{
    EastAsianWidth: true,
    ControlSequences: true,
}

width := myOptions.String("Hello, 世界!")

ControlSequences

ControlSequences specifies whether to ignore ECMA-48 escape sequences when calculating the display width. When false (default), ANSI escape sequences are treated as just a series of characters. When true, they are treated as a single zero-width unit.

EastAsianWidth

EastAsianWidth defines how East Asian Ambiguous characters are treated.

When false (default), East Asian Ambiguous characters are treated as width 1. When true, they are treated as width 2.

You may wish to configure this based on environment variables or locale. go-runewidth, for example, does so during package initialization. displaywidth does not do this automatically, we prefer to leave it to you.

Technical standards and compatibility

This package implements the Unicode East Asian Width standard (UAX #11), and handles version selectors, and regional indicator pairs (flags). We implement Unicode TR51 for emojis. We are keeping an eye on emerging standards.

For control sequences, we implement the ECMA-48 standard for 7-bit ASCII control sequences.

clipperhouse/displaywidth, mattn/go-runewidth, and rivo/uniseg will give the same outputs for most real-world text. Extensive details are in the compatibility analysis.

Prior Art

mattn/go-runewidth

rivo/uniseg

x/text/width

x/text/internal/triegen

Benchmarks

cd comparison
go test -bench=. -benchmem

goos: darwin
goarch: arm64
pkg: github.com/clipperhouse/displaywidth/comparison
cpu: Apple M2

BenchmarkString_Mixed/clipperhouse/displaywidth-8             5784 ns/op	      291.69 MB/s	      0 B/op	   0 allocs/op
BenchmarkString_Mixed/mattn/go-runewidth-8                   14751 ns/op	      114.36 MB/s	      0 B/op	   0 allocs/op
BenchmarkString_Mixed/rivo/uniseg-8                          19360 ns/op	       87.14 MB/s	      0 B/op	   0 allocs/op

BenchmarkString_ASCII/clipperhouse/displaywidth-8               54.60 ns/op	     2344.32 MB/s	      0 B/op	   0 allocs/op
BenchmarkString_ASCII/mattn/go-runewidth-8                    1195 ns/op	      107.08 MB/s	      0 B/op	   0 allocs/op
BenchmarkString_ASCII/rivo/uniseg-8                           1578 ns/op	       81.13 MB/s	      0 B/op	   0 allocs/op

BenchmarkString_EastAsian/clipperhouse/displaywidth-8         5837 ns/op	      289.01 MB/s	      0 B/op	   0 allocs/op
BenchmarkString_EastAsian/mattn/go-runewidth-8               24418 ns/op	       69.09 MB/s	      0 B/op	   0 allocs/op
BenchmarkString_EastAsian/rivo/uniseg-8                      19339 ns/op	       87.23 MB/s	      0 B/op	   0 allocs/op

BenchmarkString_Emoji/clipperhouse/displaywidth-8             3225 ns/op	      224.51 MB/s	      0 B/op	   0 allocs/op
BenchmarkString_Emoji/mattn/go-runewidth-8                    4851 ns/op	      149.25 MB/s	      0 B/op	   0 allocs/op
BenchmarkString_Emoji/rivo/uniseg-8                           6591 ns/op	      109.85 MB/s	      0 B/op	   0 allocs/op

BenchmarkRune_Mixed/clipperhouse/displaywidth-8               3385 ns/op	      498.34 MB/s	      0 B/op	   0 allocs/op
BenchmarkRune_Mixed/mattn/go-runewidth-8                      5354 ns/op	      315.07 MB/s	      0 B/op	   0 allocs/op

BenchmarkRune_EastAsian/clipperhouse/displaywidth-8           3397 ns/op	      496.56 MB/s	      0 B/op	   0 allocs/op
BenchmarkRune_EastAsian/mattn/go-runewidth-8                 15673 ns/op	      107.64 MB/s	      0 B/op	   0 allocs/op

BenchmarkRune_ASCII/clipperhouse/displaywidth-8                255.7 ns/op	      500.53 MB/s	      0 B/op	   0 allocs/op
BenchmarkRune_ASCII/mattn/go-runewidth-8                       261.5 ns/op	      489.55 MB/s	      0 B/op	   0 allocs/op

BenchmarkRune_Emoji/clipperhouse/displaywidth-8               1371 ns/op	      528.22 MB/s	      0 B/op	   0 allocs/op
BenchmarkRune_Emoji/mattn/go-runewidth-8                      2267 ns/op	      319.43 MB/s	      0 B/op	   0 allocs/op

BenchmarkTruncateWithTail/clipperhouse/displaywidth-8         3229 ns/op	       54.82 MB/s	    192 B/op	  14 allocs/op
BenchmarkTruncateWithTail/mattn/go-runewidth-8                8408 ns/op	       21.05 MB/s	    192 B/op	  14 allocs/op

BenchmarkTruncateWithoutTail/clipperhouse/displaywidth-8      3554 ns/op	       64.43 MB/s	      0 B/op	   0 allocs/op
BenchmarkTruncateWithoutTail/mattn/go-runewidth-8            11189 ns/op	       20.47 MB/s	      0 B/op	   0 allocs/op

Here are some notes on how to make Unicode things fast.

Documentation

Index

• Variables
• func Bytes(s []byte) int
• func Rune(r rune) int
• func String(s string) int
• func TruncateBytes(s []byte, maxWidth int, tail []byte) []byte
• func TruncateString(s string, maxWidth int, tail string) string
• type Graphemes
  • func BytesGraphemes(s []byte) Graphemes[[]byte]
  • func StringGraphemes(s string) Graphemes[string]
  • func (g *Graphemes[T]) Next() bool
  • func (g *Graphemes[T]) Value() T
  • func (g *Graphemes[T]) Width() int
• type Options
  • func (options Options) Bytes(s []byte) int
  • func (options Options) BytesGraphemes(s []byte) Graphemes[[]byte]
  • func (options Options) Rune(r rune) int
  • func (options Options) String(s string) int
  • func (options Options) StringGraphemes(s string) Graphemes[string]
  • func (options Options) TruncateBytes(s []byte, maxWidth int, tail []byte) []byte
  • func (options Options) TruncateString(s string, maxWidth int, tail string) string

Variables

var DefaultOptions = Options{EastAsianWidth: false, ControlSequences: false}

DefaultOptions is the default options for the display width calculation, which is EastAsianWidth false and ControlSequences false.

Functions

func Bytes

func Bytes(s []byte) int

Bytes calculates the display width of a []byte, by iterating over grapheme clusters in the byte slice and summing their widths.

func Rune

func Rune(r rune) int

Rune calculates the display width of a rune. You should almost certainly use String or Bytes for most purposes.

The smallest unit of display width is a grapheme cluster, not a rune. Iterating over runes to measure width is incorrect in many cases.

func String

func String(s string) int

String calculates the display width of a string, by iterating over grapheme clusters in the string and summing their widths.

func TruncateBytes

func TruncateBytes(s []byte, maxWidth int, tail []byte) []byte

TruncateBytes truncates a []byte to the given maxWidth, and appends the given tail if the []byte is truncated.

It ensures the total width, including the width of the tail, is less than or equal to maxWidth.

func TruncateString

func TruncateString(s string, maxWidth int, tail string) string

TruncateString truncates a string to the given maxWidth, and appends the given tail if the string is truncated.

It ensures the total width, including the width of the tail, is less than or equal to maxWidth.

Types

type Graphemes

type Graphemes[T ~string | []byte] struct {
	// contains filtered or unexported fields
}

Graphemes is an iterator over grapheme clusters.

Iterate using the Next method, and get the width of the current grapheme using the Width method.

func BytesGraphemes

func BytesGraphemes(s []byte) Graphemes[[]byte]

BytesGraphemes returns an iterator over grapheme clusters for the given []byte.

Iterate using the Next method, and get the width of the current grapheme using the Width method.

func StringGraphemes

func StringGraphemes(s string) Graphemes[string]

StringGraphemes returns an iterator over grapheme clusters for the given string.

Iterate using the Next method, and get the width of the current grapheme using the Width method.

func (*Graphemes[T]) Next

func (g *Graphemes[T]) Next() bool

Next advances the iterator to the next grapheme cluster.

func (*Graphemes[T]) Value

func (g *Graphemes[T]) Value() T

Value returns the current grapheme cluster.

func (*Graphemes[T]) Width

func (g *Graphemes[T]) Width() int

Width returns the display width of the current grapheme cluster.

type Options

type Options struct {
	// EastAsianWidth specifies whether to treat ambiguous East Asian characters
	// as width 1 or 2. When false (default), ambiguous East Asian characters
	// are treated as width 1. When true, they are width 2.
	EastAsianWidth bool

	// ControlSequences specifies whether to ignore ECMA-48 escape sequences
	// when calculating the display width. When false (default), ANSI escape
	// sequences are treated as just a series of characters. When true, they are
	// treated as a single zero-width unit.
	//
	// Note that this option is about *sequences*. Individual control characters
	// are already treated as zero-width. With this option, ANSI sequences such as
	// "\x1b[31m" and "\x1b[0m" do not count towards the width of a string.
	ControlSequences bool
}

Options allows you to specify the treatment of ambiguous East Asian characters and ANSI escape sequences.

func (Options) Bytes

func (options Options) Bytes(s []byte) int

Bytes calculates the display width of a []byte, for the given options, by iterating over grapheme clusters in the slice and summing their widths.

func (Options) BytesGraphemes

func (options Options) BytesGraphemes(s []byte) Graphemes[[]byte]

BytesGraphemes returns an iterator over grapheme clusters for the given []byte, with the given options.

Iterate using the Next method, and get the width of the current grapheme using the Width method.

func (Options) Rune

func (options Options) Rune(r rune) int

Rune calculates the display width of a rune, for the given options.

You should almost certainly use String or Bytes for most purposes.

The smallest unit of display width is a grapheme cluster, not a rune. Iterating over runes to measure width is incorrect in many cases.

func (Options) String

func (options Options) String(s string) int

String calculates the display width of a string, for the given options, by iterating over grapheme clusters in the string and summing their widths.

func (Options) StringGraphemes

func (options Options) StringGraphemes(s string) Graphemes[string]

StringGraphemes returns an iterator over grapheme clusters for the given string, with the given options.

Iterate using the Next method, and get the width of the current grapheme using the Width method.

func (Options) TruncateBytes

func (options Options) TruncateBytes(s []byte, maxWidth int, tail []byte) []byte

TruncateBytes truncates a []byte to the given maxWidth, and appends the given tail if the []byte is truncated.

It ensures the visible width, including the width of the tail, is less than or equal to maxWidth.

When [Options.ControlSequences] is true, ANSI escape sequences that appear after the truncation point are preserved in the output. This ensures that escape sequences such as SGR resets are not lost, preventing color bleed in terminal output.

func (Options) TruncateString

func (options Options) TruncateString(s string, maxWidth int, tail string) string

TruncateString truncates a string to the given maxWidth, and appends the given tail if the string is truncated.

It ensures the visible width, including the width of the tail, is less than or equal to maxWidth.

When [Options.ControlSequences] is true, ANSI escape sequences that appear after the truncation point are preserved in the output. This ensures that escape sequences such as SGR resets are not lost, preventing color bleed in terminal output.