displaywidth

README

displaywidth

A high-performance Go package for measuring the monospace display width of strings, UTF-8 bytes, and runes.

Install

go get github.com/clipperhouse/displaywidth

Usage

package main

import (
    "fmt"
    "github.com/clipperhouse/displaywidth"
)

func main() {
    width := displaywidth.String("Hello, 世界!")
    fmt.Println(width)

    width = displaywidth.Bytes([]byte("🌍"))
    fmt.Println(width)

    width = displaywidth.Rune('🌍')
    fmt.Println(width)
}

For most purposes, you should use the String or Bytes methods.

Options

You can specify East Asian Width settings. When false (default), East Asian Ambiguous characters are treated as width 1. When true, East Asian Ambiguous characters are treated as width 2.

myOptions := displaywidth.Options{
    EastAsianWidth: true,
}

width := myOptions.String("Hello, 世界!")
fmt.Println(width)

Technical details

This package implements the Unicode East Asian Width standard (UAX #11), and handles version selectors, and regional indicator pairs (flags). We implement Unicode TR51.

clipperhouse/displaywidth, mattn/go-runewidth, and rivo/uniseg will give the same outputs for most real-world text. See extensive details in the compatibility analysis.

If you wish to investigate the core logic, see the lookupProperties and width functions in width.go. The essential trie generation logic is in buildPropertyBitmap in unicode.go.

I (@clipperhouse) am keeping an eye on emerging standards and test suites.

Prior Art

mattn/go-runewidth

rivo/uniseg

x/text/width

x/text/internal/triegen

Benchmarks

cd comparison
go test -bench=. -benchmem

goos: darwin
goarch: arm64
pkg: github.com/clipperhouse/displaywidth/comparison
cpu: Apple M2

BenchmarkString_Mixed/clipperhouse/displaywidth-8         10929 ns/op	    154.36 MB/s	      0 B/op     0 allocs/op
BenchmarkString_Mixed/mattn/go-runewidth-8                14540 ns/op	    116.02 MB/s	      0 B/op     0 allocs/op
BenchmarkString_Mixed/rivo/uniseg-8                       19751 ns/op	     85.41 MB/s	      0 B/op     0 allocs/op

BenchmarkString_EastAsian/clipperhouse/displaywidth-8     10885 ns/op	    154.98 MB/s	      0 B/op     0 allocs/op
BenchmarkString_EastAsian/mattn/go-runewidth-8            23969 ns/op	     70.38 MB/s	      0 B/op     0 allocs/op
BenchmarkString_EastAsian/rivo/uniseg-8                   19852 ns/op	     84.98 MB/s	      0 B/op     0 allocs/op

BenchmarkString_ASCII/clipperhouse/displaywidth-8          1103 ns/op	    116.01 MB/s	      0 B/op     0 allocs/op
BenchmarkString_ASCII/mattn/go-runewidth-8                 1166 ns/op	    109.79 MB/s	      0 B/op     0 allocs/op
BenchmarkString_ASCII/rivo/uniseg-8                        1584 ns/op	     80.83 MB/s	      0 B/op     0 allocs/op

BenchmarkString_Emoji/clipperhouse/displaywidth-8          3108 ns/op	    232.93 MB/s	      0 B/op     0 allocs/op
BenchmarkString_Emoji/mattn/go-runewidth-8                 4802 ns/op	    150.76 MB/s	      0 B/op     0 allocs/op
BenchmarkString_Emoji/rivo/uniseg-8                        6607 ns/op	    109.58 MB/s	      0 B/op     0 allocs/op

BenchmarkRune_Mixed/clipperhouse/displaywidth-8            3456 ns/op	    488.20 MB/s	      0 B/op     0 allocs/op
BenchmarkRune_Mixed/mattn/go-runewidth-8                   5400 ns/op	    312.39 MB/s	      0 B/op     0 allocs/op

BenchmarkRune_EastAsian/clipperhouse/displaywidth-8        3475 ns/op	    485.41 MB/s	      0 B/op     0 allocs/op
BenchmarkRune_EastAsian/mattn/go-runewidth-8              15701 ns/op	    107.44 MB/s	      0 B/op     0 allocs/op

BenchmarkRune_ASCII/clipperhouse/displaywidth-8             257.0 ns/op	    498.13 MB/s	      0 B/op     0 allocs/op
BenchmarkRune_ASCII/mattn/go-runewidth-8                    266.4 ns/op	    480.50 MB/s	      0 B/op     0 allocs/op

BenchmarkRune_Emoji/clipperhouse/displaywidth-8            1384 ns/op	    523.02 MB/s	      0 B/op     0 allocs/op
BenchmarkRune_Emoji/mattn/go-runewidth-8                   2273 ns/op	    318.45 MB/s	      0 B/op     0 allocs/op

Documentation

Index

• Variables
• func Bytes(s []byte) int
• func Rune(r rune) int
• func String(s string) int
• type Options
  • func (options Options) Bytes(s []byte) int
  • func (options Options) Rune(r rune) int
  • func (options Options) String(s string) int

Variables

var DefaultOptions = Options{EastAsianWidth: false}

DefaultOptions is the default options for the display width calculation, which is EastAsianWidth: false.

Functions

func Bytes

func Bytes(s []byte) int

Bytes calculates the display width of a []byte, by iterating over grapheme clusters in the byte slice and summing their widths.

func Rune

func Rune(r rune) int

Rune calculates the display width of a rune. You should almost certainly use String or Bytes for most purposes.

The smallest unit of display width is a grapheme cluster, not a rune. Iterating over runes to measure width is incorrect in most cases.

func String

func String(s string) int

String calculates the display width of a string, by iterating over grapheme clusters in the string and summing their widths.

Types

type Options

type Options struct {
	EastAsianWidth bool
}

Options allows you to specify the treatment of ambiguous East Asian characters. When EastAsianWidth is false (default), ambiguous East Asian characters are treated as width 1. When EastAsianWidth is true, ambiguous East Asian characters are treated as width 2.

func (Options) Bytes

func (options Options) Bytes(s []byte) int

Bytes calculates the display width of a []byte, for the given options, by iterating over grapheme clusters in the byte slice and summing their widths.

func (Options) Rune

func (options Options) Rune(r rune) int

Rune calculates the display width of a rune, for the given options.

The smallest unit of display width is a grapheme cluster, not a rune. Iterating over runes to measure width is incorrect in most cases.

func (Options) String

func (options Options) String(s string) int

String calculates the display width of a string, for the given options, by iterating over grapheme clusters and summing their widths.