bidi

README

bidi

A pure-Go, CGO_ENABLED=0, standard-library-only implementation of the Unicode Bidirectional Algorithm (UAX #9) for laying out mixed left-to-right / right-to-left text.

Unlike most Go bidi implementations, go-opentype/bidi does not depend on golang.org/x/text. The Bidi_Class and paired-bracket Unicode properties are compiled into small generated lookup tables (see cmd/genbidi), so the package builds anywhere the standard library does.

Install

go get github.com/go-opentype/bidi

Usage

package main

import (
	"fmt"

	"github.com/go-opentype/bidi"
)

func main() {
	// A logical-order string mixing English and Hebrew.
	s := "abc אבג def"

	// Visual (left-to-right) order for display.
	fmt.Println(bidi.VisualOrder(s, bidi.Auto))

	// Or work with levels directly.
	runes := []rune(s)
	levels := bidi.ResolveLevels(runes, bidi.LeftToRight)
	order := bidi.Reorder(runes, levels) // visual-order permutation of indices
	fmt.Println(levels, order)

	// Inspect a single rune's Bidi_Class.
	fmt.Println(bidi.ClassOf('א')) // R
}

API

Symbol	Purpose
ClassOf(r rune) Class	Bidi_Class of a rune
Class enum	L R AL EN ES ET AN CS NSM BN B S WS ON LRE RLE LRO RLO PDF LRI RLI FSI PDI
ResolveLevels(text []rune, base Direction) []Level	resolved embedding level per rune
BaseLevel(text []rune, base Direction) Level	paragraph level (rules P2/P3)
Reorder(text []rune, levels []Level) []int	rule L2 visual-order permutation
VisualOrder(text string, base Direction) string	resolve + reorder convenience
Direction	LeftToRight, RightToLeft, Auto
Level	embedding level (even = LTR, odd = RTL)

Implemented vs deferred

Implemented — the algorithm runs through rule L2, the full extent covered by the Unicode conformance file BidiCharacterTest.txt:

• P2, P3 base paragraph level from the first strong character.
• X1–X8 explicit embeddings and isolates (with overflow handling and the directional status stack); X9 removal of the deprecated formatting characters and BN; X10 isolating run sequences with sos/eos.
• W1–W7 weak types.
• N0 paired-bracket resolution (BD16, incl. the U+2329/U+232A canonical equivalence), N1–N2 neutral types.
• I1, I2 implicit levels.
• L1 separator / trailing-whitespace reset, L2 reordering.

Deferred (out of scope for a bidi engine):

• L3 (combining marks) and L4 (glyph mirroring of paired brackets and other mirrored characters) — these belong to the rendering / shaping stage, so VisualOrder does not substitute mirrored glyphs.
• Arabic cursive shaping / joining — the job of a shaper, not bidi.
• P1 paragraph splitting — the caller drives it; the API operates per paragraph (an inline Paragraph_Separator is still handled by X8/L1).

Conformance

The package is validated against the entire BidiCharacterTest.txt (all cases pass: paragraph level, per-character levels and visual order). A curated representative subset is embedded under testdata and run by TestConformance. CI enforces exactly 100% statement coverage, go vet, gofmt, and cross-compilation for the six 64-bit architectures plus js/wasm, darwin/arm64 and windows/amd64.

Regenerating the tables

go run ./cmd/genbidi .

This fetches the latest DerivedBidiClass.txt and BidiBrackets.txt from the Unicode Character Database and rewrites bidiclass_table.go and bidibrackets_table.go.

License

BSD-3-Clause. See LICENSE.

Documentation

Overview

Package bidi is a pure-Go, CGO=0, standard-library-only implementation of the Unicode Bidirectional Algorithm (UAX #9) for laying out mixed left-to-right / right-to-left text.

It has no dependency on golang.org/x/text: the Bidi_Class property and the paired-bracket property are shipped as generated lookup tables (see cmd/genbidi), so the package builds anywhere the standard library does.

Public API

• ClassOf reports the Bidi_Class of a rune, and the Class enum enumerates every Bidi_Class value.
• ResolveLevels runs the algorithm over a paragraph and returns the resolved embedding Level of each rune.
• BaseLevel reports the paragraph embedding level chosen by rules P2/P3.
• Reorder applies rule L2 to a level slice, returning the visual-order index permutation.
• VisualOrder is a convenience that resolves and reorders a string in one call.
• Direction selects the base direction: LeftToRight, RightToLeft or Auto.

Implemented rules

The algorithm is implemented through rule L2, which is the full extent verified by the Unicode conformance file BidiCharacterTest.txt:

• P2, P3: paragraph embedding level from the first strong character.
• X1–X8: explicit embeddings (LRE/RLE/LRO/RLO/PDF) and isolates (LRI/RLI/FSI/PDI), including overflow handling and the directional status stack, with FSI resolved per X5c.
• X9: the deprecated embedding/override/PDF formatting characters and BN are removed (reported as carrying the preceding character's level).
• X10: isolating run sequences with their sos/eos boundary types.
• W1–W7: the weak-type rules.
• N0: paired-bracket resolution (BD16), including the U+2329/U+232A ~ U+3008/U+3009 canonical equivalence.
• N1, N2: the neutral-type rules.
• I1, I2: the implicit level rules.
• L1: resetting separators and trailing whitespace to the paragraph level.
• L2: reordering to visual order.

The package is validated against the full BidiCharacterTest.txt (all cases pass); a curated subset is embedded under testdata for the committed test suite.

Deferred

• L3 (combining marks) and L4 (mirroring of paired-bracket and other mirrored glyphs) are out of scope: they belong to the rendering/shaping stage. VisualOrder therefore does not substitute mirrored glyphs.
• Arabic cursive shaping and joining are the job of a shaper, not the bidirectional algorithm, and are out of scope here.
• Rule P1 (splitting text into paragraphs on Paragraph_Separator) is the caller's responsibility; the API operates on a single paragraph, though a Paragraph_Separator encountered inline is handled by rule X8/L1.

Index

• func Reorder(text []rune, levels []Level) []int
• func VisualOrder(text string, base Direction) string
• type Class
  • func ClassOf(r rune) Class
  • func (c Class) String() string
• type Direction
• type Level
  • func BaseLevel(text []rune, base Direction) Level
  • func ResolveLevels(text []rune, base Direction) []Level

Functions

func Reorder

func Reorder(text []rune, levels []Level) []int

Reorder implements rule L2. Given the per-rune levels (as returned by ResolveLevels for the same text), it returns the visual left-to-right order as a permutation of indices into text.

func VisualOrder

func VisualOrder(text string, base Direction) string

VisualOrder is a convenience that resolves levels for text and returns the characters in visual (left-to-right) order. Characters removed by rule X9 are dropped. Glyph mirroring (rule L4) is not applied; see the package documentation.

Types

type Class

type Class uint8

Class is a Unicode Bidi_Class property value, as defined by UAX #9 and UAX #44. Every rune has exactly one Bidi_Class.

const (
	// Strong types.
	L  Class = iota // Left_To_Right
	R               // Right_To_Left
	AL              // Arabic_Letter

	// Weak types.
	EN  // European_Number
	ES  // European_Separator
	ET  // European_Terminator
	AN  // Arabic_Number
	CS  // Common_Separator
	NSM // Nonspacing_Mark
	BN  // Boundary_Neutral

	// Neutral types.
	B  // Paragraph_Separator
	S  // Segment_Separator
	WS // White_Space
	ON // Other_Neutral

	// Explicit formatting types.
	LRE // Left_To_Right_Embedding
	RLE // Right_To_Left_Embedding
	LRO // Left_To_Right_Override
	RLO // Right_To_Left_Override
	PDF // Pop_Directional_Format
	LRI // Left_To_Right_Isolate
	RLI // Right_To_Left_Isolate
	FSI // First_Strong_Isolate
	PDI // Pop_Directional_Isolate

)

The Bidi_Class values. The ordering matches the enumeration used throughout the Unicode Bidirectional Algorithm; do not rely on the numeric values other than for equality.

func ClassOf

func ClassOf(r rune) Class

ClassOf returns the Bidi_Class of r. Runes outside the assigned ranges follow the Unicode defaults declared by the @missing lines of DerivedBidiClass.txt (L in general, R/AL in unassigned right-to-left blocks, ET in the currency block, BN for default-ignorable and non-characters).

It is named ClassOf rather than Class because Go does not allow a function to share the name of the Class type it returns.

func (Class) String

func (c Class) String() string

String returns the short Bidi_Class abbreviation (for example "AL"). An out-of-range Class value renders as "Class(N)".

type Direction

type Direction int

Direction selects the base paragraph direction passed to the algorithm.

const (
	// LeftToRight forces a base paragraph embedding level of 0.
	LeftToRight Direction = iota
	// RightToLeft forces a base paragraph embedding level of 1.
	RightToLeft
	// Auto derives the base level from the first strong character (rules P2
	// and P3), defaulting to left-to-right.
	Auto
)

type Level

type Level int

Level is a bidirectional embedding level as defined by UAX #9. Even levels are left-to-right; odd levels are right-to-left.

func BaseLevel

func BaseLevel(text []rune, base Direction) Level

BaseLevel returns the resolved paragraph embedding level chosen for text under base (rules P2/P3): 0 for left-to-right, 1 for right-to-left.

func ResolveLevels

func ResolveLevels(text []rune, base Direction) []Level

ResolveLevels runs the Unicode Bidirectional Algorithm over text and returns the resolved embedding level of every rune. Even levels are left-to-right, odd levels right-to-left. Characters removed by rule X9 (the deprecated embedding/override formatting characters and BN) carry the level of the preceding retained character.