shape

README

go-opentype/shape

A pure-Go, CGO=0, standard-library-only complex-text shaper — a HarfBuzz-lite for the go-opentype stack. It turns a run of Unicode text into positioned glyphs in visual order, so Arabic, Indic, Southeast-Asian, CJK vertical and Egyptian Hieroglyph text renders correctly instead of as isolated, unattached, unreordered glyphs.

It composes two siblings and adds nothing else: bidi reordering from go-opentype/bidi and GSUB/GPOS from go-opentype/opentype. No golang.org/x/*, no third-party modules; it builds for every Go target including GOOS=js GOARCH=wasm.

Install

go get github.com/go-opentype/shape

Quick start

package main

import (
	"fmt"

	"github.com/go-opentype/fonts/notosansarabic"
	"github.com/go-opentype/opentype"
	"github.com/go-opentype/shape"
)

func main() {
	f, err := opentype.Parse(notosansarabic.TTF)
	if err != nil {
		panic(err)
	}
	face := f.NewFace(32) // 32px per em

	for _, g := range shape.Shape(face, "بيت", shape.Options{}) {
		// g.GID              glyph to draw
		// g.Cluster          source rune index it derives from
		// g.XOffset, g.YOffset   placement relative to the pen (px)
		// g.XAdvance, g.YAdvance advance to move the pen by (px)
		// g.Scale            draw scale (< 1 inside an Egyptian quadrat)
		fmt.Println(g)
	}
}

The base direction defaults to Auto (from the first strong character) and the script is auto-detected from the text (any Arabic-block rune selects Arabic, Indic/USE runes select their shaper, ...) unless you set Options.Script or Options.Direction.

Features

• Bidirectional reordering — resolves UAX #9 embedding levels (via go-opentype/bidi) and lays glyphs out left-to-right, so a right-to-left run is emitted in drawing order.
• Arabic cursive joining — resolves each letter's joining form (isolated / initial / medial / final) and applies the font's isol/init/medi/fina GSUB features positionally, each only at the glyphs in that form, so letters actually connect (including fonts built on the rasm-skeleton-plus-dots architecture, such as Noto Sans Arabic).
• Indic shaping — Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada, Malayalam and Sinhala, via the HarfBuzz "indic" model: syllable splitting, base/reph detection, two reordering passes (pre-base matras and reph), the full basic + presentation GSUB feature pipeline, and GPOS mark/mkmk/abvm/blwm attachment.
• Universal Shaping Engine (USE) — the general complex-script model for the scripts without a bespoke shaper: Thai, Lao, Khmer, Myanmar, Tibetan, Javanese, Balinese, Buginese, Tai Tham and more. Runs are classified into USE syllabic categories, split into clusters, reordered (pre-base vowels and modifiers before the base, repha after it) and run through the USE GSUB/GPOS pipeline, with sakot/halant joining, split-vowel decomposition and dotted-circle insertion for defective clusters.
• Egyptian Hieroglyph quadrats — the Unicode format-control characters U+13430–U+1345F (joiners, corner insertions, overlays, segment/enclosure delimiters) are parsed into a two-dimensional quadrat tree and laid out geometrically, so a run of signs renders as compact blocks.
• Vertical writing mode (CJK tategaki) — Options.Vertical selects the vert/vrt2 upright glyph forms and stacks glyphs top-to-bottom using the font's vertical metrics (vmtx/VORG).
• Ligatures, mark attachment, kerning — GSUB ccmp/rlig/liga/calt then GPOS kern/mark/mkmk/curs for every script path, so diacritics sit on their base and pairs kern.

API tour

type Glyph struct {
    GID      opentype.GlyphIndex // glyph to draw
    Cluster  int                 // source rune index
    XAdvance int
    YAdvance int
    XOffset  int
    YOffset  int
    Scale    float64 // 1.0 normally; < 1 inside an Egyptian quadrat
}

type Options struct {
    Direction bidi.Direction // LeftToRight, RightToLeft, Auto (default)
    Script    string         // "arab", "latn", "dflt", a script tag, or "" to auto-detect
    Features  []string       // extra feature tags applied over the whole run
    Vertical  bool           // CJK tategaki: top-to-bottom, vert/vrt2 forms
}

func Shape(face *opentype.Face, text string, opts Options) []Glyph

// DeriveUSECategory maps Unicode Indic_Syllabic_Category / Indic_Positional_Category /
// General_Category to a USE syllabic category; exported for cmd/genuse.
func DeriveUSECategory(uisc, uipc, ugc string) string

See example_test.go for runnable examples (an Arabic word with cursive joining, plain Latin, and forcing Options), and go doc github.com/go-opentype/shape for the full reference, including the Indic, USE, Egyptian Hieroglyph and Vertical sections.

Scope

Implemented: Arabic, the ten dedicated-shaper Indic scripts, the Universal Shaping Engine (Thai, Lao, Khmer, Myanmar, Tibetan, Javanese, Balinese, Buginese, Tai Tham, ...), Egyptian Hieroglyph quadrat layout, vertical writing mode, and Latin/default (Latin, Cyrillic, Greek, CJK horizontal, ...) shaping.

Cluster indices are exact for one-to-one substitutions (the Arabic positional forms) and best-effort, monotonic, when a substitution changes the run length (ligatures, decomposition, Indic/USE reordering).

Testing

Most branches are exercised with synthetic in-memory fonts; a handful of real-font sanity checks run against bundled go-opentype/fonts families (Noto Sans Arabic for cursive joining, and Noto Sans Balinese/Khmer/Myanmar/ Tai Tham under testdata for USE) to confirm the shaper actually joins, reorders and positions glyphs in production fonts, not just the synthetic test doubles. CI enforces 100.0% statement coverage, go vet, and cross-compilation for the six 64-bit architectures plus js/wasm, darwin/arm64 and windows/amd64.

Part of the go-opentype pure-Go text stack

go-opentype/shape is the HarfBuzz-lite shaping layer of a dependency-free text stack:

• opentype — the parsing, GSUB/GPOS shaping-primitives and rasterising engine this package builds its Shape call on.
• bidi — the Unicode Bidirectional Algorithm (UBA) implementation that resolves this package's base direction and reordering.
• shape (this repo) — the complex-script shaper.
• fonts — 36 bundled OFL/BSD font families, per-family lazily go:embed-ed, used throughout this repo's real-font tests and examples.

License

BSD-3-Clause. See LICENSE.

Documentation

Overview

Package shape is a HarfBuzz-lite complex-text shaper for the go-opentype stack. It turns a run of Unicode text into positioned glyphs in visual (left-to-right) order, ready to blit, applying the three things a naive cmap-then-GSUB pass gets wrong for real text:

• Bidirectional reordering (via github.com/go-opentype/bidi): resolve the UAX #9 embedding levels and lay the glyphs out left-to-right, so a right-to-left Arabic run is emitted in the order it is drawn.
• Arabic cursive joining: each letter's Unicode joining form (isolated, initial, medial, final) is resolved, then the font's isol/init/medi/fina GSUB features are applied positionally — each only at the glyphs in that form — via opentype's ApplyMasked. Without this, Arabic renders as disconnected isolated letters.
• Ligatures, mark attachment and kerning: GSUB ccmp/rlig/liga/calt then GPOS kern/mark/mkmk/curs, so diacritics sit on their base and pairs kern.
• Indic syllable reordering: Devanagari and the other Indic scripts are split into syllables, each glyph is given a reorder position, the Indic GSUB feature pipeline runs, and a stable sort moves pre-base (left) matras before the base consonant and the reph to its script-specific slot — the reordering a naive cmap-then-GSUB pass cannot express. See the Indic section below.
• Egyptian Hieroglyph quadrats: the format-control characters U+13430..U+1345F (vertical/horizontal joiners, corner insertions, overlays, segment and enclosure delimiters, plus the Unicode 15 blank and mirror additions) are parsed into a two-dimensional quadrat tree and laid out geometrically, so a run of signs renders as compact blocks rather than a flat row. See "Egyptian Hieroglyphs" below.
• Vertical writing mode (CJK tategaki): with Options.Vertical the vert/vrt2 features select upright vertical glyph forms and glyphs are stacked top-to-bottom using the font's vertical metrics. See "Vertical" below.
• Universal Shaping Engine (USE): the general complex-script model for the many scripts without a bespoke shaper (Thai, Lao, Khmer, Myanmar, Tibetan, Javanese, Balinese, Buginese, Tai Tham, ...). Runs are classified into USE syllabic categories, split into clusters, reordered (pre-base vowels and modifiers before the base, repha after it), then run through the USE GSUB/GPOS feature pipeline. See "USE" below.

Usage

face := font.NewFace(32)
glyphs := shape.Shape(face, "بيت", shape.Options{})
for _, g := range glyphs {
	// g.GID is the glyph to draw; advance the pen by g.XAdvance,
	// offset the glyph by (g.XOffset, g.YOffset). All in pixels.
}

The base direction defaults to Auto (derived from the first strong character); the script is auto-detected from the text (any Arabic-block rune selects the Arabic shaper) unless Options.Script forces it.

Indic

Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada, Malayalam and Sinhala are shaped with the HarfBuzz "indic" model (the dev2/bng2/... v2 feature set; the old deva/beng/... tags are accepted too and normalized). Each run is split into syllables and shaped in HarfBuzz's two reordering passes:

• Two-part dependent vowels (the Bengali/Tamil/Malayalam/... O and AU signs) are first decomposed into their canonical components, so each part reaches its own reorder position.
• The base consonant is found with the script's base-position model (BASE_POS_LAST for every script but Sinhala, which uses BASE_POS_LAST_SINHALA), any reph is found in the script's encoding mode (the implicit Ra + halant, Telugu's explicit Ra + halant + ZWJ, or Malayalam's logical-order U+0D4E repha), and a defective cluster — one opening with a dependent mark and no base — has a dotted circle (U+25CC) inserted to carry the marks.
• Initial reordering: a stable sort by ladder position moves pre-base matras before the base and parks the reph Ra at the front.
• The basic GSUB features run in order (locl, nukt, akhn, rphf masked to the reph Ra, rkrf, pref masked to the pre-base-reordering Ra, blwf, half, pstf, vatu, cjct), and whether rphf and pref actually fired is observed by re-shaping the masked run.
• Final reordering: a fired reph moves to its per-script slot (otherwise the Ra stays a pre-base consonant) and a fired pre-base-reordering Ra moves ahead of the base.
• The presentation features run (init, pres, abvs, blws, psts, haln).

GPOS then applies kern, dist, abvm, blwm, mark and mkmk. Character categories come from a table generated from the Unicode Character Database (Indic_Syllabic_Category and Indic_Positional_Category) by cmd/genindic.

Indic edges deliberately left simple:

• Only the old (deva) and v2 (dev2) OpenType tags are recognized; a font that files its features solely under a real script still resolves, since GSUB/GPOS use the script-agnostic default fallback.

Scope

Arabic, Indic, the Universal Shaping Engine (USE) and Latin/default (Latin, Cyrillic, Greek, CJK, ...) shaping are implemented, plus Egyptian Hieroglyph quadrat layout and vertical writing mode. USE covers the complex scripts that lack a bespoke shaper (Thai, Lao, Khmer, Myanmar, Tibetan, Javanese, ...), including the per-script behaviours sakot handling, split-vowel decomposition, pref/rphf feature-based reordering and dotted-circle insertion (see "USE" below). Cluster indices are exact for one-to-one substitutions (the Arabic positional forms) and best-effort, monotonic, when a substitution changes the run length (ligatures, decomposition, Indic and USE reordering).

Egyptian Hieroglyphs

A run whose script is "egyp" (or that contains any Egyptian rune, U+13000 and above) is shaped as quadrats: the format controls U+13430..U+1345F are an infix notation grouping signs two-dimensionally. Vertical joiners stack signs, horizontal joiners set them side by side, overlays place one over another, insertions drop a sign into a corner or edge of a host, and segment/enclosure delimiters bracket sub-groups; the Unicode 15 blank/shading code points are treated as space-occupying blanks and the mirror control is recognised as a geometric no-op. Each top-level quadrat is laid out inside one em block: every sign gets a Scale below 1 and an X/Y offset placing it in the block, and the block's advance is carried on its last glyph.

The quadrat layout is geometric: OpenType has no standard font-driven quadrat mechanism, so the control-character model above is implemented directly. The font's GSUB ccmp feature is consulted to pick font-preferred sign forms position-for-position before layout (a substitution that would change the sign count, such as a ligature, is suppressed to preserve the quadrat structure); font GPOS is not applied, the geometric placement supersedes it.

Vertical

Options.Vertical selects vertical writing mode (CJK tategaki). The vert/vrt2 GSUB features swap horizontal glyph forms for their upright vertical variants, and each glyph is positioned top-to-bottom: YAdvance is the glyph's vertical advance (from vmtx, or one em when the font has none), XOffset centres it on the vertical baseline, and YOffset is its vertical origin (VORG when present, otherwise the vhea ascender). Bidi reordering is not applied — a vertical column reads top-to-bottom in logical order.

USE

The Universal Shaping Engine handles the complex scripts Unicode supports without a dedicated shaper — Thai, Lao, Khmer, Myanmar, Tibetan, Javanese, Balinese, Buginese, Tai Tham and many more. A run is forced to USE with an explicit USE script tag (thai, khmr, lana, ...) or auto-detected from its runes. Each rune is assigned a USE syllabic category (base, halant, pre/above/ below/post vowel, vowel modifier, repha, ...), derived from the Unicode Indic_Syllabic_Category, Indic_Positional_Category and General_Category by cmd/genuse into the generated use_table.go; DeriveUSECategory is the shared derivation. The run is split into clusters by the USE syllable grammar (standard, virama-terminated, number-joiner, symbol and independent clusters), the default/basic GSUB features run (locl, ccmp, nukt, akhn, rphf, pref, rkrf, abvf, blwf, half, pstf, vatu, cjct), the run is reordered, the presentation GSUB features run (isol, init, medi, fina, abvs, blws, haln, pres, psts) and GPOS applies kern, dist, abvm, blwm, mark and mkmk.

Reordering is both property- and feature-based: pre-base vowels (VPre) and pre-base vowel modifiers (VMPre) move ahead of the base, a consonant the pref feature turns into a pre-base form moves just before it, and a leading repha — a static Consonant_Preceding_Repha (R) or a cluster head the rphf feature ligates into a repha — moves after it. The other per-script behaviours are reproduced too: a sakot (the Tai Tham U+1A60, and the general Sakot class) or a halant joins a consonant to the following one across an optional ZWJ/ZWNJ so they stay one cluster; two-part dependent vowels (Tibetan, Balinese and the Chakma pair) are decomposed into their components before shaping so each part is classified and positioned on its own; and a defective cluster of bare combining marks receives an inserted U+25CC dotted-circle base. The ten dedicated-shaper Indic scripts (Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada, Malayalam and Sinhala) are routed to the Indic shaper rather than USE.

Index

• func DeriveUSECategory(uisc, uipc, ugc string) string
• type Glyph
  • func Shape(face *opentype.Face, text string, opts Options) []Glyph
• type Options

Examples

• Options
• Shape
• Shape (Latin)

Functions

func DeriveUSECategory

func DeriveUSECategory(uisc, uipc, ugc string) string

DeriveUSECategory maps a rune's Unicode Indic_Syllabic_Category (uisc), Indic_Positional_Category (uipc) and General_Category (ugc) to its USE category, returned as the specification sigla (for example "B", "VAbv" or "VMPst"), or "O" for characters USE treats as Other. Positional families are refined from uipc. It implements the derivation table of the OpenType USE specification (minus the Arabic-joining and hieroglyph clauses, which are handled by dedicated shapers) and is exported so cmd/genuse can build the committed classification table without duplicating the mapping.

Types

type Glyph

type Glyph struct {
	GID      opentype.GlyphIndex
	Cluster  int
	XAdvance int
	YAdvance int
	XOffset  int
	YOffset  int
	// Scale is the factor to draw the glyph at relative to the face's size:
	// 1.0 for ordinary shaping, a fraction for the scaled-down signs of an
	// Egyptian Hieroglyph quadrat.
	Scale float64
}

Glyph is one positioned glyph of a shaped run, in visual (left-to-right) order. GID is the glyph to draw; Cluster is the logical (rune) index in the source text the glyph derives from; the advances move the pen after drawing and the offsets place the glyph relative to the pen — all in whole pixels at the face's size.

func Shape

func Shape(face *opentype.Face, text string, opts Options) []Glyph

Shape turns text into a positioned glyph run in visual order. It resolves the bidirectional embedding levels, maps each rune to a glyph, applies GSUB (positionally for Arabic cursive joining, whole-run for ligatures and contextual alternates), positions the result with GPOS (kerning and mark attachment), and emits the glyphs left-to-right with per-glyph advances and offsets in pixels. An empty text, or a face whose font lacks GSUB/GPOS, simply skips the corresponding stage.

Example

ExampleShape shapes an Arabic word, بيت ("house"): beh-yeh-teh. Noto Sans Arabic decomposes each letter into a rasm skeleton plus dot marks via GSUB ccmp, so the shaper emits two glyphs (rasm + dots) per letter — six glyphs for three letters. Bidi reordering also runs: the run is emitted left-to-right in drawing order even though the source text is right-to-left, so the glyphs for the last logical letter (Cluster 2, teh) are emitted first.

package main

import (
	"fmt"

	"github.com/go-opentype/fonts/notosansarabic"
	"github.com/go-opentype/opentype"
	"github.com/go-opentype/shape"
)

func main() {
	f, err := opentype.Parse(notosansarabic.TTF)
	if err != nil {
		panic(err)
	}
	face := f.NewFace(32)

	glyphs := shape.Shape(face, "بيت", shape.Options{})
	fmt.Println("glyph count:", len(glyphs))
	fmt.Println("first glyph's source rune (cluster):", glyphs[0].Cluster)
	fmt.Println("last glyph's source rune (cluster):", glyphs[len(glyphs)-1].Cluster)
}

Output:
glyph count: 6
first glyph's source rune (cluster): 2
last glyph's source rune (cluster): 0

Example (Latin)

ExampleShape_latin shapes plain Latin text with the default shaper: one glyph per rune, left-to-right, kerned and advanced by GPOS/hmtx.

package main

import (
	"fmt"

	"github.com/go-opentype/fonts/goregular"
	"github.com/go-opentype/opentype"
	"github.com/go-opentype/shape"
)

func main() {
	f, err := opentype.Parse(goregular.TTF)
	if err != nil {
		panic(err)
	}
	face := f.NewFace(32)

	glyphs := shape.Shape(face, "Wave", shape.Options{})
	total := 0
	for _, g := range glyphs {
		total += g.XAdvance
	}
	fmt.Println("glyph count:", len(glyphs))
	fmt.Println("total advance:", total)
}

Output:
glyph count: 4
total advance: 82

type Options

type Options struct {
	// Direction is the base paragraph direction (bidi.LeftToRight,
	// bidi.RightToLeft or bidi.Auto). The zero value is bidi.LeftToRight.
	Direction bidi.Direction
	// Script forces the shaping script: "arab" for Arabic, "latn"/"dflt" (or
	// any other value) for the default shaper. Empty auto-detects: any
	// Arabic-block rune selects "arab", otherwise "dflt".
	Script string
	// Features lists extra OpenType feature tags to activate, applied over the
	// whole run in both the substitution and positioning stages (a tag with no
	// matching lookups is a no-op).
	Features []string
	// Vertical selects vertical writing mode (CJK tategaki): glyphs are laid
	// out top to bottom, the vert/vrt2 features select upright vertical forms,
	// and each glyph carries its vertical advance in YAdvance. Egyptian
	// Hieroglyph runs always use quadrat layout regardless of this flag.
	Vertical bool
}

Options configures a Shape call. The zero value shapes with an automatic base direction (from the first strong character), a script auto-detected from the text, and no extra features.

Example

ExampleOptions forces the script and base direction instead of relying on auto-detection, useful when the caller already knows the text's script (for example, from higher-level document metadata).

package main

import (
	"fmt"

	"github.com/go-opentype/bidi"
	"github.com/go-opentype/fonts/notosansarabic"
	"github.com/go-opentype/opentype"
	"github.com/go-opentype/shape"
)

func main() {
	f, err := opentype.Parse(notosansarabic.TTF)
	if err != nil {
		panic(err)
	}
	face := f.NewFace(32)

	opts := shape.Options{
		Script:    "arab",
		Direction: bidi.RightToLeft,
	}
	glyphs := shape.Shape(face, "بيت", opts)
	fmt.Println("glyph count:", len(glyphs))
}

Output:
glyph count: 6