hyphenation

README

hyphenation

Where may a hyphen fall in a word — pure Go, no cgo, no dependencies.

It answers that one question, for any language you have patterns for.

How it works

The method is Liang's (1983). A pattern carries priority digits between letters: hy3ph says a break after hy scores 3. Every pattern matching any substring of the word contributes (a . in a pattern anchors it to a word boundary), the highest value at each position wins, and an odd value means a break is allowed there.

Two limits then forbid breaks too close to either end — by default 2 letters before the first hyphen and 3 after the last, which is why a five-letter word is never hyphenated. Both are settable.

Patterns

The library reads the pattern format used by the freely available pattern files covering some seventy languages, distributed as hyph-*.tex (for example hyph-en-gb, hyph-fr, hyph-de-1996). Feed it the pattern lines; nothing else about those files is needed.

Use

import "github.com/go-typeset/hyphenation"

h := hyphenation.New()
for _, p := range patterns {   // "hy3ph", "he2n", "hena4", …
    h.AddPattern(p)
}

h.Points("hyphenation")   // → [2 6]: hy-phen-ation
h.SetMins(2, 2)           // fewer letters required after the last hyphen

Each value t means a hyphen may follow the first t letters.

Pair it with linebreak to turn those positions into candidate breakpoints.

Tests

go test ./... — 100% statement coverage, run on six 64-bit architectures (amd64, arm64, riscv64, loong64, ppc64le, s390x), three operating systems, and both wasm targets.

Licence

BSD-3-Clause.

Documentation

Index

• type Hyphenator
  • func New() *Hyphenator
  • func (h *Hyphenator) AddPattern(p string)
  • func (h *Hyphenator) Mins() (left, right int)
  • func (h *Hyphenator) Points(word string) []int
  • func (h *Hyphenator) SetMins(left, right int)

Types

type Hyphenator

type Hyphenator struct {
	// contains filtered or unexported fields
}

Hyphenator holds the loaded patterns and the min-affix limits.

func New

func New() *Hyphenator

func (*Hyphenator) AddPattern

func (h *Hyphenator) AddPattern(p string)

addPattern parses one Liang pattern (e.g. "a1bc3d" or ".ach4") into its letter key and inter-letter value array (length = letters+1).

func (*Hyphenator) Mins

func (h *Hyphenator) Mins() (left, right int)

Mins reports the current minimum number of letters before the first hyphen and after the last.

func (*Hyphenator) Points

func (h *Hyphenator) Points(word string) []int

points returns the break positions in word: each value t means a hyphen is allowed after the first t letters (so between word[t-1] and word[t]).

func (*Hyphenator) SetMins

func (h *Hyphenator) SetMins(left, right int)

SetMins sets how many letters must remain before the first hyphen and after the last. The defaults are 2 and 3 — the long-standing English convention, and the reason a five-letter word is never hyphenated; other languages, and other house styles, want otherwise.