engine

README

engine — go-tex

A pure-Go (no cgo) TeX engine, under construction — aimed at functional parity with a TeX distribution, not a subset. This module is the engine's mouth and gullet: a faithful re-implementation of TeX's category-code tokenizer, the equivalents table (eqtb) with grouping/scoping, macro definition with delimited parameters, and the full expansion machinery.

It is developed the way parity is actually reachable — reimplement the engine faithfully, gated by TeX's own conformance oracle — and later run the real LaTeX kernel and packages on it (they are TeX macros), diffing output against pdftex/xetex. The primary gate here is the conformance ratchet (TestConformance): TeX snippets checked byte-for-byte against real-TeX output.

Working today (verified, faithful)

• Category-code tokenizer — full catcode table, comments, control words/symbols.
• Macros — \def (undelimited, delimited, and grouped parameters, with backtracking on partial delimiter matches), \edef/\gdef/\xdef, \let (to macros, primitives, undefined, and character tokens), \global.
• Expansion — \expandafter, \csname/\endcsname, \noexpand, \string, \the, \number, \romannumeral, \meaning, \uppercase/\lowercase.
• Conditionals — \if, \ifnum, \ifx, \ifcat, \ifodd, \ifcase, \iftrue/\iffalse, with \else/\or/\fi and nesting.
• Registers & arithmetic — \count, \advance, \multiply, \chardef, \catcode, read via \the/\count.
• Grouping — {…}, \begingroup/\endgroup, save/restore of meanings, registers, and catcodes; \global escapes the current group.

Faithfulness is checked on subtleties only real TeX gets right (a control word absorbing its following space; significant spaces in conditional branches).

out, _ := engine.New().Run(`\def\twice#1{#1#1}\message{\twice{\twice A}}`)
// out == "AAAA"

Status & roadmap to parity

This is stage 1 (mouth + gullet). The remaining stages, each gated by an objective oracle:

1. ✅ Mouth + gullet — tokenizer, eqtb, macros, expansion (this module).
2. ⬜ Stomach — box/glue/penalty model, h/v lists, Knuth–Plass line breaking, page builder, \halign — gated by the TRIP test.
3. ⬜ Math (Appendix G) — go-tex/math is the starting point.
4. ⬜ Fonts — TFM + OpenType (via go-opentype).
5. ⬜ Output — DVI → PDF (via go-pdfkit), then run the real latex.ltx + packages, gated by PDF-diff vs pdftex/xetex.

Because it is under construction, line coverage (currently ~84%) rises as subsystems land; the meaningful gate is the growing conformance ratchet, not a fixed coverage figure. Pure Go, CGO=0, go vet clean, green across six 64-bit arches plus js/wasm and wasip1/wasm.

License

BSD-3-Clause — see LICENSE. Copyright the go-tex/engine authors.

Documentation

Overview

Package engine is the core of a pure-Go (CGO=0) TeX engine: a faithful re-implementation of TeX's mouth and gullet — category-code tokenization, the equivalents table (eqtb) with grouping/scoping, macro definition with delimited parameters, and the expansion machinery (\def, \edef, \let, \expandafter, \csname, \noexpand, \string, \the, \number, conditionals, integer registers). It is the foundation on which the real LaTeX kernel and packages will run, gated by TeX's own conformance suite (the TRIP test) — the path to functional parity with a TeX distribution, not a subset.

Index

• Constants
• type Engine
  • func New() *Engine
  • func (e *Engine) Run(src string) (string, error)
• type Item
  • func Box(w float64) Item
  • func Glue(w, stretch, shrink float64) Item
  • func Penalty(w, p float64, flagged bool) Item
• type ItemKind
• type Line
  • func KnuthPlass(items []Item, lineWidth, tolerance, linePenalty float64) ([]Line, bool)

Constants

const InfPenalty = 10000.0

InfPenalty is TeX's "infinite" penalty (∞ = forbidden break, −∞ = forced).

Types

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine holds all TeX state: the input stack, the eqtb (control-sequence meanings), integer registers, category codes, the grouping save stack, and the \message output buffer.

func New

func New() *Engine

New builds an engine with TeX's default category codes and primitives loaded.

func (*Engine) Run

func (e *Engine) Run(src string) (string, error)

Run tokenizes src as the base input and processes it to completion, returning the accumulated \message output.

type Item

type Item struct {
	Kind            ItemKind
	Width           float64
	Stretch, Shrink float64 // glue only
	Penalty         float64 // penalty only
	Flagged         bool    // penalty only (e.g. a hyphen) — consecutive flags are penalised
}

Item is one element of a horizontal list.

func Box

func Box(w float64) Item

Box, Glue and Penalty are constructors.

func Glue

func Glue(w, stretch, shrink float64) Item

func Penalty

func Penalty(w, p float64, flagged bool) Item

type ItemKind

type ItemKind uint8

ItemKind classifies a horizontal-list item.

const (
	KBox     ItemKind = iota // a box of fixed width
	KGlue                    // stretchable/shrinkable space (a legal breakpoint after a box)
	KPenalty                 // a penalty (a legal breakpoint; ±InfPenalty = forbidden/forced)
)

type Line

type Line struct {
	Start, End int     // item index range [Start, End) actually set on the line
	Ratio      float64 // glue adjustment ratio r (−1 fully shrunk … +tolerance stretched)
}

Line describes one output line of a broken paragraph.

func KnuthPlass

func KnuthPlass(items []Item, lineWidth, tolerance, linePenalty float64) ([]Line, bool)

KnuthPlass breaks items into lines of the given width, minimising total demerits (linePenalty is TeX's \linepenalty). It returns the chosen lines in order and ok=false if no sequence of feasible breaks exists within tolerance.