engine

README

engine — go-tex

A pure-Go (no cgo) TeX engine aimed at functional parity with a TeX distribution, not a subset. It is a faithful re-implementation of TeX — the category-code tokenizer, the equivalents table (eqtb) with grouping/scoping, macro definition with delimited parameters and the full expansion machinery (the mouth and gullet); a scaled-point box/glue/penalty stomach with Knuth–Plass line breaking and a cost-based page builder; math via go-tex/math; OpenType fonts; and PDF + SVG output — and on top of it, it loads and runs the genuine LaTeX classes: \documentclass{article}, {report} and {book} execute the real, embedded .cls files, in native builds and in the browser (js/wasm), with no TeXLive.

It is developed the way parity is actually reachable — reimplement the engine faithfully, gated by objective oracles — then run the real LaTeX classes and packages on it (they are TeX macros). Two gates hold the line: the conformance ratchet (TestConformance, TeX snippets checked byte-for-byte against real-TeX output) and a fidelity check that compares whole-document prose against a real LaTeX engine (tectonic).

Working today (verified, faithful)

• Category-code tokenizer — full catcode table, comments, control words/symbols.
• Macros — \def (undelimited, delimited, and grouped parameters, with backtracking on partial delimiter matches), \edef/\gdef/\xdef, \let (to macros, primitives, undefined, and character tokens), \global.
• Expansion — \expandafter, \csname/\endcsname, \noexpand, \string, \the, \number, \romannumeral, \meaning, \uppercase/\lowercase.
• Conditionals — \if, \ifnum, \ifx, \ifcat, \ifodd, \ifcase, \iftrue/\iffalse, with \else/\or/\fi and nesting.
• Registers & arithmetic — \count, \advance, \multiply, \chardef, \catcode, read via \the/\count.
• Grouping — {…}, \begingroup/\endgroup, save/restore of meanings, registers, and catcodes; \global escapes the current group.

Faithfulness is checked on subtleties only real TeX gets right (a control word absorbing its following space; significant spaces in conditional branches).

out, _ := engine.New().Run(`\def\twice#1{#1#1}\message{\twice{\twice A}}`)
// out == "AAAA"

Real-world documents (lenient mode)

A real third-party paper pulls in classes, packages, fonts, figures and .bib files that a from-scratch engine does not carry. Strict mode aborts on the first such gap (as TeX does). Lenient mode (gotex -lenient, or Options{Lenient: true}) turns those gaps into best-effort no-ops so an editor preview shows the typesettable content instead of one hard error:

• an undefined command is skipped, along with its likely [opt]{arg} block;
• an unloadable figure becomes a framed placeholder of the requested size;
• a math macro go-tex/math doesn't know drops that one equation;
• a \setlength on an unmodelled length, and a missing \input/\bibliography/ \font file, are ignored.

Every skipped construct is tallied ((*Engine).SkippedCommands) so a caller can report what was dropped. On a sample of 54 real arXiv sources, strict mode compiled 0 end-to-end (each hit a package command in the preamble); lenient mode produces a multi-page PDF for all 54, with real, selectable prose text. It is a preview aid, not a fidelity claim — the roadmap below is how the gaps close for real.

Loading real classes and packages

\documentclass and \usepackage (and \RequirePackage, \LoadClass, \LoadClassWithOptions) do more than emulate: they resolve and load the real .cls/.sty — from the document's own directory (an arXiv paper's bundled class/package), a TEXINPUTS/GOTEX_TEXMF search path, or an embedded base set — making @ a letter and running the file's own \newcommand/\def/… on the engine. The LaTeX2e option mechanism runs too: \DeclareOption, \DeclareOption*, \ProcessOptions, \ExecuteOptions, \CurrentOption, \PassOptionsToPackage/\PassOptionsToClass, plus \IfFileExists/ \InputIfFileExists. A file loads tolerantly — a command the engine lacks is skipped, so a real class contributes what it can — and a runaway-expansion guard bounds macro expansion so a pathological or partially-supported file can never hang (it stops with partial output in lenient mode, an error in strict). Distribution-heavy packages the engine emulates natively or better as stubs (geometry, tikz, hyperref, graphicx, encodings, …) are not loaded from disk.

The standard base classes run for real. \documentclass{article}, {report} and {book} load and execute the genuine, embedded LaTeX classes (article.cls/report.cls/book.cls + their size option files, LPPL, verbatim) — not an emulation. Everything they need is in place: the LaTeX2e kernel helpers, a class-kernel substrate (constants, registers, \if@ flags, NFSS font-switch aliases), \newcommand*/\DeclareOldFontCommand, the rubber-glue and <factor><internal-dimen> length scanner, numbered \@startsection with \@tocentry, \secdef via \@dblarg (so \chapter works), \@float figure/table captions, \@starttoc bridged to the engine's two-pass contents table, and — the keystone — stable source lines (loading a 644-line class no longer shifts the line numbers the editor maps glyphs back to). A real \documentclass{article} document typesets a numbered title, a dotted \tableofcontents, numbered sections, and numbered figure/table captions, and it reproduces the reference engine's prose on the fidelity gate. Because the class files are go:embeded and the resolver needs no filesystem, the real classes also run in the js/wasm build — genuine LaTeX class rendering in the browser, with no TeXLive and no server. amsart is embedded and its class loads (its \maketitle even drove real token-register support — \toks/\newtoks — into the engine), but \documentclass{amsart} is kept on the emulation for now: its own \newtheorem…[section] machinery loops on the engine, so it waits on that fix before it is routed to the real class.

Status & roadmap to parity

Each stage is gated by an objective oracle:

1. ✅ Mouth + gullet — tokenizer, eqtb, macros, expansion.
2. ✅ Stomach — box/glue/penalty model in scaled points, h/v lists, Knuth–Plass line breaking with an emergency pass, cost-based page builder, \halign.
3. ✅ Math — $…$ and the display environments delegated to go-tex/math (vector output).
4. ✅ Fonts — OpenType via go-opentype; a built-in font so it runs with no assets, with kerning and ligatures.
5. ✅ Output — PDF (via go-pdfkit, embedded subset fonts, selectable text) and self-contained SVG pages; the SVG carries a source map for click-to-line.
6. ✅ Real classes — \documentclass{article|report|book} loads and runs the genuine embedded LaTeX class (see above), reproducing the reference engine's prose on the fidelity gate — in native builds and in js/wasm.

Next: amsart's \newtheorem fix (then route it to the real class), more real packages (amsmath, hyperref, graphicx), a broader real-document conformance corpus (PDF-diff vs pdftex/xetex), and the TRIP test. Coverage ~91%; the meaningful gate is the conformance ratchet plus the fidelity check against a real LaTeX engine, not a fixed coverage figure. Pure Go, CGO=0, go vet clean, green across three 64-bit arches under qemu plus js/wasm and wasip1/wasm.

License

BSD-3-Clause — see LICENSE. Copyright the go-tex/engine authors.

Documentation

Overview

Package engine is the core of a pure-Go (CGO=0) TeX engine: a faithful re-implementation of TeX's mouth and gullet — category-code tokenization, the equivalents table (eqtb) with grouping/scoping, macro definition with delimited parameters, and the expansion machinery (\def, \edef, \let, \expandafter, \csname, \noexpand, \string, \the, \number, conditionals, integer registers). It is the foundation on which the real LaTeX kernel and packages will run, gated by TeX's own conformance suite (the TRIP test) — the path to functional parity with a TeX distribution, not a subset.

Index

• Constants
• Variables
• func CompileToPDF(src []byte, opt Options, w io.Writer) (int, error)
• func CompileToPDFReport(src []byte, opt Options, w io.Writer) (int, map[string]int, error)
• func CompileToSVGPages(src []byte, opt Options) ([]string, error)
• func CompileToSVGPagesReport(src []byte, opt Options) ([]string, map[string]int, error)
• func LineAt(spans []SourceSpan, x, y float64) int
• type Diagnostics
  • func CompileToPDFDiag(src []byte, opt Options, w io.Writer) (int, Diagnostics, error)
  • func CompileToSVGPagesDiag(src []byte, opt Options) ([]string, Diagnostics, error)
• type Engine
  • func New() *Engine
  • func NewDocument(opt Options) (*Engine, error)
  • func (e *Engine) Diagnostics() Diagnostics
  • func (e *Engine) LoadFormat(src string) error
  • func (e *Engine) LoadLaTeX() error
  • func (e *Engine) LoadPlain() error
  • func (e *Engine) Page() *boxNode
  • func (e *Engine) Pages() []*boxNode
  • func (e *Engine) Position() (line, col int)
  • func (e *Engine) RenderBox(i int, margin float64) string
  • func (e *Engine) RenderPDF(w io.Writer, margin float64) error
  • func (e *Engine) RenderPage(margin float64) string
  • func (e *Engine) RenderPages(margin float64) []string
  • func (e *Engine) Run(src string) (string, error)
  • func (e *Engine) SetFont(f fontFace)
  • func (e *Engine) SkippedCommands() map[string]int
  • func (e *Engine) SourceSpans(margin float64) [][]SourceSpan
• type Item
• type Line
• type OpenTypeFont
  • func NewOpenTypeFont(fontBytes []byte, sizePx int) (*OpenTypeFont, error)
  • func (o *OpenTypeFont) CharDims(r rune) (float64, float64, float64)
  • func (o *OpenTypeFont) Space() (float64, float64, float64)
• type Options
• type SourceError
  • func (s SourceError) Error() string
• type SourceSpan
  • func RectsForLine(spans []SourceSpan, line int) []SourceSpan

Constants

const AMSClassSubstrate = `` /* 7834-byte string literal not displayed */

AMSClassSubstrate is the register-and-macro layer amsart builds on beyond the article substrate: plain-TeX scratch registers, the penalty/spacing parameters a class assigns, and best-effort no-ops for NFSS font selection and a few text commands. Loaded last by LoadLaTeX.

const BeamerClassKernel = `` /* 5548-byte string literal not displayed */

BeamerClassKernel is the built-in emulation of the beamer presentation class. The engine EMULATES beamer's document structure rather than running the real class: every frame becomes a page, frame titles become headings, blocks render as titled paragraphs, and overlay specifications (\pause, <1->, \only, \uncover, …) are shown STATICALLY — all the material of a frame at once. Themes and pgf styling are gobbled. The goal is that a beamer talk RENDERS its content as a sequence of pages, not pixel-fidelity to a themed slide. Loaded by doDocumentClass when \documentclass{beamer} is seen.

The real beamer.cls is being brought up alongside this; it now LOADS in full, and GOTEX_BEAMER switches to it — see realBeamer in packages.go for exactly how far that has got and why the emulation is still the default.

const (
	InfPenalty = linebreak.InfPenalty
)

const LaTeX2eClassKernel = `` /* 21841-byte string literal not displayed */

LaTeX2eClassKernel is the LaTeX2e class-kernel substrate, loaded by LoadLaTeX right after LaTeX2eKernelHelpers.

const LaTeX2eClassLead = `` /* 7557-byte string literal not displayed */

LaTeX2eClassLead is the best-effort high-level class machinery a real class builds on top of the kernel: sectioning (\@startsection and its helpers), the generic \list, and small no-ops. It is intentionally simplified — headings are typeset in their requested style without the exact spacing/numbering of real LaTeX — so that loading a real class produces readable structured output rather than aborting. It is loaded after LaTeX2eClassKernel.

const LaTeX2eKernelHelpers = `
\catcode64=11
% ── expansion primitives / gobblers / selectors ─────────────────────────────
\def\@empty{}
\def\@iden#1{#1}
\def\@firstofone#1{#1}
\def\@gobble#1{}
\def\@gobbletwo#1#2{}
\def\@gobblethree#1#2#3{}
\def\@gobblefour#1#2#3#4{}
\def\@firstoftwo#1#2{#1}
\def\@secondoftwo#1#2{#2}
% LaTeX kernel while-loops. \@whilenum <test> \do {<body>} repeats <body> while
% the \ifnum test holds; \@whiledim is the \ifdim analogue; \@whilesw <switch>\fi
% {<body>} loops on a boolean switch. Classes drive frontmatter box splitting and
% list machinery with these (e.g. imsart's \close@fm \@whiledim…\vsplit loop);
% defined exactly as latex.ltx so the delimited \do argument parses correctly.
\long\def\@whilenum#1\do #2{\ifnum #1\relax #2\relax\@iwhilenum{#1\relax #2\relax}\fi}
\long\def\@iwhilenum#1{\ifnum #1\expandafter\@iwhilenum\else\expandafter\@gobble\fi{#1}}
\long\def\@whiledim#1\do #2{\ifdim #1\relax#2\@iwhiledim{#1\relax#2}\fi}
\long\def\@iwhiledim#1{\ifdim #1\expandafter\@iwhiledim\else\expandafter\@gobble\fi{#1}}
\long\def\@whilesw#1\fi#2{#1#2\@iwhilesw{#1#2}\fi}
\long\def\@iwhilesw#1\fi{#1\expandafter\@iwhilesw\else\@gobbletwo\fi{#1}\fi}
% \@nil / \@nnil : pure delimiter tokens for list-scanning macros. Their meaning
% is never executed; \@nnil's body is the single token \@nil so that a loop macro
% \def'd to \@nil compares \ifx-equal to \@nnil (meaning-equality of bodies).
\def\@nil{}
\def\@nnil{\@nil}
% ── \csname-based (re)naming ────────────────────────────────────────────────
\def\@namedef#1{\expandafter\def\csname #1\endcsname}
\def\@nameuse#1{\csname #1\endcsname}
% ── numeric / length constants used pervasively by the kernel ───────────────
% Plain-TeX register constants. \z@ is BOTH the number 0 and the dimen 0pt (as in
% plain.tex it is a \dimen); \@ne/\tw@/\thr@@ are \chardef'd small integers.
\newcount\m@ne \m@ne=-1
\chardef\@ne=1
\chardef\tw@=2
\chardef\thr@@=3
\newdimen\z@ \z@=0pt
\newskip\z@skip \z@skip=0pt plus0pt minus0pt
\newskip\@tempskipa
\newskip\@tempskipb
\newdimen\@tempdima
\newdimen\@tempdimb
\newcount\@tempcnta
\newcount\@tempcntb
% ── \newif (the engine has no \newif primitive) ─────────────────────────────
% \@stripif\iffoo expands to the letters "foo": \string\iffoo gives the six
% catcode-12 chars \ i f f o o (the engine has no \escapechar, so \string always
% keeps the leading backslash), and \@gobblethree drops the "\if" prefix. \newif
% then \let's the switch false and builds \footrue / \foofalse via \csname.
\def\@stripif#1{\expandafter\@gobblethree\string#1}
\def\newif#1{%
  \let#1\iffalse
  \expandafter\def\csname\@stripif#1true\endcsname{\let#1\iftrue}%
  \expandafter\def\csname\@stripif#1false\endcsname{\let#1\iffalse}}
\newif\if@tempswa
\newif\ifin@
% xcolor's own switches, which packages built on it read: xxcolor — beamer's
% colour layer — toggles \ifglobalcolors around every colour it sets.
\newif\ifglobalcolors
\newif\ifXC@keepwhite
% ── definability / undefined tests ──────────────────────────────────────────
% \@ifundefined{name}{then}{else}: \csname name\endcsname is \relax when the name
% is undefined (doCsname), so \ifx…\relax selects the branch. NOTE the standard
% LaTeX side effect: after the test an undefined name becomes \relax (no longer
% "undefined") — matching real latex.ltx.
% (One line: any space/newline in the body would leak into the branch that runs
% inside a \message, so it is written without stray space tokens.)
\def\@ifundefined#1{\expandafter\ifx\csname #1\endcsname\relax\expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi}
% \@ifdefinable\name{def}: run {def} if \name is currently undefinable-safe (i.e.
% \relax/undefined); otherwise a no-op (LIMITATION: real LaTeX raises an error and
% also rejects a handful of reserved names — the engine's \newcommand path is a Go
% primitive and does its own checking, so this is only a best-effort fallback).
\def\@ifdefinable#1#2{%
  \expandafter\ifx\csname\expandafter\@gobble\string#1\endcsname\relax
    #2%
  \fi}
% ── list / string dissection ────────────────────────────────────────────────
\def\@car#1#2\@nil{#1}
\def\@cdr#1#2\@nil{#2}
% \zap@space<text> \@empty strips ALL spaces from <text>; the trailing \@empty is
% the sentinel (\zap@space a b c\@empty -> abc).
\def\zap@space#1 #2{#1\ifx#2\@empty\else\expandafter\zap@space\fi#2}
% \@backslashchar / \@percentchar : a single catcode-12 "\" / "%" usable inside
% \edef (\string keeps the backslash, \@gobble removes it).
\edef\@backslashchar{\expandafter\@gobble\string\\}
\edef\@percentchar{\expandafter\@gobble\string\%}
% ── two-argument expansion + substring membership (\in@) ─────────────────────
\def\@expandtwoargs#1#2#3{\edef\reserved@a{\noexpand#1{#2}{#3}}\reserved@a}
\def\reserved@a{}
% \@ifinlist{item}{comma-list} sets \ifin@ true iff <item> (after \edef expansion)
% equals one of the comma-separated entries of <comma-list> (also \edef-expanded
% first, so a macro like \opt@foo.sty is spread into its entries). A self-contained
% scanner terminated by the \@inliststop sentinel — it always halts.
%
% LIMITATION: the classic substring \in@ from latex.ltx is NOT provided. Its
% definition builds a macro whose parameter text is self-delimited by its own name
% (\def\in@@#1<sub>#2#3\in@@{…}); this engine's delimited-parameter matcher loops
% on that construction. \@ifinlist covers the comma-option-list case that
% \@ifpackagewith needs; packages that call \in@ directly are the option-processing
% layer's concern (handled in Go by the lead).
\def\@inliststop{\@inliststop}
\def\@ifinlist#1#2{\in@false\edef\@inlistwant{#1}\edef\@inlisttmp{#2}\expandafter\@inlist@scan\@inlisttmp,\@inliststop,}
\def\@inlist@scan#1,{\def\@inlistcur{#1}\ifx\@inlistcur\@inliststop\else\ifx\@inlistcur\@inlistwant\in@true\fi\expandafter\@inlist@scan\fi}
% ── appending tokens to a macro ─────────────────────────────────────────────
% \g@addto@macro (global) / \@addto@macro (local): append #2 to the body of the
% macro #1. Uses an \expandafter chain instead of \toks@ (the engine has no toks
% registers): \expandafter…\def\expandafter#1\expandafter{#1#2} redefines #1 to be
% its old body followed by #2.
\def\@addto@macro#1#2{\expandafter\def\expandafter#1\expandafter{#1#2}}
\def\g@addto@macro#1#2{\expandafter\gdef\expandafter#1\expandafter{#1#2}}
% ── list iteration (\@for over a comma list, \@tfor over tokens) ─────────────
% Authentic latex.ltx definitions (they need only \ifx, \def, \expandafter, the
% \@nil/\@nnil/\@empty sentinels and delimited parameters — all available).
\def\@fornoop#1\@@#2#3{}
\def\@for#1:=#2\do#3{%
  \expandafter\def\expandafter\@fortmp\expandafter{#2}%
  \ifx\@fortmp\@empty \else
    \expandafter\@forloop#2,\@nil,\@nil\@@#1{#3}\fi}
\def\@forloop#1,#2,#3\@@#4#5{\def#4{#1}\ifx #4\@nnil \else
       #5\def#4{#2}\ifx #4\@nnil \else#5\@iforloop #3\@@#4{#5}\fi\fi}
\def\@iforloop#1,#2\@@#3#4{\def#3{#1}\ifx #3\@nnil
       \expandafter\@fornoop \else
    #4\relax\expandafter\@iforloop\fi#2\@@#3{#4}}
\def\@tfor#1:=#2\do#3{%
  \def\@fortmp{#2}\ifx\@fortmp\@empty \else
    \expandafter\@tforloop#2\@nil\@nil\@@#1{#3}\fi}
\def\@tforloop#1#2\@@#3#4{\def#3{#1}\ifx #3\@nnil
       \expandafter\@fornoop \else
    #4\relax\expandafter\@tforloop\fi#2\@@#3{#4}}
\def\@break@tfor#1\@@#2#3{\fi\fi}
% ── star/long dispatch ──────────────────────────────────────────────────────
% \@star@or@long\cmd peeks for a '*' (via \@ifstar) and runs \cmd either way, and
% sets \l@ngrel@x to the prefix the definition should carry — \relax after a star,
% \long without one, exactly as the LaTeX kernel does. \long is an accepted no-op
% prefix here (the engine does not model the \par-in-argument restriction), but the
% MEANING of \l@ngrel@x is read: etoolbox's \newrobustcmd branches on
% \ifx\l@ngrel@x\relax to decide between \protected and \protected\long, so a
% \l@ngrel@x that is neither sends every starred command down the long branch.
\let\l@ngrel@x\relax
\def\@star@or@long#1{\@ifstar{\let\l@ngrel@x\relax#1}{\let\l@ngrel@x\long#1}}
% \@ifnextchar is a native primitive (real \ifx-based look-ahead over any target
% token, including control sequences) — see the engine's primitive table.
% \@testopt{<cmd>}{<default>} runs <cmd> on a following [optional argument], or on
% [<default>] when there is none — the kernel's optional-argument dispatcher, which
% every \newcommand-with-a-default and etoolbox's \newrobustcmd are built on. The
% default is BRACED so a multi-token default arrives as one argument.
\def\@testopt#1#2{\@ifnextchar[{#1}{#1[{#2}]}}
% \@protected@testopt\cmd is the same dispatcher guarded by \protect: in the real
% kernel the guard chooses between typesetting and writing the command to a file.
% This engine has one \protect meaning (typesetting — nothing is written to .aux
% for a later run), so the guard is always true and the argument, which only the
% write branch uses, is dropped.
\def\@typeset@protect{}
\def\@protected@testopt#1{\@testopt}
% ── \@argdef / \@yargdef: build an n-argument definition ─────────────────────
% These are the kernel's own definition builders, and packages call them directly
% — etoolbox's \newrobustcmd routes every command it defines through \@argdef (no
% optional argument) or \@yargdef (with one). They were missing, so etoolbox
% defined NOTHING: \mode, on which every line of beamer stands, simply did not
% exist, and the class's own \mode<all> printed "<all>" onto the page.
%
% Written exactly as the LaTeX kernel writes them (read back from a real TeX with
% \meaning). \@yargd@f assembles the parameter text #1#2…#n by matching against a
% ready-made run of nine parameters, and stops at the requested count using a
% parameter text that ends in "#{" — the argument runs up to the opening brace,
% which stays behind to open the body.
\long\def\@argdef#1[#2]#3{\@ifdefinable#1{\@yargdef#1\@ne{#2}{#3}}}
\long\def\@reargdef#1[#2]{\@yargdef#1\@ne{#2}}
\long\def\@yargdef#1#2#3{%
  \ifx#2\tw@
    \def\reserved@b##11{[####1]}%
  \else
    \let\reserved@b\@gobble
  \fi
  \expandafter\@yargd@f\expandafter{\number#3}#1}
\long\def\@yargd@f#1#2{%
  \def\reserved@a##1#1##2##{\expandafter\def\expandafter#2\reserved@b##1#1}%
  \l@ngrel@x\reserved@a 0##1##2##3##4##5##6##7##8##9###1}
% \kernel@ifnextchar is the kernel's own name for \@ifnextchar (the kernel keeps a
% private copy so a package that redefines \@ifnextchar cannot break the kernel).
% beamer's overlay decoder calls it by that name.
% \maxdimen is plain TeX's largest dimension. Package code uses it as "no limit"
% (\vbox to\maxdimen, \dimen0=\maxdimen), so its absence is not a rounding matter:
% the assignment simply does not happen.
\newdimen\maxdimen \maxdimen=16383.99998pt
% \@cons\list{\item} appends an \@elt-separated item to a LaTeX list macro, with
% \@elt made harmless while the list is rebuilt.
\def\@cons#1#2{\begingroup\let\@elt\relax\xdef#1{#1\@elt #2}\endgroup}
% \@onelevel@sanitize\cs rewrites \cs's content as ordinary characters, which is
% how the kernel makes a name safe to compare or write out. \strip@prefix drops
% the "macro:->" that \meaning prints in front of the body.
\def\strip@prefix#1>{}
\def\@onelevel@sanitize#1{\edef#1{\expandafter\strip@prefix\meaning#1}}
\let\kernel@ifnextchar\@ifnextchar
% \in@{<a>}{<b>} sets \ifin@ true iff the token list <a> occurs inside <b>. The
% kernel builds it as a delimited-match macro; keyval and beamer's option handling
% both ask it.
\def\in@#1#2{%
  \def\in@@##1#1##2##3\in@@{%
    \ifx\in@##2\in@false\else\in@true\fi}%
  \in@@#2#1\in@\in@@}
% \@makeother makes a character ordinary — how a package reads text verbatim — and
% \dospecials is plain TeX's list of the characters that need it.
\def\@makeother#1{\catcode` + "`" + `#1=12\relax}
\def\dospecials{\do\ \do\\\do\{\do\}\do\$\do\&\do\#\do\^\do\_\do\%\do\~}
% \@onlypreamble{\cmd} records a command as preamble-only. The list is kept, so
% code that walks \@preamblecmds finds what it expects; the engine does not
% disable the commands at \begin{document}, which only costs a worse error message
% for a document that misuses one.
\def\@preamblecmds{}
\def\@onlypreamble#1{%
  \expandafter\gdef\expandafter\@preamblecmds\expandafter{\@preamblecmds\do#1}}
\long\def\@xargdef#1[#2][#3]#4{%
  \@ifdefinable#1{%
    \expandafter\def\expandafter#1\expandafter{%
      \expandafter\@protected@testopt\expandafter#1%
      \csname\string#1\endcsname{#3}}%
    \expandafter\@yargdef\csname\string#1\endcsname\tw@{#2}{#4}}}
% ── logging / diagnostics: never abort a .sty, route everything to \message ──
% \MessageBreak/\protect/\on@line/\@spaces are the tokens warnings interpolate;
% keep them harmless inside the \message expansion. \@ehc/\@ehd are the "error
% help" tokens callers pass as the last argument of \PackageError/\ClassError.
\def\MessageBreak{ }
\def\protect{}
\def\on@line{}
\def\@spaces{}
\def\@ehc{}
\def\@ehd{}
\def\wlog#1{\message{#1}}
\def\typeout#1{\message{#1}}
\def\PackageWarning#1#2{\message{Package #1 Warning: #2}}
\def\PackageWarningNoLine#1#2{\message{Package #1 Warning: #2}}
\def\PackageInfo#1#2{\message{Package #1 Info: #2}}
\def\PackageError#1#2#3{\message{Package #1 Error: #2}}
\def\ClassWarning#1#2{\message{Class #1 Warning: #2}}
\def\ClassWarningNoLine#1#2{\message{Class #1 Warning: #2}}
\def\ClassInfo#1#2{\message{Class #1 Info: #2}}
\def\ClassError#1#2#3{\message{Class #1 Error: #2}}
\def\@latex@warning#1{\message{LaTeX Warning: #1}}
\def\@latex@warning@no@line#1{\message{LaTeX Warning: #1}}
\def\@latex@info#1{\message{LaTeX Info: #1}}
\def\@latex@info@no@line#1{\message{LaTeX Info: #1}}
\def\@latex@error#1#2{\message{LaTeX Error: #1}}
\def\@warning#1{\message{Warning: #1}}
% ── begin/end-document and package/class hooks ──────────────────────────────
% \AtBeginDocument / \AtEndDocument accumulate tokens fired by \document /
% \enddocument (redefined here to run the hook; the originals in MiniLaTeXKernel
% only toggled the @ catcode / added \vfill). \AtEndOfPackage / \AtEndOfClass
% accumulate into hooks the Go loader runs+resets after each load (see CONTRACT).
\def\@begindocumenthook{}
\def\@enddocumenthook{}
\def\@endofpackagehook{}
\def\@endofclasshook{}
\def\AtBeginDocument#1{\g@addto@macro\@begindocumenthook{#1}}
\def\AtEndDocument#1{\g@addto@macro\@enddocumenthook{#1}}
\def\AtEndOfPackage#1{\g@addto@macro\@endofpackagehook{#1}}
\def\AtEndOfClass#1{\g@addto@macro\@endofclasshook{#1}}
% \document / \enddocument run BOTH the classic \AtBeginDocument / \AtEndDocument
% accumulators and the named hooks of the 2020 format (see hooks.go), in the order
% the real format uses: begindocument/before, the \AtBeginDocument code, then
% begindocument/end, then the document environment's own env/document/begin.
\def\document{\catcode64=12 \UseHook{begindocument/before}\@begindocumenthook
  \UseHook{begindocument}\UseHook{begindocument/end}\UseHook{env/document/begin}}
\def\enddocument{\UseHook{env/document/end}\@enddocumenthook\UseHook{enddocument}%
  \UseHook{enddocument/afterlastpage}\UseHook{enddocument/afteraux}%
  \UseHook{enddocument/info}\UseHook{enddocument/end}%
  \par\vfill\penalty-10000 }
% ── loaded-package / loaded-class registry (see CONTRACT above) ─────────────
\def\@ifl@aded#1#2{\@ifundefined{ver@#2.#1}\@secondoftwo\@firstoftwo}
% \@ptionlist{<file>} expands to the options that <file> was loaded with (the Go
% loader records them in opt@<file>, see the CONTRACT above); empty if it was not.
\def\@ptionlist#1{\@ifundefined{opt@#1}\@empty{\csname opt@#1\endcsname}}
\def\@ifpackageloaded#1{\@ifl@aded{sty}{#1}}
\def\@ifclassloaded#1{\@ifl@aded{cls}{#1}}
% \@ifpackagewith{pkg}{opts}{then}{else}: true iff every option in {opts} is in
% the recorded option list opt@pkg.sty. If that list was never recorded (Go side
% did not populate it), fall back to the else-branch.
% \@ifpackagewith{pkg}{opt}: true iff <opt> is one of the options recorded for the
% package in opt@pkg.sty. LIMITATION: checks a SINGLE option (the dominant real
% use); a multi-option {opt} is compared as one string and will not match.
\def\@ifpackagewith#1#2{\@ifundefined{opt@#1.sty}{\@secondoftwo}{\@ifinlist{#2}{\@nameuse{opt@#1.sty}}\ifin@\expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi}}
\catcode64=11
`

LaTeX2eKernelHelpers is the LaTeX2e low-level kernel helper layer, loaded by LoadLaTeX right after MiniLaTeXKernel.

const LaTeXHooks = `` /* 4286-byte string literal not displayed */

This file holds two things a package written after 2020 asks about before it does anything else: HOW OLD THE FORMAT IS, and the HOOK system that a modern format offers instead of patching kernel macros.

The question is not academic. etoolbox ends with

\IfFormatAtLeastTF{2020-10-01}
  {\newrobustcmd*{\AtEndPreamble}{\AddToHook{begindocument/before}}%
   … \endinput}
  {}

and then, in the branch for an OLD format, prepends code to \document and \patchcm ds \enddocument. With no \IfFormatAtLeastTF at all the engine fell through to that old branch, whose \patchcmd cannot match this kernel's macros: the patches failed, their code leaked into the page, and the rest of the document was swallowed. beamer's compatibility layer branches the same way.

Answering "yes, at least 2020-10-01" is both the truthful answer — the hook interface below is exactly what that release introduced — and the one that puts packages on the code path that asks the format for a service rather than rewriting the format's internals.

── What the hook layer models, and what it does not ─────────────────────────

A hook is a named token list that the format executes at a defined moment. The engine stores one macro per hook (gotex@hook@<name>) and runs it at that moment. Modelled:

• \NewHook / \NewReversedHook / \ProvideHook / \NewMirroredHookPair — declare.
• \AddToHook{<name>}[<label>]{<code>} — append; the hook need not exist yet.
• \AddToHookNext{<name>}{<code>} — code that fires at the NEXT use only.
• \UseHook / \UseOneTimeHook — run (and clear the next-use list).
• \RemoveFromHook, \IfHookEmptyTF, \ShowHook / \LogHook, \ActivateGenericHook.
• The document hooks are WIRED: begindocument/before, begindocument, begindocument/end and env/document/begin all fire at \begin{document}; enddocument and its four sub-hooks fire at \end{document}.
• package/<name>/after and file/<name>/after fire from the package loader when that package or file finishes loading (see loadTeXFile).

NOT modelled, deliberately:

• ORDERING. \DeclareHookRule and the [<label>] argument are accepted and ignored; code runs in the order it was added. \RemoveFromHook therefore empties the whole hook rather than one label's contribution.
• cmd/<name>/before and cmd/<name>/after. Those ask the format to inject code into the BODY of an existing command, which needs the command's argument count to place the "after" part. The engine records such a hook and never fires it. The one caller that reaches this in a beamer talk is the pdfpages integration, which needs pdfpages itself.
• Reversed hooks run in the order added, like ordinary ones (ordering again).

const MiniLaTeX = `` /* 167-byte string literal not displayed */

MiniLaTeX is a small LaTeX-flavoured kernel written *in TeX* — the engine runs these macro definitions through its gullet exactly as a real format does; they are not reimplemented in Go. It is deliberately tiny (the road to parity is to grow this by loading the real latex.ltx, not to hand-code commands).

const MiniLaTeXKernel = `` /* 36707-byte string literal not displayed */

MiniLaTeXKernel is the LaTeX-flavoured macro layer loaded by LoadLaTeX (after the Plain macros).

const NoProgressLimitHeavy = tightLoopStepsHeavy

NoProgressLimitHeavy is the ceiling a program that renders whole documents should put in Options.NoProgressLimit. See tightLoopStepsHeavy for what it costs and what it buys.

const Plain = `` /* 2333-byte string literal not displayed */

Plain is a small set of plain-TeX structural macros, written *in TeX* on top of the box/glue primitives (\hbox to, \hfil, \vskip). Loaded with LoadPlain, they let a document use the familiar commands without any Go-side support — the same growth path as the kernel: add macros, do not hand-code commands.

Variables

var (
	Box        = linebreak.Box
	Glue       = linebreak.Glue
	Glyph      = linebreak.Glyph
	Penalty    = linebreak.Penalty
	KnuthPlass = linebreak.KnuthPlass
)

var RasterizePDF func(data []byte, dpi float64) (image.Image, error)

RasterizePDF, when set by a consumer, rasterises a PDF figure (the bytes of an included .pdf) to an image at the given DPI. It is the seam for \includegraphics of vector PDFs: a pure-Go PDF renderer is a heavy dependency, so the engine core stays free of it — the CLI and loom inject one (see go-tex/pdfrender), while the browser/wasm build leaves it nil and shows a placeholder. EPS is not handled here.

Functions

func CompileToPDF

func CompileToPDF(src []byte, opt Options, w io.Writer) (int, error)

CompileToPDF processes TeX source and writes a PDF to w, returning the page count. A document's own \font/\hsize/… override the option defaults.

func CompileToPDFReport

func CompileToPDFReport(src []byte, opt Options, w io.Writer) (int, map[string]int, error)

CompileToPDFReport is CompileToPDF that also returns the skipped-command tally (see CompileToSVGPagesReport / SkippedCommands).

func CompileToSVGPages

func CompileToSVGPages(src []byte, opt Options) ([]string, error)

CompileToSVGPages processes TeX source and returns one SVG string per page — the form an editor preview pane consumes directly.

func CompileToSVGPagesReport

func CompileToSVGPagesReport(src []byte, opt Options) ([]string, map[string]int, error)

CompileToSVGPagesReport is CompileToSVGPages that also returns the tally of undefined control sequences the (lenient) compile skipped — see SkippedCommands. It lets a caller surface the feature gaps a best-effort render would otherwise hide: an unimplemented command silently dropped can take a document's whole body with it (a class's frontmatter macro, \subfile, …) while the page still looks plausible. Ranked over a corpus, the tally points straight at what is worth implementing.

func LineAt

func LineAt(spans []SourceSpan, x, y float64) int

LineAt returns the source line of the last glyph span whose box contains (x, y), or 0 when the point is over no glyph. Last-wins so nested/overlapping content (a table cell over its row) resolves to the innermost glyph painted there.

Types

type Diagnostics

type Diagnostics struct {
	Skipped    map[string]int // undefined control sequences, by count (internal markers removed)
	Runaway    bool           // the expansion/argument runaway guard tripped
	OpenGroups int            // groups still open at end of the document (a likely swallow)
	PageCapHit bool           // pagination hit the maxPages backstop (a page-count explosion)
	// UndefinedEnvs counts \begin{env} whose environment was undefined — a silent
	// no-op that never appears in Skipped (\csname coerces the missing \env to
	// \relax). Aggregated over a corpus it surfaces unimplemented environments
	// (math/float/theorem) whose bodies were then typeset in the wrong mode.
	UndefinedEnvs map[string]int

	// MathDropped tallies whole equations the go-tex/math layer refused and the engine
	// dropped, keyed by the unknown math command that triggered it ("\X") or "$math$"
	// for a non-command math error. This is invisible content loss INSIDE a formula:
	// one unrecognised token drops the entire equation. These are lifted out of Skipped
	// (which stays text-mode undefined commands) so a math feature gap is not conflated
	// with a missing text macro. nil/empty when no equation was dropped.
	MathDropped map[string]int
}

Diagnostics summarises what a lenient compile may have quietly lost. Beyond the undefined commands it skipped, it flags the signals of a SILENT swallow — content dropped with no undefined command at all: a runaway loop or exponential scan that tripped the guard, groups left open at the end of the document (an unbalanced { or \begingroup, the fingerprint of an eager-box or delimited-scan swallow), and a pagination explosion capped at maxPages. Aggregated over a corpus these surface problems SkippedCommands cannot see.

func CompileToPDFDiag

func CompileToPDFDiag(src []byte, opt Options, w io.Writer) (int, Diagnostics, error)

CompileToPDFDiag is CompileToPDF that also returns the compile's Diagnostics.

func CompileToSVGPagesDiag

func CompileToSVGPagesDiag(src []byte, opt Options) ([]string, Diagnostics, error)

CompileToSVGPagesDiag is CompileToSVGPages that also returns the compile's Diagnostics (undefined commands plus the silent-swallow flags — see Diagnostics), for a preview UI's log panel or a corpus report.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine holds all TeX state: the input stack, the eqtb (control-sequence meanings), integer registers, category codes, the grouping save stack, and the \message output buffer.

func New

func New() *Engine

New builds an engine with TeX's default category codes and primitives loaded.

func NewDocument

func NewDocument(opt Options) (*Engine, error)

NewDocument builds an engine configured from opts: the Plain macros loaded (unless NoPlain) and the text font family set. It is the starting point for programmatic use when callers want to Run source incrementally before rendering.

func (*Engine) Diagnostics

func (e *Engine) Diagnostics() Diagnostics

Diagnostics returns the compile's Diagnostics (see the type). Internal markers (the page-cap tally and the like) are lifted out of Skipped into their own flags.

func (*Engine) LoadFormat

func (e *Engine) LoadFormat(src string) error

LoadFormat executes a string of TeX definitions (a format/preamble) through the gullet, defining its macros in the engine's eqtb without typesetting.

func (*Engine) LoadLaTeX

func (e *Engine) LoadLaTeX() error

LoadLaTeX loads the Plain macros (if not already) and the minimal LaTeX kernel.

func (*Engine) LoadPlain

func (e *Engine) LoadPlain() error

LoadPlain defines the Plain structural macros in the engine.

func (*Engine) Page

func (e *Engine) Page() *boxNode

Page vpacks the main vertical list (everything contributed at top level) into a single vbox at natural height. Empty (nil) if nothing was contributed.

func (*Engine) Pages

func (e *Engine) Pages() []*boxNode

func (*Engine) Position

func (e *Engine) Position() (line, col int)

Position returns the 1-based line and 0-based column of the input the engine is currently reading — the location a diagnostic should point at.

func (*Engine) RenderBox

func (e *Engine) RenderBox(i int, margin float64) string

RenderBox renders box register i to an SVG string with a uniform margin (pt). Empty if the register is void.

func (*Engine) RenderPDF

func (e *Engine) RenderPDF(w io.Writer, margin float64) error

RenderPDF writes the main vertical list, split into \vsize pages, as a PDF to w. Each page is (content width + 2·margin) × (content height + 2·margin) points. A current OpenType font (with embeddable bytes) is required to draw text.

func (*Engine) RenderPage

func (e *Engine) RenderPage(margin float64) string

RenderPage renders the main vertical list to an SVG page with the given margin.

func (*Engine) RenderPages

func (e *Engine) RenderPages(margin float64) []string

RenderPages renders each page of the main vertical list to its own SVG string.

func (*Engine) Run

func (e *Engine) Run(src string) (string, error)

Run tokenizes src as the base input and processes it to completion, returning the accumulated \message output.

func (*Engine) SetFont

func (e *Engine) SetFont(f fontFace)

SetFont sets the current font used to measure and render characters in horizontal mode. Passing an *OpenTypeFont (or any fontFace) is the Go-level stand-in for TeX's \font primitive until font-file loading via \font lands.

func (*Engine) SkippedCommands

func (e *Engine) SkippedCommands() map[string]int

SkippedCommands returns, for a lenient compile, how many times each undefined control sequence was skipped (empty when strict or when none were undefined). It lets a caller surface "these commands were dropped" after a preview compile.

func (*Engine) SourceSpans

func (e *Engine) SourceSpans(margin float64) [][]SourceSpan

SourceSpans returns, for each page of the built document, the glyph spans that tie output back to source — the data behind click-to-source and jump-to-line. margin matches the value passed to the SVG/PDF renderers so coordinates align.

type Item

type Item = linebreak.Item

Two pieces of this engine are algorithms in their own right, useful to anyone laying out text and not only to a TeX engine, so they now live in their own repositories and this file is the seam:

• github.com/go-typeset/linebreak — Knuth–Plass optimal line breaking: the box/glue/penalty model and the paragraph builder.
• github.com/go-typeset/hyphenation — Liang's algorithm: where a hyphen may fall in a word, reading TeX's own pattern files.

The aliases below keep the engine's own code reading as it did (Item, Line, KnuthPlass, …) while the definitions live upstream. What stays HERE is what is genuinely TeX's rather than the algorithm's: the \patterns primitive that loads a pattern file, and the walk that turns a node list into discretionary breaks.

type Line

type Line = linebreak.Line

Two pieces of this engine are algorithms in their own right, useful to anyone laying out text and not only to a TeX engine, so they now live in their own repositories and this file is the seam:

• github.com/go-typeset/linebreak — Knuth–Plass optimal line breaking: the box/glue/penalty model and the paragraph builder.
• github.com/go-typeset/hyphenation — Liang's algorithm: where a hyphen may fall in a word, reading TeX's own pattern files.

The aliases below keep the engine's own code reading as it did (Item, Line, KnuthPlass, …) while the definitions live upstream. What stays HERE is what is genuinely TeX's rather than the algorithm's: the \patterns primitive that loads a pattern file, and the walk that turns a node list into discretionary breaks.

type OpenTypeFont

type OpenTypeFont struct {
	// contains filtered or unexported fields
}

OpenTypeFont adapts a go-opentype face for the engine: it measures glyphs (in points via CharDims and in scaled points via charDimsSP), supplies the interword glue, and returns glyph outline paths for the SVG driver.

func NewOpenTypeFont

func NewOpenTypeFont(fontBytes []byte, sizePx int) (*OpenTypeFont, error)

NewOpenTypeFont builds a metrics source from a font and a pixel size.

func (*OpenTypeFont) CharDims

func (o *OpenTypeFont) CharDims(r rune) (float64, float64, float64)

CharDims returns a glyph's advance width and its ink height above and depth below the baseline, in pixels.

func (*OpenTypeFont) Space

func (o *OpenTypeFont) Space() (float64, float64, float64)

Space returns TeX-like interword glue derived from the space advance.

type Options

type Options struct {
	Font       []byte  // roman text font (.ttf/.otf); nil ⇒ the built-in default
	BoldFont   []byte  // optional bold face, bound to \bf (so \textbf really bolds)
	ItalicFont []byte  // optional italic face, bound to \it (so \textit/\emph slant)
	MonoFont   []byte  // optional monospace face, bound to \tt (so \texttt is fixed-width)
	SansFont   []byte  // optional sans-serif face, bound to \sf (so \textsf is sans)
	Size       int     // font size in points (0 ⇒ 10)
	Margin     float64 // page margin in points (0 ⇒ 72)
	NoPlain    bool    // set to omit the Plain macros
	Date       string  // \today text (a pure-Go wasm build has no clock; supply it here)

	// Lenient turns an undefined control sequence from a fatal error into a
	// skipped command: the engine drops the unknown \cs (and its likely
	// [optional]/{mandatory} argument block) and carries on, recording the name
	// for reporting. This is best-effort "produce something" behaviour for real
	// third-party documents that pull in packages/classes gotex does not load —
	// an editor preview shows the typesettable content instead of one hard error.
	// The default (false) is strict: an undefined cs aborts, as TeX does.
	Lenient bool

	// NoProgressLimit raises the ceiling on expansion steps taken with no new
	// base input consumed — the guard that stops a non-terminating expansion.
	// Zero keeps the engine's own, which is tight enough that a loop aborts in a
	// tenth of a second.
	//
	// A program that renders whole documents should ask for
	// NoProgressLimitHeavy: two million steps is too tight for a real
	// package, and pgfplots — which needs three to four million both to load and
	// to draw — stops the document dead without the headroom. It is not the
	// default because a genuine loop then takes ten times as long to abort, and a
	// caller exercising the guard wants it tight.
	NoProgressLimit int

	// Resolve supplies class, package and \input files that the process cannot
	// read from disk, so a host can hand the engine a texmf tree it holds in
	// memory. It is given the file name the document asked for, already carrying
	// an extension (article.cls, beamerbasetitle.sty, pgfcore.code.tex), and
	// returns the file's bytes.
	//
	// It is consulted AFTER the document's own directory and TEXINPUTS/
	// GOTEX_TEXMF — a file next to the document still overrides — and BEFORE the
	// small base set embedded in the binary, so a host that supplies a real
	// article.cls gets its own rather than the built-in one.
	//
	// This is what a build with no filesystem needs: in js/wasm every os.ReadFile
	// fails, so without it a browser host can compile nothing beyond the embedded
	// set. Such a host fetches the files it needs (driven by a scan of the source
	// for \usepackage and friends), keeps them in a map, and answers from it —
	// the engine's own reading stays synchronous.
	//
	// nil (the default) means no host resolver: disk and the embedded set only.
	Resolve func(name string) ([]byte, bool)
}

Options configures a compile. The zero value is valid: a built-in font at 10pt with a 72pt (1 inch) margin.

type SourceError

type SourceError struct {
	Line, Col int // 1-based line, 0-based column (0/0 = unknown)
	Msg       string
}

SourceError is an engine error carrying the source location it occurred at, so a caller (a CLI, loom's compile panel) can point the user at the exact line.

func (SourceError) Error

func (s SourceError) Error() string

type SourceSpan

type SourceSpan struct {
	Line       int
	X, Y, W, H float64
}

SourceSpan is one rendered glyph's bounding box on a page (points, SVG coordinates: origin top-left, Y is the box top) tagged with the source line it came from. It is the programmatic form of the SVG's data-l groups: a caller maps a click (x, y) → line, or a line → its output rectangles.

func RectsForLine

func RectsForLine(spans []SourceSpan, line int) []SourceSpan

RectsForLine returns every glyph span originating from the given source line — the boxes an editor highlights when the cursor sits on that line.