decant

package module
v1.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 4, 2026 License: MIT Imports: 21 Imported by: 0

README

decant

Reconstruct semantic, reflowable EPUB 3 from fixed-layout PDF.

PDF stores positioned glyphs. EPUB needs paragraphs, headings, and reading order. decant is a single static Go binary that recovers the second from the first, plus an importable library so a TUI can drive the same code path without shelling out.

Status: M5. Table detection, text extraction, column detection, paragraph reconstruction, heading classification, outline-driven chapter splitting, images with figures and captions, furniture removal, dehyphenation, lists, blockquotes, code blocks, and linked footnotes. Every output passes epubcheck with zero errors. See Milestones.

MIT licensed. Full design in spec.md.

Install

go install github.com/sroberts/decant/cmd/decant@latest

Or build from a checkout:

go build -o decant ./cmd/decant

Pure Go, no cgo. Cross-compiles to a static binary for linux/amd64, linux/arm64, darwin/arm64, and windows/amd64.

Use

decant book.pdf                          # writes book.epub
decant convert book.pdf -o out.epub
decant convert book.pdf --profile=crosspoint
decant convert book.pdf --pages 5-200,210 --report report.json
decant meta book.pdf                     # metadata, no conversion
decant probe book.pdf --stage=lines --page=12

convert is the default verb. Flags may appear before or after the input path.

Device profiles
Setting standard crosspoint minimal
Images keep, RGB 16-level grayscale; JPEG photos, PNG line art drop
Image max width 1600 480 n/a
Max chunk bytes 262144 262144 65536
Table mode auto text text
TOC depth unlimited 2 2
CSS base reduced none

crosspoint targets an Xteink X4 running CrossPoint firmware: a 480x800 E Ink panel driven by an ESP32-C3 with roughly 380 KB of usable RAM.

Chapter size is not a memory constraint there, contrary to the obvious guess. The firmware streams XHTML through expat in 1 KB chunks and serializes each laid-out page to the SD card as it completes, so a chapter never lands in RAM; only a 12-byte-per-page lookup table scales with its length, about 0.9% of the chapter's bytes. The out-of-memory crashes in its release notes were the CSS parser, now guarded at 128 KB, and decant emits under 1 KB of CSS. The profile therefore keeps the standard 256 KB chunk.

The same reading settled the image formats. CrossPoint's EPUB path decodes JPG and PNG — including the indexed form — and has no BMP decoder at all, so line art ships as a paletted PNG where it is smaller and stays sharp, and only photographs dither to JPEG. See spec section 5.1.

Exit codes
Code Meaning
0 Success
1 Runtime failure
2 Usage error
3 Encrypted PDF (unsupported in v1)
4 No usable text layer (scanned document)
5 Converted with warnings and --strict was set
6 Malformed PDF beyond repair

Diagnostics go to stderr. --json output goes to stdout, unmixed, so -o - can stream an EPUB to a pipe.

Library

conv, err := decant.New(decant.Options{
    Profile:    decant.ProfileCrossPoint,
    Heuristics: decant.DefaultHeuristics(),
})

// Analyze runs stages 1 through 6 and returns the block tree.
doc, err := conv.Analyze(ctx, reader, size)

// Callers may correct structure before committing the EPUB.
doc.Blocks[0].Kind = decant.KindHeading
doc.Blocks[0].Level = 1

report, err := conv.Write(ctx, doc, out)
fmt.Println(report.QualityScore) // 0 to 100, a triage signal

Convert is Analyze followed by Write. The split exists so a caller can preview detected structure and fix it first.

Everything below the root package is under internal/, so the public surface stays small. Full documentation and runnable examples are on pkg.go.dev.

API stability

The API is unstable until v1.0.0, which is cut at M6 once the CrossPoint TUI has exercised Analyze and Write against real files. From v1.0.0, normal Go compatibility applies to the root package, and these behaviours are part of the contract rather than incidental:

  • Write does not mutate the Document and does not accumulate into the Report, so it may be called repeatedly on one tree. A TUI writes a preview, keeps editing, and writes again.
  • A Converter holds no mutable state and is reusable across documents.
  • Edits to Document.Blocks reach the output. Heading levels drive chapter splitting and the navigation document.
  • Heading levels outside 1 through 6 are clamped, not rejected, because XHTML has no other heading elements. A stray edit cannot fail a conversion or emit an element that does not exist.
  • A document with no blocks is refused rather than written as an empty but structurally valid EPUB.
  • New copies its Options; mutating the caller's copy afterwards does not affect the converter.

api_test.go pins each of these.

What is explicitly not covered: the exact structure any given PDF converts to. Layout reconstruction is heuristic, and heuristics improve. Block counts, heading levels, and chapter boundaries may change in any release. Pin thresholds through Heuristics if you need stability there, and watch testdata/corpus_manifest.json for how a change moves 34 real documents.

Custom device profiles

The three built-in profiles cover what decant knows about. For a device it has never seen, dump the closest one, edit it, and pass the file:

decant profile --dump crosspoint -o kobo.json
decant convert book.pdf --profile-file kobo.json

Keys under options and heuristics are the field names from go doc decant.Options and go doc decant.Heuristics, so the API reference is the format reference. Anything the file omits keeps its base profile's value. Unknown keys are an error rather than a silent no-op.

Precedence runs defaults, then the base profile, then the file, then any explicit flag — so a shared profile can be adopted wholesale and still overridden one setting at a time.

decant will not fetch a profile over the network. Identical input and flags have to produce identical output, and a remote file can change between runs; the file is the shareable unit.

A profile for a device nobody has tested is a guess. The crosspoint numbers come from reading that firmware's source. Anything you write for other hardware is worth checking on the hardware.

Guarantees

Deterministic output. Identical input plus identical flags produces byte-identical EPUB, independent of worker count or wall clock. Anchor IDs derive from content hashes rather than counters. dc:identifier is a UUIDv5 over the input's SHA-256, so reconverting a file yields the same identifier. ZIP entries sort by name, carry no extra fields, and share one fixed timestamp taken from --date, then SOURCE_DATE_EPOCH, then the PDF ModDate.

No OCR, ever. Not embedded, not by subprocess. Scanned PDFs are detected after glyph extraction and exit 4 with the metrics that triggered the decision, pointing at an external OCR pass.

Fail loud, degrade gracefully. Every heuristic that fires records a diagnostic in the conversion report. decant probe dumps the intermediate model at any stage.

What works today

  • Content stream interpretation with full graphics and text state
  • Simple and Type0/CID fonts; /ToUnicode, /Differences, the standard encoding tables, Adobe glyph names, and reverse lookup through an embedded font's cmap
  • Base-14 metrics for documents that reference fonts without embedding them
  • Page rotation, form XObject recursion, inline image skipping
  • Line assembly with space reconstruction; ligature, soft hyphen, and NFC normalization
  • Column detection from a per-row projection profile, with correct reading order and full-width headings preserved across the gutter
  • Bottom-up block segmentation using horizontal overlap and running leading
  • Paragraph reconstruction from indent, leading, and terminal punctuation
  • Heading classification against a document-wide body font, ranked to h1-h6
  • PDF outline reconciliation, hierarchical TOC, chapter splitting at headings
  • Image extraction with placement from the CTM, deduplication by pixel digest, Catmull-Rom scaling, JPEG/paletted-PNG selection by colour count, and DCT passthrough when nothing needs the pixels
  • Background and watermark rejection, size floors, figures placed in reading order, caption binding, and grayscale dithering for the crosspoint profile
  • Running head and folio removal by repeated text and repeated position
  • Dehyphenation by inverted Liang pattern matching in eight languages
  • Ordered and unordered lists with inferred start, blockquotes, code blocks
  • Table detection from ruling lines and column alignment, with colspan
  • Inline bold and italic as <strong> and <em>, with TeX math italic excluded so a formula's variables do not become emphasis
  • Internal cross-references rewritten from PDF /Link annotations to href fragments, anchoring only the blocks something points at
  • Superscript detection and footnotes linked with epub:type noteref
  • Deterministic EPUB 3.3 output with an EPUB 2 NCX fallback
  • Encrypted, scanned, and malformed input detection with distinct exit codes
  • convert, probe, meta, and version subcommands

Not implemented yet

--table-mode=image has been removed. It needed the vector renderer that spec §13.1 leaves for after v1, so it only ever degraded to text with a warning; shipping it would have frozen a mode that silently does something else into the v1 API. Asking for it is now a usage error.

--jobs is reserved: it is accepted, prints a notice, and does nothing. Page processing is sequential and stays that way. Stages 1 and 2 run inside pdfcpu, which mutates its cross-reference table on every dereference with no lock, and they are two thirds of per-page time; the rest is about 4% of a conversion. Options.Jobs was removed from the library rather than shipped as a permanent no-op. See spec §4.

Hyphenation language comes from --language, then the PDF's /Lang, then XMP dc:language, then English.

Dehyphenation ships patterns for English, German, Spanish, French, Italian, Dutch, Polish, and Portuguese. Russian and Swedish are deliberately absent: their hyph-utf8 files are LPPL-only, and spec §4.6 says to drop the language rather than take on a share-alike or renaming condition. Those documents convert normally with dehyphenation disabled and a diagnostic. See THIRD_PARTY.md.

Right-to-left and vertical CJK documents convert, with a warning. The text is extracted correctly, but decant emits it in logical order: it does not run the bidirectional algorithm and does not set vertical columns, so lines may read in the wrong direction. The report carries rtl_letter_ratio and vertical_text_pages, and the warning fires once the document is substantially right-to-left rather than on a single quoted phrase.

JPEG 2000 and JBIG2 images drop with a diagnostic: neither has a pure-Go decoder, and spec principle 2 rules out cgo. Inline (BI) images have their position recorded for scan detection but are not extracted.

Vector artwork is not rendered, so a chart drawn as paths is lost. It is reported rather than dropped silently: the conversion report counts painted paths per page and warns when a page carries enough of them to be a diagram. Rasterization was considered and declined for v1 (spec §13, closed 2026-08-03). §1 permits either rasterizing or dropping, and dropping is what decant does.

In practice that means a diagram-heavy academic PDF loses its figures. On the sample corpus the warning fires on 39 of one mathematics textbook's 117 pages and on nothing else, so the exposure is narrow, but if your library is mostly papers with plotted charts, expect to lose them and to be told so. The text around them converts normally.

Table detection still over-fires on mathematical typesetting. On the corpus's LaTeX textbook it reports eight medium-confidence tables that are really plotted axes and matrix-like displays; --table-mode=auto renders those as space-preserved text rather than as <table>, so no false table markup reaches the reader, but the layout is still wrong. --table-mode=drop turns detection off entirely and leaves the text as paragraphs. Three guards already narrow this — a fill ratio, a rejection of grids whose cells hold one character each, and a rule that a table may not straddle the page's own columns — and further tuning needs a corpus with more real tables in it than this one has.

Milestones

Scope Status
M1 Parse, glyph extraction, line assembly, plain paragraphs done
M2 Block segmentation, column detection, headings, outline TOC, chapter splitting done
M3 Image extraction, placement, re-encoding, figures and captions done
M4 Furniture removal, dehyphenation, footnotes, lists, blockquotes done
M5 Table detection, device profiles, conversion report, probe done
M6 Public API stabilization, content fidelity, remaining spec gaps done

M1 through M3 ship before anything gets optimized. Layout heuristics need real-corpus feedback; tuning thresholds against three test files produces overfitted garbage.

Releasing

git tag -a v1.2.0 -m "..."   # the message becomes the release notes
git push origin v1.2.0

The release workflow re-runs the gate, cross-compiles five targets, checks the built binary reports the tag, and publishes with checksums. make dist VERSION=v1.2.0 rehearses the build without publishing.

Development

go test ./...                  # unit and integration tests
go test -race ./...            # CI runs this
go vet ./... && staticcheck ./...

epubcheck on PATH enables validation tests against generated output; they skip without it. CI installs it and enforces zero errors as a merge gate.

Real-world corpus
make corpus        # fetch py-pdf/sample-files, pinned to a commit
make corpus-test   # run the corpus tests
make manifest      # regenerate the golden, then review the diff

The corpus is py-pdf/sample-files: 34 PDFs from pdfTeX, LibreOffice, Google Docs, ReportLab, PDFKit, and others, including a 117-page LaTeX book, a two-column paper, Arabic text, an encrypted file, and a damaged one. It is not vendored — those files are CC-BY-SA-4.0 and decant ships MIT — so it is fetched on demand and the tests skip without it.

testdata/corpus_manifest.json records what decant produces for each file: outcome, block and heading counts, columns detected, a decode-failure bucket, and structure and text digests. It is the regression gate, and the tool for judging whether a heuristic change helps or hurts across real documents rather than across three fixtures.

Current state: 27 files convert (25 with zero decode failures), 5 are correctly rejected as image-only with no text layer, 1 is correctly rejected as encrypted, and 1 damaged file is not yet recoverable.

Fuzz targets cover the content stream lexer, the interpreter, the CMap parser, and the xref parser:

go test -run XXX -fuzz FuzzLexer -fuzztime 60s ./internal/pdf/

Run a single test:

go test -run TestDeterministicOutput ./...

Dependencies

Package License Role
github.com/pdfcpu/pdfcpu Apache-2.0 xref parsing, object model
golang.org/x/image BSD-3 font/sfnt metrics, Catmull-Rom resampling
golang.org/x/text BSD-3 Unicode normalization
github.com/hhrutter/tiff BSD-3 CMYK TIFF decode, which x/image rejects

Hyphenation patterns are vendored from hyph-utf8 under MIT, BSD, or unrestricted terms only; THIRD_PARTY.md records the per-file audit.

unidoc/unipdf (AGPL or paid), go-fitz and other MuPDF bindings (cgo plus AGPL), and rsc.io/pdf (no font or positioning support) are all ruled out by the pure-Go static binary requirement or the license.

Documentation

Overview

Package decant converts fixed-layout, text-layer PDF into semantic, reflowable EPUB 3.

PDF stores positioned glyphs; EPUB needs paragraphs, headings, and reading order. Recovering the second from the first is the whole problem, and it is heuristic: decant infers structure that the source file does not record. Every inference it makes is tunable through Heuristics, inspectable through Converter.Probe, and reported through Report.

The pipeline

Conversion runs in eight stages: 

parse → glyphs → lines → blocks → furniture → classify → assemble → serialize

Stages 1 through 6 produce a Document, an editable block tree. Stage 7 and 8 turn that tree into an EPUB container. The split is deliberate and is the main reason this is a library rather than only a command: a caller can inspect and correct the inferred structure before committing to output.

Basic use

Converter.Convert runs the whole pipeline: 

conv, err := decant.New(decant.DefaultOptions())
if err != nil {
	return err
}
rep, err := conv.Convert(ctx, in, size, out)

A Converter holds no mutable state, so one instance is safe to reuse across documents and across goroutines.

Correcting structure before writing

To review or edit what was inferred, call Converter.Analyze, modify the returned Document, then call Converter.Write: 

doc, err := conv.Analyze(ctx, in, size)
if err != nil {
	return err
}
for i := range doc.Blocks {
	if doc.Blocks[i].Text == "Appendix A" {
		doc.Blocks[i].Kind = decant.KindHeading
		doc.Blocks[i].Level = 1
	}
}
rep, err := conv.Write(ctx, doc, out)

Edits reach the output: heading levels drive chapter splitting and the navigation document, so promoting a paragraph to a level-1 heading starts a new chapter. Levels outside 1 through 6 are clamped rather than rejected, because XHTML has no other heading elements. Write does not mutate the Document, so it may be called more than once on the same tree.

A block carries more than its text. Block.Styles holds the bold and italic runs as character ranges, and Block.Links the internal cross-references, both editable the same way: narrowing a range or clearing a target suppresses the markup without touching the words.

Device profiles

Options.ApplyProfileDefaults applies one of the built-in profiles. For a device decant has never seen, LoadProfileDoc reads a profile document and Options.ApplyProfileDoc layers it over a chosen base, so a profile can be shared as a file rather than compiled in. WriteProfileDoc emits a built-in as a starting point.

Determinism

Identical input and options produce byte-identical output. Anchor IDs are content hashes rather than counters, the package identifier is a UUIDv5 over the input's SHA-256, and ZIP entries carry a fixed timestamp taken from Options.Deterministic, the PDF ModDate, or the Unix epoch, in that order. Reconverting a file therefore yields the same bytes, which is what makes output diffable and cacheable.

Failure

decant fails loudly rather than emitting silently corrupt EPUB. Four conditions are reported as typed errors, each of which the command maps to a distinct exit code:

  • EncryptedError, for a PDF carrying an /Encrypt dictionary
  • NoTextLayerError, for a scan. decant does not OCR, ever
  • MalformedError, for damage beyond what xref reconstruction recovers. A table that can be rebuilt is repaired instead, and the report says so
  • UsageError, for invalid options

Everything short of those degrades gracefully and records a Diagnostic in the Report instead. A conversion that emitted warnings still produced a valid EPUB; Report.QualityScore summarizes how much to trust it.

Stability

Every package below this one is internal, so the surface here is the whole supported API. See the repository README for what the v1 compatibility promise does and does not cover.

Example

Convert a PDF to EPUB with the default settings. 

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/sroberts/decant"
)

// openPDF is the boilerplate every example needs: decant reads through
// io.ReaderAt and needs the size, so a caller passes both.
func openPDF(path string) (*os.File, int64, error) {
	f, err := os.Open(path)
	if err != nil {
		return nil, 0, err
	}
	st, err := f.Stat()
	if err != nil {
		f.Close()
		return nil, 0, err
	}
	return f, st.Size(), nil
}

func main() {
	in, size, err := openPDF("book.pdf")
	if err != nil {
		log.Fatal(err)
	}
	defer in.Close()

	out, err := os.Create("book.epub")
	if err != nil {
		log.Fatal(err)
	}
	defer out.Close()

	conv, err := decant.New(decant.DefaultOptions())
	if err != nil {
		log.Fatal(err)
	}

	rep, err := conv.Convert(context.Background(), in, size, out)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%d chapters, quality %d/100\n", rep.Chapters, rep.QualityScore)
}

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func WriteProfileDoc added in v1.1.0 

func WriteProfileDoc(w io.Writer, p Profile) error

WriteProfileDoc writes a built-in profile as a document, which is the starting point for adapting one to a new device.

Types

type Block 

type Block struct {
	Kind BlockKind
	// Level is the heading rank 1 through 6, and 0 for non-headings.
	Level int
	// Text is the block's plain text content.
	Text string

	// Page is the zero-based source page index.
	Page int
	// Bounds is the block's bounding box in page space.
	Bounds Rect

	// ID is the anchor identifier. It derives from a content hash rather than
	// a counter so it stays stable across runs, which internal cross-
	// reference rewriting in spec section 4.9 depends on.
	ID string

	// Size is the block's median font size in points.
	Size float64
	// Font is the block's dominant font family name.
	Font string

	// ImageID names the image a figure block carries, matching Document.Images.
	// Empty for every other kind.
	ImageID string
	// Caption is a figure's caption text, empty when it has none.
	Caption string
	// InlineImage marks a figure narrow enough, and inside a paragraph, to
	// flow in the text rather than stand as a block. Spec 4.7.
	InlineImage bool

	// ListItems holds a list block's items, one per entry. Empty for every
	// other kind.
	ListItems []string
	// ListOrdered selects ol over ul.
	ListOrdered bool
	// ListStart is the first item's number, inferred from its marker. Zero
	// means the list starts at one.
	ListStart int

	// Styles are the bold and italic runs of Text, as character ranges.
	// Spec section 4.6.
	Styles []StyleRun

	// Links are internal cross-references originating in this block, as
	// character ranges of Text. Spec section 4.9.
	Links []CrossRef

	// Superscripts are the superscript run labels found in Text, in order.
	Superscripts []string
	// NoteRefs maps a superscript label to the ID of the footnote block it
	// references. Spec 4.6 links these with epub:type noteref and footnote.
	NoteRefs map[string]string
	// NoteLabel is a footnote block's own marker, e.g. "1" or "†".
	NoteLabel string

	// TableRows holds a table block's cells, row by row. Empty for every
	// other kind.
	TableRows [][]TableCell
	// TableConfidence records which detection signals fired, which
	// --table-mode=auto keys on.
	TableConfidence string
}

type BlockKind 

type BlockKind string

BlockKind classifies a block after structure classification. 

const (
	// KindParagraph is flowed body text.
	KindParagraph BlockKind = "paragraph"
	// KindHeading is a heading; see Block.Level for its rank.
	KindHeading BlockKind = "heading"
	// KindList is a bullet or numbered list.
	KindList BlockKind = "list"
	// KindQuote is a blockquote.
	KindQuote BlockKind = "blockquote"
	// KindCode is a fixed-pitch code block.
	KindCode BlockKind = "code"
	// KindCaption is a figure or table caption.
	KindCaption BlockKind = "caption"
	// KindFootnote is a footnote body.
	KindFootnote BlockKind = "footnote"
	// KindFigure is an image with optional caption.
	KindFigure BlockKind = "figure"
	// KindTable is a detected table.
	KindTable BlockKind = "table"
)

type Converter 

type Converter struct {
	// contains filtered or unexported fields
}

Converter runs the conversion pipeline. It holds no mutable state, so one Converter is safe to reuse across documents.

func New 

func New(opts Options) (*Converter, error)

New validates options and returns a Converter.

func (*Converter) Analyze 

func (c *Converter) Analyze(ctx context.Context, r io.ReaderAt, size int64) (*Document, error)

Analyze runs stages 1 through 6 and returns the intermediate model.

The returned Document exposes the block tree so a caller can correct structure before Write serializes it.

Example

Correct the inferred structure before writing.

This is why Analyze and Write are separate. Headings drive chapter splitting and the navigation document, so promoting a block to a level-1 heading starts a new chapter in the output. 

package main

import (
	"context"
	"log"
	"os"

	"github.com/sroberts/decant"
)

// openPDF is the boilerplate every example needs: decant reads through
// io.ReaderAt and needs the size, so a caller passes both.
func openPDF(path string) (*os.File, int64, error) {
	f, err := os.Open(path)
	if err != nil {
		return nil, 0, err
	}
	st, err := f.Stat()
	if err != nil {
		f.Close()
		return nil, 0, err
	}
	return f, st.Size(), nil
}

func main() {
	in, size, err := openPDF("book.pdf")
	if err != nil {
		log.Fatal(err)
	}
	defer in.Close()

	conv, err := decant.New(decant.DefaultOptions())
	if err != nil {
		log.Fatal(err)
	}

	doc, err := conv.Analyze(context.Background(), in, size)
	if err != nil {
		log.Fatal(err)
	}

	// An epigraph set in a large face is a classic false heading. Demote it,
	// and promote an appendix the classifier read as body text.
	for i := range doc.Blocks {
		switch doc.Blocks[i].Text {
		case "It was a bright cold day in April":
			doc.Blocks[i].Kind = decant.KindParagraph
			doc.Blocks[i].Level = 0
		case "Appendix A":
			doc.Blocks[i].Kind = decant.KindHeading
			doc.Blocks[i].Level = 1
		}
	}

	out, err := os.Create("book.epub")
	if err != nil {
		log.Fatal(err)
	}
	defer out.Close()

	if _, err := conv.Write(context.Background(), doc, out); err != nil {
		log.Fatal(err)
	}
}

func (*Converter) Convert 

func (c *Converter) Convert(ctx context.Context, r io.ReaderAt, size int64, w io.Writer) (*Report, error)

Convert is Analyze followed by Write.

Example (Errors)

Distinguish the four failure modes.

Each maps to a distinct exit code at the command layer. Everything short of these degrades gracefully and lands in the report as a diagnostic instead. 

package main

import (
	"context"
	"errors"
	"fmt"
	"log"
	"os"

	"github.com/sroberts/decant"
)

// openPDF is the boilerplate every example needs: decant reads through
// io.ReaderAt and needs the size, so a caller passes both.
func openPDF(path string) (*os.File, int64, error) {
	f, err := os.Open(path)
	if err != nil {
		return nil, 0, err
	}
	st, err := f.Stat()
	if err != nil {
		f.Close()
		return nil, 0, err
	}
	return f, st.Size(), nil
}

func main() {
	in, size, err := openPDF("book.pdf")
	if err != nil {
		log.Fatal(err)
	}
	defer in.Close()

	conv, err := decant.New(decant.DefaultOptions())
	if err != nil {
		log.Fatal(err)
	}

	_, err = conv.Convert(context.Background(), in, size, os.Stdout)

	var encrypted *decant.EncryptedError
	var noText *decant.NoTextLayerError
	var malformed *decant.MalformedError
	switch {
	case errors.As(err, &encrypted):
		fmt.Println("password-protected; decant does not decrypt")
	case errors.As(err, &noText):
		fmt.Println("scanned; decant does not OCR")
	case errors.As(err, &malformed):
		fmt.Println("damaged beyond repair")
	case err != nil:
		log.Fatal(err)
	}
}

func (*Converter) Options 

func (c *Converter) Options() Options

Options returns the converter's resolved options.

func (*Converter) Probe 

func (c *Converter) Probe(ctx context.Context, r io.ReaderAt, size int64, stage ProbeStage, page int) (*ProbeResult, error)

Probe runs the pipeline as far as the requested stage and returns the intermediate model. A page of -1 probes every selected page.

Example

Inspect the intermediate model at one pipeline stage.

Probe is what makes the heuristics auditable: it dumps what decant saw at the point a decision was made, for one page or the whole document. 

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/sroberts/decant"
)

// openPDF is the boilerplate every example needs: decant reads through
// io.ReaderAt and needs the size, so a caller passes both.
func openPDF(path string) (*os.File, int64, error) {
	f, err := os.Open(path)
	if err != nil {
		return nil, 0, err
	}
	st, err := f.Stat()
	if err != nil {
		f.Close()
		return nil, 0, err
	}
	return f, st.Size(), nil
}

func main() {
	in, size, err := openPDF("book.pdf")
	if err != nil {
		log.Fatal(err)
	}
	defer in.Close()

	conv, err := decant.New(decant.DefaultOptions())
	if err != nil {
		log.Fatal(err)
	}

	res, err := conv.Probe(context.Background(), in, size, decant.StageLines, 12)
	if err != nil {
		log.Fatal(err)
	}
	for _, p := range res.Pages {
		for _, l := range p.Lines {
			fmt.Printf("%.1f  %s\n", l.Baseline, l.Text)
		}
	}
}

func (*Converter) Write 

func (c *Converter) Write(ctx context.Context, doc *Document, w io.Writer) (*Report, error)

Write serializes an analyzed Document to EPUB.

type CrossRef 

type CrossRef struct {
	// Start and End are byte offsets into the owning block's Text, half-open.
	Start, End int

	// TargetPage is the zero-based destination page and TargetY its vertical
	// position in page space, or NaN when the destination names none.
	TargetPage int
	TargetY    float64

	// TargetID is the ID of the block the destination resolved to. Empty
	// means the destination pointed somewhere with no block, in which case
	// the range renders as plain text and the report records it.
	TargetID string
}

CrossRef is one internal cross-reference: a range of a block's text that links to somewhere else in the document.

Spec section 4.9 rewrites PDF /Link annotations into these. A caller may edit the range or clear TargetID to suppress the link, the same way it may edit any other field of the block tree.

type Diagnostic 

type Diagnostic struct {
	Severity Severity `json:"severity"`
	// Stage names the pipeline stage, e.g. "glyphs" or "assemble".
	Stage string `json:"stage"`
	// Page is the zero-based page index, or -1 for document-level entries.
	Page    int    `json:"page"`
	Message string `json:"message"`
}

Diagnostic records one heuristic firing or one quality problem. Spec principle 3 requires every heuristic that fires to leave one of these.

type Document 

type Document struct {
	// Title, Author, and Language are resolved from PDF metadata and any
	// caller overrides.
	Title    string
	Author   string
	Language string

	// Source is the input filename, recorded as dc:source.
	Source string

	// Blocks is the reconstructed content in reading order.
	Blocks []Block

	// Outline is the PDF bookmark tree, empty when the document has none.
	Outline []OutlineItem

	// Images are the pictures carried into the EPUB, referenced by
	// Block.ImageID.
	Images []Image

	// PageCount is the number of pages in the source document, before any
	// page range applies.
	PageCount int

	// Digest is the hex SHA-256 of the input file. The EPUB identifier is a
	// UUIDv5 over it, which makes reconversion produce the same identifier.
	Digest string

	// Modified is the timestamp written to dcterms:modified and every ZIP
	// header.
	Modified time.Time
	// contains filtered or unexported fields
}

Document is the intermediate model produced by Analyze: the block tree plus the metadata needed to serialize it.

func (*Document) Report 

func (d *Document) Report() *Report

Report returns the diagnostics gathered so far. Analyze fills the per-page metrics; Write adds serialization results and the quality score.

type DocumentInfo 

type DocumentInfo struct {
	Source    string `json:"source"`
	PageCount int    `json:"page_count"`

	Title    string `json:"title,omitempty"`
	Author   string `json:"author,omitempty"`
	Subject  string `json:"subject,omitempty"`
	Keywords string `json:"keywords,omitempty"`
	Creator  string `json:"creator,omitempty"`
	Producer string `json:"producer,omitempty"`
	Language string `json:"language,omitempty"`

	Created  time.Time `json:"created,omitempty"`
	Modified time.Time `json:"modified,omitempty"`

	// Digest is the hex SHA-256 of the input file.
	Digest string `json:"digest"`
	// Identifier is the EPUB dc:identifier a conversion would produce.
	Identifier string `json:"identifier"`

	// OutlineEntries is the number of top-level bookmarks.
	OutlineEntries int           `json:"outline_entries"`
	Outline        []OutlineItem `json:"outline,omitempty"`

	// PageSizes lists the distinct page dimensions in points.
	PageSizes []PageSize `json:"page_sizes,omitempty"`
}

DocumentInfo is the metadata the meta subcommand reports. Reading it does not run the conversion pipeline.

func Meta 

func Meta(ctx context.Context, r io.ReaderAt, size int64) (*DocumentInfo, error)

Meta reads document metadata without running the conversion pipeline.

Example

Read metadata without converting.

Meta parses only the trailer and page tree, so it is cheap enough to run across a whole library to build an index. 

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/sroberts/decant"
)

// openPDF is the boilerplate every example needs: decant reads through
// io.ReaderAt and needs the size, so a caller passes both.
func openPDF(path string) (*os.File, int64, error) {
	f, err := os.Open(path)
	if err != nil {
		return nil, 0, err
	}
	st, err := f.Stat()
	if err != nil {
		f.Close()
		return nil, 0, err
	}
	return f, st.Size(), nil
}

func main() {
	in, size, err := openPDF("book.pdf")
	if err != nil {
		log.Fatal(err)
	}
	defer in.Close()

	info, err := decant.Meta(context.Background(), in, size)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s by %s, %d pages\n", info.Title, info.Author, info.PageCount)
}

type EncryptedError 

type EncryptedError struct {
	// Handler names the security handler, e.g. "Standard".
	Handler string
	// Revision is the /R value identifying the algorithm generation.
	Revision int
}

EncryptedError reports a PDF carrying an /Encrypt dictionary. Spec section 1 puts decryption out of scope for v1; the CLI maps this to exit code 3.

func (*EncryptedError) Error 

func (e *EncryptedError) Error() string

type Heuristics 

type Heuristics struct {
	// BaselineTolerance is the fraction of median glyph height within which
	// glyphs share a baseline. Default 0.3.
	BaselineTolerance float64

	// SpaceGapRatio is the fraction of a font's space width that an
	// inter-glyph gap must exceed to become a space. Default 0.25.
	SpaceGapRatio float64

	// RotationTolerance is the maximum baseline angle in degrees still
	// treated as horizontal text. Default 5.
	RotationTolerance float64

	// KeepRotated retains rotated runs rather than dropping them with a
	// warning. Default false.
	KeepRotated bool

	// BlockOverlapRatio is the horizontal overlap, as a fraction of the
	// narrower line, required to merge a line into a block. Default 0.5.
	BlockOverlapRatio float64

	// BlockGapRatio is the multiple of running median leading beyond which a
	// vertical gap breaks a block. Default 1.5.
	BlockGapRatio float64

	// BlockSizeChangeRatio is the relative font size change that breaks a
	// block. Default 0.15.
	BlockSizeChangeRatio float64

	// ParagraphGapRatio is the fractional overshoot of the running leading
	// that starts a new paragraph. Default 0.25.
	ParagraphGapRatio float64

	// ParagraphIndentEm is the indent above the block median, in em, that
	// starts a new paragraph. Default 0.5.
	ParagraphIndentEm float64

	// ShortLineRatio is the fraction of block width below which a line
	// ending in terminal punctuation ends a paragraph. Default 0.8.
	ShortLineRatio float64

	// ScanMedianGlyphs is the median glyphs per page below which a document
	// is a candidate for the scanned classifier. Default 20.
	ScanMedianGlyphs int

	// ScanImagePageRatio is the fraction of sampled pages that must carry
	// page-covering images to confirm a scan. Default 0.8.
	ScanImagePageRatio float64

	// ScanSamplePages is how many pages the scanned classifier samples.
	// Default 20.
	ScanSamplePages int

	// ScanImageCoverRatio is the share of a page images must cover for it to
	// count as image-covered by the scanned classifier. Spec 6 describes
	// "full-page images"; 0.8 allows for the margins a scan usually keeps.
	ScanImageCoverRatio float64

	// MaxColumns caps automatic column detection. Default 3.
	MaxColumns int

	// GutterMinWidthSpaces is how many median space widths a whitespace band
	// must span to count as a column gutter. Default 2.
	GutterMinWidthSpaces float64

	// GutterMinHeightRatio is the fraction of text-carrying rows a band must
	// be empty across to count as a gutter. Default 0.6.
	GutterMinHeightRatio float64

	// ColumnMinGlyphRatio is the minimum share of a page's glyphs each
	// detected column must hold for the split to be believed. Default 0.1.
	//
	// Not in the spec. Section 4.4 notes the column heuristic misfires on
	// tables and figures; this guard rejects a split that would leave a
	// column nearly empty.
	ColumnMinGlyphRatio float64

	// ColumnMinRows is the number of text-carrying rows a page needs before
	// its projection profile is trusted at all. Default 8.
	//
	// Not in the spec, and the guard that matters most in practice. Asking
	// whether a band is empty across 60% of rows is meaningless on a page
	// with four rows, so title pages and figure pages otherwise produce
	// phantom gutters.
	ColumnMinRows int

	// ColumnMinLines is the number of assembled lines each detected column
	// must contain for the split to survive. Default 3.
	//
	// Not in the spec. Checked after lines are split at the gutters, which is
	// stronger evidence than the glyph share.
	ColumnMinLines int

	// HeadingSizeRatio is how far a block's font size must exceed the body
	// font to be a heading. Default 0.15, meaning 15% larger.
	HeadingSizeRatio float64

	// HeadingBoldMaxWords is the word count below which a bold block with no
	// terminal punctuation is a heading. Default 15.
	HeadingBoldMaxWords int

	// HeadingMaxWords caps the size-based heading test. Default 50.
	//
	// Not in the spec. Section 4.6 makes size alone sufficient, which would
	// turn a long epigraph or pull quote set slightly large into a heading
	// and split the book at it.
	HeadingMaxWords int

	// BackgroundCoverRatio is the share of the page an image must cover,
	// when painted beneath the text, to be dropped as a background or
	// watermark. Default 0.95.
	BackgroundCoverRatio float64

	// MinImagePoints is the smallest edge an image may have before it is
	// dropped. Default 16.
	MinImagePoints float64

	// MinImageAreaRatio is the share of page area below which an image is
	// dropped. Default 0.02.
	MinImageAreaRatio float64

	// InlineImageWidthRatio is the fraction of the text column width below
	// which an image inside a paragraph flows inline. Default 0.4.
	InlineImageWidthRatio float64

	// CaptionGapLines is how many line heights a caption may sit from its
	// figure. Default 1.5.
	CaptionGapLines float64

	// CaptionSizeRatio is how far below the body font a caption is set.
	// Default 0.05.
	CaptionSizeRatio float64

	// CaptionOverlapRatio is the horizontal overlap a block must share with
	// an image to be treated as its caption. Default 0.3.
	//
	// Not in the spec: without it a sidebar level with a figure binds to it.
	CaptionOverlapRatio float64

	// FurnitureBandRatio is the fraction of page height at the top and bottom
	// in which running heads and folios live. Default 0.08.
	FurnitureBandRatio float64

	// FurnitureRepeatRatio is the fraction of sampled pages a block must
	// repeat on to be removed as furniture. Default 0.6.
	FurnitureRepeatRatio float64

	// FurnitureSamplePages is how many pages the furniture sampler examines.
	// Default 20.
	FurnitureSamplePages int

	// FurnitureMinPages is the document length below which furniture removal
	// is skipped. Default 5.
	FurnitureMinPages int

	// QuoteIndentEm is how far both margins must be inset beyond the body, in
	// em, for a blockquote. Default 1.5.
	QuoteIndentEm float64

	// FootnoteBandRatio is the fraction of page height, from the bottom, in
	// which a footnote may sit. Default 0.2.
	FootnoteBandRatio float64

	// FootnoteSizeRatio is how far below the body font a footnote is set.
	// Default 0.1.
	FootnoteSizeRatio float64

	// SuperscriptRiseEm is the baseline offset, as a fraction of em, above
	// which a glyph counts as a superscript. Default 0.2.
	SuperscriptRiseEm float64

	// SuperscriptSizeRatio is the size ratio below which a raised glyph
	// counts as a superscript. Default 0.85.
	SuperscriptSizeRatio float64

	// RuleMaxThickness is the stroke width above which a painted segment is a
	// bar rather than a ruling line. Default 2.
	RuleMaxThickness float64

	// RuleClusterTolerance is how far apart two rules may sit and still count
	// as the same table boundary. Default 2.
	RuleClusterTolerance float64

	// RuleRowCoverRatio is the fraction of a row's height a vertical rule
	// must span to separate its cells. Default 0.6.
	RuleRowCoverRatio float64

	// TableRegionGap is the vertical gap between ruling lines beyond which
	// they belong to separate tables. Default 36.
	TableRegionGap float64

	// TableColumnTolerance is how close two column starts must be to count as
	// the same boundary. Default 2.
	TableColumnTolerance float64

	// TableMinSharedColumns is how many boundaries every row must share for
	// the alignment signal to fire. Default 2.
	TableMinSharedColumns int

	// TableMinRows is the number of consecutive tabulated lines the alignment
	// signal needs. Default 3.
	TableMinRows int

	// TableMinFilledRatio is the fraction of a ruled grid's cells that must
	// carry text for it to be a table. Default 0.5.
	//
	// Not in the spec. Diagrams draw axis-aligned lines that form apparent
	// grids; without this guard a mathematics textbook yields phantom tables.
	TableMinFilledRatio float64

	// StyleMinLetters is the shortest bold or italic run emitted as emphasis,
	// in letters. Default 2.
	//
	// Not in spec section 4.6, and a guard against its rule misfiring. A
	// single italic letter mid-sentence is a variable or a symbol rather than
	// emphasis, and a mathematics document is made of them.
	StyleMinLetters int
	// RTLLetterRatio is the fraction of a document's letters that must belong
	// to a right-to-left script before spec section 1's scope warning fires.
	// Default 0.2.
	//
	// A ratio rather than any occurrence at all: a Latin book quoting a line
	// of Hebrew is not a bidirectional document, and warning on it would
	// train the reader to ignore the warning that matters.
	RTLLetterRatio float64
	// VectorMinPaints is the number of painted paths a page must carry before
	// its vector artwork is reported as dropped content. Default 24.
	//
	// Not in the spec. Almost every PDF paints some paths for rules,
	// underlines, table borders, and form fields; reporting those as lost
	// artwork would be noise a reader cannot act on. A page actually drawing
	// a diagram paints far more, and the distributions do not overlap: across
	// the sample corpus incidental decoration runs from 7 to 20 paths per
	// page, while a mathematics textbook full of geometry figures paints
	// about forty. The default sits in that gap.
	//
	// The bias is deliberate. Reporting a form border as lost artwork is
	// noise; missing a dropped chart is the silent content loss this
	// diagnostic exists to end, so the threshold is set to catch artwork
	// rather than to catch every path.
	VectorMinPaints int
}

Heuristics holds every tunable threshold from spec section 4. Each field documents its default; DefaultHeuristics returns them all.

Spec principle 5 requires these to be inspectable as well as tunable, which is what the probe subcommand dumps.

Example

Tune a heuristic.

Every threshold decant infers structure from is exposed and documented. This one raises how far a block must exceed the body font to read as a heading, from the default 15% to 30%, which suppresses false headings in a document set slightly large throughout. 

package main

import (
	"log"

	"github.com/sroberts/decant"
)

func main() {
	opts := decant.DefaultOptions()
	opts.Heuristics = decant.DefaultHeuristics()
	opts.Heuristics.HeadingSizeRatio = 0.30

	conv, err := decant.New(opts)
	if err != nil {
		log.Fatal(err)
	}
	_ = conv
}

func DefaultHeuristics 

func DefaultHeuristics() Heuristics

DefaultHeuristics returns the documented defaults from spec section 4.

type HyphenDecision 

type HyphenDecision struct {
	Left    string `json:"left"`
	Right   string `json:"right"`
	Dropped bool   `json:"dropped"`
	Reason  string `json:"reason"`
}

HyphenDecision records one line-break hyphen and the reasoning behind it.

type HyphenationReport 

type HyphenationReport struct {
	// Language is the pattern set that was used, empty when dehyphenation
	// was disabled.
	Language string `json:"language,omitempty"`
	// Patterns is the number of patterns in that set.
	Patterns int `json:"patterns,omitempty"`
	// Dropped counts hyphens removed as typesetting artifacts, Kept counts
	// those judged lexical.
	Dropped int `json:"dropped"`
	Kept    int `json:"kept"`
	// Decisions is a bounded sample of the individual calls.
	Decisions []HyphenDecision `json:"decisions,omitempty"`
}

HyphenationReport summarizes what dehyphenation did.

Spec section 4.6 asks for every decision to be recorded. A full-length book makes thousands, so the counts cover all of them and Decisions carries a bounded sample; the full trail would make a report file unreadable.

type Image 

type Image struct {
	// ID is the manifest id and the basename of the file, e.g. "img001".
	ID string
	// Data is the encoded bytes.
	Data []byte
	// MediaType is the manifest media type.
	MediaType string
	// Ext is the filename extension without a dot.
	Ext string
	// Width and Height are the final pixel dimensions.
	Width, Height int
	// Passthrough reports that the original JPEG bytes were kept unmodified.
	Passthrough bool
}

Image is one picture carried into the EPUB.

func (Image) Href 

func (i Image) Href() string

Href returns the image's path relative to the OEBPS directory.

type ImageMode 

type ImageMode string

ImageMode selects how images are handled. 

const (
	// ImagesKeep retains images in their original color.
	ImagesKeep ImageMode = "keep"
	// ImagesGrayscale converts images to grayscale.
	ImagesGrayscale ImageMode = "grayscale"
	// ImagesDrop discards images.
	ImagesDrop ImageMode = "drop"
)

func (ImageMode) Valid 

func (m ImageMode) Valid() bool

Valid reports whether m is a known image mode.

type MalformedError 

type MalformedError struct {
	Detail string
	Err    error
}

MalformedError reports a PDF damaged beyond what xref reconstruction can recover. The CLI maps this to exit code 6.

func (*MalformedError) Error 

func (e *MalformedError) Error() string

func (*MalformedError) Unwrap 

func (e *MalformedError) Unwrap() error

type Metadata 

type Metadata struct {
	Title    string
	Author   string
	Language string
}

Metadata overrides Dublin Core values otherwise taken from the PDF.

type NoTextLayerError 

type NoTextLayerError struct {
	// MedianGlyphs is the median glyph count across sampled pages.
	MedianGlyphs float64
	// SampledPages is how many pages were examined.
	SampledPages int
	// ImagePageFraction is the fraction of sampled pages covered by
	// page-scale images.
	ImagePageFraction float64
}

NoTextLayerError reports a scanned document. decant does not OCR; spec section 6 fails fast and points at an external OCR pass. The CLI maps this to exit code 4.

func (*NoTextLayerError) Error 

func (e *NoTextLayerError) Error() string

type Options 

type Options struct {
	Profile    Profile
	Metadata   Metadata
	Pages      PageRange
	Heuristics Heuristics

	// SplitAt selects chapter boundaries. Default SplitAtH1.
	SplitAt SplitMode
	// MaxChunkBytes forces a split of oversized XHTML at a paragraph
	// boundary. Default 262144; the minimal profile lowers it to 65536.
	//
	// The crosspoint profile keeps 262144: its firmware streams XHTML rather
	// than parsing it into memory, so chapter size is not a memory constraint
	// there. See ApplyProfileDefaults.
	MaxChunkBytes int

	// KeepHeaders retains running heads and folios. Default false.
	KeepHeaders bool
	// NoDehyphenate preserves line-break hyphens verbatim.
	NoDehyphenate bool

	// Columns forces a column count. Zero detects from the page's projection
	// profile; 1, 2, or 3 override it. Spec 3: --columns.
	Columns int

	// Images selects image handling. Default ImagesKeep.
	Images ImageMode
	// KeepSmallImages retains images the size rules in spec 4.7 would drop.
	KeepSmallImages bool

	// Tables selects how detected tables are emitted. Default TableAuto; the
	// crosspoint and minimal profiles lower it to TableText.
	Tables TableMode
	// ImageMaxWidth is the longest edge in pixels; 0 disables scaling.
	ImageMaxWidth int

	// Deterministic fixes the output timestamp. When zero, the PDF ModDate
	// is used, then the Unix epoch.
	Deterministic time.Time

	// Strict makes quality threshold breaches an error at the CLI layer.
	Strict bool
}

Options configures a Converter. The zero value is not usable; start from DefaultOptions.

func DefaultOptions 

func DefaultOptions() Options

DefaultOptions returns the documented defaults from spec section 3.

func (*Options) ApplyProfileDefaults 

func (o *Options) ApplyProfileDefaults()

ApplyProfileDefaults overwrites the image, chunk-size, and related fields with the device profile defaults from spec section 5.

It overwrites unconditionally, so a caller that wants an explicit value to win must re-apply it afterward. The CLI does exactly that, using flag.Visit to learn which flags the user actually passed.

Example

Target a constrained reading device.

A profile sets image handling, chunk size, and table mode together. ApplyProfileDefaults overwrites those fields unconditionally, so set it before any value you want to keep. 

package main

import (
	"fmt"
	"log"

	"github.com/sroberts/decant"
)

func main() {
	opts := decant.DefaultOptions()
	opts.Profile = decant.ProfileCrossPoint
	opts.ApplyProfileDefaults()

	// Overrides go after, or the profile wins.
	opts.ImageMaxWidth = 800

	conv, err := decant.New(opts)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(conv.Options().Tables)
}

func (*Options) ApplyProfileDoc added in v1.1.0 

func (o *Options) ApplyProfileDoc(d *ProfileDoc) error

ApplyProfileDoc layers a document's overrides onto o.

It applies Options and Heuristics only. The document's Base is advisory and the caller chooses the starting point, because the caller is the only one that knows whether a profile was also named explicitly: 

o := DefaultOptions()
o.Profile = doc.Base          // or an explicit choice, which wins
o.ApplyProfileDefaults()
err := o.ApplyProfileDoc(doc)

That order gives defaults, then base profile, then document, then whatever the caller sets afterwards.

type OutlineItem 

type OutlineItem struct {
	Title string
	// Page is the zero-based destination page, or -1 when unresolved.
	Page int
	// Y is the destination's vertical position in PDF user space, or NaN when
	// the destination specifies none. It is not page space: user space runs
	// y-up from the crop box's lower-left corner.
	Y        float64
	Children []OutlineItem
}

OutlineItem is one node of the source PDF's bookmark tree.

type PageMetrics 

type PageMetrics struct {
	Page int `json:"page"`
	// Glyphs is the number of glyphs extracted after invisible-text
	// filtering.
	Glyphs int `json:"glyphs"`
	// DecodeFailures counts glyphs that mapped to U+FFFD.
	DecodeFailures int `json:"decode_failures"`
	// Lines and Blocks count the stage 3 and stage 4 output.
	Lines  int `json:"lines"`
	Blocks int `json:"blocks"`
	// Columns is the number of text columns detected on the page.
	Columns int `json:"columns"`
	// Images is the number of images placed from the page, after the drop
	// rules in spec section 4.7.
	Images int `json:"images"`
	// Tables counts tables detected on the page.
	Tables int `json:"tables,omitempty"`
	// VectorPaints counts painted path operations on the page. decant does
	// not render vector artwork, so a high count means content was lost.
	VectorPaints int `json:"vector_paints,omitempty"`
	// RotatedDropped counts rotated runs discarded.
	RotatedDropped int `json:"rotated_dropped"`
	// Letters and RTLLetters count the page's letters and how many of them
	// belong to a right-to-left script. Spec section 1 puts bidirectional
	// layout out of scope and asks for it to be detected.
	Letters    int `json:"letters,omitempty"`
	RTLLetters int `json:"rtl_letters,omitempty"`
	// VerticalText marks a page using a font with a vertical writing mode.
	VerticalText bool `json:"vertical_text,omitempty"`
	// UsedInvisibleText marks a page whose only text was a mode-3 layer,
	// which is the searchable-scan case.
	UsedInvisibleText bool `json:"used_invisible_text"`
}

PageMetrics holds the per-page numbers the report surfaces.

func (PageMetrics) DecodeFailureRate 

func (m PageMetrics) DecodeFailureRate() float64

DecodeFailureRate returns failures as a fraction of glyphs.

type PageRange 

type PageRange struct {
	// contains filtered or unexported fields
}

PageRange selects a subset of pages. The zero value selects every page.

func ParsePageRange 

func ParsePageRange(s string) (PageRange, error)

ParsePageRange parses the CLI form, e.g. "5-200,210". Page numbers are one-based in the text form and stored zero-based.

func (PageRange) All 

func (p PageRange) All() bool

All reports whether the range selects every page.

func (PageRange) Contains 

func (p PageRange) Contains(i int) bool

Contains reports whether the zero-based page index is selected.

func (PageRange) String 

func (p PageRange) String() string

String renders the range in the CLI form.

type PageSize 

type PageSize struct {
	Width  float64 `json:"width"`
	Height float64 `json:"height"`
	Rotate int     `json:"rotate"`
	Pages  int     `json:"pages"`
}

PageSize is one distinct page geometry and how many pages share it.

type ProbeBlock 

type ProbeBlock struct {
	Bounds     Rect     `json:"bounds"`
	Lines      int      `json:"lines"`
	Kind       string   `json:"kind,omitempty"`
	Level      int      `json:"level,omitempty"`
	Paragraphs []string `json:"paragraphs"`
}

ProbeBlock is one segmented block with its paragraphs.

type ProbeGlyph 

type ProbeGlyph struct {
	Rune     string  `json:"rune"`
	X        float64 `json:"x"`
	Y        float64 `json:"y"`
	Advance  float64 `json:"advance"`
	Size     float64 `json:"size"`
	Rise     float64 `json:"rise,omitempty"`
	Rotation float64 `json:"rotation,omitempty"`
	Font     string  `json:"font,omitempty"`
	Mode     int     `json:"render_mode,omitempty"`
	Missing  bool    `json:"missing,omitempty"`
}

ProbeGlyph is one positioned character.

type ProbeLine 

type ProbeLine struct {
	Text     string  `json:"text"`
	Baseline float64 `json:"baseline"`
	Bounds   Rect    `json:"bounds"`
	Size     float64 `json:"size"`
	Font     string  `json:"font,omitempty"`
	Glyphs   int     `json:"glyphs"`
}

ProbeLine is one assembled line.

type ProbePage 

type ProbePage struct {
	Page   int     `json:"page"`
	Width  float64 `json:"width"`
	Height float64 `json:"height"`
	Rotate int     `json:"rotate"`

	Glyphs []ProbeGlyph `json:"glyphs,omitempty"`
	Lines  []ProbeLine  `json:"lines,omitempty"`
	Blocks []ProbeBlock `json:"blocks,omitempty"`
}

ProbePage is one page of probe output.

type ProbeResult 

type ProbeResult struct {
	Stage ProbeStage  `json:"stage"`
	Pages []ProbePage `json:"pages"`
	Notes []string    `json:"notes,omitempty"`
}

ProbeResult is the intermediate model dump, per spec principle 5.

type ProbeStage 

type ProbeStage string

ProbeStage names an inspectable point in the pipeline. 

const (
	// StageGlyphs dumps positioned glyphs straight out of the content stream
	// interpreter.
	StageGlyphs ProbeStage = "glyphs"
	// StageLines dumps assembled lines.
	StageLines ProbeStage = "lines"
	// StageBlocks dumps segmented blocks.
	StageBlocks ProbeStage = "blocks"
	// StageStructure dumps classified blocks with their kinds and levels.
	StageStructure ProbeStage = "structure"
)

func (ProbeStage) Valid 

func (s ProbeStage) Valid() bool

Valid reports whether s names a known stage.

type Profile 

type Profile string

Profile constrains output for a class of reading device. See spec section 5. 

const (
	// ProfileStandard is the unconstrained default.
	ProfileStandard Profile = "standard"
	// ProfileCrossPoint targets the Xteink X4 running CrossPoint firmware:
	// a 480x800 E Ink panel on an ESP32-C3 with roughly 380 KB of usable RAM.
	ProfileCrossPoint Profile = "crosspoint"
	// ProfileMinimal drops images and stylesheets entirely.
	ProfileMinimal Profile = "minimal"
)

func (Profile) Valid 

func (p Profile) Valid() bool

Valid reports whether p is a known profile.

type ProfileDoc added in v1.1.0 

type ProfileDoc struct {
	// Name identifies the profile. It is not matched against anything and
	// exists so a file says what it is.
	Name string `json:"name"`
	// Description is free text for whoever reads the file next.
	Description string `json:"description,omitempty"`

	// Base names the built-in profile to start from. Empty means standard.
	Base Profile `json:"base,omitempty"`

	// Options overrides fields of [Options]; Heuristics overrides fields of
	// [Heuristics]. Both are applied over the base profile's defaults.
	Options    json.RawMessage `json:"options,omitempty"`
	Heuristics json.RawMessage `json:"heuristics,omitempty"`
}

ProfileDoc is a device profile in serializable form, so one can be written once and shared rather than compiled in.

The three built-in profiles in spec section 5 stay compiled in, because their values encode findings this repository is responsible for: the crosspoint numbers come from reading the firmware. A document extends that set for a device decant has never seen, without a rebuild.

Keys under Options and Heuristics are the field names of Options and Heuristics, matched without regard to case. That is deliberate: the field documentation is then the format documentation, and `go doc decant.Options` is the reference. Unknown keys are an error rather than a silent no-op, since a typo in a threshold would otherwise look like a threshold that did nothing.

A field the document omits keeps the value the base profile gave it, so a document states only what it changes.

func LoadProfileDoc added in v1.1.0 

func LoadProfileDoc(r io.Reader) (*ProfileDoc, error)

LoadProfileDoc reads a profile document.

Only the document's own shape is checked here. Whether its values make a usable option set is decided by Options.ApplyProfileDoc and New.

type Rect 

type Rect struct {
	MinX, MinY, MaxX, MaxY float64
}

Rect is an axis-aligned rectangle in page space, with y increasing downward from the top-left of the page.

func (Rect) Height 

func (r Rect) Height() float64

Height returns the vertical extent.

func (Rect) Width 

func (r Rect) Width() float64

Width returns the horizontal extent.

type Report 

type Report struct {
	// Source is the input filename.
	Source string `json:"source"`
	// PageCount is the source document's page count.
	PageCount int `json:"page_count"`
	// PagesConverted is how many pages the page range selected.
	PagesConverted int `json:"pages_converted"`

	Pages []PageMetrics `json:"pages"`

	// Blocks counts blocks by kind.
	Blocks map[BlockKind]int `json:"blocks"`
	// Headings counts headings by level, indexed 1 through 6.
	Headings map[int]int `json:"headings,omitempty"`
	// BodyFont describes the document's computed body font, which every
	// structure decision in spec section 4.6 is measured against.
	BodyFont string `json:"body_font,omitempty"`
	// MultiColumnPages counts pages where more than one column was detected.
	MultiColumnPages int `json:"multi_column_pages"`
	// ImagesPlaced counts distinct images carried into the EPUB, after
	// deduplication.
	ImagesPlaced int `json:"images_placed"`
	// ImageBytes is the total encoded size of those images.
	ImageBytes int `json:"image_bytes"`
	// Hyphenation summarizes the dehyphenation decisions in spec 4.6.
	Hyphenation HyphenationReport `json:"hyphenation"`
	// Tables counts detected tables by confidence, which is what
	// --table-mode=auto keys on.
	Tables map[string]int `json:"tables,omitempty"`
	// RTLLetterRatio is the fraction of the document's letters belonging to a
	// right-to-left script, and VerticalTextPages counts pages using a
	// vertical writing mode. Spec section 1 puts both out of scope beyond
	// basic text extraction.
	RTLLetterRatio    float64 `json:"rtl_letter_ratio,omitempty"`
	VerticalTextPages int     `json:"vertical_text_pages,omitempty"`
	// FurnitureRemoved counts blocks dropped as running heads or folios.
	FurnitureRemoved int `json:"furniture_removed"`
	// VectorPagesDropped counts pages carrying vector artwork that decant did
	// not render, and VectorPaintsDropped the painted paths on them.
	//
	// Spec section 1 puts vector conversion out of scope for v1 and section
	// 13 keeps rasterization open. Reporting the loss is what principle 3
	// requires in the meantime: a chart drawn as paths otherwise disappears
	// with no trace in the output or the report.
	VectorPagesDropped  int `json:"vector_pages_dropped,omitempty"`
	VectorPaintsDropped int `json:"vector_paints_dropped,omitempty"`
	// Chapters is the number of XHTML files written.
	Chapters int `json:"chapters"`
	// OutputBytes is the size of the EPUB.
	OutputBytes int64 `json:"output_bytes"`
	// LargestChapterBytes is the biggest single XHTML document, which is the
	// dominant failure mode on the crosspoint profile.
	LargestChapterBytes int `json:"largest_chapter_bytes"`

	Diagnostics []Diagnostic `json:"diagnostics"`

	// QualityScore is a 0 to 100 summary. See Finish for its derivation.
	QualityScore int `json:"quality_score"`
}

Report describes one conversion. It is written as JSON by --report and surfaced by the CrossPoint TUI to flag conversions worth reviewing.

Example

Act on what the conversion reported.

A warning does not mean failure: the EPUB is still valid. It means a heuristic fired somewhere worth reviewing. 

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/sroberts/decant"
)

// openPDF is the boilerplate every example needs: decant reads through
// io.ReaderAt and needs the size, so a caller passes both.
func openPDF(path string) (*os.File, int64, error) {
	f, err := os.Open(path)
	if err != nil {
		return nil, 0, err
	}
	st, err := f.Stat()
	if err != nil {
		f.Close()
		return nil, 0, err
	}
	return f, st.Size(), nil
}

func main() {
	in, size, err := openPDF("book.pdf")
	if err != nil {
		log.Fatal(err)
	}
	defer in.Close()

	conv, err := decant.New(decant.DefaultOptions())
	if err != nil {
		log.Fatal(err)
	}

	out, err := os.Create("book.epub")
	if err != nil {
		log.Fatal(err)
	}
	defer out.Close()

	rep, err := conv.Convert(context.Background(), in, size, out)
	if err != nil {
		log.Fatal(err)
	}

	if rep.QualityScore < 80 {
		fmt.Printf("quality %d/100, worth reviewing\n", rep.QualityScore)
	}
	for _, d := range rep.Diagnostics {
		if d.Severity == decant.SeverityWarning {
			fmt.Printf("%s: %s\n", d.Stage, d.Message)
		}
	}
}

func (*Report) DecodeFailureRate 

func (r *Report) DecodeFailureRate() float64

DecodeFailureRate returns the document-wide rate.

func (*Report) Finish 

func (r *Report) Finish()

Finish computes the quality score. It runs at the end of Write.

The score starts at 100 and subtracts for the failure modes that most often indicate output worth reviewing by hand. It is a triage signal, not a measurement: the TUI uses it to decide which conversions to surface.

func (*Report) MedianGlyphsPerPage 

func (r *Report) MedianGlyphsPerPage() float64

MedianGlyphsPerPage returns the median across converted pages, which the scanned-document classifier in spec section 6 keys on.

func (*Report) Warnings 

func (r *Report) Warnings() int

Warnings returns the number of warning-level diagnostics, which --strict turns into a non-zero exit.

type Severity 

type Severity string

Severity ranks a diagnostic. 

const (
	// SeverityInfo records a decision worth auditing but not worrying about.
	SeverityInfo Severity = "info"
	// SeverityWarning records degraded output. Under --strict these make the
	// run exit non-zero.
	SeverityWarning Severity = "warning"
)

type SplitMode 

type SplitMode string

SplitMode selects where chapter files break. 

const (
	// SplitAtH1 breaks at top-level headings. This is the default.
	SplitAtH1 SplitMode = "h1"
	// SplitAtH2 breaks at second-level headings.
	SplitAtH2 SplitMode = "h2"
	// SplitAtPage breaks at every source page.
	SplitAtPage SplitMode = "page"
	// SplitAtNone emits one chapter, subject to MaxChunkBytes.
	SplitAtNone SplitMode = "none"
)

func (SplitMode) Valid 

func (s SplitMode) Valid() bool

Valid reports whether s is a known split mode.

type StyleRun 

type StyleRun struct {
	// Start and End are byte offsets into the owning block's Text, half-open.
	Start, End int
	// Bold and Italic are derived from the font's /FontDescriptor flags and
	// its family-name suffix, per spec section 4.6.
	Bold, Italic bool
}

Block is one unit of reconstructed content in reading order.

Document exposes these so a caller can correct structure before serialization; the CrossPoint TUI uses that to let a reader fix heading levels. Mutating Kind, Level, or Text is supported. Page and Bounds are provenance and should be left alone. StyleRun is a range of a block's text set in bold, italic, or both.

A caller may drop or narrow a run the same way it may edit any other field of the block tree; the renderer emits strong and em from these alone.

type TableCell 

type TableCell struct {
	Text string
	// ColSpan is 1 unless the cell spans columns, which happens where a
	// vertical ruling line is absent between two boundaries.
	ColSpan int
}

TableCell is one cell of a detected table.

type TableMode 

type TableMode string

TableMode selects how a detected table is emitted, per spec section 4.8. 

const (
	// TableAuto picks by detection confidence: a real table when both
	// signals fire, a rasterized region at medium confidence, and
	// space-preserved text at low.
	TableAuto TableMode = "auto"
	// TableHTML always emits a table element.
	TableHTML TableMode = "html"
	// TableText emits space-preserved text inside a pre element.
	TableText TableMode = "text"
	// TableDrop discards detected tables, leaving their text as paragraphs.
	TableDrop TableMode = "drop"
)

func (TableMode) Valid 

func (m TableMode) Valid() bool

Valid reports whether m is a known table mode.

type UsageError 

type UsageError struct{ Err error }

UsageError reports an invalid option combination. The CLI maps this to exit code 2.

func (*UsageError) Error 

func (e *UsageError) Error() string

func (*UsageError) Unwrap 

func (e *UsageError) Unwrap() error

Source Files

View all Source files

Directories

Path Synopsis
cmd
decant command
Command decant converts fixed-layout PDF to reflowable EPUB 3.
 Command decant converts fixed-layout PDF to reflowable EPUB 3.
internal
epub
Package epub serializes an EPUB 3.3 container with an EPUB 2 NCX fallback.
 Package epub serializes an EPUB 3.3 container with an EPUB 2 NCX fallback.
hyphen
Package hyphen implements Liang's hyphenation algorithm over TeX pattern sets, which spec section 4.6 uses inverted to decide whether a line-break hyphen is a typesetting artifact or part of the word.
 Package hyphen implements Liang's hyphenation algorithm over TeX pattern sets, which spec section 4.6 uses inverted to decide whether a line-break hyphen is a typesetting artifact or part of the word.
images
Package images re-encodes extracted PDF images for EPUB delivery.
 Package images re-encodes extracted PDF images for EPUB delivery.
layout
Package layout turns positioned glyphs into lines, blocks, and structured content.
 Package layout turns positioned glyphs into lines, blocks, and structured content.
pdf
testpdf
Package testpdf builds small, valid PDFs in memory for tests.
 Package testpdf builds small, valid PDFs in memory for tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.