pdf

README

pdf

Robust PDF text extraction in pure Go — with glyph positions, hardened against real-world (and hostile) files.

import "github.com/giraffesyo/pdf"

doc, err := pdf.Extract(ctx, readerAt, size)
if err != nil { ... }

text := doc.Text()             // whole document, plain text
page := doc.Pages[0]
line := page.Text()            // per-page reconstruction
glyphs := page.Glyphs          // []Glyph{Text, X, Y, Advance, Size}

Page.Text reconstructs reading order from glyph positions: lines are clustered by baseline, and word boundaries are recovered from glyph gaps and font metrics — so PDFs that encode no space characters at all still come out readable.

Why another PDF text extractor

This package grew out of converting large corpora of real-world documents with ledongthuc/pdf, whose built-in text extraction breaks on files that are common in the wild:

Real-world file	ledongthuc/pdf	this package
Google Docs exports (text inside Form XObjects)	extracts nothing	full text
Word exports (Identity-H fonts + ToUnicode CMaps)	U+FFFD soup	decoded
Type3 fonts with ToUnicode but an /Encoding dict	control-char garbage	decoded
Td/T*-positioned text (most PDFs)	positions lost	positioned
Array token split across /Contents segments	lexer hangs forever	handled
Decompression bombs, stalled filter chains	unbounded CPU/memory	hard budgets
Cyclic page trees	stack overflow (crash)	rejected upfront
Inline images (BI/ID/EI)	derails the lexer	skipped cleanly

It still uses ledongthuc/pdf's object/xref parsing and standard font encodings underneath (aliased internally); everything from the content stream down — lexer, interpreter, text state machine, CTM/Form-XObject handling, ToUnicode parsing, CID widths — is implemented here. Replacing the remaining object layer with a native implementation is on the roadmap.

Undecodable glyphs (no ToUnicode, no standard encoding) are dropped, not emitted as garbage, so image-only or outlined-text PDFs yield empty output you can detect, instead of plausible-looking noise.

Limitations

• No OCR: scanned pages and text converted to vector outlines have no text to extract.
• No layout analysis beyond line/word reconstruction: complex multi-column layouts may interleave.
• Encrypted files are not decrypted.

pdftest

github.com/giraffesyo/pdf/pdftest builds minimal synthetic PDFs for tests (in the spirit of httptest): document skeletons, composite fonts, ToUnicode CMaps, Form XObjects, segmented content streams. This package's own regression suite is built on it — every failure pattern above is reproduced synthetically, with fuzzing on top.

License

MIT. Uses ledongthuc/pdf (BSD-3-Clause).

Documentation

Overview

Package pdf extracts text from PDF files, with glyph positions.

Extract returns per-page positioned glyphs; Page.Text reconstructs plain text by clustering glyphs into lines and inserting word spacing derived from glyph gaps and font metrics.

The package implements its own content-stream lexer, interpreter, and font decoding on top of the object-parsing primitives of github.com/ledongthuc/pdf (BSD-3-Clause). That library's built-in text extraction has gaps that matter on real-world files: it never descends into Form XObjects (losing all text in e.g. Google Docs exports), it ignores /ToUnicode when an /Encoding dictionary is present, it tracks positions only through the Tm operator, its lexer spins forever on tokens that straddle /Contents array segments, and it has no defenses against decompression bombs, stalled filter chains, or cyclic page trees. This package addresses all of the above and is validated against corpora of real-world documents; replacing the remaining object layer with a native implementation is on the roadmap.

Index

• type Document
  • func Extract(ctx context.Context, r io.ReaderAt, size int64) (doc *Document, err error)
  • func (d *Document) Text() string
• type Glyph
• type Page
  • func (p Page) Text() string

Types

type Document

type Document struct {
	Pages []Page
}

Document is the extraction result for a whole file.

func Extract

func Extract(ctx context.Context, r io.ReaderAt, size int64) (doc *Document, err error)

Extract parses the PDF in r and extracts every page's glyphs. Malformed pages yield whatever was decoded before the failure; a malformed file returns an error. The context is checked between pages.

func (*Document) Text

func (d *Document) Text() string

Text reconstructs the whole document's plain text: pages separated by blank lines, empty pages skipped.

type Glyph

type Glyph struct {
	Text    string  // decoded text, non-empty
	X, Y    float64 // origin
	Advance float64 // horizontal advance
	Size    float64 // effective font size
}

Glyph is one decoded glyph (or glyph cluster) with its position in unrotated page space.

type Page

type Page struct {
	Glyphs []Glyph
}

Page holds the glyphs of one page in content order.

func (Page) Text

func (p Page) Text() string

Text reconstructs a page's plain text from glyph positions: glyphs are clustered into lines by Y, ordered by X, and spaces are inserted where the horizontal gap between glyphs is too wide to be kerning.