Documentation
¶
Overview ¶
Package extract turns a page's content stream into positioned text.
This is the use case the rest of the repo exists to serve, and the reason it is its own package is the failure it has to avoid. Every column of the benchmark in docs/DESIGN.md §1 — 0.01% spaces, a 4,069-character "word", 6% of words over 25 characters — is one bug: PDF does not record words, only glyphs and the positions they are painted at, so a reader that does not reconstruct the gaps between them produces text that is character-for-character correct and unreadable.
Reconstruction needs three things at once, and no two of them live in the same package: the glyph's advance width (font), the composed text rendering matrix (content), and one threshold policy for deciding when a gap is a space (geom.Tolerance). This package is where they meet. It owns no parsing and no tables of its own.
What it produces is doc.Page — the same type the OCR path produces — so a scanned page and a born-digital one reach the sinks the same way.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var DefaultOptions = Options{Tol: geom.DefaultTolerance, KeepHidden: true}
DefaultOptions is extraction as the CLI runs it: artifacts dropped, hidden text kept, default tolerances.
Functions ¶
This section is empty.
Types ¶
type Extractor ¶
type Extractor struct {
// contains filtered or unexported fields
}
Extractor reads pages from one document.
It is not safe for concurrent use, because objects.Store makes no such promise and the font cache is unsynchronized. Page-level parallelism uses one Extractor per worker over its own Store, which is what the --jobs flag will do.
func New ¶
New returns an Extractor over s. The Store is borrowed, not owned: closing it remains the caller's job, because the caller opened it and may still be probing it.
func (*Extractor) Document ¶
Document extracts every page.
A page that fails to extract yields an empty page rather than an error for the document. One malformed page out of a thousand must not cost the other 999, and an empty page in the output is visible where a missing one would silently renumber everything after it.
type Options ¶
type Options struct {
// Tol is the threshold policy. The zero value is replaced by
// geom.DefaultTolerance, because a zero SpaceFrac would infer a space between
// every pair of glyphs.
Tol geom.Tolerance
// KeepArtifacts includes text inside Artifact marked-content regions — running
// headers, folios, watermarks. Off by default: on a 1,023-page specification
// that text is the same header a thousand times, and it interleaves with body
// prose at every page boundary. The blocks are still produced and still
// carry doc.RoleArtifact when this is on, so the judgement stays inspectable
// rather than being made silently at read time.
KeepArtifacts bool
// KeepHidden includes text drawn in rendering mode 3 or 7. On by default,
// because invisible text is the text layer under a scanned page and it is
// exactly what an extractor wants; set false only when comparing against what
// a viewer displays.
KeepHidden bool
}
Options configures extraction.
Source Files