tag

Documentation

Overview

Package tag reads a PDF's logical structure tree: the /StructTreeRoot graph described in ISO 32000-2 §14.7.

This package is why the toolkit can produce sectioned Markdown without layout heuristics. A structure tree states the document's heading hierarchy and reading order outright, and it is not page-scoped, so a section spanning several pages is one contiguous subtree. Where a tree exists, walking it beats inferring structure from font sizes and coordinates.

Untagged files get nothing from this package; they take the layout path.

Index

• type Elem
  • func (e *Elem) Depth() int
  • func (e *Elem) MCIDs() []int
• type MCRef
• type Role
  • func (r Role) HeadingLevel() int
  • func (r Role) IsHeading() bool
• type Stats
• type Tree
  • func Read(s objects.Store) (*Tree, error)
  • func (t *Tree) ResolvePages(s objects.Store) error
  • func (t *Tree) Stats() Stats
  • func (t *Tree) Walk(fn func(e *Elem, depth int) bool)

Types

type Elem

type Elem struct {
	// Role is the type after /RoleMap normalization.
	Role Role

	// RawType is the /S value as written, before normalization. Kept because a
	// custom role can carry document-specific meaning worth preserving.
	RawType Role

	// Title is /T, a human-readable label. Producers of long specifications
	// often put the clause number and heading here.
	Title string

	// Lang is /Lang if present on this element.
	Lang string

	// ActualText is /ActualText: replacement text for the element, used when
	// the glyphs do not spell what the content means (ligatures, artwork).
	ActualText string

	// Alt is /Alt, alternate description, typically on a Figure.
	Alt string

	// Page is the 1-based page this element is anchored to, or 0 when neither it
	// nor any ancestor names one. Structure elements need not be page-scoped,
	// which is exactly what lets a section span pages.
	//
	// Inherited, not just read: §14.7.4.3 says an element's page is its own /Pg
	// or, absent that, its nearest ancestor's. A producer that puts /Pg on a Sect
	// and omits it from every P inside is conformant, and reading only the
	// element's own entry leaves all of them anchored nowhere.
	Page int

	// Content is the marked-content this element owns, in order, each identifier
	// paired with the page it was drawn on. Together they are the join key
	// between the structure tree and the text found in content streams.
	//
	// The page is per reference rather than per element because a marked-content
	// reference may carry its own /Pg, and that is precisely how one paragraph
	// continues across a page break. Rare — 5 of 2,035 references in
	// Well-Tagged-PDF-WTPDF-1.0.pdf, none in ISO 32000-2 — but every one of those
	// 5 names a page other than its element's, so treating the element's page as
	// authoritative loses exactly the content that spans pages.
	Content []MCRef

	// Kids are child elements in logical reading order.
	Kids []*Elem

	// Parent is the enclosing element, nil at the root.
	Parent *Elem
	// contains filtered or unexported fields
}

Elem is one node of the structure tree.

func (*Elem) Depth

func (e *Elem) Depth() int

Depth returns the heading level for a heading element.

H1..H6 report their own level. A bare H per §14.8.4.4 takes its level from nesting: count enclosing grouping elements, since those are what express hierarchy for documents that use H throughout.

func (*Elem) MCIDs

func (e *Elem) MCIDs() []int

MCIDs returns the identifiers in Content, for a caller that only needs to know which marked content an element owns and not where it sits.

type MCRef

type MCRef struct {
	MCID int
	Page int
	// contains filtered or unexported fields
}

MCRef is one marked-content identifier together with the page it was drawn on.

Page is 0 until ResolvePages has run, and stays 0 for a reference whose element chain never names a page — that content cannot be joined to page text, because an MCID is unique only within a page.

type Role

type Role string

Role is a normalized structure element type.

PDF lets a document define arbitrary element names and map them onto the standard set through the catalog's /RoleMap, so the raw /S value cannot be trusted. Callers see normalized roles; the original is kept in Elem.RawType.

const (
	RoleDocument   Role = "Document"
	RolePart       Role = "Part"
	RoleArt        Role = "Art"
	RoleSect       Role = "Sect"
	RoleDiv        Role = "Div"
	RoleH          Role = "H"
	RoleH1         Role = "H1"
	RoleH2         Role = "H2"
	RoleH3         Role = "H3"
	RoleH4         Role = "H4"
	RoleH5         Role = "H5"
	RoleH6         Role = "H6"
	RoleP          Role = "P"
	RoleL          Role = "L"
	RoleLI         Role = "LI"
	RoleLbl        Role = "Lbl"
	RoleLBody      Role = "LBody"
	RoleTable      Role = "Table"
	RoleTR         Role = "TR"
	RoleTH         Role = "TH"
	RoleTD         Role = "TD"
	RoleTHead      Role = "THead"
	RoleTBody      Role = "TBody"
	RoleTFoot      Role = "TFoot"
	RoleSpan       Role = "Span"
	RoleQuote      Role = "Quote"
	RoleNote       Role = "Note"
	RoleCode       Role = "Code"
	RoleFigure     Role = "Figure"
	RoleFormula    Role = "Formula"
	RoleCaption    Role = "Caption"
	RoleTOC        Role = "TOC"
	RoleTOCI       Role = "TOCI"
	RoleLink       Role = "Link"
	RoleArtifact   Role = "Artifact"
	RoleNonStruct  Role = "NonStruct"
	RoleBlockQuote Role = "BlockQuote"
)

Standard structure types from ISO 32000-2 §14.8.4. Only those meaningful to text extraction are named; anything else keeps its raw value.

func (Role) HeadingLevel

func (r Role) HeadingLevel() int

HeadingLevel returns the heading depth for H1..H6, or 0 for anything else.

A bare H is level 0 here, not 1. ISO 32000-2 §14.8.4.4 defines H as a heading whose level comes from nesting depth in the structure hierarchy rather than from its name, so resolving it needs tree context the Role alone lacks. Depth handles it.

func (Role) IsHeading

func (r Role) IsHeading() bool

IsHeading reports whether the role denotes a heading, including a bare H.

type Stats

type Stats struct {
	Elements int
	Headings int
	Paras    int
	Tables   int
	Figures  int
	Lists    int
	MCIDs    int
	MaxDepth int
	Roles    map[Role]int
}

Stats summarizes a tree, for probe output and for confirming that a tree is substantive rather than a near-empty stub some producers emit.

type Tree

type Tree struct {
	Root *Elem

	// RoleMap is the catalog's /RoleMap, custom type to standard type.
	RoleMap map[Role]Role
}

Tree is a document's structure tree.

func Read

func Read(s objects.Store) (*Tree, error)

Read returns the structure tree, or nil with a nil error when the document has none.

An absent tree is the normal case for most PDFs in the wild and is not a failure: it selects the layout path instead. Callers check for nil.

func (*Tree) ResolvePages

func (t *Tree) ResolvePages(s objects.Store) error

ResolvePages fills in Elem.Page and the page of every MCRef.

Separate from Read because it costs a walk of the page tree to build the object-number-to-page-number map, and a caller that only wants the heading outline never needs page numbers. Reading the structure tree of a 1000-page document should not pay for page resolution it will not use.

Pages are inherited: an element with no /Pg takes its nearest ancestor's, and a marked-content reference with no /Pg takes its element's. That is §14.7.4.3, and it is what lets a producer put /Pg once on a Sect instead of on all 300 paragraphs inside it.

func (*Tree) Stats

func (t *Tree) Stats() Stats

Stats walks the tree and counts what it contains.

func (*Tree) Walk

func (t *Tree) Walk(fn func(e *Elem, depth int) bool)

Walk calls fn for every element in logical reading order, depth first. A fn returning false stops the traversal.