Documentation
¶
Overview ¶
Package content reads PDF content streams: the token stream, and the graphics and text state machine that turns operators into positioned glyphs.
A content stream is a sequence of operands followed by an operator, in postfix order (ISO 32000-2 §7.8.2). The syntax is the same as the object syntax in §7.3 with two differences: there are no indirect references, and bare keywords are operators rather than errors.
This package is deliberately tolerant. Content streams are the part of a PDF most often malformed, because they are generated by drawing code rather than written by a serializer, and a stream that stops tokenizing at the first oddity loses the rest of the page.
Index ¶
- type GraphicsState
- type Kind
- type Lexer
- type Machine
- func (m *Machine) Advance(tx float64)
- func (m *Machine) AdvanceVertical(ty float64)
- func (m *Machine) Apply(op Op) bool
- func (m *Machine) InArtifact() bool
- func (m *Machine) MCID() int
- func (m *Machine) MarkedTag() Name
- func (m *Machine) NextLine()
- func (m *Machine) RenderMatrix() geom.Matrix
- func (m *Machine) SetMCID(id int)
- func (m *Machine) Visible() bool
- type Name
- type Op
- type Scanner
- type TextState
- type Token
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type GraphicsState ¶
type GraphicsState struct {
CTM geom.Matrix
Text TextState
// ClipDepth counts how many clipping paths are active. It is carried so a
// consumer can tell that text may be clipped away, without this package
// having to evaluate path geometry.
ClipDepth int
}
GraphicsState is the part of the graphics state that affects text placement. The full state includes color, line width, and dash patterns, none of which change where a glyph lands, so none of them are tracked.
type Kind ¶
type Kind uint8
Kind distinguishes the token classes a content stream can hold.
const ( // KindEOF marks the end of the stream. KindEOF Kind = iota // KindObject is a complete direct object: number, string, name, boolean, null. KindObject // KindArrayOpen and the three that follow are structural punctuation. They are // tokens rather than assembled objects because the lexer does not recurse; // assembly is the scanner's job, which keeps nesting depth bounded by the // scanner's explicit limit instead of by the call stack. KindArrayOpen KindArrayClose KindDictOpen KindDictClose // KindOperator is a bare keyword: an operator such as Tj, or a stray token. KindOperator )
type Lexer ¶
type Lexer struct {
// contains filtered or unexported fields
}
Lexer tokenizes a content stream. It holds no state beyond a position, so a caller may record Pos and resume.
func NewLexer ¶
NewLexer returns a Lexer over data. The slice is not copied and must not be modified while lexing.
type Machine ¶
type Machine struct {
// GS is the current graphics state.
GS GraphicsState
// Tm is the text matrix, Tlm the text line matrix. Both are only meaningful
// between BT and ET, and both are reset by BT (§9.4.1).
Tm geom.Matrix
Tlm geom.Matrix
// InText reports whether the machine is inside a BT/ET pair. Text-showing
// operators outside one are malformed but common, and are still processed
// because the text is real.
InText bool
// contains filtered or unexported fields
}
Machine tracks graphics and text state across a content stream.
It is separate from Scanner because the same operator sequence drives both text extraction and image extraction, and because state tracking is where the subtle spec rules live. Keeping it apart from tokenizing means each can be tested against its own failure modes.
func NewMachine ¶
NewMachine returns a Machine initialized for a page whose media box maps to base with the given CTM.
func (*Machine) Advance ¶
Advance moves the text matrix horizontally by tx unscaled text-space units, as showing a glyph does. Callers compute tx from font metrics.
func (*Machine) AdvanceVertical ¶
AdvanceVertical moves the text matrix by ty, for vertical writing mode.
func (*Machine) Apply ¶
Apply updates state for op and reports whether it was a state operator.
Text-showing operators (Tj, TJ, ', ") are not handled here: they need font metrics to advance the text matrix, which this package does not own. A caller handles them and calls Advance with the displacement it computed.
func (*Machine) InArtifact ¶
InArtifact reports whether any enclosing marked-content region is an Artifact. Checking the whole stack, not just the innermost, because an Artifact region can contain nested marked content.
func (*Machine) MCID ¶
MCID returns the innermost marked-content identifier, or -1 when none is active. Innermost rather than outermost: nested BDCs occur, and the innermost is the one the structure tree references.
func (*Machine) MarkedTag ¶
MarkedTag returns the innermost marked-content tag, or "" when none is active. Artifact regions are tagged this way, and are what must be dropped to keep running headers and page numbers out of extracted prose.
func (*Machine) NextLine ¶
func (m *Machine) NextLine()
NextLine moves to the next line using the current leading, as T* does.
func (*Machine) RenderMatrix ¶
RenderMatrix returns the composed matrix mapping text space to device space, per ISO 32000-2 §9.4.4.
The parameter matrix folds in font size, horizontal scaling, and rise, and is composed with the text matrix and then the CTM. Getting this composition wrong is the usual cause of extracted text with plausible characters in an implausible order, because every position and every width derives from it.
func (*Machine) SetMCID ¶
SetMCID overrides the innermost region's MCID. A BDC whose properties are a named resource rather than an inline dictionary needs this, since resolving the name requires the page resources that this package does not hold.
type Name ¶
type Name string
Name is a resource name, kept as a distinct type so a font resource key is not confused with a font's PostScript name.
type Op ¶
Op is one operator with its operands, in the order they appeared.
Operands are only valid until the next call to Scanner.Next, which reuses the backing array. An operator that needs to retain them copies. This saves one slice allocation per operator, which is worth having across the millions an entire document runs, but it is not the whole cost: each operand value is still boxed into an objects.Object, and strings and names each own a buffer. See the note on Scanner.operands.
type Scanner ¶
type Scanner struct {
// contains filtered or unexported fields
}
Scanner groups a content stream's tokens into operators with their operands.
It assembles nested arrays and dictionaries iteratively with an explicit stack, so a stream nesting a million arrays hits maxNestDepth instead of the call stack. Content streams are untrusted input.
func NewScanner ¶
NewScanner returns a Scanner over a decoded content stream.
func (*Scanner) InlineData ¶
InlineData returns the image data of the last BI/ID/EI operator scanned. It is valid until the next call to Next.
type TextState ¶
type TextState struct {
// Font is the resource name from Tf, and Size its operand. Size may be
// negative, which mirrors the glyph, and may be zero, which makes text
// invisible but still advancing.
Font Name
Size float64
// CharSpace is Tc, added to every glyph's advance in unscaled text units.
CharSpace float64
// WordSpace is Tw, added to the advance of single-byte code 32 only. The
// single-byte restriction is why a CID font with a two-byte code 32 is
// unaffected, a rule that silently breaks naive extractors.
WordSpace float64
// Scale is Th from Tz, as a percentage. It scales horizontal advances and
// glyph widths but not vertical movement.
Scale float64
// Leading is TL, the baseline-to-baseline distance used by T*, ', and ".
// Positive leading moves down the page.
Leading float64
// Rise is Ts, the baseline offset for superscripts and subscripts.
Rise float64
// Render is Tr, the rendering mode. Mode 3 is invisible, and mode 7 adds to
// a clipping path without painting. Both still advance the text position,
// and both are how OCR layers hide text under a scanned image.
Render int
}
TextState holds the text-related parameters of the graphics state (ISO 32000-2 §9.3). These persist across BT/ET blocks: a Tf outside any text object still applies to the next one.
Source Files