layout

package
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 20, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package layout provides PDF document layout analysis functionality.

Index

Examples

Constants

View Source 
const (
	ElementTypeHeader     = types.ElementTypeHeader
	ElementTypeParagraph  = types.ElementTypeParagraph
	ElementTypeCodeBlock  = types.ElementTypeCodeBlock
	ElementTypeList       = types.ElementTypeList
	ElementTypeTable      = types.ElementTypeTable
	ElementTypeAdmonition = types.ElementTypeAdmonition
	ElementTypeImage      = types.ElementTypeImage
)

Re-export element type constants

Variables

This section is empty.

Functions

func ApplyLinksToBlocks 

func ApplyLinksToBlocks(blocks []extractor.TextBlock, links []types.Link) []extractor.TextBlock

ApplyLinksToBlocks applies links to raw text blocks before they are merged into elements. This allows for more granular linking (e.g. linking just "here" in "Click here").

func CleanMathContent 

func CleanMathContent(text string) string

CleanMathContent attempts to clean up fragmented math expressions e.g., "$x$$y$" -> "$xy$" and "$a$ = $b$" -> "$a = b$"

func CreateONNXConfig 

func CreateONNXConfig(cfg *Config) *onnx.Config

CreateONNXConfig creates an onnx.Config from layout Config.

func DetectHorizontalSeparators 

func DetectHorizontalSeparators(graphics []types.VectorGraphic, pageWidth float64) []float64

DetectHorizontalSeparators finds horizontal lines that could be table row separators

func DetectVerticalSeparators 

func DetectVerticalSeparators(graphics []types.VectorGraphic, pageHeight float64) []float64

DetectVerticalSeparators finds vertical lines that could be table column separators

func FindTableCellsFromLines 

func FindTableCellsFromLines(hLines, vLines []Line, blocks []extractor.TextBlock) [][]TableCell

FindTableCellsFromLines uses horizontal and vertical lines to define cell boundaries and maps text blocks to those cells

func IsLikelyEquation 

func IsLikelyEquation(blocks []extractor.TextBlock) bool

IsLikelyEquation checks if blocks look more like an equation than a table This helps disambiguate tables from mathematical expressions

func SortBlocks 

func SortBlocks(blocks []extractor.TextBlock) []extractor.TextBlock

SortBlocks sorts text blocks using the Recursive XY-Cut algorithm This handles multi-column layouts by recursively splitting the page into horizontal rows and vertical columns.

Types

type Analyzer 

type Analyzer struct {
	ColumnGapThreshold float64
	HeaderSizeRatio    float64
	Rules              []Rule
	Exclusion          ExclusionZone
	Config             *Config
}

Analyzer analyzes the layout of text blocks

func NewAnalyzer 

func NewAnalyzer() *Analyzer

NewAnalyzer creates a new Analyzer with default configuration

func NewAnalyzerWithConfig 

func NewAnalyzerWithConfig(config *Config) *Analyzer

NewAnalyzerWithConfig creates a new Analyzer with the specified configuration

func (*Analyzer) Analyze 

func (a *Analyzer) Analyze(content *extractor.PageContent) []Element

Analyze converts raw text blocks into structured elements

Example 
// 1. Create an Analyzer
analyzer := NewAnalyzer()

// 2. Define raw text blocks (simulating extraction)
blocks := []extractor.TextBlock{
	{Text: "1. Introduction", X: 10, Y: 800, FontSize: 14},      // Increased font size
	{Text: "This is a paragraph.", X: 10, Y: 770, FontSize: 12}, // Gap 30 > Threshold
	{Text: "Another paragraph.", X: 10, Y: 740, FontSize: 12},   // Gap 30 > Threshold
	{Text: "More body text.", X: 10, Y: 710, FontSize: 12},      // Gap 30 > Threshold
	{Text: "func main() {", X: 10, Y: 680, FontSize: 10},
}

// 3. Analyze layout
content := &extractor.PageContent{
	TextBlocks: blocks,
}
elements := analyzer.Analyze(content)

// 4. Print results
for _, el := range elements {
	fmt.Printf("Type: %s, Content: %s\n", el.Type, el.Content)
}

Output:
Type: header, Content: 1. Introduction
Type: paragraph, Content: This is a paragraph.
Type: paragraph, Content: Another paragraph.
Type: paragraph, Content: More body text.
Type: code_block, Content: func main() {

func (*Analyzer) CleanFragmentedMath 

func (a *Analyzer) CleanFragmentedMath(elements []Element) []Element

CleanFragmentedMath applies CleanMathContent to fix fragmented math expressions

func (*Analyzer) CleanMathSymbols 

func (a *Analyzer) CleanMathSymbols(elements []Element) []Element

CleanMathSymbols merges consecutive duplicate math symbols

func (*Analyzer) FilterPageNumbers 

func (a *Analyzer) FilterPageNumbers(elements []Element) []Element

FilterPageNumbers removes standalone page numbers from the output

func (*Analyzer) FilterRepeatedPageHeaders 

func (a *Analyzer) FilterRepeatedPageHeaders(elements []Element) []Element

FilterRepeatedPageHeaders removes repeated page headers/footers These are text elements that appear multiple times with the exact same content, typically at similar Y positions (running headers/footers)

func (*Analyzer) MergeCodeBlocks 

func (a *Analyzer) MergeCodeBlocks(elements []Element) []Element

MergeCodeBlocks merges consecutive code block elements into a single block

func (*Analyzer) MergeDropCaps 

func (a *Analyzer) MergeDropCaps(elements []Element) []Element

MergeDropCaps merges drop cap letters with their following text Drop caps are large initial letters that appear at the start of paragraphs

func (*Analyzer) MergeElements 

func (a *Analyzer) MergeElements(elements []Element) []Element

MergeElements merges consecutive text elements on the same line

func (*Analyzer) MergeListContinuations 

func (a *Analyzer) MergeListContinuations(elements []Element) []Element

MergeListContinuations merges paragraph elements that follow list items into those list items This handles cases where multi-line list items are split across multiple elements

func (*Analyzer) MergeParagraphLines 

func (a *Analyzer) MergeParagraphLines(elements []Element) []Element

MergeParagraphLines merges consecutive paragraph lines

func (*Analyzer) MergeSameLinkElements 

func (a *Analyzer) MergeSameLinkElements(elements []Element) []Element

MergeSameLinkElements merges consecutive elements that share the same LinkURI

func (*Analyzer) MergeTableRows 

func (a *Analyzer) MergeTableRows(elements []Element) []Element

MergeTableRows merges consecutive table row elements into a single table block

func (*Analyzer) NormalizeHeaderLevels 

func (a *Analyzer) NormalizeHeaderLevels(elements []Element) []Element

NormalizeHeaderLevels adjusts header levels so the smallest level becomes H1

func (*Analyzer) RemoveTableOfContentsRange 

func (a *Analyzer) RemoveTableOfContentsRange(elements []Element) []Element

RemoveTableOfContentsRange removes the entire TOC section by finding the start and end markers.

type AnalyzerWithONNX 

type AnalyzerWithONNX struct {
	*Analyzer
	// contains filtered or unexported fields
}

AnalyzerWithONNX extends Analyzer with ONNX detection capabilities.

func NewAnalyzerWithONNX 

func NewAnalyzerWithONNX(config *Config, detector LayoutDetector) *AnalyzerWithONNX

NewAnalyzerWithONNX creates an analyzer with ONNX support. If detector is nil, falls back to rule-based analysis only.

func (*AnalyzerWithONNX) AnalyzeWithImage 

func (a *AnalyzerWithONNX) AnalyzeWithImage(content *extractor.PageContent, pageImage image.Image) []Element

AnalyzeWithImage analyzes page content with optional ONNX detection. If pageImage is provided and ONNX detector is available, uses ML detection. Otherwise falls back to rule-based analysis.

type Config 

type Config struct {
	// Column detection
	ColumnGapThreshold float64 // Minimum gap to consider as column boundary (default: 10.0)

	// Header detection
	HeaderSizeRatio float64 // Minimum ratio to body font to detect as header (default: 1.2)

	// Header level thresholds (ratio of font size to body font)
	H1Ratio float64 // Default: 2.0
	H2Ratio float64 // Default: 1.75
	H3Ratio float64 // Default: 1.5
	H4Ratio float64 // Default: 1.3
	H5Ratio float64 // Default: 1.15
	H6Ratio float64 // Default: 1.0

	// Table detection
	TableMinColumns int     // Minimum columns to detect as table (default: 2)
	TableMinSpaces  int     // Minimum consecutive spaces for wide gap (default: 4)
	WideGapMultiple float64 // Multiplier for wide gap detection (default: 3.0)

	// Image filtering
	MinImageSize float64 // Minimum width/height to include images (default: 50.0)

	// Text merging
	LineHeightMultiplier  float64 // Multiplier for line height threshold (default: 1.6)
	IndentThreshold       float64 // Threshold to detect indentation (default: 5.0)
	WordGapMultiplier     float64 // Multiplier for word gap detection (default: 0.2)
	SentenceGapMultiplier float64 // Stricter gap for sentence boundaries (default: 1.5)

	// Equation detection
	EquationScoreThreshold int // Minimum score to classify as equation (default: 2)

	// ONNX Layout Detection
	// ONNX detection provides ML-based layout analysis using DocLayout-YOLO model.
	// When enabled, it overrides rule-based classification for detected regions.
	EnableONNX        bool    // Enable ONNX layout detection (default: true)
	ONNXModelPath     string  // Path to ONNX model file (auto-detected if empty)
	ONNXRuntimePath   string  // Path to ONNX Runtime library (auto-detected if empty)
	ONNXConfThreshold float64 // Minimum confidence for ONNX detections (default: 0.25)
	ONNXNMSThreshold  float64 // IoU threshold for Non-Maximum Suppression (default: 0.45)
	MinONNXConfidence float64 // Minimum confidence to override rule-based classification (default: 0.5)
	ONNXInputSize     int     // Model input size in pixels (default: 1024)
	ONNXStride        int     // Model stride for padding alignment (default: 32)
	ONNXUseCoreML     bool    // Enable CoreML acceleration on macOS (default: true on darwin)
}

Config holds configurable thresholds for layout analysis. These values can be adjusted to fine-tune the analysis for different document types and formatting styles.

func ConfigForAcademicPapers 

func ConfigForAcademicPapers() *Config

ConfigForAcademicPapers returns configuration optimized for academic papers. Academic papers often have: - Smaller column gaps (two-column layouts) - More math/equations - Section numbering patterns

func ConfigForScannedDocuments 

func ConfigForScannedDocuments() *Config

ConfigForScannedDocuments returns configuration optimized for scanned PDFs. Scanned documents often have: - Inconsistent spacing - OCR artifacts - Noisier text positioning

func ConfigForTechnicalDocs 

func ConfigForTechnicalDocs() *Config

ConfigForTechnicalDocs returns configuration optimized for technical documentation. Technical docs often have: - Code blocks with keywords - Wider spacing - Clear header hierarchy

func DefaultConfig 

func DefaultConfig() *Config

DefaultConfig returns the default layout configuration. These defaults work well for most PDF documents. ONNX detection is enabled by default for improved layout analysis.

type Element 

type Element = types.Element

Element is an alias for types.Element for backward compatibility. 

func ProcessLinks(elements []Element, links []types.Link) []Element

ProcessLinks maps links to text blocks and formats them as Markdown links

type ElementType 

type ElementType = types.ElementType

ElementType is an alias for types.ElementType for backward compatibility.

type ExclusionZone 

type ExclusionZone struct {
	Top    float64 // Height from top to ignore
	Bottom float64 // Height from bottom to ignore
}

ExclusionZone defines areas to ignore (e.g., headers, footers)

type GridRegion 

type GridRegion struct {
	X, Y            float64
	Width, Height   float64
	HorizontalLines []float64 // Y positions of horizontal lines
	VerticalLines   []float64 // X positions of vertical lines
}

GridRegion represents a region bounded by grid lines

func MergeRectanglesIntoCells 

func MergeRectanglesIntoCells(graphics []types.VectorGraphic) []GridRegion

MergeRectanglesIntoCells detects rectangles (from graphics fill operations) and uses them as cell boundaries

type LayoutDetector 

type LayoutDetector interface {
	DetectLayout(pageImage image.Image, pageWidth, pageHeight float64) (*types.PageDetections, error)
	Close() error
	IsAvailable() bool
}

LayoutDetector defines the interface for ONNX-based layout detection. This interface is defined here (consumer side) per Go idioms.

type Line 

type Line struct {
	X1, Y1       float64
	X2, Y2       float64
	Length       float64
	IsHorizontal bool
	IsVertical   bool
}

Line represents a detected line (horizontal or vertical)

type Rule 

type Rule struct {
	Name      string
	Condition func(text string, fontSize, bodyFontSize float64) bool
	Type      ElementType
}

Rule defines a classification rule for the layout analyzer

type TableCell 

type TableCell struct {
	Content  string
	Row      int
	Col      int
	RowSpan  int
	ColSpan  int
	X, Y     float64
	Width    float64
	Height   float64
	IsHeader bool
}

TableCell represents a single cell in a table

type TableDetector 

type TableDetector struct {
	MinColumns      int     // Minimum columns to be considered a table (default: 2)
	MinRows         int     // Minimum rows to be considered a table (default: 2)
	ColumnTolerance float64 // X-position tolerance for column alignment (default: 5.0)
	RowTolerance    float64 // Y-position tolerance for row alignment (default: 3.0)
	MinConfidence   float64 // Minimum confidence score (default: 0.5)
	GapThreshold    float64 // Minimum gap ratio between columns (default: 2.0)
}

TableDetector detects tables from text blocks and optional graphics

func NewTableDetector 

func NewTableDetector() *TableDetector

NewTableDetector creates a new table detector with default settings

func (*TableDetector) DetectTables 

func (td *TableDetector) DetectTables(blocks []extractor.TextBlock, graphics []types.VectorGraphic) ([]TableStructure, map[int]bool)

DetectTables finds tables in a set of text blocks Returns detected tables and the blocks that were consumed by tables

type TableStructure 

type TableStructure struct {
	Cells      [][]TableCell // [row][col]
	NumRows    int
	NumCols    int
	HasHeader  bool
	X, Y       float64
	Width      float64
	Height     float64
	Confidence float64 // 0.0 to 1.0
}

TableStructure represents a detected table

func DetectTableFromGraphics 

func DetectTableFromGraphics(graphics []types.VectorGraphic, textBlocks []extractor.TextBlock) *TableStructure

DetectTableFromGraphics attempts to detect a table structure from graphics and text blocks

func (*TableStructure) ToMarkdown 

func (ts *TableStructure) ToMarkdown() string

ToMarkdown converts a TableStructure to markdown format

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.