ocr

README

PaddleOCR-VL Go SDK

A pure Go SDK for running PaddleOCR-VL and other VLM OCR models via LM Studio (OpenAI-compatible API).

Features

• No external dependencies — uses only Go standard library
• LM Studio / OpenAI-compatible — works with any OpenAI API endpoint
• LOC Token Parsing — converts <|LOC_xxx|> tokens to pixel-accurate polygons
• Layout Analysis — reading order sorting (Y-cluster → X-sort)
• Multiple Output Formats — Markdown, JSON, HTML, Plain Text
• Pipeline Architecture — extensible for future VLM models (Qwen2.5-VL, InternVL3, etc.)
• Streaming Support — handles SSE streaming responses

CLI Installation

go install github.com/schaepher/ocr/cmd/ocr@latest

CLI Usage

# Basic OCR to Markdown (default)
ocr --image screenshot.png

# Output as HTML with overlays
ocr --image screenshot.png --format html

# Other formats
ocr --image screenshot.png --format json
ocr --image screenshot.png --format text

# Custom output path
ocr --image screenshot.png --format html --output result.html

# Custom LM Studio endpoint / model
ocr --image screenshot.png --base-url http://127.0.0.1:1234/v1 --model PaddleOCR-VL-1.6

Flags

Flag	Default	Description
--image	(required)	Path to image file
--format	markdown	Output format: markdown, json, html, text
--output	same dir as image, auto extension	Output file path
--base-url	http://127.0.0.1:1234/v1	LM Studio API base URL
--model	PaddleOCR-VL-1.6	Model name

SDK Usage

package main

import (
    "context"
    "fmt"

    "github.com/schaepher/ocr"
    "github.com/schaepher/ocr/provider/paddleocrvl"
)

func main() {
    ctx := context.Background()

    doc, err := ocr.New(paddleocrvl.New()).
        LMStudio("http://127.0.0.1:1234/v1").
        ParseImage(ctx, "screenshot.png")
    if err != nil {
        panic(err)
    }

    // Output as Markdown
    md, _ := ocr.Markdown(doc)
    fmt.Println(md)
}

Package Structure

├── ocr.go                  # Top-level API (New(provider).ParseImage())
├── client/                 # OpenAI-compatible HTTP client
├── cmd/
│   └── ocr/                # CLI binary
├── decoder/                # Decoder interface
│   └── paddleocrvl/        # PaddleOCR-VL token parser
├── document/               # Core data types (Document, Block, Polygon)
├── layout/                 # Layout analysis (sort, paragraph merge)
├── output/                 # Output renderers (Markdown, JSON, HTML, Text)
├── pipeline/               # Pipeline orchestrator
└── provider/               # Provider interface
    └── paddleocrvl/        # PaddleOCR-VL provider (model, prompt, decoder)

How It Works

1. Image is read and base64-encoded
2. LM Studio API receives the image via OpenAI-compatible chat completion
3. Raw output contains text with <|LOC_xxx|> location tokens
4. Decoder parses LOC tokens into polygons and scales to pixel coordinates
5. Layout sorts blocks into natural reading order (Y-cluster → X-sort)
6. Output renders the structured Document as Markdown/JSON/HTML/Text

Coordinate Conversion

PaddleOCR-VL uses a discrete 0–1000 grid for coordinates. The SDK converts:

pixelX = locX * imageWidth / 1000
pixelY = locY * imageHeight / 1000

License

MIT

Documentation

Overview

Package ocr is a Go SDK for running OCR via VLM models (PaddleOCR-VL, Qwen2.5-VL, etc.) through LM Studio or any OpenAI-compatible API.

Usage:

doc, err := ocr.New(paddleocrvl.New()).
    LMStudio("http://127.0.0.1:1234/v1").
    ParseImage(ctx, "demo.png")

md, _ := ocr.Markdown(doc)

Index

• Variables
• func HTML(doc *document.Document, imageSrc string) (string, error)
• func JSON(doc *document.Document) (string, error)
• func Markdown(doc *document.Document) (string, error)
• func RecordFail(imagePath string)
• func Text(doc *document.Document) (string, error)
• type Client
  • func New(p provider.Provider) *Client
  • func (c *Client) Debug(path string) *Client
  • func (c *Client) LMStudio(url string) *Client
  • func (c *Client) MaxHeight(h int) *Client
  • func (c *Client) MaxRetries(n int) *Client
  • func (c *Client) Model(name string) *Client
  • func (c *Client) OnProgress(fn ProgressFunc) *Client
  • func (c *Client) Overlap(px int) *Client
  • func (c *Client) Page(n int) *Client
  • func (c *Client) ParseImage(ctx context.Context, imagePath string) (*document.Document, error)
  • func (c *Client) ParseImageReader(ctx context.Context, r io.Reader, imagePath string) (*document.Document, error)
  • func (c *Client) SaveSlices(v bool) *Client
  • func (c *Client) SystemPrompt(prompt string) *Client
• type ProgressFunc

Variables

var ErrSkipped = errors.New("skipped")

Functions

func HTML

func HTML(doc *document.Document, imageSrc string) (string, error)

HTML renders the document as positioned HTML. imageSrc is used as the src attribute of the background <img> tag.

func JSON

func JSON(doc *document.Document) (string, error)

JSON renders the document as indented JSON.

func Markdown

func Markdown(doc *document.Document) (string, error)

Markdown renders the document as Markdown.

func RecordFail

func RecordFail(imagePath string)

func Text

func Text(doc *document.Document) (string, error)

Text renders the document as plain text.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

func New

func New(p provider.Provider) *Client

New creates a new Client with the given provider. The provider supplies the default model name, system prompt, and decoder.

func (*Client) Debug

func (c *Client) Debug(path string) *Client

Debug enables debug mode: raw model output is saved to path (JSON array). If path already exists, the model is skipped and cached raw output is replayed.

func (*Client) LMStudio

func (c *Client) LMStudio(url string) *Client

LMStudio sets the LM Studio API base URL.

func (*Client) MaxHeight

func (c *Client) MaxHeight(h int) *Client

MaxHeight sets the maximum image height before slicing. 0 (default) means no slicing. Overlap between slices is 200px by default.

func (*Client) MaxRetries

func (c *Client) MaxRetries(n int) *Client

MaxRetries sets the max retries when the model output has no LOC tokens. Default is 3.

func (*Client) Model

func (c *Client) Model(name string) *Client

Model overrides the default model name.

func (*Client) OnProgress

func (c *Client) OnProgress(fn ProgressFunc) *Client

OnProgress sets a callback invoked for each slice during slicing.

func (*Client) Overlap

func (c *Client) Overlap(px int) *Client

Overlap sets the vertical overlap between adjacent slices.

func (*Client) Page

func (c *Client) Page(n int) *Client

Page sets which page (1-based slice index) to OCR. 0 means all pages.

func (*Client) ParseImage

func (c *Client) ParseImage(ctx context.Context, imagePath string) (*document.Document, error)

ParseImage runs OCR on an image file and returns a structured Document. If MaxHeight is set and the image exceeds it, the image is split into overlapping horizontal slices, each processed separately, then merged.

func (*Client) ParseImageReader

func (c *Client) ParseImageReader(ctx context.Context, r io.Reader, imagePath string) (*document.Document, error)

ParseImageReader runs OCR on an image from an io.Reader.

func (*Client) SaveSlices

func (c *Client) SaveSlices(v bool) *Client

SaveSlices enables saving slice JPEGs and per-slice raw.json/html to disk.

func (*Client) SystemPrompt

func (c *Client) SystemPrompt(prompt string) *Client

SystemPrompt overrides the default system prompt.

type ProgressFunc

type ProgressFunc func(current, total int, y int)

Client is the convenient top-level API for OCR. ProgressFunc is called with the current slice index and total during slicing.