downmark

README

downmark

Convert documents to Markdown. Pure Go — no Python, no cgo, no external tools. One static binary; also usable as a library where you only link the formats you import.

downmark is a native Go re-imagining of Microsoft's markitdown: the same job (get documents into clean, LLM-friendly Markdown) without the Python runtime and native dependencies.

Supported formats

Format	Package	Notes
PDF	convert/pdf	Text extraction via github.com/giraffesyo/pdf: Form XObjects (Google Docs exports), Identity-H composite fonts, ToUnicode CMaps, segmented content streams, inline images; hard budgets against decompression bombs. No OCR: scanned PDFs return a clear error.
DOC	convert/doc	Word 97–2003 binary documents: bounded Compound Binary parsing, CLX piece-table reconstruction, ANSI/UTF-16 text, and displayed field results. Main-document text only; legacy formatting is not preserved.
DOCX	convert/docx	Headings, bold/italic/strikethrough, sub/superscript, nested lists, tables (incl. gridSpan/vMerge), hyperlinks, image placeholders, tracked changes. Equations degrade to plain text.
XLSX	convert/xlsx	Every sheet as ## SheetName + a Markdown table.
PPTX	convert/pptx	Slides in order with <!-- Slide number: N --> markers, position-based reading order, tables, chart data tables, image alt text, speaker notes.
HTML	convert/html	DOM sanitization (hidden/inert content dropped, javascript: links unwrapped, data URIs truncated) + GFM tables.
CSV	convert/csv	Charset-aware (incl. Shift-JIS/cp932), delimiter sniffing (, ; \t |).
ZIP	convert/zipfile	Supported members are converted through the engine and emitted under per-file headings; hard limits cap compressed archives at 64 MiB, total member data at 128 MiB (10 MiB/member), conversion at 64 members, scanning at 1,024 entries, and output at 32 MiB.
Plain text	built into core	Charset-detected passthrough (UTF-8/16, legacy encodings).

Install the CLI

go install github.com/giraffesyo/downmark/cmd/downmark@latest

usage: downmark [flags] [file]      # stdin if file omitted or "-"; Markdown → stdout
  -o file           write output to file
  -x .ext           extension hint (leading dot optional)
  -m type           MIME type hint
  -c charset        charset hint, e.g. shift_jis
  -keep-data-uris   keep full data: URIs
  -version          print version

Exit codes: 0 success, 1 conversion failed, 2 usage error.

Library

Pay for what you import: the core engine pulls only lightweight detection dependencies; each format package brings its own.

import (
    "github.com/giraffesyo/downmark"
    "github.com/giraffesyo/downmark/convert/pdf"
)

e := downmark.New() // core engine (plain-text passthrough builtin)
pdf.Register(e)     // link only the PDF stack

res, err := e.ConvertFile(ctx, "report.pdf")
// res.Markdown, res.Title

Everything wired up (what the CLI uses):

import "github.com/giraffesyo/downmark/all"

res, err := all.ConvertFile(ctx, "report.docx")
// or: e := all.New(all.Options{KeepDataURIs: true})

Indicative binary sizes (-trimpath -ldflags "-s -w", darwin/arm64): a PDF-only consumer ≈ 4.9 MB, CSV-only ≈ 4.5 MB, everything ≈ 7.5 MB.

MarkItDown comparison

Downmark and MarkItDown solve the same end-to-end problem, so the comparison uses their CLI interfaces against the same fixtures. Each value is the median of ten complete conversions, including process startup and Markdown written to stdout. The runner alternates which CLI runs first on each measured pair to reduce order bias. Both tools returned successfully for every fixture.

Measured 2026-07-15 from a worktree based on commit 152d4a8 with github.com/giraffesyo/pdf v0.2.1, using Go 1.26.4, MarkItDown 0.1.6, Python 3.14.2, and macOS 26.5.1 on an Apple M5 Pro (darwin/arm64). Re-run on your target environment before making a performance decision.

Fixture	Downmark	MarkItDown
PDF	7.0 ms	394.5 ms
PDF (Form XObject + ToUnicode)	5.6 ms	337.7 ms
DOCX	7.2 ms	350.0 ms
XLSX	6.7 ms	342.1 ms
PPTX	7.6 ms	338.7 ms
HTML	6.5 ms	341.2 ms
Shift-JIS CSV	5.5 ms	339.8 ms

Markdown is allowed to differ when both renderings are valid. The comparison runner records each output's byte count and digest so changes are visible; Downmark's golden tests define its output contract. Run the benchmark and inspect its raw output with the commands in benchmarks/README.md.

Custom converters implement downmark.Converter (use StreamInfo.Matches in Accepts) and register with e.Register(conv, downmark.PrioritySpecific); converters registered later at the same priority are tried first, so yours shadow the builtins.

Errors: errors.Is(err, downmark.ErrUnsupportedFormat) when nothing matched; *downmark.ConversionError (with per-converter attempts) when converters matched but failed.

PDF extraction

PDF text extraction lives in its own module, github.com/giraffesyo/pdf, usable without the converter framework — positioned glyphs, line/word reconstruction, and hardening against malformed and hostile files.

Limitations

• No OCR. Scanned/image-only PDFs and text-converted-to-outlines fail with "no extractable text" rather than silently emitting nothing.
• DOC: Word 97–2003 main-document text is extracted, but formatting, tables, images, headers/footers, footnotes, comments, and text boxes are not reconstructed. Word 6/95 files are not supported.
• PDF: complex multi-column layouts may interleave; fonts lacking both a ToUnicode map and a standard encoding are dropped rather than emitted as garbage; borderless table-layout reconstruction is not yet implemented.
• DOCX: headers/footers, footnotes, comments, and text boxes are skipped; equations render as plain text; nested tables flatten to text.
• Encrypted Office files, PDFs, and ZIP archives are not decrypted.
• OOXML archives (DOCX/XLSX/PPTX) are capped at 64 MiB compressed, 128 MiB total uncompressed, 16 MiB per decompressed part, and 1,024 entries; XML structure is also bounded to prevent small parts from creating huge trees.
• ZIP archives nested inside ZIP archives are skipped; archive conversion does not recurse.
• Non-seekable inputs (stdin, network streams) are buffered in memory. Matching ZIP and Office inputs are rejected while buffering at their 64 MiB hard limit; formats without an input-limit converter remain fully buffered.

Development

go test ./...                       # unit + vector + golden tests
go test -run TestGolden -update .   # refresh golden files
golangci-lint run ./...             # lint (config in .golangci.yml)

Test fixtures under testdata/ partly come from microsoft/markitdown (MIT); see testdata/NOTICE.md.

License

MIT

Documentation

Overview

Package downmark converts documents to Markdown.

This package is the core: the Engine, converter registry, content-type detection, and a plain-text converter. Format converters live in separate packages so binaries link only the formats they use:

• github.com/giraffesyo/downmark/convert/pdf
• github.com/giraffesyo/downmark/convert/doc
• github.com/giraffesyo/downmark/convert/docx
• github.com/giraffesyo/downmark/convert/xlsx
• github.com/giraffesyo/downmark/convert/pptx
• github.com/giraffesyo/downmark/convert/html
• github.com/giraffesyo/downmark/convert/csv
• github.com/giraffesyo/downmark/convert/zipfile

The github.com/giraffesyo/downmark/all package wires every converter into one engine for consumers who want the full matrix. Standalone PDF text extraction lives in the separate github.com/giraffesyo/pdf module.

Picking formats

e := downmark.New()          // engine with plain-text passthrough only
pdf.Register(e)              // + PDF
docx.Register(e, docx.Options{}) // + DOCX

res, err := e.ConvertFile(ctx, "report.pdf")
if err != nil { ... }
fmt.Println(res.Markdown)

Streams work too; pass whatever type hints you have and detection fills in the rest by sniffing content:

res, err := e.Convert(ctx, resp.Body, downmark.StreamInfo{
	MIMEType: resp.Header.Get("Content-Type"),
})

Derive the context with WithResultLimit to reject oversized Markdown before the engine performs final normalization. Converters can inspect ResultLimit to enforce the same budget while constructing their result. Converters with hard compressed-input budgets can also implement InputLimitConverter so non-seekable streams are bounded while the engine buffers them.

Custom converters

Implement Converter and register it; among converters with equal priority the most recently registered wins, so user converters shadow builtins. StreamInfo.Matches is the standard Accepts building block. A converter may also implement interface{ Name() string } to control how it is reported in errors.

Errors

When no converter accepts an input, errors.Is(err, ErrUnsupportedFormat) reports true. When converters accepted but all failed, the error is a *ConversionError carrying each attempt.

Index

• Variables
• func ResultLimit(ctx context.Context) (int, bool)
• func WithResultLimit(ctx context.Context, maxBytes int) context.Context
• type AttemptError
  • func (e *AttemptError) Error() string
  • func (e *AttemptError) Unwrap() error
• type ConversionError
  • func (e *ConversionError) Error() string
  • func (e *ConversionError) Unwrap() []error
• type Converter
• type Engine
  • func New(opts ...Option) *Engine
  • func (e *Engine) CanConvert(hints StreamInfo) bool
  • func (e *Engine) Convert(ctx context.Context, r io.Reader, hints StreamInfo) (*Result, error)
  • func (e *Engine) ConvertFile(ctx context.Context, path string) (*Result, error)
  • func (e *Engine) Register(c Converter, p Priority)
• type InputLimitConverter
• type Option
  • func WithHTTPClient(c *http.Client) Option
  • func WithoutBuiltins() Option
• type Priority
• type Result
• type StreamInfo
  • func (s StreamInfo) Matches(exts []string, mimePrefixes []string) bool

Variables

var ErrInputTooLarge = errors.New("downmark: input exceeds size limit")

ErrInputTooLarge is returned when a converter's hard input budget is exceeded, including while the engine buffers a non-seekable stream.

var ErrResultTooLarge = errors.New("downmark: result exceeds size limit")

ErrResultTooLarge is returned when a conversion result exceeds the maximum size carried by its context via WithResultLimit.

var ErrUnsupportedFormat = errors.New("downmark: unsupported format")

ErrUnsupportedFormat is returned when no registered converter accepted the input under any detected interpretation of its type.

Functions

func ResultLimit

func ResultLimit(ctx context.Context) (int, bool)

ResultLimit reports the maximum accepted Markdown result size carried by ctx. Converters may use it to avoid constructing a result that the engine will reject.

func WithResultLimit

func WithResultLimit(ctx context.Context, maxBytes int) context.Context

WithResultLimit returns a derived context that limits accepted Markdown results to maxBytes. A child context cannot loosen an existing limit. It panics if ctx is nil or maxBytes is not positive.

Types

type AttemptError

type AttemptError struct {
	// Converter is the name of the converter that failed, e.g. "pdf".
	Converter string
	// Err is the error the converter returned.
	Err error
}

AttemptError records a single converter's failed conversion attempt.

func (*AttemptError) Error

func (e *AttemptError) Error() string

func (*AttemptError) Unwrap

func (e *AttemptError) Unwrap() error

type ConversionError

type ConversionError struct {
	Attempts []*AttemptError
}

ConversionError is returned when at least one converter accepted the input but every attempt failed. Attempts lists each failure in the order tried.

func (*ConversionError) Error

func (e *ConversionError) Error() string

func (*ConversionError) Unwrap

func (e *ConversionError) Unwrap() []error

type Converter

type Converter interface {
	// Accepts reports whether the converter wants to try this stream.
	Accepts(info StreamInfo) bool
	// Convert reads input (positioned at offset 0) and produces Markdown.
	// Implementations that can construct large results must consult
	// ResultLimit(ctx) and stop before allocating output the engine will reject.
	Convert(ctx context.Context, input io.ReadSeeker, info StreamInfo) (*Result, error)
}

Converter converts one class of documents to Markdown.

Implementations must judge Accepts from the StreamInfo alone, never by reading the stream; content sniffing happens in the Engine's detection layer before Accepts is called.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine holds a converter registry and conversion options. Create one with New; the zero value is not usable.

func New

func New(opts ...Option) *Engine

New returns an Engine with only the builtin plain-text converter registered (unless WithoutBuiltins is given). Register format converters explicitly from the convert/... packages, or use the all package for an engine with every format wired up.

func (*Engine) CanConvert

func (e *Engine) CanConvert(hints StreamInfo) bool

CanConvert reports whether a registered format-specific converter would accept input described by hints, judged from the hints alone (no stream is read). Catch-all fallback converters are excluded, so this means "a dedicated converter claims this format" rather than "Convert would produce something"; callers routing files can use it to send only real documents through Convert and handle plain text themselves. An empty Extension is derived from Filename. Because no content is sniffed, a mislabeled file that Convert would still classify by its bytes can read as false here.

func (*Engine) Convert

func (e *Engine) Convert(ctx context.Context, r io.Reader, hints StreamInfo) (*Result, error)

Convert converts an arbitrary reader to Markdown. hints may be the zero value. If r is not an io.ReadSeeker its content is buffered into memory; seekable inputs are rewound to offset 0.

func (*Engine) ConvertFile

func (e *Engine) ConvertFile(ctx context.Context, path string) (*Result, error)

ConvertFile opens path and converts it, deriving Extension, Filename, and LocalPath from the path.

func (*Engine) Register

func (e *Engine) Register(c Converter, p Priority)

Register adds a converter at the given priority. Among converters with equal priority, the most recently registered is tried first, so user converters shadow builtins.

type InputLimitConverter

type InputLimitConverter interface {
	InputLimit() int64
}

InputLimitConverter is optionally implemented by converters with a hard compressed-input budget. The engine applies the smallest matching limit while buffering a non-seekable input, before conversion begins.

type Option

type Option func(*Engine)

Option configures an Engine created by New.

func WithHTTPClient

func WithHTTPClient(c *http.Client) Option

WithHTTPClient sets the HTTP client used for URL-based conversions.

func WithoutBuiltins

func WithoutBuiltins() Option

WithoutBuiltins starts the Engine with an empty converter registry (omitting even the plain-text converter). Use Register to add converters.

type Priority

type Priority int

Priority orders converters: lower values are tried first.

const (
	// PrioritySpecific is for converters keyed to a specific file format
	// (doc, docx, xlsx, pptx, pdf, csv).
	PrioritySpecific Priority = 0
	// PriorityArchive is for archive walkers, which must run only after
	// converters for formats that use an archive as their container (such as
	// docx, xlsx, and pptx) have declined the input.
	PriorityArchive Priority = 50
	// PriorityGeneric is for near-catch-all converters (html, plain text).
	PriorityGeneric Priority = 100
)

type Result

type Result struct {
	// Markdown is the normalized Markdown output.
	Markdown string
	// Title is the document title if the format provides one, else "".
	Title string
}

Result is the outcome of a successful conversion.

type StreamInfo

type StreamInfo struct {
	// MIMEType is the media type without parameters, e.g. "application/pdf".
	MIMEType string
	// Extension is lowercase with a leading dot, e.g. ".docx".
	Extension string
	// Charset is an IANA charset name, e.g. "utf-8" or "shift_jis".
	Charset string
	// Filename is the base name of the source, if known.
	Filename string
	// LocalPath is the path of the source file, if the input came from disk.
	LocalPath string
	// URL is the source URL, if the input came from the network.
	URL string
}

StreamInfo carries what is known or guessed about an input stream. The zero value means "nothing known"; every field is optional.

func (StreamInfo) Matches

func (s StreamInfo) Matches(exts []string, mimePrefixes []string) bool

Matches reports whether the stream's extension is one of exts (lowercase, with leading dot) or its MIME type starts with any of mimePrefixes. It is the standard building block for Converter.Accepts implementations.