par2go

README

par2go

Pure Go library for creating PAR2 parity/recovery files. PAR2 files let you repair or verify data using Reed–Solomon erasure coding. Recovery files produced by par2go are compatible with par2cmdline, MultiPar, and other PAR2-compliant tools.

Installation

go get github.com/javi11/par2go

Requires Go 1.26 or later.

SIMD-accelerated encoding

par2go uses pre-built ParPar static libraries for SIMD-optimized Reed-Solomon encoding (SSE2, AVX2, AVX-512, NEON, SVE2, etc. with runtime CPU detection). The static libraries are committed to the repo, so no C++ compiler is needed.

CGO must be enabled (the default). On Windows, a MinGW-w64 toolchain is required for linking — install via MSYS2: pacman -S mingw-w64-x86_64-gcc.

Quick start

package main

import (
	"context"
	"log"

	"github.com/javi11/par2go"
)

func main() {
	ctx := context.Background()
	outputPath := "/path/to/myfile.par2"
	inputFiles := []string{"/path/to/myfile.bin"}

	opts := par2go.Options{
		SliceSize:   32768,  // 32 KB blocks (must be a multiple of 4)
		NumRecovery: 5,      // number of recovery blocks
	}

	if err := par2go.Create(ctx, outputPath, inputFiles, opts); err != nil {
		log.Fatal(err)
	}
	// Creates myfile.par2 and volume files like myfile.vol00+01.par2, ...
}

Options

Option	Description
SliceSize	Block size in bytes. Must be a positive multiple of 4 (e.g. 32768 for 32 KB).
NumRecovery	Number of recovery blocks to create. You can recover up to this many missing/corrupt blocks.
MemoryBudget	Max memory for recovery buffers (default: 512 MB).
NumGoroutines	Parallel workers (default: runtime.NumCPU()).
OnProgress	Optional callback func(phase string, pct float64) with phase "hashing", "encoding", or "writing" and pct in 0.0–1.0.
Creator	Creator string stored in the PAR2 file (default: "Postie").

Progress and cancellation

Use OnProgress for UI or logging:

opts := par2go.Options{
	SliceSize:   32768,
	NumRecovery: 5,
	OnProgress: func(phase string, pct float64) {
		fmt.Printf("%s: %.0f%%\n", phase, pct*100)
	},
}

Pass a cancellable context to stop creation early:

ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// call cancel() to abort
err := par2go.Create(ctx, outputPath, inputFiles, opts)

Multiple input files

You can protect several files in one recovery set. All are hashed and encoded together; the main .par2 and volume files reference the whole set.

inputFiles := []string{
	"/path/to/file1.bin",
	"/path/to/file2.bin",
	"/path/to/file3.bin",
}
err := par2go.Create(ctx, "/path/to/set.par2", inputFiles, opts)

Output files

• Main file: the path you pass (e.g. myfile.par2). Contains metadata only (Main, Creator, File Description, IFSC packets).
• Volume files: created next to the main file with names like myfile.vol00+01.par2, myfile.vol01+01.par2, myfile.vol02+02.par2, … and contain the recovery slice packets.

Use the main .par2 with any PAR2 repair/verify tool; it will find the volume files by naming convention.

Performance

Benchmarked on Apple M4, 1 GiB input file, 768 KB slice size, 10 recovery blocks:

Tool	Throughput
par2go	~710 MB/s
parpar v0.4.5 (x64, Rosetta 2)	~550 MB/s

par2go achieves ~125% of parpar's throughput on the same machine. parpar is an x64 binary running under Rosetta 2 emulation; par2go runs natively on arm64.

The Go orchestration layer uses a single-pass pipeline:

• One read per file — file data is read exactly once; hashing and encoding happen in the same pass.
• Parallel IFSC hash pool — per-slice MD5/CRC32 computation runs on numCPU/2 goroutines concurrently with disk I/O and GF16 compute.
• Parallel file scanning — the initial 16 KB scan (needed for file IDs) runs concurrently across all input files.
• Parallel volume writes — .vol*.par2 files are written concurrently.

Run the included benchmark CLI to compare on your own machine:

go run ./cmd/benchmark -input /path/to/largefile -parpar /path/to/parpar

Compatibility

par2go only creates PAR2 files. To verify or repair, use an existing tool (e.g. par2cmdline or MultiPar) that reads the same PAR2 format. The generated packets and volume layout follow the usual PAR2 conventions.

License

See LICENSE.

Documentation

Overview

Package par2go implements a pure Go PAR2 parity file creator.

It creates PAR2 recovery files compatible with par2cmdline, MultiPar, and other PAR2-compliant tools, without requiring any external binaries.

Index

• func Create(ctx context.Context, outputPath string, inputFiles []string, opts Options) error
• func CreateWithNames(ctx context.Context, outputPath string, inputs []InputFile, opts Options) error
• type InputFile
• type Options

Functions

func Create

func Create(ctx context.Context, outputPath string, inputFiles []string, opts Options) error

Create creates PAR2 parity files for the given input files.

outputPath is the path for the main .par2 file (e.g., "/path/to/file.par2"). Volume files will be created alongside with names like file.vol00+01.par2.

FileDesc filenames are set to filepath.Base of each input path. Use CreateWithNames to embed relative paths instead.

func CreateWithNames

func CreateWithNames(ctx context.Context, outputPath string, inputs []InputFile, opts Options) error

CreateWithNames is like Create but allows callers to specify the logical filename (including a relative path) for each input file. All inputs are bundled into a single par2 set written to outputPath.

Types

type InputFile

type InputFile struct {
	Path string
	Name string
}

InputFile pairs an on-disk path with the logical filename to embed in the PAR2 FileDesc packet.

Name should use forward-slash separators for relative paths (e.g. "folder2/file.txt"). When Name is empty, filepath.Base(Path) is used, matching the behavior of Create.

Embedding relative paths makes downloaders such as SABnzbd and NZBGet reconstruct the original folder tree on disk after par2 verification (they use the FileDesc filename to rename downloaded flat files into subdirectories). This matches the default behavior of ParPar.

Name is rejected if it contains a backslash, is absolute, or contains a ".." segment, to prevent path-traversal in downstream consumers.

type Options

type Options struct {
	// SliceSize is the block/slice size in bytes. Must be a multiple of 4.
	SliceSize int
	// NumRecovery is the number of recovery blocks to create.
	NumRecovery int
	// NumGoroutines is the number of parallel GF compute threads (default: runtime.NumCPU()).
	// Pass 0 to use hardware_concurrency() auto-detection.
	NumGoroutines int
	// MemoryLimit caps the memory used for recovery output buffers (in bytes).
	// When NumRecovery * SliceSize exceeds this limit, encoding is split into
	// multiple passes over the input files, each processing a subset of recovery
	// blocks that fits within the budget. This dramatically improves cache locality
	// for large recovery counts at the cost of re-reading input files.
	// Default 0 means automatic (min of 75% physical RAM or 4 GiB).
	MemoryLimit int64
	// Method forces a specific GF16 SIMD method (0 = auto-detect).
	// Use parpar.GF16Auto, parpar.GF16ShuffleAVX2, parpar.GF16ClmulNEON, etc.
	Method int
	// InputGrouping controls the input batch size for the GF16 encoder.
	// 0 means auto-detect (typically ~12). Higher values use more memory
	// but may improve throughput.
	InputGrouping int
	// ChunkLen controls the sub-slice chunk length for parallel GF16 processing.
	// 0 means auto-detect based on the SIMD method's ideal chunk size.
	ChunkLen int
	// StagingAreas controls the number of double-buffered staging areas in the
	// GF16 encoder. Default 0 means 2. More areas can overlap I/O and compute.
	StagingAreas int
	// OnProgress reports progress: phase is "hashing", "encoding", or "writing", pct is 0.0-1.0.
	OnProgress func(phase string, pct float64)
	// Creator is the creator string embedded in the PAR2 file (default: "Postie").
	Creator string
	// Logger is the structured logger used internally. Defaults to discarding all output.
	// Set to slog.Default() or a custom *slog.Logger to enable logging.
	Logger *slog.Logger
}

Options configures PAR2 creation.