agrep

README

agrep

agrep is an ultrafast, Unicode-aware search tool tailored for Arabic-script text. It matches across spelling variants, diacritics (tashkil), letter forms, consonant skeletons (rasm), and transliterated queries while preserving original source lines and exact byte offsets.

# Matches across diacritics, alef forms, and ta-marbuta:
agrep 'مدرسه' book.txt                         # matches مَدْرَسَة and مدرسة
agrep -r -n --include '*.txt' 'كتاب' books/    # recursive search with line numbers
agrep --translit=buckwalter 'ktAb' corpus.txt  # Latin keyboard query → كتاب
agrep --json 'احمد' people.txt                 # JSON Lines with exact byte spans

With no path, agrep reads standard input. Pass - as a path argument to mix standard input with files. Exit status follows standard grep conventions: 0 for match, 1 for no match, and 2 for an error.

Install

Quick Install (macOS, Linux, BSD)

Install the latest pre-compiled binary via the automated installer script:

curl -fsSL https://raw.githubusercontent.com/MohamedElashri/agrep/main/scripts/install.sh | bash

Custom installation directory (e.g., ~/.local/bin):

BINDIR=~/.local/bin curl -fsSL https://raw.githubusercontent.com/MohamedElashri/agrep/main/scripts/install.sh | bash

Pre-compiled Binaries

Download standalone archives for Linux, macOS, Windows, FreeBSD, OpenBSD, and NetBSD (amd64, arm64) from GitHub Releases, and verify hashes against checksums.txt.

Using GitHub CLI:

gh release download v0.1.0 -R MohamedElashri/agrep

Direct download (Linux amd64 example):

curl -fsSL https://github.com/MohamedElashri/agrep/releases/download/v0.1.0/agrep_0.1.0_linux_amd64.tar.gz | tar -xz
sudo mv agrep /usr/local/bin/

Go Toolchain

go install github.com/MohamedElashri/agrep/cmd/agrep@v0.1.0

Build from Source

git clone https://github.com/MohamedElashri/agrep.git
cd agrep && go build -ldflags="-s -w" ./cmd/agrep

HTML and EPUB extraction can be included by adding -tags formats during compilation. See INPUT.md.

Updating

Check for new releases or self-update in place:

agrep --check-update    # check if a newer release is available
agrep --update          # download, verify, and update in place
agrep --update=v0.2.0   # update or switch to a specific version

Search modes

Need	Option	Detail
Exact spelling distinctions	--profile=strict	Keeps hamza, ta-marbuta, and alef-maksura distinct
High recall / skeleton	--profile=loose or --rasm	Folds consonants to dotless skeletons
Typographical / OCR errors	--fuzzy[=N]	Levenshtein edit distance over normalized codepoints
Regular expressions	--regex	Pattern syntax evaluated over normalized text
Non-Arabic orthographies	--lang=fa, --lang=ur, etc.	Language-specific equivalences (Persian, Urdu, Kurdish)
Legacy encodings	--encoding=auto	Automatic CP1256, ISO-8859-6, and UTF-16 decoding

The matching line is always printed with its original source spelling. Emitted JSON spans and -o refer to decoded UTF-8 byte offsets in that original line.

Documentation

Topic	Document	Description
CLI Reference	CLI.md	Full list of flags, options, and JSON Lines format
Normalization	NORMALIZATION.md	Unicode rules and profiles (search, strict, loose, lucene, camel)
Matching Engines	MATCHING.md	Dotless rasm, Myers fuzzy Levenshtein distance, regex
Languages & Encodings	LANGUAGES.md, INPUT.md	Persian, Urdu, Kurdish orthographies and transliteration
Web Playground	PLAYGROUND.md	Live interactive demo running in WebAssembly
Benchmarks	BENCHMARKS.md	Performance metrics and comparison suites
Agent Integration	Agent Skill	Ready-to-use search skill for Codex and Claude Code

Go packages

agrep is also available as modular Go packages: arabic for Unicode normalization, match for literal, regex, and fuzzy engines, and scan for line-by-line streaming.

import "github.com/MohamedElashri/agrep/arabic"

// Normalized comparison treats diacritics and letter variants as equivalent:
same := arabic.Normalize("مَدْرَسَةٌ") == arabic.Normalize("مدرسه") // true

Documentation: arabic · match · scan.

License

MIT License. See LICENSE.