README
¶
agrep
agrep searches Arabic-script text across spelling and Unicode variants. It can
ignore diacritics, fold common letter forms, search transliterated queries, and
return original-text spans as JSON Lines.
Install
Download the archive for your operating system from
GitHub Releases, verify its
SHA-256 hash against checksums.txt, extract it, and put agrep on your PATH.
Check the installation with agrep --version.
With Go installed, you can also install the latest published version:
go install github.com/MohamedElashri/agrep/cmd/agrep@latest
The standard binary searches text files. HTML and EPUB extraction requires a
source build with -tags formats.
Start searching
agrep 'مدرسه' book.txt # also finds مَدْرَسَة
agrep -r -n --include '*.txt' 'كتاب' books/ # recursive search
agrep --translit=buckwalter 'ktAb' corpus.txt # Latin keyboard → كتاب
agrep --json 'احمد' people.txt # JSON Lines with byte spans
With no path, agrep reads standard input. Use - as a path to mix standard
input with files. Exit status is 0 for a selection, 1 for none, and 2 for
an error.
Choose a search mode
| Need | Option | Detail |
|---|---|---|
| Exact spelling distinctions | --profile=strict |
Keeps hamza, ta-marbuta, and alef-maksura distinct. |
| More recall | --profile=loose or --rasm |
Can produce extra matches. |
| Misspellings or OCR errors | --fuzzy[=N] |
Edit distance over normalized Unicode codepoints. |
| Regular expressions | --regex |
Pattern syntax is evaluated over normalized text. |
| Persian, Urdu, and other orthographies | --lang=fa, --lang=ar,fa, etc. |
Selects language-specific equivalences. |
| Legacy files | --encoding=auto or an explicit encoding |
Decodes before searching. |
The matching line is displayed with its original spelling. JSON spans and
-o refer to decoded UTF-8 byte offsets in that original line.
Reference
- All CLI options and output formats
- Normalization rules and profiles
- Languages, rasm and fuzzy matching, encodings and transliteration
- Terminal color and normalization playground
- Benchmarks and development
- Agent search skill for Codex and Claude Code
Go packages
arabic normalizes text, match builds reusable literal, regex, or fuzzy
matchers, and scan streams matching lines. The CLI uses the same packages.
import "github.com/MohamedElashri/agrep/arabic"
same := arabic.Normalize("مَدْرَسَةٌ") == arabic.Normalize("مدرسه") // true
_ = same
See the package documentation for
arabic,
match, and
scan.
License
MIT. See LICENSE.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package arabic normalizes Arabic-script text into a lossy comparison key: presentation and selected language forms are handled before Unicode NFD, then tashkil and other combining marks, tatweel, and selected orthographic variants are folded.
|
Package arabic normalizes Arabic-script text into a lossy comparison key: presentation and selected language forms are handled before Unicode NFD, then tashkil and other combining marks, tatweel, and selected orthographic variants are folded. |
|
gen
command
|
|
|
cmd
|
|
|
agrep
command
|
|
|
playground
command
|
|
|
internal
|
|
|
decode
Package decode converts supported input encodings to UTF-8 before scanning.
|
Package decode converts supported input encodings to UTF-8 before scanning. |
|
inputformat
Package inputformat opens plain-text input in the default build.
|
Package inputformat opens plain-text input in the default build. |
|
playground
Package playground prepares normalization examples for the WebAssembly UI.
|
Package playground prepares normalization examples for the WebAssembly UI. |
|
translit
Package translit converts between Arabic text and supported Latin schemes.
|
Package translit converts between Arabic text and supported Latin schemes. |
|
walk
Package walk discovers input files for recursive searches.
|
Package walk discovers input files for recursive searches. |
|
Package match finds occurrences of a search pattern inside text that has already been normalized by package arabic.
|
Package match finds occurrences of a search pattern inside text that has already been normalized by package arabic. |
|
Package scan reads arbitrary-length logical lines from an io.Reader and reports the ones whose normalized text a match.Matcher finds, without bufio.Scanner's token-size ceiling.
|
Package scan reads arbitrary-length logical lines from an io.Reader and reports the ones whose normalized text a match.Matcher finds, without bufio.Scanner's token-size ceiling. |