file-search-on

README

file-search-on

Content-type aware file search with CEL-powered attribute filtering.

file-search-on walks a directory tree and returns files matching a CEL expression evaluated over each file's metadata and content-type-specific attributes. Instead of grepping by name, ask things like:

file-search-on 'is_pdf && page_count > 10 && author == "Jane Doe"'
file-search-on 'is_image && gps_lat > 51.4 && gps_lat < 51.6'        # photos near home
file-search-on 'is_audio && artist == "Radiohead" && year < 2000'
file-search-on 'is_video && video_height >= 2160 && video_codec == "h265"'
file-search-on 'is_office && language == "fr"'
file-search-on 'is_markdown && "longread" in tags && word_count > 1000'

# Or match fuzzily — typos in the data are no longer fatal:
file-search-on 'is_audio && levenshtein(artist, "Radiohead") <= 2'                # catches "Radiohad", "Radiohea"
file-search-on 'is_image && soundex(camera_make) == soundex("Nikon")'             # phonetic match across capitalisation / spelling
file-search-on 'is_markdown && ngram_similarity(title, "kubernetes", 2) > 0.6'    # substring-tolerant title match

Across 74 file formats organised into thirteen content-type families (documents, data, images, audio, video, office, ebooks, plain text, archives, compiled binaries, email, source code, notebooks), with format-specific metadata extraction.

Built in the open — issues, PRs, and feature requests warmly welcomed. See Contributing.

Features

• Pluggable content-type detection — extension-first with magic-byte fallback. New formats are a single registration call.

• Thirteen content-type families, each with its own metadata extractors:

  Family	Formats	Bundle of attributes
  Documents	PDF, EPUB	title, author, language, page_count
  Markup	Markdown, HTML, XML	title, word_count, frontmatter, language, root_element
  Data	JSON, YAML, TOML, CSV, TSV	json_kind, yaml_kind, yaml_document_count, column_count, csv_columns
  Plain text	TXT, log, …	line_count, word_count
  Images	JPEG, PNG, GIF, WebP, TIFF, BMP, SVG, HEIC	dimensions + EXIF: camera, lens, GPS, ISO, focal_length, taken_at
  Audio	MP3, M4A, FLAC, OGG	tags (artist, album, genre, year, …) + duration, bitrate / nominal_bitrate, sample_rate, channels, bit_depth, ReplayGain
  Video	MP4, MOV, MKV, WebM, AVI	duration, bitrate / nominal_bitrate, video_codec, audio_codec, video_width/height, frame_rate, rotation, HDR / colour-space, subtitles
  Office	DOCX, XLSX, PPTX, ODT	title, author, language (Dublin Core)
  Archives	ZIP (incl. JAR / WAR / EAR), TAR, TAR.GZ, GZIP	entry_count, uncompressed_size, top_level_entries, has_root_dir
  Binaries	ELF (Linux/BSD), Mach-O (macOS, incl. universal), PE (Windows)	architectures, bitness, binary_format, binary_type, is_dynamically_linked, is_stripped, entry_point
  Email	RFC 5322 (.eml), Unix mbox (.mbox)	title (subject), author (from), email_to, email_cc, sent_at, attachment_count, email_count
  Source code	Go, Python, JS/TS, Rust, C/C++, Java, Ruby, Swift, Kotlin, Scala, Shell, Lua, Elixir, Clojure, Haskell, OCaml, Zig	language, line_count, loc, comment_loc, blank_loc
  Notebooks	Jupyter .ipynb, Apache Zeppelin .zpln	cell_count, code_cell_count, markdown_cell_count, kernel, language, title

  Type predicates (is_pdf, is_image, is_audio, is_video, is_office, is_epub, …) light up automatically from the registered content type. See examples/ for recipes by family.

• Exact-name content types for common repo files — Dockerfile, Makefile, LICENSE, .gitignore, go.mod, package.json, Cargo.toml, Pipfile, Gemfile, requirements.txt, Procfile, Vagrantfile, and more — with per-type predicates (is_dockerfile, is_gomod, is_node_manifest, …) plus family predicates (is_build, is_repo_meta, is_ignore, is_manifest, is_platform). Predicates cross-fire: package.json is both is_node_manifest and is_json. See examples/repo-files.md.

• Project-type detection — detect-project / find-projects subcommands identify Go / Node / Rust / Python / Ruby / Java / .NET / Terraform / Docker Compose projects. Pair with --resolve-projects (file-level project_type filter) and --prune-build-artefacts (skip vendor/node_modules/target/__pycache__ automatically). Define custom project types via CEL in YAML — see examples/projects.md.

• First-class Markdown front-matter — YAML (---), TOML (+++), and JSON ({ ... }) are recognised by leading bytes. Common keys (title, author, language, tags, categories, draft, date) become top-level CEL variables; everything else lives in a generic frontmatter map. See examples/markdown.md.

• CEL expressions — the full Common Expression Language: comparisons, &&/||, string functions, list membership, timestamp arithmetic. Composes naturally with structural attributes.

• Fuzzy, phonetic, and geographic matching — built-in levenshtein, soundex, ngrams, ngram_similarity, and point_in_polygon (for GPS bboxes / city outlines) let you write typo-tolerant and "sounds-like" queries against any string attribute. EXIF camera make in Nikkon instead of Nikon? Artist tag mistyped as Radiohad? Same query catches all of them. See examples/fuzzy-search.md.

• Multiple output formats — bare (paths only), default, verbose (multi-line), json (NDJSON), or a Go text/template via --format.

• MCP server mode — same binary doubles as a Model Context Protocol server (stdio, HTTP, or SSE). Eleven tools exposed: search, read_attributes, read_lines, stats, find_duplicates, find_matches, detect_project, find_projects, resolve_project_for_path, list_attributes, index_stats.

• Pure Go, no CGO — cross-compiles cleanly to all six release targets. No image/audio/video decoder dependencies.

• Parallel walking — files are evaluated across a worker pool (defaults to NumCPU).

Install

Homebrew (macOS / Linux)

brew install richardwooding/tap/file-search-on

The cask is published from this repo on every tagged release to richardwooding/homebrew-tap.

macOS note: the binary isn't signed with an Apple Developer ID (yet — happy to accept a sponsor!). The Homebrew cask's post-install hook strips the quarantine xattr automatically. If macOS still blocks it on first run:

sudo xattr -dr com.apple.quarantine $(brew --prefix)/bin/file-search-on

Container (Docker / Podman)

OCI images are published to GitHub Container Registry on every tag, with linux/amd64 and linux/arm64 manifests:

docker run --rm -v "$PWD:/work" ghcr.io/richardwooding/file-search-on:latest \
  'is_markdown && draft' -d /work

Pin to a specific version with :vX.Y.Z. The base image is cgr.dev/chainguard/static, so the container has the binary and nothing else (no shell).

Pre-built binaries

Pre-built archives for Linux, macOS, and Windows on amd64 and arm64 are attached to every GitHub Release, along with a checksums.txt you should verify.

From source

Requires Go 1.26.2 or newer.

go install github.com/richardwooding/file-search-on/cmd/file-search-on@latest

Or build from a clone:

git clone https://github.com/richardwooding/file-search-on.git
cd file-search-on
go build -o file-search-on ./cmd/file-search-on

Usage

search is the default subcommand. Pass a CEL expression and a directory:

file-search-on 'is_markdown && word_count > 500' -d ./docs
file-search-on 'is_image && iso > 1600' -d ~/Pictures -o json
file-search-on 'is_video && duration > 1800 && video_height >= 2160' -d ~/Movies
file-search-on -d .                                   # empty expression matches every file

Subcommands

Command	Purpose	Deep dive
search (default)	CEL expression over file metadata	every page in examples/
attrs <path>	Print attributes for one file (no walk, no CEL)	examples/cookbook.md
stats [expr]	Histogram + totals, bucketed by group_by	examples/group-by.md
duplicates [expr]	Byte-identical files by sha256	examples/duplicates.md
find-matches <re> --expr <cel> -C N	Line-level regex hits with context	examples/find-matches.md
lines <path> --start --end	Print a line range	examples/read-lines.md
detect-project [dir]	Identify project type(s) of a directory	examples/projects.md
find-projects [root]	Walk a tree listing every project subdirectory	examples/projects.md
which-project <path>	Walk UP from a file/dir to its nearest enclosing project root	examples/projects.md
config-paths	Print platform-specific project-type config paths	examples/projects.md
mcp	Run as a Model Context Protocol server	MCP server mode

file-search-on --list prints the canonical schema (every attribute, every built-in function, every registered content type) — useful for "what can I filter on?" exploration.

Output formats

file-search-on '...' -o bare        # paths only — pipes well into xargs / fzf
file-search-on '...' -o default     # path \t [content-type] \t size
file-search-on '...' -o verbose     # multi-line per match with every attribute
file-search-on '...' -o json        # NDJSON, one match per line
file-search-on '...' --format '{{.Path}} ({{.WordCount}} words)'

Content search

CEL's standard string methods (contains, startsWith, endsWith, matches) work on every string attribute. Pass --body to populate the body variable from text-based files (markdown, source, csv, json, xml, html, plus is_text) and filter on full file content:

file-search-on 'is_source && body.contains("panic")' --body -d ./internal
file-search-on 'is_source && body.matches("(?i)\\bTODO\\b")' --body
file-search-on '...' --sort word_count --order desc --limit 5

Top-K queries (--sort + --limit) buffer the full result set, sort, then truncate. Without --sort, --limit returns the first N in walk order.

Stats and reconnaissance

file-search-on stats -d ~/Downloads                                    # by content_type (default)
file-search-on stats 'is_image' -d ~/Pictures --group-by camera_make
file-search-on stats 'is_source' -d ./src --group-by language
file-search-on stats 'is_image' -d ~/Pictures --group-by taken_at_year
file-search-on stats --dir ~/docs --dir ~/posts --group-by ext         # multi-root aggregation

group_by keys: content_type (default), ext, dir, language, camera_make, camera_model, lens, artist, album, genre, kernel, binary_format, binary_type, frontmatter_format, plus time-bucket keys (mtime_year/month/day, taken_at_*, sent_at_*, date_*). Unrecognised keys silently fall back to content_type.

Project-type detection

file-search-on detect-project ~/my-app
file-search-on find-projects ~/Code --type go --type rust
file-search-on 'is_source && project_type == "go"' \
    --resolve-projects --prune-build-artefacts -d ~/Code
file-search-on config-paths                       # where to drop user-wide / per-project YAML

--resolve-projects walks up from each file's directory to the nearest project root and sets project_type (string) + project_types (list). --prune-build-artefacts does a pre-walk to discover all project subdirectories under the search root and skips their canonical artefact directories (vendor, node_modules, target, __pycache__, .venv, bin, obj, .terraform, …). Custom project types are user-definable via CEL — drop a YAML at the path printed by config-paths. Full guide: examples/projects.md.

Duplicates and disk-eaters

file-search-on duplicates -d ~/Pictures                   # all duplicates under a tree
file-search-on duplicates 'is_image' -d ~/Pictures        # scope to photos
file-search-on duplicates -d /Volumes/backup --min-size 1048576  # skip files < 1 MiB
file-search-on duplicates -d ~/Downloads -o json

Two-pass: files with unique sizes are skipped before any hashing. With --index-path, hashes are cached alongside (size, mtime) so repeat runs are free.

Common flags

-d <dir> (repeatable for multi-root walks), --exclude <glob> (basename, repeatable), --respect-gitignore, --timeout 30s (partial results returned on expiry), --workers N, --index-path <file.db> (persistent attribute cache — see examples/indexing.md).

Recipes

Focused recipe collections live under examples/:

Recipe file	What's in it
examples/markdown.md	Front-matter (YAML / TOML / JSON), draft flags, tag membership, custom keys
examples/images.md	EXIF camera/lens, GPS bounding boxes, ISO / aperture / focal length, taken-at ranges
examples/audio.md	Artist / album / genre / year, bitrate, sample rate, hi-res filtering
examples/video.md	Codec, resolution, frame rate, duration, MKV vs MP4
examples/office.md	DOCX / XLSX / PPTX / ODT — title, author, language
examples/epub.md	EPUB books — title, author, language; XMP fallback
examples/data.md	JSON arrays vs objects, CSV column membership, XML root elements
examples/text.md	Plain text / log files — line count, word count, big-line caps
examples/notebooks.md	Jupyter (.ipynb) and Apache Zeppelin (.zpln) — cell_count, code_cell_count, kernel, language
examples/projects.md	Project type detection — detect-project / find-projects for go / node / rust / python / terraform / docker-compose / …
examples/cookbook.md	Cross-cutting recipes — dedupe, mixed media filters, pipeline integration
examples/fuzzy-search.md	Fuzzy / phonetic / n-gram similarity matching — levenshtein, soundex, ngrams, ngram_similarity
examples/indexing.md	Persistent attribute index (--index-path) — cold/warm CLI runs, MCP auto-on cache, refresh + inspection
examples/timeouts.md	Timeouts and partial results — CLI --timeout, MCP timeout_seconds, exit codes, cancellation semantics
examples/top-k.md	Top-K queries — --sort + --limit for "biggest 5 videos", "10 most recent photos", etc.
examples/snippets.md	Body previews — --snippet returns the first N lines of text files alongside metadata
examples/exclude.md	Pruning the walk — --exclude basename globs and --respect-gitignore
examples/body-search.md	Content filters — --body exposes file body to CEL; pair with contains / matches (RE2) / startsWith
examples/stats.md	Directory reconnaissance — file-search-on stats aggregates a content-type histogram with totals
examples/group-by.md	Stats bucketed by any attribute — --group-by camera_make, --group-by language, --group-by taken_at_year, etc.
examples/read-lines.md	Print a specific line range from a file — pairs with search to fetch match context
examples/duplicates.md	Find byte-identical files by sha256 — file-search-on duplicates [--min-size N]

A handful of representative one-liners:

# All Markdown files larger than 500 words
file-search-on 'is_markdown && word_count > 500' -d ./docs

# 4K HEVC videos longer than 30 minutes
file-search-on 'is_video && video_height >= 2160 && video_codec == "h265" && duration > 1800' -d ~/Videos

# Photos taken in 2024 with a Sony camera at high ISO
file-search-on 'is_image && camera_make == "SONY" && iso > 1600 && taken_at > timestamp("2024-01-01T00:00:00Z")' -d ~/Pictures

# CSVs with a "revenue" column
file-search-on 'is_csv && csv_columns.exists(c, c == "revenue")' -d ./reports

# French-language office documents
file-search-on 'is_office && language == "fr"' -d ~/Documents

# Audio tracks ≥ 96 kHz (hi-res)
file-search-on 'is_audio && sample_rate >= 96000' -d ~/Music

# Fuzzy: artist tag within 2 edits of "Radiohead" (catches typos)
file-search-on 'is_audio && levenshtein(artist, "Radiohead") <= 2' -d ~/Music

# Phonetic: any author whose name sounds like "Smith"
file-search-on 'is_markdown && soundex(author) == soundex("Smith")' -d ./posts

Combine paths and types — find HTML files inside a build/ directory:

file-search-on 'is_html && dir.contains("build")'

Available attributes

file-search-on --list prints the canonical schema with descriptions. The summary below names every attribute so you know what you can reach in a CEL expression; for recipes and detailed semantics see the per-family pages under examples/.

On every file

name, path, dir, size, ext, content_type, mod_time.

Type predicates

By format — is_markdown, is_json, is_yaml, is_toml, is_xml, is_html, is_pdf, is_csv, is_text, is_image, is_audio, is_video, is_office, is_epub, is_archive, is_binary, is_email, is_source, is_notebook.

By exact filename — is_dockerfile, is_makefile, is_justfile, is_rakefile, is_license, is_changelog, is_contributing, is_codeowners, is_gitignore, is_dockerignore, is_gomod, is_node_manifest, is_cargo_manifest, is_pipfile, is_python_reqs, is_gemfile, is_procfile, is_vagrantfile.

By family — is_build, is_repo_meta, is_ignore, is_manifest, is_platform. Fire alongside the per-type predicate (a Dockerfile is both is_dockerfile and is_build). Same shape as is_image covering every image/* subtype.

Cross-firing: a package.json matches is_node_manifest AND is_json; Cargo.toml matches is_cargo_manifest AND is_toml; LICENSE / CHANGELOG / CONTRIBUTING / requirements.txt match their per-type predicate AND is_text.

Per-family attributes

Family	Attributes
Documents / markup	title, author, language, word_count, line_count, page_count, column_count
Data	json_kind, yaml_kind, yaml_document_count, csv_columns, root_element
Markdown frontmatter	tags, categories, draft, date, frontmatter, frontmatter_format (plus the document title/author/language keys are promoted)
Body filter	body (text content types; opt-in via --body CLI / include_body MCP). Use CEL string methods: body.contains(...), body.matches(...) (RE2), body.startsWith(...), size(body).
Images	img_width, img_height, camera_make, camera_model, lens, taken_at, orientation, gps_lat, gps_lon, iso, focal_length, f_stop, exposure_time
Audio	artist, album, album_artist, composer, year, track, genre, duration, bitrate, nominal_bitrate, sample_rate, channels, bit_depth, replaygain_track_gain, replaygain_album_gain
Video	video_codec, audio_codec, video_width, video_height, frame_rate, rotation, duration, bitrate, nominal_bitrate, is_hdr, color_primaries, color_transfer, subtitles, subtitle_languages
Archives	entry_count, uncompressed_size, top_level_entries, has_root_dir
Binaries	architectures, bitness, binary_format, binary_type, is_dynamically_linked, is_stripped, entry_point
Email	email_to, email_cc, email_message_id, email_in_reply_to, sent_at, attachment_count, email_count (plus shared title / author)
Source code	language, line_count, loc, comment_loc, blank_loc
Notebooks	cell_count, code_cell_count, markdown_cell_count, kernel (plus shared language / title)
Project context	module, go_version, base_image, project_types, project_type (the last two populated by --resolve-projects)

Built-in CEL functions

Function	Returns	What it does
levenshtein(a, b)	int	Edit distance, rune-aware
soundex(s)	string	NARA-standard phonetic 4-char code
ngrams(s, n)	list<string>	Character n-grams as a list
ngram_similarity(a, b, n)	double	Jaccard similarity over n-gram sets, 0.0–1.0
point_in_polygon(lat, lon, polygon)	bool	Ray-casting; polygon is a flat lat,lon,lat,lon,… list

CEL's standard string methods (contains, startsWith, endsWith, matches, size) work on every string attribute. Recipes: examples/fuzzy-search.md.

MCP server mode

The same binary can run as a Model Context Protocol server, exposing the search to any MCP-compatible client (Claude Desktop, IDE plugins, agents). Three transports:

file-search-on mcp                                       # stdio (default; for desktop clients)
file-search-on mcp --transport http --addr :8080         # Streamable HTTP (MCP 2025-03-26)
file-search-on mcp --transport sse  --addr :8080         # HTTP+SSE (DEPRECATED — MCP 2024-11-05)
file-search-on mcp --timeout 90s                         # raise the per-call default (60s out of the box)

Transport	Spec version	When to use
stdio	all	Desktop clients (Claude Desktop, IDE plugins) — the agent spawns the binary as a subprocess.
http	2025-03-26	Network-accessible servers, multi-client, or Docker deployments.
sse	2024-11-05	Legacy clients only. The HTTP+SSE transport was deprecated in the 2025-03-26 spec; new deployments should pick http.

For HTTP and SSE, --addr (default :8080) is the bind address and --path (default /) is the URL prefix. --timeout (default 60s) sets the per-tool-call deadline; per-call timeout_seconds on the search tool input overrides it.

Eleven tools are exposed:

Tool	What it does
search	CEL expression over a directory tree. Supports sort_by / limit (top-K), include_body (full body filter), include_snippet (preview), resolve_projects (project_type per match), prune_build_artefacts, fields (project response to a subset of attributes; path / content_type / size always-on). Returns matches with the full attribute set + partial-result fields.
read_attributes	Attributes for a single path — same shape as one search match. Accepts fields for the same token-saving projection.
read_lines	A specific line range of a file — pairs with search for context around matches.
stats	Histogram + totals for a directory tree, bucketed by group_by (default content_type; full set documented in Usage § Stats).
find_duplicates	Byte-identical files keyed by sha256 — two-pass (size-bucket then hash). Sorted by wasted_bytes desc.
find_matches	Line-level regex (RE2) hits across a tree with context_before / context_after windows. CEL pre-prune (e.g. is_source && language == "go") keeps the regex pass narrow. Replaces the search-then-read_lines dance with one call.
detect_project	Project type(s) of one directory.
find_projects	Walk a tree, list every project subdirectory.
resolve_project_for_path	Walk UP from a file/dir path to the nearest enclosing project root. Useful when an agent has a stray path and needs to know the project context.
list_attributes	The full canonical schema (common, type_specific, frontmatter, functions) plus registered content types.
index_stats	Cache counters for the running server (hits, misses, puts, stales, errors).

Every walking tool (search, stats, find_duplicates, find_matches, find_projects) honours the same partial-result contract: on timeout the call returns cancelled=true with the results gathered so far, never an error. Agents inspect the flag rather than catching exceptions.

The MCP server keeps an attribute cache for its process lifetime — repeated search / read_attributes calls against the same files skip the parse step on the second and later invocations. Pass --index-path to persist the cache across restarts:

file-search-on mcp --index-path ~/.cache/fso/agent.db                    # stdio + persistent cache
file-search-on mcp --transport http --addr :8080 --index-path /var/lib/fso.db

Example Claude Desktop entry in claude_desktop_config.json (stdio):

{
  "mcpServers": {
    "file-search-on": {
      "command": "file-search-on",
      "args": ["mcp"]
    }
  }
}

For HTTP-based clients, point at http://<host>:<port>/ after starting the server with --transport http.

Built on github.com/modelcontextprotocol/go-sdk.

Contributing

The project is small enough to read in an afternoon and welcoming to first-time contributors. See CONTRIBUTING.md for setup, branch/commit conventions, the local CI matrix, and PR expectations. A few quick entry points:

• Open issues filtered by good first issue, help wanted, enhancement.
• New content type or CEL function? CLAUDE.md has step-by-step recipes — search for "Adding a new content type" and "Adding a CEL function".
• Security issue? Please don't open a public issue — see SECURITY.md for the private reporting channel.

Local CI matrix:

go build ./...
go test -race ./...
go vet ./...
golangci-lint run
go fix -diff ./...   # CI enforces empty diff

That's the whole CI matrix locally. Tests run in under 10 seconds; the race detector is on by default.

Architecture map

CLAUDE.md is the canonical architecture map — five internal packages, the CEL evaluator's data shape, the walker's cancellation contract, the MCP server's tool surface, the release pipeline, and where every gotcha is documented. Written for both human and LLM contributors; either audience should find it readable.

The repo also ships with .claude/skills/ — step-by-step templates for the repetitive contributions: adding a content type, extending the CEL schema, adding an MCP tool, cutting a release. Useful whether you're working solo or pairing with an LLM agent.

Releases

Tag-driven via GoReleaser v2 + ko. Pushing vX.Y.Z to main triggers six platform archives, an OCI image at ghcr.io/richardwooding/file-search-on:X.Y.Z, and an auto-commit to the Homebrew tap. Full pipeline documented in CLAUDE.md § Releases.

License

MIT