README
¶
ajq
ajq is semantic jq for JSON and NDJSON streams: a fuzzy JSON filter and semantic grep that keeps ordinary jq byte-deterministic, then calls a language model only for explicit semantic operations such as fuzzy matching, bounded classification, limited scoring, and limited normalization.
Use it when you need to classify JSON streams, find records by meaning instead of exact text, or add LLM-enhanced jq predicates without turning the whole pipeline into a prompt.
Usage
Start with ajq --help, then run ajq examples for categorized, copy-pasteable
safe workflows. Coding agents should first run ajq capabilities --json to
inspect the static machine-readable contract. --backend mock is the
deterministic, no-network, no-model path to exercise semantic query syntax
before selecting a real backend.
# Help and version
ajq --help
ajq --version
# Pure jq over JSON stays deterministic
printf '{"users":[{"name":"Ada"}]}' | ajq -r '.users[].name'
# Ada
# Semantic grep for JSON with the deterministic mock backend: safe agent probe
# (no model, network, or API key)
printf '[{"id":1,"msg":"please keep this"},{"id":2,"msg":"drop it"}]' \
| ajq --backend mock -c '.[] | select(.msg =~ "keep") | .id'
# 1
# Inspect semantic plan and estimated backend calls before running an LLM-enhanced jq query
printf '[{"msg":"refund demanded"}]' \
| ajq --backend mock --explain '.[] | select(.msg =~ "angry/frustrated") | .msg'
Run ajq provision once before using --backend local; then the same semantic queries can run against the managed local llama.cpp backend.
Install
Use Homebrew, the release script for prebuilt archives, or Go source:
brew install --cask ricardocabral/tap/ajq
curl -fsSL https://raw.githubusercontent.com/ricardocabral/ajq/main/scripts/install.sh | sh
# manual download: https://github.com/ricardocabral/ajq/releases/latest
go install github.com/ricardocabral/ajq/cmd/ajq@latest
The Homebrew cask is published to the public ricardocabral/tap tap by the
release workflow.
Status
| Area | What works today |
|---|---|
| Backends | Six semantic backends ship: local, mock, ollama, openai, openrouter, and Anthropic via --cloud / --backend anthropic. |
| Cost controls | --explain estimates model calls, --max-calls caps post-dedup judgements, and paid/cloud backends default to a 100-call guardrail. |
| Persistent cache | Semantic judgements are stored on disk under the ajq cache directory; --no-cache disables reads/writes for sensitive runs. |
| Local provisioning | ajq provision downloads or locates the llama.cpp engine and default GGUF model for --backend local on supported platforms. |
| Model management | ajq models list, ajq models pull, and ajq models use manage checksum-pinned local GGUF catalog models. |
| Semantic operators | Fuzzy matching (=~ / sem_match) and bounded sem_classify ship for filters and labels; sem_score and sem_norm are limited to supported contexts. Standalone sem_extract and sem_redact are registered but currently unsupported. |
| Determinism contract | Pure jq paths stay byte-reproducible and never contact AI backends; only explicit semantic operators make schema-constrained, cache-keyed model calls by backend/model/spec/value. |
Docs
Everything beyond the quick start lives on the website:
- Install details
- First pipeline tutorial
- Filter JSON by meaning
- Classify JSON and NDJSON streams
- Semantic functions reference
- CLI reference
- Split execution and determinism
Contributor verification
make test
make build
make website-build
make bench-phase2 runs the CI-safe benchmark harness with the deterministic mock backend. Real local-inference benchmarks are opt-in and require provisioned assets; see the website docs and internal/bench package for details.
License
MIT. See LICENSE.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
ajq
command
|
|
|
internal
|
|
|
backend
Package backend defines the minimal semantic judgment seam used by the split executor.
|
Package backend defines the minimal semantic judgment seam used by the split executor. |
|
backend/anthropicbk
Package anthropicbk implements the official Anthropic Messages API backend.
|
Package anthropicbk implements the official Anthropic Messages API backend. |
|
backend/conformance
Package conformance defines the backend acceptance suite shared by every ajq semantic backend implementation.
|
Package conformance defines the backend acceptance suite shared by every ajq semantic backend implementation. |
|
backend/local
Package local implements a semantic backend that talks to a warm localhost llama-server daemon over HTTP.
|
Package local implements a semantic backend that talks to a warm localhost llama-server daemon over HTTP. |
|
backend/oai
Package oai implements a thin OpenAI-compatible chat/completions backend.
|
Package oai implements a thin OpenAI-compatible chat/completions backend. |
|
backend/ollamabk
Package ollamabk implements a thin client for Ollama's native /api/chat API.
|
Package ollamabk implements a thin client for Ollama's native /api/chat API. |
|
bench
Package bench is the Phase 2.5 latency and throughput benchmark harness for ajq.
|
Package bench is the Phase 2.5 latency and throughput benchmark harness for ajq. |
|
cache
Package cache provides in-memory semantic result memoization for split execution.
|
Package cache provides in-memory semantic result memoization for split execution. |
|
cachepath
Package cachepath resolves the shared ajq cache root used by provisioning, daemon-managed assets, and persistent judgement cache files.
|
Package cachepath resolves the shared ajq cache root used by provisioning, daemon-managed assets, and persistent judgement cache files. |
|
config
Package config loads ajq's optional TOML configuration file and resolves settings precedence across command-line flags, environment variables, config file values, and built-in defaults.
|
Package config loads ajq's optional TOML configuration file and resolves settings precedence across command-line flags, environment variables, config file values, and built-in defaults. |
|
daemon
Package daemon manages the lifecycle of a local llama-server process used as a warm localhost daemon by ajq.
|
Package daemon manages the lifecycle of a local llama-server process used as a warm localhost daemon by ajq. |
|
desugar
Package desugar rewrites ajq-only surface syntax into jq function-core syntax before gojq parses the query.
|
Package desugar rewrites ajq-only surface syntax into jq function-core syntax before gojq parses the query. |
|
engine
Package engine wires ajq's stdin framing, pure-jq evaluation, and output serialization together.
|
Package engine wires ajq's stdin framing, pure-jq evaluation, and output serialization together. |
|
explain
Package explain renders ajq's stable query explanation output.
|
Package explain renders ajq's stable query explanation output. |
|
jq
Package jq wraps github.com/itchyny/gojq behind the small execution contract ajq needs for deterministic pure-jq queries.
|
Package jq wraps github.com/itchyny/gojq behind the small execution contract ajq needs for deterministic pure-jq queries. |
|
plan
Package plan builds ajq's semantic execution plan from gojq queries.
|
Package plan builds ajq's semantic execution plan from gojq queries. |
|
pricing
Package pricing provides static, dated cost estimates for semantic backend calls.
|
Package pricing provides static, dated cost estimates for semantic backend calls. |
|
provision
Package provision implements first-run provisioning of local inference assets for ajq: it locates or downloads a platform-appropriate llama-server engine and a default GGUF model into the ajq cache directory, verifies their integrity, and reports progress.
|
Package provision implements first-run provisioning of local inference assets for ajq: it locates or downloads a platform-appropriate llama-server engine and a default GGUF model into the ajq cache directory, verifies their integrity, and reports progress. |
|
schema
Package schema turns semantic operator contracts into deterministic, grammar/schema-constrained output shapes for local inference.
|
Package schema turns semantic operator contracts into deterministic, grammar/schema-constrained output shapes for local inference. |
|
semantics
Package semantics defines ajq's semantic operator metadata without any jq parser or executor dependencies.
|
Package semantics defines ajq's semantic operator metadata without any jq parser or executor dependencies. |
|
testharness
Package testharness verifies ajq golden-output fixtures.
|
Package testharness verifies ajq golden-output fixtures. |
Click to show internal directories.
Click to hide internal directories.