README
¶
open-nitpick
Self-hosted, model-agnostic pull request review.
Referral link: Synthetic is the route this project recommends and the link pays the author referral credit. Below says what that is worth and what the plain link is.
Documentation: https://jdziat.github.io/open-nitpick/
open-nitpick reads a pull request, reviews it with a model you choose, and posts inline comments. Run it as a GitHub Action, as a CLI in any CI system, as MCP tools inside an agent session, or against your uncommitted working tree.
Nothing is hosted here. You supply the model, which can be one running on your own machine. The prompts and the configuration that drive the review are files in your repository that you can read and edit.
Quick start
# A signed binary for your platform. Releases carry linux and darwin on amd64
# and arm64, plus windows on amd64, each with a Sigstore bundle and a
# checksums file.
v=$(gh release view --repo jdziat/open-nitpick --json tagName -q .tagName)
os=$(uname -s | tr 'A-Z' 'a-z'); arch=$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/')
curl -fsSLo nitpick "https://github.com/jdziat/open-nitpick/releases/download/$v/nitpick_${v}_${os}_${arch}"
chmod +x nitpick && sudo mv nitpick /usr/local/bin/
export LLM_PROVIDER=synthetic LLM_MODEL=hf:moonshotai/Kimi-K3
export SYNTHETIC_API_KEY=syn_...
nitpick init # writes .nitpick.yaml for this repository
nitpick review # reviews your uncommitted changes
Verifying that download is one command and
Development has it. Building from source
instead is go install github.com/jdziat/open-nitpick/cmd/nitpick@latest,
which needs Go 1.25.5 or newer: the go directive in go.mod is a
patch-level floor, so 1.25.4 refuses.
init is optional: with those two variables set, a review runs with no config
file at all. What it buys is a file that names the model, the analyzers this
checkout's languages call for, and the dozen settings most worth changing,
commented. It is a starting point, not the whole surface: the configuration
reference lists every key the loader accepts.
Every value written is already the one in force except two, so deleting a key
changes nothing: the model, which no default supplies, and linters.enabled,
which is matched to this checkout rather than copied from the default. A
Go-only tree is written [golangci-lint] and a Python-only one [ruff], where
the shipped default is both, and under mode: strict that list is the one
whose absence fails a run. It refuses to overwrite an existing file without
-force, and -workflow writes the Actions workflow beside it.
The quickstart uses Synthetic, which serves open-weight models on a flat subscription. Their pricing page read $30 a month for one pack on 2026-09-05, so a review costs nothing per token. That link carries the author's referral code and pays the author referral credit if you sign up through it. Plain https://synthetic.new is the same service at the same price.
You can spend nothing first. nitpick explain-config prints what a review would
send without sending it, and LLM_PROVIDER=ollama runs against a local model.
For OpenRouter, an OpenAI-compatible endpoint or a local model, see
Providers.
Why this exists
Most review bots wrap one vendor's model in a hosted service, priced per seat, with a prompt you cannot read. This one is built the other way round.
- Any model. 16 providers via llm-go-sdk, plus built-in
synthetic(open-weight models on a subscription) andopenrouter(the rest of the catalogue on one key) for 18 in all, plus any OpenAI-compatible endpoint throughbase_url, plus local models viaollamaandllamacpp.nitpick providersprints the list. - Different models for different jobs. A cheap model triages and deduplicates while an expensive one reviews. Most of the available cost saving comes from that split.
- Prompts as configuration. Path-scoped instructions live in your repository
next to the code they describe.
nitpick explain-configprints the exact prompt that will be sent, before you spend a token on it. - Linters as evidence. Findings from golangci-lint, ruff, eslint and semgrep go to the model for triage. Your pull request never receives the raw output.
- Runs offline. The whole engine works against a local git checkout with no credentials and no network beyond the model call.
Documentation
| page | what it covers |
|---|---|
| Usage | reviewing a change or a whole repository, the remediation plan and score, the slop class, and the MCP server for agent sessions |
| GitHub Actions and other CI | the Action, its inputs and permissions, incremental review, forks, and running the CLI in any other CI |
| Configuration | .nitpick.yaml: models per role, budget, related context, personality and instructions, model-family notes, severities |
| Configuration reference | every key the loader accepts, with its type and shipped default, generated from the binary |
| Analyzers | the 33 deterministic tools, how they are detected, isolated and fed to the model as evidence, and what strict mode means |
| Providers and models | Synthetic, OpenRouter, choosing a model by price, routing and ensembles, pinning an upstream, stalls, other gateways, local models |
| Trust model | what a pull request can and cannot change about its own review, and why |
| How a review runs | the pipeline from diff to posted comments |
| Development | tests, commits and releases, iterating cheaply, and evaluating prompts against real models |
Measurement
This project makes empirical claims about review quality, so how those numbers are produced is part of the product.
- Measurement: the fifteen rules a number out of the eval harness has to satisfy before it is worth acting on. Each rule was written after the harness produced a confident wrong number that something believed.
- Against Incumbent: capabilities and measured results by language, with what each iteration changed.
- Remediation: every miss on the benchmark repository, its cause read from the pull request, and the plan.
- Findings: every measurement taken and what it supports. It also tables the instrument bugs found so far. Some of those bugs flattered one side of a comparison, and one put a claim on this page that had to be retracted.
Prefer the judge-free columns. The LLM judge runs at temperature 0 and still scores byte-identical input anywhere from 3.66 to 3.98.
Status
Early. Everything described above is built and runs.
It out-detects a hosted incumbent on both counted corpora: 33 of 41 planted defects against 30 of 41 across 44 real pull requests, and 10 of 13 against 4 of 13 on the held-out fixtures. Noise is the open question: a later run took detection to 36 of 41 and noise findings from 1 to 17. Findings has both comparisons, what they do not support, and the twenty instrument bugs found along the way. Against Incumbent has the costs: about two cents a review on the default reviewer, under a fifth of a cent on the cheap one.
GitHub and a local checkout are the only forges. There is no GitLab provider.
Reviews default to advisory, so nothing blocks a merge until you set fail_on.
License
Apache-2.0.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
benchrepo
command
Command benchrepo turns the eval corpora into a real repository, so the GitHub Action can be benchmarked end to end on real pull requests.
|
Command benchrepo turns the eval corpora into a real repository, so the GitHub Action can be benchmarked end to end on real pull requests. |
|
nitpick
command
Command nitpick reviews a pull request or a local change with a configured model.
|
Command nitpick reviews a pull request or a local change with a configured model. |
|
internal
|
|
|
bundle
Package bundle assembles review context: it selects which changed files to review, attaches their contents, and groups them into batches that fit a token budget.
|
Package bundle assembles review context: it selects which changed files to review, attaches their contents, and groups them into batches that fit a token budget. |
|
config
Package config loads, merges, and validates open-nitpick configuration.
|
Package config loads, merges, and validates open-nitpick configuration. |
|
config/docgen
Package docgen renders the configuration reference from the configuration itself.
|
Package docgen renders the configuration reference from the configuration itself. |
|
converse
Package converse is the @open-nitpick conversation: a comment on a pull request that mentions the reviewer is read as a command (review again, resolve this thread) or as a question the model answers in the same thread, with the change and the thread as its context.
|
Package converse is the @open-nitpick conversation: a comment on a pull request that mentions the reviewer is read as a command (review again, resolve this thread) or as a question the model answers in the same thread, with the change and the thread as its context. |
|
diff
Package diff parses unified diffs into files, hunks, and lines, and maps new-file line numbers onto the positions review APIs require.
|
Package diff parses unified diffs into files, hunks, and lines, and maps new-file line numbers onto the positions review APIs require. |
|
evals
Package evals runs open-nitpick against real models and scores the result.
|
Package evals runs open-nitpick against real models and scores the result. |
|
fix
Package fix applies a published finding by asking a model to rewrite the files the finding names.
|
Package fix applies a published finding by asking a model to rewrite the files the finding names. |
|
fullreview
Package fullreview renders what a whole-tree review found in the order a reader acts on it.
|
Package fullreview renders what a whole-tree review found in the order a reader acts on it. |
|
knowledge
Package knowledge holds what a reviewing model was not necessarily taught.
|
Package knowledge holds what a reviewing model was not necessarily taught. |
|
linters
Package linters runs deterministic analyzers over the changed files and normalizes their output into review findings.
|
Package linters runs deterministic analyzers over the changed files and normalizes their output into review findings. |
|
llm
Package llm builds configured model clients and constrains their output to a schema.
|
Package llm builds configured model clients and constrains their output to a schema. |
|
prompt
Package prompt renders the prompts that drive a review.
|
Package prompt renders the prompts that drive a review. |
|
review
Package review runs a pull request review end to end: select and batch the change, analyze each batch with a model, triage the combined findings, and publish.
|
Package review runs a pull request review end to end: select and batch the change, analyze each batch with a model, triage the combined findings, and publish. |
|
slop
Package slop finds the tells of generated prose without a model: the habits a reader can check on the page, each with a fix.
|
Package slop finds the tells of generated prose without a model: the habits a reader can check on the page, each with a fix. |
|
vcs
Package vcs abstracts the forge a review runs against.
|
Package vcs abstracts the forge a review runs against. |