sast-triage

README

sast-triage

You turned on a SAST scanner — Semgrep, CodeQL, SonarQube — got 400 findings, and turned it off. Most were false positives; nobody had time to check. The AI triage that fixes that exists, but it's a paid tier: Semgrep, GitHub, and Snyk all charge $25–30 per developer per month for it — $15k–18k a year for a 50-developer team.

How it works

graph LR
    subgraph T["sast-triage agent"]
        B["cache lookup"] -->|hit| V["benign | exploitable | uncertain"]
        B -->|miss| E["🤖 LLM reads the code<br/>(read-only tools, bounded loop)"]
        E --> V
    end
    A["findings.sarif<br/>(generated by your SAST scanner)"] --> B
    V --> O["triage-report.md<br/>triage-cache.json"]
    O --> G["Actions run summary —<br/>every verdict + evidence"]
    O --> C["PR check —<br/>fails on new exploitable"]
    O --> H["Security tab · Issues —<br/>opt-in"]
    O --> P["cache diff in a PR —<br/>you approve each suppression"]

The agent only reads code and writes verdicts to two files; your workflow decides where they land. A run that finds nothing new is a green check and a report nobody has to open.

Quick Start

Hosted API (Claude, DeepSeek, Kimi, OpenAI, …)

Add your Claude key as an ANTHROPIC_API_KEY repo secret, then save this as .github/workflows/triage.yml:

name: Triage
on: [pull_request]
permissions:
  contents: read
jobs:
  triage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7

      # Your scanner goes here — CodeQL, gosec, Snyk Code, anything emitting SARIF 2.1.0
      - name: Scan → findings.sarif
        run: |
          pipx install semgrep
          semgrep scan --config=auto --dataflow-traces --sarif-output=findings.sarif

      - name: Triage
        uses: alexpermiakov/sast-triage@v1
        with:
          provider: anthropic
          model: claude-sonnet-5
          api-key: ${{ secrets.ANTHROPIC_API_KEY }}

      - name: Publish the report
        if: always()
        run: cat triage-digest.md >> "$GITHUB_STEP_SUMMARY"

On a different provider? Drop provider and name the endpoint instead:

with:
  base-url: https://api.deepseek.com/v1
  model: DeepSeek-V4-Pro
  api-key: ${{ secrets.DEEPSEEK_API_KEY }}

Self-hosted model (Ollama, vLLM, LM Studio) — no API key, no cost, nothing leaves the runner

Built for your own on-premise runners: the model sits next to the code and nothing crosses the fence.

name: Triage
on: [pull_request]
permissions:
  contents: read
jobs:
  triage:
    runs-on: ubuntu-latest
    services:
      ollama:
        image: ollama/ollama:latest
        ports: ["11434:11434"]
    steps:
      - uses: actions/checkout@v7
      - run: curl -fsS http://localhost:11434/api/pull -d '{"name":"qwen2.5-coder:1.5b-instruct"}'

      # Your scanner goes here — CodeQL, gosec, Snyk Code, anything emitting SARIF 2.1.0
      - name: Scan → findings.sarif
        run: |
          pipx install semgrep
          semgrep scan --config=auto --dataflow-traces --sarif-output=findings.sarif

      - name: Triage
        uses: alexpermiakov/sast-triage@v1
        with:
          base-url: http://localhost:11434/v1
          model: qwen2.5-coder:1.5b-instruct

base-url + model point at any OpenAI-compatible server — swap in vLLM or LM Studio by changing the URL. That 1.5b model on ubuntu-latest proves the plumbing, not the judgment — for real local verdicts use runs-on: [self-hosted, gpu] and the biggest code model your hardware fits, and expect more uncertain than a frontier model leaves behind.

Run it locally on your machine — one-off triage, no CI

Nothing is sent anywhere you didn't name: -base-url is always explicit, so pointing it at local Ollama keeps everything on your machine.

go install github.com/alexpermiakov/sast-triage/cmd/sast-triage@latest

# 1. Scan — anything emitting SARIF 2.1.0 works; semgrep needs no rules setup
pipx install semgrep
semgrep scan --config=auto --dataflow-traces --sarif-output=findings.sarif

# 2a. Triage with a local model via Ollama…
ollama serve &                        # http://localhost:11434
ollama pull qwen2.5-coder:7b
sast-triage -sarif findings.sarif -repo . \
  -base-url http://localhost:11434/v1 -model qwen2.5-coder:7b

# 2b. …or with Claude (needs ANTHROPIC_API_KEY in your environment)
export ANTHROPIC_API_KEY=sk-ant-...
sast-triage -sarif findings.sarif -repo . \
  -provider anthropic -model claude-sonnet-5

cat triage-report.md

What you get

Every run writes triage-report.md — every verdict with its reasoning and clickable file:line evidence, proposed suppressions first so vetoing one is a 30-second action. That one file is where you read the results; keep it with actions/upload-artifact, since it is complete and uncapped.

It also writes triage-digest.md, the same verdicts bounded to 50,000 bytes — that is what the Quick Start pipes to the run summary, because GitHub drops a step summary over 1 MiB and rejects a PR body over 65,536 characters, and a 2,000-finding backlog clears both. The digest leads with exploitable rather than benign, and when findings do not fit it drops the least urgent first and says so in the footer: _Digest truncated to fit: 320 benign omitted._ Set digest: "" to skip it, digest-bytes: to move the cap. Around these:

• ✅ A PR gate — the check fails only on new exploitable findings, so the 400-finding backlog baselined in the committed cache never blocks a merge again (on by default; fail-on-new-exploitable: false turns it off)
• ✅ Human-approved verdicts — the run updates triage-cache.json; commit it from a bot branch and every suppression becomes a readable PR diff nobody can skip
• ✅ Security tab integration (set triaged-sarif:, then upload it) — benign findings arrive dismissed, with the reason as justification
• ✅ GitHub issues for confirmed vulnerabilities (opt-in create-issues:) — one per finding, deduped across runs, with the evidence in the body
• ✅ Any SARIF 2.1.0 scanner — Semgrep, CodeQL, Snyk Code, gosec, Bandit, …

The cache PR, Security tab, and Issues are wiring the Quick Start leaves out to stay short — the workflow this repo runs on itself has all three.

Reference

Everything below has a working default — the quick starts above set model, api-key, and nothing else.

What it costs — a full run on OWASP BenchmarkJava, 2,376 findings

BenchmarkJava is the corpus this is measured on: one scan produces 2,376 findings, and a finding takes ~25k tokens to triage at medium effort — the agent re-sends the conversation each turn, so nearly all of that is input.

Model	Per finding	All 2,376 findings
DeepSeek-V4-Pro	$0.003	$7.60 (measured)
Claude Sonnet 5	~$0.09	~$220 (estimated)
Local Ollama	$0	$0

Only the first run costs anything. Every finding after that is a cache hit until the code it cites changes, so re-running the same 2,376 findings is ~$0 and a PR that adds one new finding costs one finding.

That is the whole backlog, once — against $15k–18k a year for a 50-developer seat licence.

All flags & action inputs

The GitHub Action exposes every flag as an input of the same name, minus the leading dash — -base-url becomes base-url:, -fail-on-new-exploitable becomes fail-on-new-exploitable: — with identical defaults:

Flag	Default	Purpose
-provider	inferred	Only needed for anthropic (Claude's native API); -base-url alone implies openai
-base-url	—	The endpoint. No default — the tool only ever talks to the host you name
-model	—	Required, no default — e.g. claude-sonnet-5 (anthropic), qwen2.5-coder:7b (openai)
-sarif	findings.sarif	SARIF 2.1.0 input
-repo	.	Repository root the findings refer to
-cache	triage-cache.json	Verdict cache (commit it to git)
-report	triage-report.md	Markdown report output — complete, uncapped
-digest	triage-digest.md	Size-bounded report for the step summary / a PR body; "" skips it
-digest-bytes	50000	Digest cap — clears both the 1 MiB summary and 65,536-char PR body limits
-triaged-sarif	—	Verdict-annotated SARIF copy for Code Scanning upload
-effort	medium	Depth: small, medium, large
-max-findings-budget	50	Max findings triaged per run (0 = unlimited)
-parallel	4	Concurrent findings
-fail-on-new-exploitable	on	Exit 3 if this run finds a new exploitable; =false for runs that must not fail (push to main)
-create-issues	off	File GitHub issues for exploitables (needs GITHUB_TOKEN)
-github-repo	$GITHUB_REPOSITORY	owner/name for issue creation
-link-base	—	E.g., https://github.com/owner/repo/blob/<sha>

The action also takes api-key (routed to whichever provider you selected), plus anthropic-api-key / openai-api-key if you'd rather be explicit.

Effort presets — reach for these when you get too many uncertain verdicts

-effort sets how much the agent may read and how long it may work on one finding; when the budget runs out, the verdict falls back to uncertain. So a run that returns more uncertain than you'd like is usually a reason to go up a preset (and a weaker model is a reason to expect them regardless):

Effort	read_file lines	grep matches	token budget	iterations
small	100	25	30k	6
medium	200	50	60k	10
large	400	100	120k	15

-token-budget and -max-iterations override the preset individually.

FAQ

What keeps the agent in bounds?

• Read-only tools — read_file and grep_repo only; no writes, no exec
• Token & iteration budgets — 10 iterations / 60k tokens per finding by default, plus a 50-findings cap per run; the loop always terminates
• Three-valued verdicts — benign requires cited file:line evidence; ambiguity or budget exhaustion → uncertain, never benign
• Cache invalidation on code change — a verdict expires the moment any line it cited changes

How accurate is it?

Accuracy is deliberately asymmetric. The dangerous mistake — suppressing a real vulnerability — has to clear three bars: cited file:line evidence the tool re-verifies, a human merging the cache review PR, and a codeHash that expires the verdict the moment any cited line changes. The cheap mistake — uncertain on something a human could resolve — costs a retry, not a missed vuln. A weaker model shifts verdicts toward uncertain, never toward silent benign.

What if it marks a real vulnerability benign?

Three independent layers have to fail at once:

• the verdict needs cited file:line evidence that the tool re-verifies — no evidence, no benign; ambiguity becomes uncertain, which never suppresses
• the suppression takes effect only after a human merges the cache review PR, where it appears as a readable diff with the reasoning inline
• any change to a cited line breaks the codeHash and expires the verdict — a wrong verdict doesn't outlive the code it misjudged

Why not Semgrep Assistant or GitHub Code Security's AI triage?

Same job, different constraints. Those are good products if you're already paying for the tier that includes them. This exists for everyone else:

• $0 per developer — MIT licence, runs as a step in the CI you already have
• bring your own model, including a local one — code never has to leave your runner
• verdicts live in your repo as a reviewable git history, not in a vendor dashboard
• consumes any SARIF scanner's output, not one vendor's

Which languages does it support?

Whatever your scanner scans. The agent doesn't parse code — it reads it the way an analyst would (read_file, grep_repo), so there is no per-language support matrix: if the scanner produced a finding, it can be triaged.

What makes a PR fail, and who approves verdicts?

PRs fail only on new exploitable findings (exit 3; the gate is on by default, -fail-on-new-exploitable=false turns it off). Verdicts are cached in git (triage-cache.json), keyed to the evidence they cite, and approved by humans via the cache review PR. The committed cache is the baseline, so pre-existing backlog never blocks a PR — only what the PR itself introduces.

What about prompt injection — a comment claiming "this is safe"?

Repo content enters the prompt as evidence, never as instructions. A benign verdict requires cited file:line evidence that the tool re-verifies — prose claims don't meet the bar. The worst case for a fooled model is a wrong verdict, and the dangerous direction (false benign) demands the most proof, is human-approved in a PR, and auto-expires when any cited line changes.

Why commit the cache to git?

• Per-finding granularity (vs. ignore files and inline suppression comments)
• Non-destructive (verdicts, not deletions)
• Carries reason, evidence, timestamps
• PR diffs are audit trails

Which scanners work?

Any of them: it consumes SARIF 2.1.0, whoever produced it.

opengrep and semgrep are the two that are tested — their matchBasedId fingerprints and dataflow traces are used directly, and this repo's own CI runs the same semgrep step the quick start shows. Anything else that speaks SARIF (CodeQL, Snyk Code, gosec, Bandit, Brakeman, SonarQube, ...) works too: when a scanner emits no stable fingerprint, a synthetic one is derived from rule + location, and scanner quirks belong in internal/sarif adapters — a parsing problem, not a prompting problem.

Which models can I use?

Any OpenAI-compatible endpoint out of the box — Ollama, vLLM, LM Studio, DeepSeek, Kimi, OpenAI itself: name it with -base-url and -model, and that's the whole configuration. Claude via -provider anthropic, the one API that isn't OpenAI-shaped. Both are thin adapters over a one-method Client interface (internal/agent/client.go); a new provider is one file implementing Complete. The verdict logic is fail-closed, so a weaker local model produces more uncertain verdicts, never silent benign ones — and the cache records which model decided each verdict.

Honest quality guidance: DeepSeek-V4-Pro is what this repo's CI uses and what produced the verdicts in triage-cache.json — chosen because it triages the whole backlog for a few dollars. The tiny CPU model in the self-hosted quick-start (qwen2.5-coder:1.5b) proves the plumbing, not the judgment — expect mostly uncertain from it. Triaging locally for real means the biggest code model your hardware runs, and still budgeting for more uncertain verdicts than a frontier model leaves behind.

Why doesn't the agent write fixes?

Scope. Triage is a judgment task with a verifiable output contract. Write access would turn a wrong verdict into a wrong commit. Judgment only.