distill

README

distill

Compile a folder of detailed per-rule markdown files into one short, AI-targeted markdown file. Uses claude --print to compress each section into bullet form; the output is regenerated end-to-end every run.

Built for CLAUDE.md generation. Source files stay human-friendly (one rule per file, with rationale, examples, anti-patterns). The output is dense, scannable, token-cheap — loaded into every AI agent session, derived, never edited by hand.

Install

go install github.com/bborbe/distill@latest

Requires the claude CLI on $PATH.

Usage

distill --source <rule-folder> --output <claude-md-file> --title "Global Preferences"

Flag	Required	Meaning
--source <dir>	yes	Folder containing source rule .md files
--output <file>	yes	Output markdown file path (overwritten every run)
--title <text>	no	Top-level # <text> heading written under the auto-generated warning
--model <name>	no	Claude model name (default: sonnet)
--verbose	no	Print per-batch prompts + Claude responses to stderr
--no-cache	no	Bypass the content-hash cache (validation and anti-injection always run)

Exit codes: 0 ok · 1 failure (parse / Claude / IO / validation) · 2 usage error (missing required flag).

Source rule format

Each rule is a markdown file with distill: frontmatter declaring which section it belongs to, plus a long-form body that the LLM compresses:

---
distill:
  section: Git
  order: 10
  id: no-git-c-flag
---

# Rule: No `git -C` Flag

Never use `git -C /path …` or `cd /path && git …` — both break the Bash
auto-approval matcher. The fix is to `cd /path` in its own Bash call, then
run `git status` / `git diff` in separate calls.

Frontmatter field	Required	Meaning
section	yes	Becomes the ## <section> heading the bullet lives under
order	no	Sort key within section (default 100)
id	no	Stable identifier; defaults to filename stem
disabled	no	Skip emission when true

Files without a distill: frontmatter block are silently skipped (they're treated as docs in the source folder).

Output shape

distill writes the file from scratch every run:

<!--
  AUTO-GENERATED by distill — do not edit by hand.
  Source: <absolute source dir>
  Regenerate: distill --source <source> --output <output>
-->

# <title>

## Git

- **No Git -C Flag.** Never `git -C /path` or `cd /path && git ...`; first `cd /path` in its own Bash call, then `git` in separate calls.

## Operational

- **English Only.** Reply in English even when the user writes German; no code-switching.
- ...

Sections are ordered by minimum order within the section, then alphabetically. Bullets within a section sort by (order, id).

The output file is owned end-to-end by distill. Don't hand-edit it — your edits will be lost on the next regeneration. Edit the source rule files instead, then re-run.

Cache

distill keeps a content-hash cache at <source-dir>/.distill-cache.json. On a re-run with unchanged sources it serves every rule from cache, spawns zero claude processes, and writes byte-identical output.

Whether to commit .distill-cache.json into your rule repo or gitignore it is an operator decision — distill's behavior is identical either way (a missing cache triggers a cold run with a stderr warning).

Changing system.md (the compression instructions) or switching --model invalidates the whole cache and forces a cold recompile on the next run, because the cache hash folds in the system prompt content and model name.

Use --no-cache to bypass cache load and save for a single run (useful when you want to force a fresh compression of all rules). Validation and the anti-injection claude flags always run regardless of --no-cache.

Make integration

A typical setup with two make generate targets — one for global ~/.claude/CLAUDE.md, one for vault ~/Documents/Obsidian/Personal/CLAUDE.md:

# ~/.claude/Makefile
SOURCE := $(HOME)/.claude/claude-md-rules
OUTPUT := $(HOME)/.claude/CLAUDE.md
TITLE  := Global Preferences

.PHONY: generate
generate:
	@distill --source "$(SOURCE)" --output "$(OUTPUT)" --title "$(TITLE)"
	@echo "wrote $(OUTPUT)"

cd ~/.claude && make generate

Edit protection hook

Generated output is overwritten on every run, so hand-edits are silently lost. hooks/deny-generated-file-edits.sh is a Claude Code PreToolUse hook that prevents this: it denies Edit / Write / NotebookEdit on any distill-generated file and tells the agent where the real source lives.

Detection is content-based, not path-based — the hook reads the first 2 KB of the target file and looks for the AUTO-GENERATED by distill banner. Any generated file is protected wherever it lives, with no per-file configuration. Because the banner also carries Source: and Regenerate:, the denial message quotes the file's own regenerate command instead of hardcoding paths.

Install:

cp hooks/deny-generated-file-edits.sh ~/.claude/hooks/
chmod +x ~/.claude/hooks/deny-generated-file-edits.sh

Wire it up in ~/.claude/settings.json:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Edit|Write|NotebookEdit",
        "hooks": [
          { "type": "command", "command": "~/.claude/hooks/deny-generated-file-edits.sh" }
        ]
      }
    ]
  }
}

Requires jq. Files without the banner pass through untouched, so the hook is safe to enable globally.

Non-goals

• No --check mode.
• No watch / daemon mode. One-shot CLI.
• No multi-target broadcasting from a single source file.

Development

make precommit   # lint + format + generate + test + checks
make test        # tests only

See docs/spec.md for the full contract and docs/dod.md for the Definition of Done.

Converting an existing hand-written CLAUDE.md to distill: docs/adopting-distill.md.

License

BSD-style — see LICENSE.

Documentation

Overview

Command distill compiles a folder of per-rule markdown files into one short AI-targeted markdown file by sending cache-miss rules through `claude --print` and assembling the returned per-rule bullets into one regenerated output file.