fak

module

v0.51.1-0...-1140fac Latest Latest Go to latest Published: Sep 6, 2026 License: Apache-2.0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/anthony-chaudhary/fak

Links

Open Source Insights

README ¶

fak — the fast local runtime for coding agents

fak is an agent runtime: one binary puts a fast, cache-accelerated boundary between your coding agent and every tool call.

In short: run coding agents locally with workflow batching and cache reuse, protected by a default-deny capability floor (blocking unauthorized actions).

Try fak

Run the offline proof with no key, model, or GPU:

go build -o fak ./cmd/fak
./fak agent --offline  # -> task completed (booked)

The poisoned result and destructive operation are blocked; safe tasks complete normally.

Or wrap the agent you already run with one command. In this example, fak forwards Codex subscription credentials with no API key required and blocks tools outside the allowed policy. The capability floor stops unsafe calls without breaking the task:

fak guard -- codex

The agent keeps working inside that boundary. See the interactive showcase for the guided tour.

Latest hardware results — 2026-09-05

The front page shows one row per supported hardware family. Latest means the newest committed performance receipt for that platform, not the newest code change. A row can be historical or held when no newer quality-complete measurement exists. The table reports measured throughput, for example 7.61 decode tok/s on Mac or 111.9 tok/s on Hopper H100, with claim boundaries beside each result and links to its receipt.

Platform	Latest witnessed result	Status	Details
Mac	Qwen3.8-27B Q4_K_M on an Apple M3 Pro: 7.61 decode tok/s (+3.1% vs llama.cpp 7.38, MLX 8.07) and 12.6 ms prefix TTFT, observed 2026-09-03.	Verified matched-envelope single-stream decode leads llama.cpp Metal; RadixAttention prefix caching eliminates repeat prefill.	Mac result
AMD	Qwen3.6-27B on an RX 7600: the measured pure-fak microbench reached 1.15–1.24 decode tok/s versus 0.99 for the local llama.cpp Vulkan baseline, observed 2026-06-19.	Witnessed in that narrow microbench; not a broad quality or full-model parity claim. Qwen3.8 awaits a comparable AMD receipt.	AMD result
NVIDIA	Hopper H100 Q8_0 decode reached 111.9 tok/s (+17.4% vs f32); live A100 Qwen3.8-27B prefix reuse achieved 4.84× TTFT speedup, observed 2026-09-05.	Witnessed on physical GCP H100 (a3-highgpu-1g) & A100; matched Q8 device GEMV and 50-agent concurrency grid (91/91 ok).	NVIDIA result

Read the status column before comparing rates: results compare matched envelopes against explicit baseline runtimes on identical hardware.

Use the benchmark index for hardware history and model-specific results. Use BENCHMARK-AUTHORITY.md for claim boundaries and canonical receipts. For newcomer Mac guidance and head-to-head Apple Silicon Metal measurements, see the Mac agent UI guide and the three-way Mac benchmark.

Open-source memory overflow landscape

Most LLM serving engines treat memory overflow as a slow host-memory fallback with multiple CPU bounce copies. fak implements hardware-native, zero-copy peer-to-peer DMA directly between NVMe storage and GPU VRAM:

Framework	Storage / Offload DMA Path	Host DRAM Copies	Predictive Prefetching	Hybrid Attention + GDN Linear State	Target Workload
fak (native)	GPU Direct NVMe P2PDMA (BaM architecture)	0 (strictly zero)	Yes (asynchronous pipeline)	Yes (bit-exact full + linear)	Interactive, real-time agent coding loops
vLLM	Host DRAM block swapping (`swap_blocks`)	2–3 copies	No (reactive)	No (Transformer KV only)	High-throughput data-center batching
DeepSpeed ZeRO	Async CPU `aio` offload via pinned DRAM buffers	2 copies	Coarse (layer-level weights)	No (static forward layers only)	Multi-node distributed training / inference
FlexGen	3-tier offload (GPU ↔ CPU ↔ Disk)	2–3 copies	Zigzag batch schedule	No (attention matrices only)	Extreme high-latency batch throughput
TensorRT-LLM	NVIDIA GPUDirect Storage (`libcufile.so`)	0 (NVIDIA only)	Yes (NVIDIA GDS)	Partial (Transformer KV)	NVIDIA enterprise data centers only
llama.cpp	OS `mmap` demand paging & CPU fallback	2 copies (OS cache)	No (kernel readahead)	Basic (CPU fallback layers)	Local desktop CPU/GPU inference

Why run coding agents on fak

Workflow batching and cache reuse: Multi-agent coding loops reuse prompt context across turns, achieving 4.1× vs tuned baselines with 86.7% cache hit rates. Instead of re-reading codebases on every turn, fak keeps shared prefixes hot and trims stale context.
Zero-copy GPU Direct storage overflow: Run models far exceeding physical GPU VRAM without CPU memory thrashing. Built on a BaM-style accelerator storage architecture, fak maps NVMe submission queues directly in GPU VRAM and streams paged KV caches and hybrid linear attention states over peer-to-peer PCIe DMA without host DRAM bounce buffering (StagingCopyCount == 0). See the GPU Direct overflow specification.
Local execution on your hardware: Run models directly with native inference across Apple Silicon, AMD, and NVIDIA. Cut per-token API bills and keep your code private on your own machine.
Default-deny capability floor: Protect your workspace from unintended terminal commands or file edits. Every tool call is checked against a default-deny (block everything unless allowed) policy before it runs. Drop-in support wraps existing agents like Claude Code, Codex, Aider, and Cursor with zero rewrites.

Native inference provides direct execution on local silicon, with external engines supported as an explicit reference; see the native inference goal for details.

Default priorities & operating modes

fak is organized around a focused four-tier default priority hierarchy:

fak all in one (serving and harness + memory — the "one touch" thing): The primary focus — a single-binary "one touch" deployment (fak up) bundling model serving, agent harness governance, and persistent memory. Verified on Terminal-Bench 4: 100.0% (5/5) solve rate vs OpenCode + llama.cpp 60.0% (3/5), reducing prompt tokens by 83.5% through in-kernel vDSO context caching (fak bench tb4).
fak serving only: High-performance model inference runtime (fak serve), disaggregated gateway, KV-cache/context MMU acceleration, and native model execution.
fak harness only: Standalone agent harness and governance substrate (fak guard), default-deny capability floor, and tool adjudication over external models.
other things: Standalone utilities, peripheral tools, benchmarks, and off-spine extensions.

Install and configure

# macOS / Linux
curl -fsSL https://raw.githubusercontent.com/anthony-chaudhary/fak/main/install.sh | sh

# Any host with Go 1.26+
go install github.com/anthony-chaudhary/fak/cmd/fak@latest

# Inspect the shipped profiles
fak agent profiles

Tune agent execution with built-in work and output profiles that cut token waste and resist unnecessary dependencies:

fak manage --output-profile caveman:medium --work-profile ponytail:high -- codex \
  "Remove the duplicate cache without adding a dependency."

Balanced defaults are ponytail:medium for work discipline and caveman:medium for concise responses. See work profiles, response profiles, or the harness guide to build a named agent around the same boundary.

Going deeper

If you want to…	Start here
Check what is shipped, limited, or planned	Status · claims · feature matrix
Browse performance evidence	Mac · AMD · NVIDIA · all benchmarks
Connect another agent or model	Codex · Claude Code · all integrations
Understand the runtime	Architecture · capability map · CLI reference
Learn in prerequisite order	Start here · learning path · documentation index
Build on fak	Go API · harness contract · contributing

Apache-2.0 licensed.

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
cmd
a2ademo command Command a2ademo is a no-key, no-model proof of fak's in-kernel agent-to-agent message channel (internal/a2achan): one capability-floored, Ref-backed mailbox delivering an addressed value from one agent to another — adjudicated by the SAME default-deny floor that gates a tool call.	Command a2ademo is a no-key, no-model proof of fak's in-kernel agent-to-agent message channel (internal/a2achan): one capability-floored, Ref-backed mailbox delivering an addressed value from one agent to another — adjudicated by the SAME default-deny floor that gates a tool call.
agentbenchdemo command Command agentbenchdemo is the performance micro-benchmark of fak's agentic spine: how much does the kernel's per-tool-call adjudication actually COST? It folds a fixed plan of tool calls through the REAL kernel (the same internal/agentdemo path cmd/timewolfdemo and `fak preflight` use — adjudicator.Default.SetPolicy + a live kernel.Fold per call), times the loop, and reports the per-call adjudication cost — the "self-tax" the safety floor adds to an agent's critical path.	Command agentbenchdemo is the performance micro-benchmark of fak's agentic spine: how much does the kernel's per-tool-call adjudication actually COST? It folds a fixed plan of tool calls through the REAL kernel (the same internal/agentdemo path cmd/timewolfdemo and `fak preflight` use — adjudicator.Default.SetPolicy + a live kernel.Fold per call), times the loop, and reports the per-call adjudication cost — the "self-tax" the safety floor adds to an agent's critical path.
agentdojoprobe command Command agentdojoprobe scores ATTACKER-PROPOSED prompt-injection attacks through the real fak red-team stack.	Command agentdojoprobe scores ATTACKER-PROPOSED prompt-injection attacks through the real fak red-team stack.
agentdojoredteam command Command agentdojoredteam runs the dynamic AgentDojo-style red-team battery (internal/agentdojo) against the stacked defense and prints the PER-ATTACK verdict stream, then folds every outcome into a frozen harvest LabelRow corpus.	Command agentdojoredteam runs the dynamic AgentDojo-style red-team battery (internal/agentdojo) against the stacked defense and prints the PER-ATTACK verdict stream, then folds every outcome into a frozen harvest LabelRow corpus.
agenticbench command Command agenticbench emits the #868 parent rollup over committed agentic benchmark artifacts.	Command agenticbench emits the #868 parent rollup over committed agentic benchmark artifacts.
archreportdemo command Command archreportdemo demonstrates fak's enforced architecture graph without a key, network, Git checkout, or mutable repository state.	Command archreportdemo demonstrates fak's enforced architecture graph without a key, network, Git checkout, or mutable repository state.
attnsnrsi command Command attnsnrsi closes the attention-S/N RSI loop in SHADOW (#867).	Command attnsnrsi closes the attention-S/N RSI loop in SHADOW (#867).
auditreceipt command
batchbench command Command batchbench measures the AGGREGATE decode throughput of the multi-user batched decode (internal/model.BatchSession) as a function of batch size B — the "continuous batching" / multi-user serving regime MODEL-BASELINE-RESULTS.md scoped out as "vLLM's claim, not fak's".	Command batchbench measures the AGGREGATE decode throughput of the multi-user batched decode (internal/model.BatchSession) as a function of batch size B — the "continuous batching" / multi-user serving regime MODEL-BASELINE-RESULTS.md scoped out as "vLLM's claim, not fak's".
benchscore command
browseractionbench command Command browseractionbench runs a browser/computer-use action-mediation smoke through fak adjudication.	Command browseractionbench runs a browser/computer-use action-mediation smoke through fak adjudication.
cachedemo command Command cachedemo renders the multi-turn shared-cache-savings demo: it reads the durable, WITNESSED gateway-usage and cache-savings ledgers and tells the per-turn story of how a single fak-guarded coding session earns cache value across turns — the provider prompt cache reading a stable prefix back, AND fak's own authored slice (per-fire compaction trim + tool prune) holding the re-sent window inside budget.	Command cachedemo renders the multi-turn shared-cache-savings demo: it reads the durable, WITNESSED gateway-usage and cache-savings ledgers and tells the per-turn story of how a single fak-guarded coding session earns cache value across turns — the provider prompt cache reading a stable prefix back, AND fak's own authored slice (per-fire compaction trim + tool prune) holding the re-sent window inside budget.
causalbench command Command causalbench is the end-to-end demonstrator for fak's CAUSAL invalidation-on-external-write: a tool RESULT cached under an external world-state witness (a git commit / blob hash / etag) is evicted byte-exact — and only it — the moment a later external write REFUTES that witness, while every sibling cached under an unrefuted witness stays warm.	Command causalbench is the end-to-end demonstrator for fak's CAUSAL invalidation-on-external-write: a tool RESULT cached under an external world-state witness (a git commit / blob hash / etag) is evicted byte-exact — and only it — the moment a later external write REFUTES that witness, while every sibling cached under an unrefuted witness stays warm.
caveman-pairwise-judge command
cfgprobe command Command cfgprobe prints the MoE/dense FFN config axes a GGUF resolves to, so a dimension bug (e.g.	Command cfgprobe prints the MoE/dense FFN config axes a GGUF resolves to, so a dimension bug (e.g.
coalescebench command Command coalescebench projects the aggregate net-tok/s(B) curve for an SSD-offloaded MoE served to B concurrent agents, by driving the deterministic cross-agent expert-cache coalescing simulator (deepseekv4moe.SimulateExpertCacheBatch, #5244) over a SYNTHETIC GLM-5.2-shaped router and applying the §2 roofline of docs/notes/MOE-SSD-MULTI-AGENT-NET-TOKS-2026-07-18.md:	Command coalescebench projects the aggregate net-tok/s(B) curve for an SSD-offloaded MoE served to B concurrent agents, by driving the deterministic cross-agent expert-cache coalescing simulator (deepseekv4moe.SimulateExpertCacheBatch, #5244) over a SYNTHETIC GLM-5.2-shaped router and applying the §2 roofline of docs/notes/MOE-SSD-MULTI-AGENT-NET-TOKS-2026-07-18.md:
codesearch command Command codesearch is a standalone front door to the epic #3434 code-intelligence engine (internal/codesearch): trigram regex/literal search, AST shape queries, call-graph traversal, and RRF-fused feature retrieval over a Go tree.	Command codesearch is a standalone front door to the epic #3434 code-intelligence engine (internal/codesearch): trigram regex/literal search, AST shape queries, call-graph traversal, and RRF-fused feature retrieval over a Go tree.
conceptbench command Command conceptbench is the runnable wrapper for the FAK-concept fidelity benchmark (epic #2721, issue #2740): it runs a set of models against a set of fak-concept tasks (stamp/lane/refusal/verdict/handoff/witness) and emits a fak.conceptbench.report.v1 carrying a per-(model x concept) pass@1 leaderboard behind the #868 result-claim honesty gate.	Command conceptbench is the runnable wrapper for the FAK-concept fidelity benchmark (epic #2721, issue #2740): it runs a set of models against a set of fak-concept tasks (stamp/lane/refusal/verdict/handoff/witness) and emits a fak.conceptbench.report.v1 carrying a per-(model x concept) pass@1 leaderboard behind the #868 result-claim honesty gate.
crossauditcalibrate command
crossauditdogfood command
crossauditfixture command
ctxbench command Command ctxbench runs the fak security gates over a corpus of tool calls and tool results.	Command ctxbench runs the fak security gates over a corpus of tool calls and tool results.
ctxdemo command Command ctxdemo is the live, on-box demo of fak's value in the MULTI-AGENT, MULTI-TURN, LONG-CONTEXT regime — the one where the context CHANGES every turn as tool calls land heterogeneous, variable-sized results.	Command ctxdemo is the live, on-box demo of fak's value in the MULTI-AGENT, MULTI-TURN, LONG-CONTEXT regime — the one where the context CHANGES every turn as tool calls land heterogeneous, variable-sized results.
ctxplanbench command Command ctxplanbench measures the ctxplan planned VIEW over REAL Claude Code session transcripts — the empirical counterpart to internal/ctxplan/scaling.go's synthetic Params model (issue #559).	Command ctxplanbench measures the ctxplan planned VIEW over REAL Claude Code session transcripts — the empirical counterpart to internal/ctxplan/scaling.go's synthetic Params model (issue #559).
ctxplandemo command Command ctxplandemo is the runnable demonstrator for the context PLANNER: it treats the current turn as an O(1) materialized VIEW over a lossless history store, and shows the three things that make that more than a slogan —	Command ctxplandemo is the runnable demonstrator for the context PLANNER: it treats the current turn as an O(1) materialized VIEW over a lossless history store, and shows the three things that make that more than a slogan —
customlintfixture command Command customlintfixture is the hostile-behavior fixture for the custom-linter ABI.	Command customlintfixture is the hostile-behavior fixture for the custom-linter ABI.
cxlpooldemo command Command cxlpooldemo is a no-model, no-GPU proof of the value a SWITCH-POOLED, multi-host shared memory tier (a CXL.mem / CXL-switch pool) adds to fak's hardware-aware cache once a KV cache is shared across a FLEET of tenants — the multi-tenant counterpart of cmd/hwcachedemo's single-stream demote-not-evict proof.	Command cxlpooldemo is a no-model, no-GPU proof of the value a SWITCH-POOLED, multi-host shared memory tier (a CXL.mem / CXL-switch pool) adds to fak's hardware-aware cache once a KV cache is shared across a FLEET of tenants — the multi-tenant counterpart of cmd/hwcachedemo's single-stream demote-not-evict proof.
dashboarddemo command
deletioncert command Command deletioncert is the end-to-end demonstrator for fak's provable-deletion receipt.	Command deletioncert is the end-to-end demonstrator for fak's provable-deletion receipt.
demorace command Command demorace is the live, on-box demo of fak's value point: REUSE.	Command demorace is the live, on-box demo of fak's value point: REUSE.
devexmeter command
devfresh command
diagtok command Command diagtok is a throwaway diagnostic: it loads a GGUF via the same path simpledemo uses, then (1) round-trips a known string through the embedded tokenizer and (2) greedily decodes a ChatML prompt while printing each raw token id next to its decode.	Command diagtok is a throwaway diagnostic: it loads a GGUF via the same path simpledemo uses, then (1) round-trips a known string through the embedded tokenizer and (2) greedily decodes a ChatML prompt while printing each raw token id next to its decode.
disambiguationdemo command
dispatchworker command guard.go — front each dispatch worker with the kernel (`fak guard`), a Go port of the dogfood-guard family in tools/dispatch_worker.py.	guard.go — front each dispatch worker with the kernel (`fak guard`), a Go port of the dogfood-guard family in tools/dispatch_worker.py.
dojorsi command Command dojorsi is the dojo-RSI loop's WORKTREE ARM — Phase 2 of docs/fak/dojo-rsi-loop.md (issue #1024).	Command dojorsi is the dojo-RSI loop's WORKTREE ARM — Phase 2 of docs/fak/dojo-rsi-loop.md (issue #1024).
dropindemo command Command dropindemo is the splashy, on-box demo of fak's DISTRIBUTION story — the "drop-in" entry point.	Command dropindemo is the splashy, on-box demo of fak's DISTRIBUTION story — the "drop-in" entry point.
extseamsdemo command Command extseamsdemo makes fak's extension choices and trust boundaries inspectable without loading third-party code.	Command extseamsdemo makes fak's extension choices and trust boundaries inspectable without loading third-party code.
fabricmapdemo command fabricmapdemo is a runnable proof of direction-agnostic, composable data movement.	fabricmapdemo is a runnable proof of direction-agnostic, composable data movement.