aq

README

#+TITLE: aq --- Ambient Agent Queue
#+SUBTITLE: Gossip layer (L1.5) for multi-agent development
#+DATE: 2026-03-13
#+STARTUP: overview
#+OPTIONS: toc:2 num:nil

#+ATTR_HTML: :width 100%
[[file:assets/aq-banner.png]]

[[https://github.com/jwalsh/aq/actions/workflows/ci.yml][https://github.com/jwalsh/aq/actions/workflows/ci.yml/badge.svg]] [[https://goreportcard.com/report/github.com/jwalsh/aq][https://img.shields.io/badge/go%20report-A+-brightgreen.svg]] [[https://pkg.go.dev/github.com/jwalsh/aq][https://pkg.go.dev/badge/github.com/jwalsh/aq.svg]] [[https://github.com/jwalsh/aq/blob/main/LICENSE][https://img.shields.io/github/license/jwalsh/aq.svg]] [[https://github.com/jwalsh/aq][https://img.shields.io/github/go-mod/go-version/jwalsh/aq.svg]]

* Overview

=aq= is a gossip layer for multi-agent development. Agents broadcast
intent --- which conjecture, what claim, what phase --- via
filesystem-backed channels so peers detect semantic conflicts before
they become merge conflicts. The conflict surface is *ideas and
architecture*, not files. It is the mDNS of coding agents:
"does anyone know this address?" Nobody is required to answer.

=aq= occupies *L1.5* in a seven-concern stack, between =bd= (L1,
authoritative work state) and =JITIR= (L2, knowledge retrieval).
Three primitives interlock: =sb= (where am I?), =cprr= (why am I
here?), =aq= (who else knows?). Broadcasts carry no obligation.
All expire via TTL. Silence is normal.

#+begin_quote
aq is gossip, not coordination. Agents broadcast presence; nobody is
required to listen.
#+end_quote

** The TRAMP Principle

Emacs TRAMP --- Transparent Remote Access, Multiple Protocols --- coined
the design pattern =aq= follows: *the consumer of an operation should
not change behavior based on where the data lives or how it got there.*
A =find-file= in TRAMP works the same on a local path, an SSH host, or
a Kubernetes pod. The caller never knows.

=aq= applies this to presence. =announce()= writes a broadcast.
=read_active()= lists what is live. Whether the underlying transport is
a local directory, an NFS mount, or a NATS subject is invisible to the
caller --- the =Channel= interface (Publish, Subscribe, Active) abstracts
it, and the filesystem is always the fallback. The agent's code never
changes. See [[file:docs/TRANSPORTS.org][docs/TRANSPORTS.org]] for the full treatment.

** What aq is NOT

- *Not file locking* (VSS, RCS, SourceSafe) --- the collaboration
  problem is conceptual and architectural, not about who has a file
  checked out. If =aq= reduces to "which files are touched," it is
  VSS circa 2000 with extra steps. The real conflict surface is
  *intent*: two agents pursuing incompatible ideas, even if they
  touch zero shared files.
- *Not an orchestrator* (Temporal, Airflow, CrewAI) --- orchestrators
  create coupling and single points of failure. =aq= is peer-to-peer
  broadcast.
- *Not a message broker* (Redis, RabbitMQ, NATS) --- brokers require a
  running service. =aq= degrades to filesystem-only with zero
  dependencies.
- *Not Google Wave's OT* (operational transform) --- Wave's value was
  the ambient presence stream, not the data model. =aq= takes the
  presence semantics and drops the complexity.
- *Not a task queue* (Celery, Bull) --- task queues assign work. =aq=
  does not assign; it broadcasts what agents are already doing.

* Quick Start

** Install

#+begin_src bash
# Install from GitHub (requires Go 1.23+)
go install github.com/jwalsh/aq@latest

# Or build from source
git clone https://github.com/jwalsh/aq.git
cd aq
make build && make install   # installs to ~/.local/bin
#+end_src

** Usage

#+begin_src bash
# Initialize the channel directory
aq init

# Announce your intent — what you're thinking about, not just what you'll touch
aq announce -c C-1 --claim "adding OAuth2 flow to replace session tokens" --phase proof

# Whisper a low-priority thought (60s TTL)
aq whisper -c C-3 --claim "considering whether Wave semantics need OT"

# Check: is anyone else working on a related idea?
aq check -c C-2 --claim "removing legacy auth middleware"

# See who's broadcasting — what are agents thinking about right now?
aq status

# Health check
aq doctor

# Machine-readable output (any command)
aq status --json
#+end_src

The primary signal is the *conjecture and claim* --- what you intend,
what you believe, what you're investigating. Files (=-f=) are optional
supporting context, not the headline. Two agents can conflict on zero
shared files if their claims are architecturally incompatible ("adding
OAuth" vs "removing auth middleware"). Two agents touching the same file
may not conflict at all if their conjectures are independent.

=aq= is about *feels, not files*. The gossip carries intent.

Wire format details (JSON schema, filename conventions, lifecycle) are
in [[file:spec-v2.org][spec-v2.org]]. You don't need to understand the protocol to use the tool.

* Three-Primitive Interlock

=aq= does not exist alone. It is the temporal layer in a three-primitive
system:

| Primitive | Dimension | Question        | Implementation |
|-----------+-----------+-----------------+----------------|
| =sb=      | spatial   | where am I?     | git worktree   |
| =cprr=    | epistemic | why am I here?  | conjecture     |
| =aq=      | temporal  | who else knows? | broadcast      |

A broadcast payload requires all three: worktree identity (=sb=),
conjecture context (=cprr=), and the broadcast itself (=aq=). They are
coupled by design. An agent without spatial context is lost; a broadcast
without epistemic context is noise; presence without a channel is
silence.

* Awareness, Not Conflict Detection

=aq= does not detect conflicts. It shows you who else is nearby.

=aq check= shows broadcasts from other agents on the same conjecture or
with similar claims. You read them. You /feel/ the overlap. You decide:
coordinate, adjust, or ignore. Maybe you start a conversation. Maybe you
split the work. Maybe you keep going because your approaches are
complementary. The tool doesn't judge --- it introduces.

#+begin_example
$ aq check -c C-10 --claim "simplifying CLI dispatch"

  Nearby (same conjecture C-10):

  agent-4  "simplifying flag parsing — consolidating into unified parser"
  agent-2  "simplifying Broadcast struct — tightening types"
  agent-9  "simplifying doctor command — cleaner output"
  ... 7 more

  You share C-10 with 9 other agents. Read their claims.
#+end_example

This is noisy. That's the point. The alternative --- silence until the
merge wall --- is worse. Two agents that see each other's claims can
collaborate: "you're rewriting dispatch? I'm rewriting flag parsing.
Let's align on the interface." Two agents that don't see each other
produce incompatible code and discover it at merge time.

Gossip is cheaper than merge conflicts. Noise is cheaper than surprise.

The exit condition: when an agent announces =status=done=, its broadcast
expires from the active set. Silence returns naturally.

* Transport Tiers

The filesystem is the only required transport. Every feature must work
with filesystem I/O alone. Higher tiers are upgrade paths, not
requirements.

| Tier  | Transport              | L1.5 Fit | Notes                                   |
|-------+------------------------+----------+-----------------------------------------|
| 0     | Filesystem (~/.aq/)    |       95 | Canonical. Debuggable with =ls= and =cat=. |
| 0.5   | NFS / SMB / WebDAV     |       50--72 | Same code, just mount and set =AQ_HOME= |
| 1     | NATS / Redis pub/sub   |    68--88 | When cross-machine latency matters      |
| 1.5   | S3 / MinIO             |       58 | Cloud or k8s-hosted agents              |
| 2     | etcd                   |       62 | Already present if running k8s          |
| 3     | MQTT (Mosquitto)       |       65 | QoS 0 = gossip; RETAIN = presence; LWT = auto-done |
| 3.5   | Meshtastic (LoRa)      |       40 | 237B payload, multi-hop mesh, off-grid. AMTP compact format. |
| 4     | IRC (#aq-presence)     |       55 | Human+agent hybrid monitoring           |
| 5     | UDP broadcast          |       72 | LAN multicast, zero config, fire-and-forget |
| 6     | Ultrasonic (ggwave)    |       30 | Data-over-sound, ~25B payload, 15-20kHz. Dogs will judge you. |
| \infin | BBS echomail           |       45 | Daily CPRR conjecture digest            |
| ---   | Carrier pigeons        |        2 | p99 RTT of ~3000s exceeds target by 6 orders of magnitude |

Key architectural insight: Tier 0 and Tier 0.5 share /identical/ code.
NFS, SMB, and WebDAV mounts are transparent to =announce()= and
=read_active()= --- the only variables are latency and consistency.
Point =AQ_HOME= at the mount and it works:

#+begin_src bash
AQ_HOME=/mnt/shared-aq aq announce -c C-1 --claim "proving filesystem transport"
#+end_src

Full evaluation of all transports (including the TRAMP design philosophy
that governs the transport layer): [[file:docs/TRANSPORTS.org][docs/TRANSPORTS.org]].

* Position in Seven Concerns

| L   | Concern         | Question                                       | Tool          |
|-----+-----------------+------------------------------------------------+---------------|
|   7 | Coordination    | Who does what, in what order?                  | Gastown       |
|   6 | Instrumentation | Which workflow governs this session?            | Organon       |
|   5 | Control         | Who is allowed to act, can we prove it?         | SEFACA        |
|   4 | Validation      | Built right, built correctly?                   | Elenctic-Spec |
|   3 | Reasoning       | What do we believe, how would we know wrong?    | CPRR          |
|   2 | Memory          | What do we already know before we re-learn it?  | JITIR         |
| 1.5 | *Presence*      | *Does anyone know what's happening right now?*  | *aq*          |
|   1 | Work            | What needs doing, what is its state?            | bd / beads    |

L1 (=bd=) is authoritative --- a bead's state is a fact. L2 (=JITIR=)
is retrieval --- "I indexed this." L1.5 (=aq=) is neither. A broadcast
carries no obligation and may expire unread. It answers probabilistically
about in-flight uncertainty.

The mDNS analogy: L2 is DNS (authoritative, query/response). L1.5 is
mDNS / Zeroconf / Bonjour (broadcast, "does anyone know this address?").
The cost is near zero and the benefit when someone /does/ know is
conflict avoidance before the merge wall.

* Research Context

** GEACL --- Gossip-Enhanced Agentic Coordination Layer

Two arXiv papers landed in 2025 formalizing the gossip substrate for
multi-agent coordination:

- *arXiv 2508.01531* (Aug 2025) --- "Revisiting Gossip Protocols: A
  Vision for Emergent Coordination in Agentic Multi-Agent Systems."
  Identifies gossip as the missing substrate for reflexivity and
  resilience in MAS. Raises unresolved challenges: semantic filtering,
  staleness, trustworthiness.

- *arXiv 2512.03285* (Dec 2025) --- Introduces *GEACL*. "MCP/A2A handle
  intent, roles, and tools; gossip maintains weakly consistent
  situational alignment." Key insight: context-aware suppression ---
  safety-critical updates propagate aggressively (=aq announce=);
  routine updates can be throttled (=aq whisper=).

Neither paper has a working implementation. Both explicitly state "rather
than proposing a complete framework." =aq= is a concrete instantiation
of the GEACL research agenda, running on a single machine with CPRR as
the semantic payload and filesystem as the canonical transport.

The lineage:

#+begin_example
Gossip protocols (Cassandra/DynamoDB, ~2007)
    |
GEACL vision papers (arXiv 2508.01531, 2512.03285 --- 2025)
    |
aq --- concrete instantiation:
    sb    = spatial scope of the gossip
    cprr  = semantic payload (not just heartbeat, but conjecture)
    TTL   = knowledge decay / suppression semantics
    files = conflict surface encoded in the rumor itself
#+end_example

** Other lineage

- [[https://www.waveprotocol.org/][Google Wave Federation Protocol]] --- presence-as-stream. Wave was
  trying to build L1.5 for documents. They never framed it that way,
  which is probably why they couldn't explain what it was for. =aq=
  takes the presence semantics and drops everything else.

- Lakatos, /Proofs and Refutations/ --- the CPRR methodology
  (conjecture, proof, refutation, refinement) that structures the
  semantic payload of every broadcast.

** The gap aq fills

From arXiv 2512.03285:

#+begin_quote
MCP lacks constructs for collaborative state convergence. There is no
framework for bottom-up task claiming, bidding, or peer negotiation.
Swarm-style organization --- patterns such as decentralized delegation,
emergent clustering, or reflexive adaptation --- are outside its
specification.
#+end_quote

=aq= is not trying to replace MCP/A2A. It is the layer beneath them
that neither defines.

* Conjectures

Each conjecture has an explicit refutation criterion. Conjectures
without measurement are decorative.

** Proven

- *C-1*: Filesystem-first transport is sufficient.
  /Proven/: chaos tests --- p99 = 77ms at 10 agents, 239ms at 50.
  Both well under the 500ms refutation threshold.
  See [[file:docs/adr/JOURNEY.md][JOURNEY.md]] and =make demo-chaos=.

- *C-3*: Wave presence semantics without Wave data model.
  /Proven/: NDJSON+TTL has expressed every coordination pattern
  observed across four dogfooding sessions. No counter-example logged.

- *C-4*: CPRR phase modulates conflict severity.
  /Proven/: both-proof=HIGH, one-proof=MEDIUM, else=LOW. Implemented
  in =checkConflicts()=. Dogfooding confirmed phase-based severity
  reduces noise vs flat severity when agents are in different phases.

** Implemented, Under Observation

- *C-2*: Conjecture identity prevents semantic conflicts.
  /Status/: implemented. =checkConflicts()= computes file overlap and
  phase severity. Same-agent broadcasts are skipped (correct: you
  don't conflict with yourself). Needs multi-worktree validation to
  confirm false negative rate.

** Resolved Without Implementation

- *C-6*: Local-first =.aq/= in cwd before =~/.aq/=.
  /Deferred/: never implemented. All agents use =~/.aq/= (global home).
  The local-first approach would cause confusion when agents operate
  from different cwd within the same project. The simpler model won.

- *C-7*: Auto-renewal / heartbeat prevents TTL cliff.
  /Superseded/: TTL bumped from 300s to 3600s. Git hooks auto-announce
  on pre-commit (prosecuting) and post-commit (done). No daemon needed.
  The hook chain solved the TTL cliff without adding heartbeat coupling.
  See [[file:docs/adr/WIRE-FORMAT-V3.1.md][WIRE-FORMAT-V3.1.md]] for the hook architecture.

** Open

- *C-8*: Function-level granularity resolves single-file false positives.
  /Refutation/: AST parsing adds complexity that exceeds the value of
  finer-grained conflict detection.
  /Status/: not implemented. File-level detection is the current floor.
  The dogfooding showed false positives when all agents touch =main.go=,
  but the single-file repo is the worst case, not the common case.

- *C-11*: Silent on repos without aq.
  /Refutation/: hook errors or delays in non-aq repos.
  /Status/: hooks check =command -v aq= and exit 0 if missing.

- *C-12*: Edit/Write fires with correct file path.
  /Refutation/: broadcast missing or wrong file after edit.
  /Status/: pre-commit hook reads =git diff --cached --name-only=.

- *C-13*: Commit hook triggers fanout to all transports.
  /Refutation/: fanout fails silently or blocks commit.
  /Status/: post-commit hook fires =aq-fanout.sh= backgrounded.
  Verified: mesh, ggwave, kbfs, mdns all fire.

- *C-14*: Fanout degrades gracefully when transports are disabled.
  /Refutation/: errors or delays when transports are off.
  /Status/: each transport wrapped in =timeout= + =is_enabled()= check.

* Ecosystem

=aq= is one tool in the DefRecord ecosystem:

| Tool   | Layer | Role                           |
|--------+-------+--------------------------------|
| =sb=   | L1.5  | Worktree auditor (spatial)     |
| =cprr= | L3    | Conjecture tracker (epistemic) |
| =bd=   | L1    | Bead-based issue tracking      |
| =JITIR= | L2   | Knowledge retrieval            |
| =aq=   | L1.5  | Ambient presence gossip        |

=sb=, =cprr=, and =aq= are single-file Go CLIs using stdlib only with
manual dispatch.

* Contrib Packages

The =contrib/= directory contains optional extensions that are not part
of the core binary:

| Package       | Description                                            |
|---------------+--------------------------------------------------------|
| =dashboard=   | WebSocket-based live dashboard (localhost:8085)        |
| =postgres=    | PostgreSQL transport (Tier 1 alternative)              |
| =mdns=        | mDNS/Bonjour broadcast demo (macOS)                   |
| =chaos=       | Chaos test suite --- 6 scenarios, subprocess-based     |
| =tramp=       | Emacs TRAMP integration (=aq-tramp.el=)                |
| =mqtt=        | MQTT transport (Tier 3, QoS 0 gossip)                 |
| =ggwave=      | Ultrasonic/audible audio transport (Python, ggwave)    |
| =keybase=     | KBFS shared-directory transport                        |
| =meshtastic=  | LoRa mesh transport (Tier 3.5, 237B frame)             |

All contrib packages are optional. The core =aq= binary has zero
dependencies beyond Go stdlib.

* Development

** Build and test

#+begin_src bash
# Go (target implementation)
make build            # Build aq binary
make test             # Run tests
make test-race        # Run tests with race detector
make lint             # go vet + fmt check + golangci-lint
make install          # Install to ~/.local/bin

# Demos and experiments
make demo-announce    # Announce presence and show status
make demo-conflict    # Two-agent conflict scenario
make demo-status      # Show active broadcasts
make demo-dashboard   # Start web dashboard (localhost:8085)
make demo-mdns        # mDNS broadcast demo (macOS)
make demo-chaos       # Run chaos tests (~2 min)
make demo-bench       # Run Go benchmarks

#+end_src

** Build order

The build is sequenced with acceptance gates. If a gate fails, stop.

1. +Protocol and broadcast struct+ --- done (Go, =main.go=)
2. +Unit tests for protocol+ --- done (=main_test.go=, =go test ./...=)
3. +Conflict detection with CPRR phase severity+ --- done
4. +CLI: announce, check, status, validate, doctor+ --- done
5. +Transport fanout: mesh, ggwave, kbfs, mdns, UDP+ --- done (v0.4.0)
6. +Wire format v3.1 with BNF grammar+ --- done (ADR accepted)
7. Integration: =sb detect= \to =aq announce= --- acceptance: auto-detect worktree
8. Benchmark at scale --- acceptance: 10 agents, 100 msg/min, p99 < 500ms

** Contributing

- Conventional commits: =feat:=, =fix:=, =docs:=, =refactor:=, =test:=
- Git notes on every commit with agent metadata (=X-Agent-Role=,
  =X-Conjectures=, =X-Testing=, etc.) --- see [[file:CLAUDE.md][CLAUDE.md]] for the full
  template
- Push notes: ~git push origin refs/notes/commits~

This project is being built by multiple Claude agents in parallel
worktrees, documenting the experience as it happens. The messy parts are
intentional. The git history, the notes, the worktree sprawl --- that is
the data.

** The dogfooding journey

Four agents. Four worktrees. One gossip tool that didn't work yet. They
tried to build =aq= using =aq=. Highlights:

- All four agents touching =main.go= --- every conflict check returned
  HIGH. Signal-to-noise ratio: zero. File-overlap detection assumes
  file-level isolation; single-file repos are the worst case.
- TTL of 300s meant every broadcast expired while agents were still
  working. Gossip with amnesia. (DefaultTTL has since been bumped to
  3600s to match real session length.)
- Agents created their /own/ worktrees instead of using the ones set up
  for them. The gossip layer had perfect information about the wrong
  worktrees. Score: Gossip 0, Chaos 1.
- Build step 7 chaos tests proved C-1 at scale: six scenarios, all
  passing. 10 agents at p99 = 77ms; 50 agents at p99 = 239ms --- both
  well under the 500ms refutation threshold. Filesystem transport is
  not the bottleneck; subprocess overhead (~60ms per =aq announce=) is.

Full account: [[file:docs/adr/JOURNEY.md][docs/adr/JOURNEY.md]].

** End-to-end acceptance test

Two agents in separate worktrees. Agent A announces =C-1, phase=proof,
files=[auth.py]=. Agent B announces =C-7, phase=proof, files=[auth.py]=.
A watcher reads active broadcasts, detects HIGH conflict (both proof,
shared file). Both agents receive the conflict signal. Agent A finishes
and announces =status=done=. Watcher confirms conflict cleared.

This is the system's definition of done.

* License

MIT. See [[file:LICENSE][LICENSE]].

Documentation

Overview

aq — ambient agent queue

Gossip layer (L1.5) for multi-agent development. Agents broadcast presence via filesystem-backed channels so peers detect semantic conflicts before they become merge conflicts.

See spec.org for the full specification. See CLAUDE.md for agent instructions.