README
¶
willow-go
A pure-Go implementation of the Willow Protocol.
Willow is a peer-to-peer protocol for synchronisable data stores with capability-based permissions. willow-go ports the data model, the Meadowcap capability layer, and the Willow'25 parameter bundle to idiomatic Go, with iOS and Android bindings via gomobile and zero cgo.
Why willow-go
The Rust reference implementation is excellent for native Rust applications. willow-go exists for everything Rust does not reach cleanly:
| willow_rs | willow-go | |
|---|---|---|
| Native iOS / Android bindings | requires cgo + cross-compile dance | gomobile bind produces an XCFramework / AAR |
| Go ecosystem (Matrix, NATS, gRPC, …) | needs FFI bridge | drop-in import |
| Container-friendly static binaries | needs musl + cross-compile | go build |
| WGPS sync protocol | YES (production) | Phase 2 — funded by NLnet grant work |
If you need WGPS sync today, use willow_rs. If you need Willow on a phone or in a Go service, this is the project.
Status
Pre-MVP. The data-model layer, the Meadowcap capability layer (including multi-step delegation chains), and the Willow'25 parameter bundle are complete and validated byte-for-byte against the Rust reference. WGPS sync is the explicit next phase and is the subject of an NLnet grant application.
53 fixtures from the upstream Rust harness pass byte-identical encode + lossless decode round-trip. 4 Meadowcap delegation chains signed by willow_rs verify under our Go IsValid. 121 tests across 7 packages.
Quick start
go get github.com/Deln0r/willow-go
package main
import (
"fmt"
"github.com/Deln0r/willow-go/datamodel"
"github.com/Deln0r/willow-go/willow25"
)
func main() {
payload := []byte("hello, willow")
path, _ := willow25.NewPath([][]byte{
[]byte("notes"),
[]byte("greeting.txt"),
})
namespace := make([]byte, 32)
subspace := make([]byte, 32)
entry, _ := willow25.NewEntry(namespace, subspace, path, 1700000000, payload)
encoded := entry.Encode()
fmt.Printf("entry encoding: %d bytes\n", len(encoded))
fmt.Printf("payload digest: %x\n", entry.PayloadDigest)
store := datamodel.NewInMemoryStore()
pruned, stored := store.Insert(entry)
fmt.Printf("stored=%v pruned=%d\n", stored, pruned)
}
What is implemented
| Layer | Component | Source |
|---|---|---|
| Data model | Paths, Entries, Range3d, Areas, Store | datamodel/ |
| Encoding | CompactU64, path encoding, path-extends-path, area-in-area | encoding/, datamodel/area.go |
| Capabilities | Communal write capability, multi-step Ed25519 delegation chains, AuthorisationToken | meadowcap/ |
| Willow'25 | 4096/4096/4096 limits, WILLIAM3 payload digest, 32-byte ids, convenience constructors | willow25/ |
| Mobile | gomobile-bindable API: PathBuilder, EntryBuilder, HashPayload | mobile/ |
| Tooling | Cross-impl smoketest CLI, end-to-end sync demo CLI | cmd/ |
Not yet implemented (tracked under "Phase 2" in the NLnet grant scope): WGPS sync, persistent store, transport encryption, owned-namespace capabilities, read capabilities. Tracked items: docs eb-1/willow-go/TECH_DEBT.md in the maintainer's private notes.
Cross-implementation validation
Every byte-producing encoder in willow-go is verified against the Rust reference by a fixture corpus generated from a pinned upstream commit:
testdata/_genfixtures/ Rust harness, pinned to willow_rs dd87996
testdata/paths/ {basic, limits, relative}.json - 26 cases
testdata/paths_rel/extends.json - 7 cases
testdata/entries/basic.json - 10 cases
testdata/areas/relative.json - 8 cases
testdata/william3/digests.json - 11 cases
testdata/meadowcap/delegation_chains.json - 4 cases (Ed25519 signed)
Total: 53 byte-compat fixtures + 4 cross-impl delegation chains.
$ make smoketest
paths (absolute) 16 pass 0 fail
paths (relative) 10 pass 0 fail
paths (extends) 7 pass 0 fail
entries 10 pass 0 fail
areas (relative) 8 pass 0 fail
TOTAL: 51 pass / 0 fail (51 cases)
(The William3 and Meadowcap fixtures are exercised by go test ./willow25/... and go test ./meadowcap/... respectively.)
To regenerate fixtures (Rust toolchain required):
cd testdata/_genfixtures
cargo run --bin gen-paths
cargo run --bin gen-entries
cargo run --bin gen-area-in-area
cargo run --bin gen-path-extends-path
cargo run --bin gen-william3
cargo run --bin gen-handover
Mobile (iOS and Android)
# Install the gomobile toolchain (one-off):
go install golang.org/x/mobile/cmd/gomobile@latest
go install golang.org/x/mobile/cmd/gobind@latest
gomobile init
# Build:
make mobile-ios # produces Mobile.xcframework (verified on Xcode 26.5)
make mobile-android # produces mobile.aar (requires Android NDK and a JDK)
After make mobile-ios, drag Mobile.xcframework into an Xcode project. The bound API:
import Mobile
let digest = MobileHashPayload("hello".data(using: .utf8)!)
// digest is a 32-byte Data (WILLIAM3)
let builder = MobilePathBuilder()
builder.addComponent("notes".data(using: .utf8)!)
builder.addComponent("greeting.txt".data(using: .utf8)!)
let pathBytes = try builder.encode()
End-to-end P2P demo
Two willow-go processes exchanging Ed25519-signed entries over a pipe:
$ go run ./cmd/willow-sync-demo --mode=gen --count=3 --tag=alice \
| go run ./cmd/willow-sync-demo --mode=recv --tag=bob
alice gen: namespace=58254987... root=a1b22c33...
alice gen: wrote 3 entries
bob recv: ACCEPT ns=58254987... path=notes/entry-000 ts=100 payload_len=16
bob recv: ACCEPT ns=58254987... path=notes/entry-001 ts=1100 payload_len=16
bob recv: ACCEPT ns=58254987... path=notes/entry-002 ts=2100 payload_len=16
bob recv: read=3 accepted=3 rejected=0 store_len=3
This is NOT WGPS (no set reconciliation, no fingerprint trees, no channel multiplexing) — that is Phase 2. This is the minimum viable proof that the data-model + capability layers compose correctly on a duplex transport.
Phase 2 roadmap (NLnet-funded)
Submitted for NLnet NGI0 funding under the implementation-diversity precedent set by Vula (Go-language post-quantum encryption) and WillowSync (the original Rust impl). Scope:
- WGPS sync protocol — set reconciliation with cumulative hash fingerprints, LCMUX channel multiplexing, PIO private interest overlap.
- Persistent store backend on top of
modernc.org/sqlite(pure Go, no cgo). - Transport encryption.
Area-encoded WGPS message framing.- Polished error handling + fuzz harness + benchmarks.
ETA: 6-9 months post-grant approval.
Architecture
willow-go/
├── encoding/ CompactU64 codec (4-bit packed + 8-bit standalone)
├── datamodel/ Paths, Entries, Range3d, Areas, Store, prefix-pruning
├── meadowcap/ Communal capabilities, delegations, AuthorisationToken
├── willow25/ Concrete 4096/4096/4096 + WILLIAM3 bundle
├── mobile/ gomobile-bindable wrappers
├── cmd/
│ ├── willow-smoketest/ Cross-impl byte-compat gate
│ └── willow-sync-demo/ Pipe-based P2P sync demonstration
└── testdata/
├── _genfixtures/ Rust harness (pinned to willow_rs dd87996)
├── paths/ paths_rel/ entries/ areas/ william3/ meadowcap/
Contributing
Bug reports and PRs welcome. See CONTRIBUTING.md for the workflow and CODE_OF_CONDUCT.md. For larger contributions or design questions, please open an issue first to discuss scope.
References
- Willow Protocol specification
- willow_rs (Rust reference)
- Meadowcap capability system
- Willow'25 parameter bundle
- bab_rs / WILLIAM3 hash function
License
MIT. See LICENSE.
Documentation
¶
Overview ¶
Package willow is a pure-Go implementation of the Willow Protocol (https://willowprotocol.org), a peer-to-peer protocol for synchronizable data stores with fine-grained capability-based permissions.
The implementation tracks the Rust reference at willow_rs (https://codeberg.org/worm-blossom/willow_rs) and the protocol spec, and is validated against fixtures generated from that reference.
Subpackages:
- datamodel: Paths, Entries, Groupings, byte-encodings, Store interface.
- meadowcap: capability-based authorisation (Ed25519 signatures).
- willow25: the 2025 parameter bundle wrapping datamodel and meadowcap for interop with default-parametrized Rust impls.
- encoding: shared binary encoders/decoders used across the above.
Pre-MVP scope deliberately excludes WGPS sync, persistent storage, and transport-encryption layers. Those are deferred to later phases.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
willow-smoketest
command
willow-smoketest is a CLI that exercises the Go impl against every committed byte-compat fixture under testdata/ and reports pass/fail counts per fixture group.
|
willow-smoketest is a CLI that exercises the Go impl against every committed byte-compat fixture under testdata/ and reports pass/fail counts per fixture group. |
|
willow-sync-demo
command
willow-sync-demo is a minimal proof-of-concept that two willow-go processes can exchange Meadowcap-authorised entries over a duplex transport.
|
willow-sync-demo is a minimal proof-of-concept that two willow-go processes can exchange Meadowcap-authorised entries over a duplex transport. |
|
Package datamodel implements the Willow data model: Paths, Entries, Groupings, and the Store interface.
|
Package datamodel implements the Willow data model: Paths, Entries, Groupings, and the Store interface. |
|
Package encoding provides shared binary primitives used by the higher-level Willow types.
|
Package encoding provides shared binary primitives used by the higher-level Willow types. |
|
Package meadowcap implements the Meadowcap capability system.
|
Package meadowcap implements the Meadowcap capability system. |
|
Package mobile is the gomobile-bindable surface of willow-go.
|
Package mobile is the gomobile-bindable surface of willow-go. |
|
Package willow25 is the Willow'25 specialisation of the generic datamodel and meadowcap layers.
|
Package willow25 is the Willow'25 specialisation of the generic datamodel and meadowcap layers. |