synapse

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 2, 2026 License: Apache-2.0 Imports: 0 Imported by: 0

README

synapse

CI

A Go event sourcing and CQRS toolkit. Composable primitives — aggregates, events, repositories, projections — plus the operational surface real services need: a Postgres backend, broker delivery for projections, sagas with timeouts, dead-letter handling, crypto-shredding, OpenTelemetry hooks, and an admin RPC for dump/restore.

Zero third-party dependencies in the root module. Anything that pulls in an external library (a SQL driver, a broker client, gRPC) lives as a sibling module so you opt in to what you use.

Status

Pre-1.0. The public surface is still moving; expect minor breaking changes between tagged versions. The Postgres and SQLite stacks (events + snapshots + checkpoints + dead-letter + timeouts + keystore, one pool / one file) are shaping up production-ready but have not yet seen production traffic.

Install

go get github.com/ianunruh/synapse

Requires Go 1.26 or newer.

Quick look

package main

import (
    "context"

    jsoncodec "github.com/ianunruh/synapse/codec/json"
    "github.com/ianunruh/synapse/es"
    "github.com/ianunruh/synapse/eventstore/memory"
)

// Define an aggregate by embedding *es.AggregateBase.

type Counter struct {
    *es.AggregateBase
    Value int
}

func NewCounter(id es.StreamID) *Counter {
    return &Counter{AggregateBase: es.NewAggregateBase(id)}
}

type CounterIncremented struct {
    By int `json:"by"`
}

func (c *Counter) Apply(env es.Envelope) {
    if inc, ok := env.Payload.(CounterIncremented); ok {
        c.Value += inc.By
    }
}

func (c *Counter) Increment(by int) {
    c.Record("counter.incremented", CounterIncremented{By: by}, c.Apply)
}

func main() {
    ctx := context.Background()

    reg := es.NewRegistry()
    es.Register(reg, "counter.incremented", jsoncodec.For[CounterIncremented]())

    repo := es.NewRepository(memory.New(), reg, NewCounter)

    c := NewCounter("counter/hits")
    c.Increment(2)
    c.Increment(3)
    _ = repo.Save(ctx, c)

    loaded, _ := repo.Load(ctx, "counter/hits")
    _ = loaded.Value // == 5
}

A full walkthrough is in docs/getting-started.md. Runnable examples live under examples/.

What's in the box

Beyond the core aggregate/repository/projection trio, the toolkit ships:

  • Storage backends — Postgres (ADR-0024) and SQLite (ADR-0017) for the event store, snapshot store, checkpoint store, dead-letter store, timeout store, and key store. Postgres uses a shared LISTEN/NOTIFY (ADR-0025) for live-tail; read-replica routing is opt-in (ADR-0038).
  • Commands — Typed es.Handler[C, A] + es.Execute (ADR-0009) for in-process callers; es/commandbus for dispatching named, byte-encoded commands from HTTP/gRPC transports (ADR-0028).
  • Process managers and sagas — es/process (ADR-0032) wraps a saga as a projection that emits commands. Saga timeouts are scheduled via outbox-style intent events on the saga's own stream, atomic with Save (ADR-0043).
  • Broker delivery — outbox.Publisher (ADR-0042) + a NATS JetStream adapter; outbox/nats.Consumer (ADR-0044) lets projections subscribe to NATS instead of polling Postgres. End-to-end demo in examples/nats-postgres-projection.
  • Dead-letter queue — projection.WithDeadLetter (ADR-0041) routes failed events to a persistent DLQ; the runner logs at Warn and continues. Admin RPCs and CLI subcommands surface entries for operators.
  • Crypto-shredding — Per-subject AES-256-GCM via a KeyStore interface (ADR-0036). Shredding a key tombstones every event for that subject in place; the rest of the log keeps replaying.
  • Observability — slog everywhere by default (ADR-0015); an opt-in otel sibling module emits spans + duration histograms around every persistence call (ADR-0034).
  • Admin RPCs — synapse-admin and a gRPC service expose stream/checkpoint/snapshot/deadletter inspection plus dump/restore in JSONL (ADR-0033, ADR-0040).
  • Liveness/readiness probes — synapse/health aggregates user-registered checks behind two HTTP handlers; stdlib only (ADR-0039).
  • Event upcasters — Old event shapes evolve forward; Apply only ever sees the latest version (ADR-0023).

Documentation

Packages

The repo is a Go workspace. Core interfaces live in es; backends and contract suites are sibling packages — or sibling modules when they pull in third-party deps.

Package Module What it is
Core
es root Aggregate, Repository, Event, Envelope, Source, Snapshotter, Projection, codec Registry, error types
es/middleware root Built-in repository middleware: PerAggregateLocking, Retry
es/projection root Runner for read-model projections (type filters, batched checkpoints, dead-letter routing)
es/commandbus root Transport-facing CommandBus + middleware (Logging, Recover, Timeout)
es/process root Process managers / sagas with timeouts (intent events, TimeoutManager)
idgen root UUIDv7 identifier generator
health root Liveness + readiness HTTP probes
crypto root Per-subject AES-256-GCM crypto-shredding (stdlib only)
Codecs
codec/json root JSON event/snapshot codec
codec/proto sibling Protobuf event/snapshot codec
codec/codectest root Codec contract test suite
Event store
eventstore/memory root In-memory event store
eventstore/postgres sibling Postgres event store (pgxpool, shared LISTEN/NOTIFY, read-replica routing)
eventstore/sqlite sibling SQLite event store
eventstore/eventstoretest root Backend contract test suite
Snapshot store
snapshotstore/memory root In-memory snapshot store
snapshotstore/postgres sibling Postgres snapshot store
snapshotstore/sqlite sibling SQLite snapshot store
snapshotstore/snapshotstoretest root Backend contract test suite
Checkpoint store
checkpointstore/memory root In-memory checkpoint store
checkpointstore/postgres sibling Postgres checkpoint store
checkpointstore/sqlite sibling SQLite checkpoint store
checkpointstore/checkpointstoretest root Backend contract test suite
Dead-letter store
deadletterstore/memory root In-memory dead-letter store
deadletterstore/postgres sibling Postgres dead-letter store
deadletterstore/sqlite sibling SQLite dead-letter store
deadletterstore/deadletterstoretest root Backend contract test suite
Timeout store (saga deadlines)
timeoutstore/memory root In-memory timeout store
timeoutstore/postgres sibling Postgres timeout store
timeoutstore/sqlite sibling SQLite timeout store
timeoutstore/timeoutstoretest root Backend contract test suite
Key store (for crypto-shredding)
keystore/memory root In-memory key store
keystore/postgres sibling Postgres key store
keystore/sqlite sibling SQLite key store
crypto/keystoretest root Backend contract test suite
Outbox
outbox root Publisher interface, AsProjection, Retrying
outbox/nats sibling NATS JetStream Publisher + Consumer adapter
Observability
otel sibling OpenTelemetry store wrappers + projection.Runner tracing
Admin / ops
admin sibling gRPC admin service + synapse-admin CLI (dump/restore, deadletter, etc.)
pgtest sibling Postgres testing harness (testcontainers-go)
Examples
examples/counter root In-memory event sourcing walkthrough
examples/order root Multi-stage aggregate with command validation
examples/projection root Projection runner walkthrough
examples/process root Process manager driving a saga
examples/saga-timeout root Saga deadlines via outbox-style intent events
examples/persistent sibling SQLite-backed end-to-end demo
examples/postgres sibling Postgres-backed end-to-end demo
examples/http-service sibling CommandBus driven by net/http, live projection
examples/nats-postgres-projection sibling Postgres event store + NATS broker + Postgres read-model table

Sibling modules each have their own go.mod. A go.work file at the repo root ties them together for local development. The root module has zero third-party deps; the SQLite backends transitively pull in modernc.org/sqlite (pure Go, no CGo).

Design principles

Recorded as Architecture Decision Records (starting at ADR-0001):

  • Go 1.26 toolchain, language features and stdlib used to current capability.
  • Zero third-party deps in the root module. Backends that need them live as sibling Go modules.
  • Modernization-clean. gopls modernize ./... exits 0 in every module.
  • Serialization-agnostic core. Codecs are registered per event type; the es package never imports a specific codec.
  • Type safety and performance are co-equal goals. Where they point the same direction (most cases), take both. Where they conflict, prefer the perf-friendly option in hot paths and document the trade-off.
  • Admin RPCs and a web UI, when they exist, will be optional sibling subpackages users opt into.

License

Apache 2.0. See LICENSE.

Documentation

Overview

Package synapse is a toolkit for building event-sourced systems in Go.

The library is organized as a set of small subpackages so applications import only what they need. This package itself is doc-only — there is nothing to import from it.

Core (root module, no third-party deps)

Codecs (per-event-type, opt-in)

  • github.com/ianunruh/synapse/codec/json — encoding/json adapter, root module, stdlib only.
  • github.com/ianunruh/synapse/codec/proto — google.golang.org/protobuf adapter, sibling module so the protobuf dep stays out of the core.

Persistence backends

Every store interface has memory / SQLite / Postgres backends. The memory backends live in the root module; SQLite and Postgres backends live as sibling modules so their drivers stay out of the core.

  • Events: eventstore/memory (root), eventstore/sqlite (sibling), eventstore/postgres (sibling).
  • Snapshots: snapshotstore/memory (root), snapshotstore/sqlite (sibling), snapshotstore/postgres (sibling).
  • Checkpoints: checkpointstore/memory (root), checkpointstore/sqlite (sibling), checkpointstore/postgres (sibling).
  • Dead-letter: deadletterstore/memory (root), deadletterstore/sqlite (sibling), deadletterstore/postgres (sibling).
  • Timeouts (saga deadlines): timeoutstore/memory (root), timeoutstore/sqlite (sibling), timeoutstore/postgres (sibling).
  • Crypto keys: keystore/memory (root), keystore/sqlite (sibling), keystore/postgres (sibling).

The Postgres event store includes shared LISTEN/NOTIFY for live tail, optional read-replica routing for catch-up reads, and parallel append support. See the eventstore/postgres package docs for the details.

Outbox + broker delivery

  • github.com/ianunruh/synapse/outbox — Publisher interface, AsProjection adapter, Retrying wrapper. Mount on a projection Runner to get checkpointing and dead-letter routing for free.
  • github.com/ianunruh/synapse/outbox/nats — NATS JetStream Publisher and Consumer adapters. Sibling module. The Consumer adapts a durable consumer into an [es.Source] so projections can subscribe to NATS instead of polling the event store.

Observability

  • github.com/ianunruh/synapse/otel — OpenTelemetry tracing + metrics wrappers for the event/snapshot/checkpoint stores plus the projection.Runner. Sibling module so the OTel SDK stays out of the core.

Admin and ops

  • github.com/ianunruh/synapse/admin — gRPC admin service plus the synapse-admin CLI for stream/checkpoint/snapshot/deadletter inspection and streaming dump/restore in JSONL. Sibling module.
  • github.com/ianunruh/synapse/pgtest — Postgres testing harness (testcontainers-go) for the sibling backend test suites.

Contract test suites

Every backend interface ships a shared contract test the in-tree backends run against; user-written backends can run the same suite: codec/codectest, eventstore/eventstoretest, snapshotstore/snapshotstoretest, checkpointstore/checkpointstoretest, deadletterstore/deadletterstoretest, timeoutstore/timeoutstoretest, crypto/keystoretest.

Architectural decisions

Architectural decisions are recorded under docs/adr/ (45 records as of this writing). Read the relevant ADR before relitigating a decision.

Source Files

View all Source files

Directories

Path Synopsis
admin module
checkpointstore
checkpointstorebench
Package checkpointstorebench provides a benchmark harness any implementation of es.CheckpointStore can run.
 Package checkpointstorebench provides a benchmark harness any implementation of es.CheckpointStore can run.
checkpointstoretest
Package checkpointstoretest provides a contract test suite that any implementation of es.CheckpointStore can run to verify the documented behavior.
 Package checkpointstoretest provides a contract test suite that any implementation of es.CheckpointStore can run to verify the documented behavior.
memory
Package memory provides an in-memory es.CheckpointStore suitable for tests, examples, and local development.
 Package memory provides an in-memory es.CheckpointStore suitable for tests, examples, and local development.
postgres module
sqlite module
codec
codectest
Package codectest provides a shared contract suite for es.TypedCodec implementations, mirroring the role eventstoretest plays for event stores (ADR-0018).
 Package codectest provides a shared contract suite for es.TypedCodec implementations, mirroring the role eventstoretest plays for event stores (ADR-0018).
json
Package json provides an es.TypedCodec backed by the standard library's encoding/json package.
 Package json provides an es.TypedCodec backed by the standard library's encoding/json package.
proto module
Package crypto wires synapse to per-subject crypto-shredding.
 Package crypto wires synapse to per-subject crypto-shredding.
keystoretest
Package keystoretest provides a contract test suite that any implementation of crypto.KeyStore can run to verify the documented behavior.
 Package keystoretest provides a contract test suite that any implementation of crypto.KeyStore can run to verify the documented behavior.
deadletterstore
deadletterstoretest
Package deadletterstoretest provides a contract test suite that every es.DeadLetterStore implementation runs to verify the documented behavior.
 Package deadletterstoretest provides a contract test suite that every es.DeadLetterStore implementation runs to verify the documented behavior.
memory
Package memory provides an in-memory es.DeadLetterStore suitable for tests, examples, and local development.
 Package memory provides an in-memory es.DeadLetterStore suitable for tests, examples, and local development.
postgres module
sqlite module
es
Package es provides event sourcing and CQRS primitives that can be composed into application-specific aggregates, command handlers, and read models.
 Package es provides event sourcing and CQRS primitives that can be composed into application-specific aggregates, command handlers, and read models.
commandbus
Package commandbus routes named, byte-encoded commands to the typed es.Handler registered for them, so HTTP and gRPC transports can dispatch commands without writing a per-route adapter by hand.
 Package commandbus routes named, byte-encoded commands to the typed es.Handler registered for them, so HTTP and gRPC transports can dispatch commands without writing a per-route adapter by hand.
middleware
Package middleware provides built-in es.Middleware implementations for common cross-cutting concerns around command execution: per-aggregate locking, retry on transient errors, and so on.
 Package middleware provides built-in es.Middleware implementations for common cross-cutting concerns around command execution: per-aggregate locking, retry on transient errors, and so on.
process
Package process provides a thin wrapper for the process-manager pattern: an aggregate that consumes events from one or more streams and emits commands to drive a multi-step workflow.
 Package process provides a thin wrapper for the process-manager pattern: an aggregate that consumes events from one or more streams and emits commands to drive a multi-step workflow.
projection
Package projection drives consumers of the event log via a Runner that subscribes to an es.Source (typically an es.EventStore, optionally a broker-backed Consumer), decodes events through a codec es.Registry, invokes a es.Projection, and (optionally) checkpoints progress to a es.CheckpointStore so consumers resume across restarts.
 Package projection drives consumers of the event log via a Runner that subscribes to an es.Source (typically an es.EventStore, optionally a broker-backed Consumer), decodes events through a codec es.Registry, invokes a es.Projection, and (optionally) checkpoints progress to a es.CheckpointStore so consumers resume across restarts.
eventstore
eventstorebench
Package eventstorebench provides a benchmark harness any implementation of es.EventStore can run to measure append and load performance under standardized workloads.
 Package eventstorebench provides a benchmark harness any implementation of es.EventStore can run to measure append and load performance under standardized workloads.
eventstoretest
Package eventstoretest provides a contract test suite that any implementation of es.EventStore can run to verify the documented behavior.
 Package eventstoretest provides a contract test suite that any implementation of es.EventStore can run to verify the documented behavior.
memory
Package memory provides an in-memory es.EventStore (and es.EventStore) suitable for tests, examples, and local development.
 Package memory provides an in-memory es.EventStore (and es.EventStore) suitable for tests, examples, and local development.
postgres module
sqlite module
examples
counter command
Command counter is an end-to-end demo of the synapse event sourcing toolkit.
 Command counter is an end-to-end demo of the synapse event sourcing toolkit.
order command
Command order is an end-to-end demo of a richer event-sourced aggregate.
 Command order is an end-to-end demo of a richer event-sourced aggregate.
process command
Command process is an end-to-end demo of the process-manager pattern: a Transfer aggregate that coordinates a multi-step workflow across two Account aggregates.
 Command process is an end-to-end demo of the process-manager pattern: a Transfer aggregate that coordinates a multi-step workflow across two Account aggregates.
projection command
Command projection demonstrates the synapse subscription / projection machinery.
 Command projection demonstrates the synapse subscription / projection machinery.
saga-timeout command
Command saga-timeout is an end-to-end demo of the outbox-style saga timeout pattern (ADR-0043).
 Command saga-timeout is an end-to-end demo of the outbox-style saga timeout pattern (ADR-0043).
Package health provides liveness and readiness HTTP probes for synapse deployments.
 Package health provides liveness and readiness HTTP probes for synapse deployments.
Package idgen provides event identifier generators for the synapse event sourcing toolkit.
 Package idgen provides event identifier generators for the synapse event sourcing toolkit.
internal
testdomain
Package testdomain provides shared test fixtures — aggregates, events, commands, and a populated codec registry — for synapse's internal tests.
 Package testdomain provides shared test fixtures — aggregates, events, commands, and a populated codec registry — for synapse's internal tests.
keystore
memory
Package memory provides an in-memory crypto.KeyStore suitable for tests, examples, and local development.
 Package memory provides an in-memory crypto.KeyStore suitable for tests, examples, and local development.
postgres module
sqlite module
otel module
Package outbox is the foundation for forwarding events from the synapse event log to external systems (Kafka, NATS, webhooks, other synapse clusters).
 Package outbox is the foundation for forwarding events from the synapse event log to external systems (Kafka, NATS, webhooks, other synapse clusters).
nats module
pgtest module
snapshotstore
memory
Package memory provides an in-memory es.SnapshotStore suitable for tests, examples, and local development.
 Package memory provides an in-memory es.SnapshotStore suitable for tests, examples, and local development.
snapshotstorebench
Package snapshotstorebench provides a benchmark harness any implementation of es.SnapshotStore can run.
 Package snapshotstorebench provides a benchmark harness any implementation of es.SnapshotStore can run.
snapshotstoretest
Package snapshotstoretest provides a contract test suite that any implementation of es.SnapshotStore can run to verify the documented behavior.
 Package snapshotstoretest provides a contract test suite that any implementation of es.SnapshotStore can run to verify the documented behavior.
postgres module
sqlite module
timeoutstore
memory
Package memory provides an in-memory process.TimeoutStore suitable for tests, examples, and local development.
 Package memory provides an in-memory process.TimeoutStore suitable for tests, examples, and local development.
timeoutstoretest
Package timeoutstoretest provides a contract test suite that every process.TimeoutStore implementation runs.
 Package timeoutstoretest provides a contract test suite that every process.TimeoutStore implementation runs.
postgres module
sqlite module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.