syncplane

module
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 20, 2026 License: MIT

README

Syncplane

Syncplane is a sync engine for local-first applications. Clients read and write a local key-value store, so an application works fully offline. A server-authoritative mutation log keeps replicas converged through optimistic rebase. It replays pending mutations on top of server-confirmed state.

Features

Sync
  • Partial sync: Sync only the rows that a user needs. Filter by predicate, keep a bounded window such as the most recently updated rows, or name the rows a device has pinned by key. Every read reports whether it saw complete data.
  • Live updates: The server pushes changed rows to connected clients, with polling as a fallback.
  • Multi-tab and multi-workspace: Multiple browser tabs share one local store and one server connection. A user in several workspaces syncs all of them over that same connection.
  • Unified query language: A query returns the same rows whether it runs against the local store or on the server. When you declare an index, the query runs as a range read on both sides instead of a scan.
  • Full-text search: Mark fields as searchable and query them with match, ranked by relevance. The client ranks synced data the same way the server does.
Collaboration
  • Collaborative text: A field can hold a text document that merges concurrent edits. Formatting and comment anchors stay attached to the characters that they cover as other users edit around them.
  • Presence and broadcast: The server delivers live cursors, presence indicators, and typing indicators without storing them. Attach a room to a row, and any user who can read the row can join the room.
  • File attachments: A row refers to an image, a PDF or a video with a BlobRef field. The bytes upload in resumable chunks beside the row, also after a reload, and every user who can read the row can read the file. See Attach files to rows.
  • Field history: Record previous field values and user identities before each editing session. You can restore an old value with a standard write.
Server
  • Authentication and permissions: Integrate your own authentication provider, manage user permissions per workspace and per write, and filter what users can read.
  • Custom mutations: In addition to generated put, patch, and del operations, define named mutations in Go that run on the server, with optional optimistic client versions.
  • Effects and connectors: Run custom code when writes commit, and keep external systems in two-way sync.
  • Runtime schemas: Register application entities at runtime instead of compiling them into the binary. One server can host multiple applications.
  • Single-node or multi-node deployments: Start with SQLite and no extra infrastructure, or switch to PostgreSQL to run multiple nodes. Application code does not change.
  • Go: local engine and client: Run the engine inside a Go desktop app or a site server and keep it converged with a remote server. It works offline, serves browsers and Go code, and the remote server decides every write. A stateless Go client reads and writes a remote server from jobs and tools. See Chained sync and Use the Go client.
  • Tunnels: Reach the server through a message channel instead of HTTP, such as a desktop shell's stream API, a MessagePort, stdio to a sidecar or a WebSocket. The engine can run in the host process or behind a proxy to a remote server.
Tooling
  • React adapter: The syncplane/react package exposes queries, rows, and mutations as React hooks, with a provider that manages the client.
  • Test kit: The syncplane/testing package runs your client against an in-process fake server. You can simulate offline mode, reject writes, and assert what the user sees.

Quick start

Go server

Set up a server with an entity schema:

store, _ := sqlitestore.Open("app.db")
defer store.Close()                  // the server does not own the store

srv, err := server.New(store,
	server.WithEntities(&todov1.Todo{}), // reads (syncplane.v1.entity) options
)
if err != nil {
	log.Fatal(err)
}
defer srv.Close()

http.ListenAndServe(":8787", srv.HTTPHandler()) // HTTP/1.1 suffices

An entity is a Protocol Buffers message with a syncplane.v1.entity annotation:

message Todo {
	option (syncplane.v1.entity) = { name: "todo" key_fields: [ "id" ] };

	string id = 1;
	string title = 2;
	bool completed = 3;
}

By default, AllowAll admits every caller into every workspace. For shared deployments, see Authentication and permissions.

TypeScript client

Connect a client and mutate entities:

const client = new SyncClient({
	url: 'https://sync.example.com',
	workspaceId: 'demo',
	entities: [ TodoSchema ],
});
await client.start(); // bootstrap + live stream (returns immediately)

// client.db gives each entity a schema-bound API, keyed by its name.
// CRUD is optimistic and offline-first:
const id = await client.db.todo.mutate.create({ title }); // mints an id
await client.db.todo.mutate.patch(id, { completed: true });
await client.db.todo.mutate.del(id);

client.db.todo.watch((rows) => render(rows));
React

Wrap your components in SyncProvider and access data through hooks:

const { SyncProvider, db } = createReactSync([ TodoSchema ]);

// The provider builds the client, starts it, and closes it on unmount.
<SyncProvider options={ { url, workspaceId, entities: [ TodoSchema ] } }>
	<TodoList />
</SyncProvider>;

function TodoList() {
	const todos = db.todo.useQuery({ where: (t) => t.completed.eq(false) });
	const { patch, del } = db.todo.useMutate();
	// ...
}

Code examples

Run queries locally against the synced subset, or on the server against all data:

const open = await client.db.todo.query({
	where: (t) => and(t.completed.eq(false), t.title.contains('milk')),
	orderBy: (t) => [ t.createdAtMs.desc() ],
	limit: 20,
});

const found = await client.db.todo.queryServer({
	where: (t) => t.title.contains('milk'),
});

Sync a subset of a workspace by predicate, by recency rank with a limit, or by row key:

client.setSubscriptions([
	subscription(TodoSchema, (t) => t.completed.eq(false)),
	subscription(CommentSchema, { limit: 500, byRecency: true }),
	subscription(DocSchema, { keys: pinnedDocIds }),
]);

Edit a collaborative text field through a live handle:

const text = await client.db.doc.text(id, 'body');
await text.insert(5, ', world');
text.subscribe((delta, value) => editor.applyDelta(delta));

Write from Go code using the same path and permissions as client writes:

err := srv.Apply(ctx, workspaceID, func(tx *server.ApplyTx) error {
	todo.Completed = true
	return tx.Patch(&todo, []string{"completed"}, &version)
})

To run system operations without authorizing a caller, opt out explicitly with server.Trusted(ctx).

Running the demo

To run the local demo application:

  1. Install dependencies and start the server and web client:

    mise install        # go, node, pnpm, buf, task, ginkgo
    task demo:server    # terminal 1: sync server on :8787
    task demo:web       # terminal 2: vite app
    
  2. Open the printed URL in two browser tabs.

Changes propagate in under one second without polling. The demo shows undo and redo, drag-to-reorder with fractional indexing, partial sync, collaborative notes with comments, and server rejection handling.

Documentation

Documentation is in the docs/ directory, organized by Diátaxis documentation type:

  • Tutorials: Build your first sync application and add collaborative text.
  • How-to guides: Configure permissions, queries, partial sync, collaborative text, server-side writes, file attachments, change feeds and effects, external store connectors, field history, multi-app hosting, multi-node setups, a local engine in a Go desktop app or site server, the Go client, observability, React, and failure handling.
  • Reference: Review the Go server API, TypeScript client API, Go client, upstream link, Protocol Buffers options, connectors, naming rules, sync status and rejection codes, storage adapter contracts, and telemetry.
  • Explanation: Read about the convergence model, partial sync coverage, collaborative text CRDTs, blobs, multi-tab behavior, chained sync, and system trade-offs.

Development

mise.toml defines tool versions. Taskfile.yml defines development tasks:

task generate          # regenerate Go + TS from proto/
task ci                # lint, breaking, vet, typecheck, all tests
task test:go           # Ginkgo suites (engine)
task test:ts           # every TS client suite: unit, react, integration
task test:ts:react     # syncplane/react hooks (jsdom)
task test:e2e          # multi-tab browser tests (Playwright; not in `task ci`)

For details about test suites and running individual tests, see Testing.

Directories

Path Synopsis
cmd
admindemo command
Command admindemo is a throwaway harness for eyeballing the built-in admin UI.
Command admindemo is a throwaway harness for eyeballing the built-in admin UI.
benchserver command
benchserver runs the sync engine with the syncplane.bench.v1 corpus for the browser benchmark suite (client/test/perf).
benchserver runs the sync engine with the syncplane.bench.v1 corpus for the browser benchmark suite (client/test/perf).
mdtodo command
mdtodo demonstrates layering the sync engine over an external source of truth: the todo app's entities live as Markdown files in a folder, kept bidirectionally converged with the engine by a connect.Connector.
mdtodo demonstrates layering the sync engine over an external source of truth: the todo app's entities live as Markdown files in a folder, kept bidirectionally converged with the engine by a connect.Connector.
syncplaned command
syncplaned runs the sync engine as a multi-app host: a server.Manager routes the public sync surface by a leading "/<appID>" path segment, while a separate, private listener serves the trusted AdminService for runtime schema registration (gated by a bearer TokenAuthorizer).
syncplaned runs the sync engine as a multi-app host: a server.Manager routes the public sync surface by a leading "/<appID>" path segment, while a separate, private listener serves the trusted AdminService for runtime schema registration (gated by a bearer TokenAuthorizer).
testserver command
testserver runs the sync engine with the test entities (Todo, Widget), the complete_all mutator and a deny-rule permission hook.
testserver runs the sync engine with the test entities (Todo, Widget), the complete_all mutator and a deny-rule permission hook.
todoserver command
todoserver backs the examples/todo web app: the sync engine with the todo entity, the complete_all mutator, the example permission policy, and permissive CORS so the Vite dev server can reach it.
todoserver backs the examples/todo web app: the sync engine with the todo entity, the complete_all mutator, the example permission policy, and permissive CORS so the Vite dev server can reach it.
engine
blob
Package blob defines how the engine names and stores blobs: the immutable bytes, such as images, PDFs and videos, that rows refer to with a syncplane.v1.BlobRef.
Package blob defines how the engine names and stores blobs: the immutable bytes, such as images, PDFs and videos, that rows refer to with a syncplane.v1.BlobRef.
blob/blobtest
Package blobtest provides a shared Ginkgo behavior suite every blob.Store implementation must pass.
Package blobtest provides a shared Ginkgo behavior suite every blob.Store implementation must pass.
blob/fsstore
Package fsstore is a blob.Store that keeps blobs in a local directory.
Package fsstore is a blob.Store that keeps blobs in a local directory.
blob/memstore
Package memstore is a blob.Store that keeps everything in memory.
Package memstore is a blob.Store that keeps everything in memory.
client
Package client is a Go client for a remote syncplane server, for code that keeps no local copy of the data: backend jobs, tools and tests.
Package client is a Go client for a remote syncplane server, for code that keeps no local copy of the data: backend jobs, tools and tests.
connect
Package connect binds the sync engine to an external source of truth — an adopter-owned store the engine does not manage (a folder of files, an existing database) — and keeps the two converged bidirectionally.
Package connect binds the sync engine to an external source of truth — an adopter-owned store the engine does not manage (a folder of files, an existing database) — and keeps the two converged bidirectionally.
connect/sqlitestate
Package sqlitestate implements connect.State on SQLite via modernc.org/sqlite (pure Go, no CGO) — the shipped default for connector bookkeeping.
Package sqlitestate implements connect.State on SQLite via modernc.org/sqlite (pure Go, no CGO) — the shipped default for connector bookkeeping.
cursorstate
Package cursorstate stores the bulky membership tail of pull cursors — the subscription set and the windowed/closure key-sets — so the cursor a client carries is a fixed-size reference instead of a restatement of everything it holds.
Package cursorstate stores the bulky membership tail of pull cursors — the subscription set and the windowed/closure key-sets — so the cursor a client carries is a fixed-size reference instead of a restatement of everything it holds.
effects
Package effects runs server-side reactions to committed changes: webhooks, notifications, search indexing, agents — anything that should happen after a row is written and is not itself part of the write.
Package effects runs server-side reactions to committed changes: webhooks, notifications, search indexing, agents — anything that should happen after a row is written and is not itself part of the write.
fractional
Package fractional makes order keys: strings that sort a list and that always have room for one more key between any two of them.
Package fractional makes order keys: strings that sort a list and that always have room for one more key between any two of them.
id
Package id generates time-ordered entity identifiers: a UUIDv7 (RFC 9562) rendered as 26-character Crockford base32.
Package id generates time-ordered entity identifiers: a UUIDv7 (RFC 9562) rendered as 26-character Crockford base32.
internal/lww
Package lww merges PATCH mutations into row snapshots using field masks.
Package lww merges PATCH mutations into row snapshots using field masks.
internal/naming
Package naming holds the naming rules of the sync engine: what an entity may be called, what a workspace may be identified by, which fields may form a row key, how their values render into one, and what the resulting key may contain.
Package naming holds the naming rules of the sync engine: what an entity may be called, what a workspace may be identified by, which fields may form a row key, how their values render into one, and what the resulting key may contain.
internal/pgtest
Package pgtest supplies the live Postgres the pgstore, pglisten, pgnotify and pgpresence suites run against.
Package pgtest supplies the live Postgres the pgstore, pglisten, pgnotify and pgpresence suites run against.
internal/pubsub
Package pubsub provides change-notification fan-out.
Package pubsub provides change-notification fan-out.
internal/pubsub/pubsubtest
Package pubsubtest provides a shared Ginkgo behavior suite every pubsub.Notifier implementation must pass.
Package pubsubtest provides a shared Ginkgo behavior suite every pubsub.Notifier implementation must pass.
internal/telemetry
Package telemetry holds what every instrumented engine package shares: the instrumentation scope, the attribute keys, and the provider resolution that makes telemetry free when nobody has configured it.
Package telemetry holds what every instrumented engine package shares: the instrumentation scope, the attribute keys, and the provider resolution that makes telemetry free when nobody has configured it.
internal/textpath
Package textpath finds the collaborative-text documents of an entity row and reads and writes them by instance path.
Package textpath finds the collaborative-text documents of an entity row and reads and writes them by instance path.
lease
Package lease hands out per-workspace work leases: advisory try-locks that let one holder of a (role, workspace) pair proceed while everyone else skips.
Package lease hands out per-workspace work leases: advisory try-locks that let one holder of a (role, workspace) pair proceed while everyone else skips.
notify/pgnotify
Package pgnotify implements the engine's change-notification fan-out over PostgreSQL LISTEN/NOTIFY, so pokes reach clients connected to any node rather than only the one that took the write.
Package pgnotify implements the engine's change-notification fan-out over PostgreSQL LISTEN/NOTIFY, so pokes reach clients connected to any node rather than only the one that took the write.
pglisten
Package pglisten is the PostgreSQL LISTEN/NOTIFY transport the engine's cross-node fan-outs are built on: one connection parked on the channels a process listens to, one issuing pg_notify, and the reconnect loop that keeps both alive.
Package pglisten is the PostgreSQL LISTEN/NOTIFY transport the engine's cross-node fan-outs are built on: one connection parked on the channels a process listens to, one issuing pg_notify, and the reconnect loop that keeps both alive.
presence
Package presence carries the sync engine's ephemeral plane: the state that only matters while someone is connected.
Package presence carries the sync engine's ephemeral plane: the state that only matters while someone is connected.
presence/pgpresence
Package pgpresence implements the ephemeral plane's fan-out over PostgreSQL LISTEN/NOTIFY, so rooms reach clients connected to any node rather than only the one holding a given session's stream.
Package pgpresence implements the ephemeral plane's fan-out over PostgreSQL LISTEN/NOTIFY, so rooms reach clients connected to any node rather than only the one holding a given session's stream.
presence/presencetest
Package presencetest provides a shared Ginkgo behavior suite every presence.Bus implementation must pass.
Package presencetest provides a shared Ginkgo behavior suite every presence.Bus implementation must pass.
query
Package query builds, compiles and evaluates syncplane.v1.Condition predicates against protoreflect messages.
Package query builds, compiles and evaluates syncplane.v1.Condition predicates against protoreflect messages.
search
Package search implements the text analyzer behind full-text search: the pipeline that turns field text and query strings into the terms the match evaluator and the search index agree on.
Package search implements the text analyzer behind full-text search: the pipeline that turns field text and query strings into the terms the match evaluator and the search index agree on.
server
Reference-following subscriptions: a subscription's synced set is its base predicate scope UNION a bounded-depth closure reached by walking typed reference relations from the base-visible rows.
Reference-following subscriptions: a subscription's synced set is its base predicate scope UNION a bounded-depth closure reached by walking typed reference relations from the base-visible rows.
storage
Package storage defines the pluggable persistence interface for the sync engine.
Package storage defines the pluggable persistence interface for the sync engine.
storage/otelstore
Package otelstore instruments any storage.Adapter with OpenTelemetry.
Package otelstore instruments any storage.Adapter with OpenTelemetry.
storage/pgstore
Package pgstore implements storage.Adapter on PostgreSQL via pgx v5.
Package pgstore implements storage.Adapter on PostgreSQL via pgx v5.
storage/prefixstore
Package prefixstore partitions one storage adapter between several servers by giving each its own prefix of the workspace-id space.
Package prefixstore partitions one storage adapter between several servers by giving each its own prefix of the workspace-id space.
storage/sqlitestore
Package sqlitestore implements storage.Adapter on SQLite via modernc.org/sqlite (pure Go, no CGO).
Package sqlitestore implements storage.Adapter on SQLite via modernc.org/sqlite (pure Go, no CGO).
storage/storagetest
Package storagetest provides a shared Ginkgo behavior suite every storage.Adapter implementation must pass.
Package storagetest provides a shared Ginkgo behavior suite every storage.Adapter implementation must pass.
text
Package text implements a convergent (CRDT) text document with Peritext-style marks and stable anchors, designed to live in a `bytes` entity field marked (syncplane.v1.field_behavior).text.
Package text implements a convergent (CRDT) text document with Peritext-style marks and stable anchors, designed to live in a `bytes` entity field marked (syncplane.v1.field_behavior).text.
tunnel
Package tunnel carries HTTP exchanges over a message channel, so an http.Handler can serve clients that have no socket to it: a webview behind a desktop shell, a process on the other end of stdio, a page across a WebSocket or a MessagePort.
Package tunnel carries HTTP exchanges over a message channel, so an http.Handler can serve clients that have no socket to it: a webview behind a desktop shell, a process on the other end of stdio, a page across a WebSocket or a MessagePort.
upstream
Package upstream keeps a local syncplane engine converged with an origin syncplane server, so that a Go process can serve, read and write synced data while offline: a desktop app whose webview syncs against the local engine, a server that syncs with another, or a local engine that forwards an origin to browsers.
Package upstream keeps a local syncplane engine converged with an origin syncplane server, so that a Go process can serve, read and write synced data while offline: a desktop app whose webview syncs against the local engine, a server that syncs with another, or a local engine that forwards an origin to browsers.
upstream/blobbridge
Package blobbridge moves blob bytes between a local engine and the origin server it syncs with.
Package blobbridge moves blob bytes between a local engine and the origin server it syncs with.
upstream/internal/rebase
Package rebase computes the rows a replica holds from the rows its origin confirmed and the local writes the origin has not confirmed yet.
Package rebase computes the rows a replica holds from the rows its origin confirmed and the local writes the origin has not confirmed yet.
upstream/presencebridge
Package presencebridge joins the rooms of a local syncplane engine to the rooms of the origin server it syncs with, so that people connected to the origin and people connected to the local engine see each other's presence and broadcasts.
Package presencebridge joins the rooms of a local syncplane engine to the rooms of the origin server it syncs with, so that people connected to the origin and people connected to the local engine see each other's presence and broadcasts.
internal
admin
Package admin is the built-in, read/write web UI for a server.Manager: it lists hosted apps, browses each app's entity data per workspace, and creates/edits/deletes rows through a generic, reflection-driven proto editor.
Package admin is the built-in, read/write web UI for a server.Manager: it lists hosted apps, browses each app's entity data per workspace, and creates/edits/deletes rows through a generic, reflection-driven proto editor.
admin/ui
templ: version: v0.3.857
templ: version: v0.3.857
benchapp
Package benchapp holds the corpus the browser sync benchmarks measure against: the syncplane.bench.v1 issue tracker, its size tiers, and a deterministic generator that seeds a workspace server-side.
Package benchapp holds the corpus the browser sync benchmarks measure against: the syncplane.bench.v1 issue tracker, its size tiers, and a deterministic generator that seeds a workspace server-side.
clilog
Package clilog builds the structured logger used by the engine's command binaries from -log-format / -log-level flags and installs it as slog.Default, so the server and connector libraries (which default to slog.Default) inherit the same handler.
Package clilog builds the structured logger used by the engine's command binaries from -log-format / -log-level flags and installs it as slog.Default, so the server and connector libraries (which default to slog.Default) inherit the same handler.
exampleapp
Package exampleapp holds the todo-app wiring shared by cmd/testserver and cmd/todoserver: entity registration, the complete_all mutator, and a small permission policy exercised by the test suites.
Package exampleapp holds the todo-app wiring shared by cmd/testserver and cmd/todoserver: entity registration, the complete_all mutator, and a small permission policy exercised by the test suites.
protoopts
Package protoopts reads the sync custom options — (syncplane.v1.entity), (syncplane.v1.reference) and (syncplane.v1.field_behavior) — off proto descriptors.
Package protoopts reads the sync custom options — (syncplane.v1.entity), (syncplane.v1.reference) and (syncplane.v1.field_behavior) — off proto descriptors.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL