ygo

README

Ygo

Pure-Go port of Yjs, the CRDT framework for collaborative applications.

Ygo speaks the Yjs V1 and V2 wire formats byte-for-byte, so JavaScript clients running yjs@13.x synchronize directly with Go servers (and vice versa) — both directions verified by 109 cross-language fixture scenarios against yjs@13.6.20. The bundled WebSocket server is Hocuspocus-compatible. No CGO; gomobile bind produces verified iOS xcframework and Android AAR.

Live demo: open ygo.deln0r.com in two browser tabs and start typing — same protocol any standard Yjs ecosystem client speaks, with a pure-Go server behind it.

Quick start

package main

import (
    "fmt"

    "github.com/Deln0r/ygo"
)

func main() {
    src := ygo.NewDoc()
    m := ygo.NewMap(src, "settings")

    txn := src.WriteTxn()
    m.Set(txn, "theme", "dark")
    m.Set(txn, "lang", "go")
    txn.Commit()

    // Encode the source doc's full state as wire bytes.
    update := ygo.EncodeStateAsUpdate(src)

    // Apply to a fresh peer doc — same bytes JS Yjs's Y.applyUpdate consumes.
    dst := ygo.NewDoc()
    if err := ygo.ApplyUpdate(dst, update); err != nil {
        panic(err)
    }

    dstMap := ygo.NewMap(dst, "settings")
    fmt.Println(dstMap.Get("theme")) // dark
}

For a collaborative server backend, see cmd/ygo-server — a stand-alone Hocuspocus-compatible WebSocket server with optional sqlite persistence:

go run ./cmd/ygo-server -addr :1234 -store data.db

Status

Alpha. Public API may change before v1.0. The CRDT engine and wire format are production-stable in the sense that they have been validated bidirectionally against yjs@13.6.20; the API surface (function signatures, package layout) may still see small refinements.

Layer	Status
internal/lib0 varint + RLE encoding	done; verified byte-equivalent vs JS lib0@0.2.93 (40 + 16 fixtures)
internal/block (Item, Content, Branch, Splice, Integrate-YATA, TrySquash, Repair, search markers)	done; full YATA conflict resolution + per-branch LRU position cache
internal/store (BlockStore, ItemSlice, Materialize)	done
internal/doc (Doc, Transaction, TransactionMut)	done; lock semantics + root-branch registry
internal/encoding (StateVector, IdSet, Update encode/decode/apply, Pending buffer, V1 + V2 codecs)	done; JS Yjs → Go proven by 29 V1 + 32 V2 fixture scenarios; Go → JS proven by 48 reverse fixtures (Map / Array / Text / XmlFragment)
internal/utf16 (UTF-16 length / byte-offset / surrogate-aware split)	done
internal/types/Map (Set / Get / Delete / Has / Len / Range / Clear + SetMap / SetArray / SetText)	done; nested-type construction supported
internal/types/Array (Insert / InsertRange / Push / Delete / Get / Len / Range / ToSlice + InsertMap / InsertArray / InsertText)	done; nested-type construction supported
internal/types/Text (Insert / Delete / String / Length + InsertWithAttributes / Format / InsertEmbed / Range / ToDelta / ApplyDelta)	done; full rich-text + Quill delta batch API
Nested-type construction (Map-in-Map, Array-in-Map, etc., to arbitrary depth)	done; ContentType wire format + Repair ParentID resolution + pending-queue retry
internal/types/Xml* (XmlFragment, XmlElement, XmlText)	done; ProseMirror/Tiptap/BlockNote unblocked. XmlHook (legacy) deferred.
Persistence (Store interface + modernc.org/sqlite reference impl)	done; append-only update log, Flush compaction, LoadDoc / GetStateVector / GetDiff helpers; pure-Go (no CGO)
y-sync protocol (internal/sync)	done; full Hocuspocus message subset (Sync + Awareness + QueryAwareness + Auth + Stateless + BroadcastStateless + Close + SyncStatus); per-document Auth permission scoping deferred (tech-debt)
Awareness (internal/awareness)	done; LWW presence map, JSON wire payload per y-protocols, self-eviction defense, SweepOutdated
server/ (WebSocket sync server)	done; http.Handler mount-anywhere shape, per-doc broadcaster, persists every applied update to optional persist.Store, awareness disconnect tombstones
cmd/ygo-server (Hocuspocus-compat binary)	done; stand-alone WS server with optional sqlite persistence via -store flag
gomobile/ (bytes-only subset for iOS/Android)	done; bindable Doc + Awareness wrappers with bytes-in/bytes-out methods only; pure-Go (no CGO). Both targets verified end-to-end on Xcode 16 + NDK 27 + Go 1.26: produces a valid Ygo.xcframework (real-device arm64 + simulator universal, 6.6 + 13 MB) and a valid Android .aar (4 archs incl. arm64-v8a / armeabi-v7a / x86 / x86_64, 8.4 MB), each drop-in for the respective IDE. See gomobile/README.md for the exact commands.
V2 update encoding	done; lib0 RLE primitives + column encoder/decoder + Update.{EncodeV2,DecodeV2} + public ygo.{EncodeStateAsUpdateV2,EncodeDiffV2,ApplyUpdateV2}; bidirectional cross-language fixtures vs yjs@13.6.20
dmonad/crdt-benchmarks B1-B4 port	done; B1.1-B1.11 / B2.1-B2.4 / B3.1+3+4 / B4 (260k-edit real-world LaTeX trace). Baseline in BENCHMARKS.md.
Undo manager / Snapshots / Subdocs / Y.Array.move / GC merging / commit-time block squash	planned for v1.0; see Roadmap

Goals

1. Binary protocol compatibility with Yjs v13.x in both V1 and V2 wire formats. Byte-for-byte. JS clients sync with Go servers and vice versa, bidirectionally verified.
2. Idiomatic Go API. Channels for events, explicit transactions, error returns.
3. Pure Go. No CGO. gomobile bind works for iOS/Android.
4. Pluggable persistence with modernc.org/sqlite reference implementation.
5. Performance within 2× of yrs on dmonad/crdt-benchmarks B1-B4. See BENCHMARKS.md.

Non-goals

• C-FFI surface. Yrs already provides this; Ygo's unique value is pure-Go native binaries.
• Drop-in replacement for the Node.js Yjs runtime. Ygo is the Go port; use yjs itself if you want a JavaScript runtime.
• Loro, Automerge, RGA, or other CRDT designs. Ygo implements the Yjs wire format, period.

Wire compatibility

The single most-important guarantee of this project is byte-level wire compatibility with yjs@13.x. This is enforced by 109 cross-language fixture scenarios:

• 29 V1 forward fixtures (testdata/yjs-updates.json) — JS Yjs encodes via Y.encodeStateAsUpdate, Go decodes and applies, state matches.
• 32 V2 forward fixtures (testdata/yjs-update-v2-fixtures.json) — same with Y.encodeStateAsUpdateV2.
• 48 reverse fixtures (testdata/go-updates.json + go-update-v2-fixtures.json) — Go encodes via EncodeStateAsUpdate / EncodeStateAsUpdateV2, JS Yjs decodes via Y.applyUpdate / Y.applyUpdateV2, state matches.

The fixtures regenerate from pinned yjs@13.6.20 + lib0@0.2.93 + y-protocols@1.0.6 on every CI run; git diff --exit-code testdata/ catches byte-level regressions.

How is this different from Hocuspocus / y-websocket / y-leveldb?

Project	Runtime	What it provides	Relationship to Ygo
yjs (npm)	Node / browser	The reference CRDT implementation	Ygo's wire-format target
y-websocket	Node	Reference WebSocket server	Ygo's cmd/ygo-server is a Go-native equivalent
Hocuspocus	Node	Production WebSocket server with auth, persistence, extensions	Ygo's cmd/ygo-server speaks the same envelope (Auth / Stateless / Close / SyncStatus)
yrs	Rust	Reference Rust port	Ygo's executable spec for porting decisions
y-leveldb, y-indexeddb	Node / browser	Persistence backends	Ygo's persist/sqlite is a Go-native equivalent
Ygo	Go	CRDT engine + WS server + persistence in one monorepo, pure-Go for native mobile	This project

If you have an existing Yjs deployment and want to move the server side to Go (no Node runtime, single static binary, native iOS / Android via gomobile) — Ygo is the path. If you're starting fresh and your team is comfortable with Node, Hocuspocus is the mature choice.

Benchmarks

See BENCHMARKS.md for the full table. Highlights from B4 (259,778-edit real-world LaTeX paper trace) on Apple M3, Go 1.26:

Metric	Ygo V1	Ygo V2	yjs (Node, Intel i5-8400)	ywasm (Intel i5-8400)
Apply all edits	10.5 s	10.5 s	5.7 s	28.7 s
Encoded doc size	1.97 MB	227 KB	160 KB	160 KB
Encode time	7.7 ms	73 ms	11 ms	3 ms
Parse time	68 ms	61 ms	39 ms	16 ms

How to read this:

• Apply throughput — within ~1.85× of native yjs on different hardware (Apple M3 vs Intel i5-8400; the M3 is generally faster so the real ratio is closer than the wall-clock suggests). Native yrs publishes sub-10-s numbers on similar hardware, putting Ygo within roughly 1.0-1.5× of yrs and comfortably under the DESIGN.md "within 2×" target. (ywasm is yrs compiled to WebAssembly and is not representative of native yrs — wasm overhead inflates it ~5×.)
• V2 doc size is competitive with yjs at 1.4× — V2's per-column RLE encoding effectively dedupes per-item overhead at the wire layer.
• V1 doc size carries a known 12× regression vs yjs's V1 because commit-time block squash is deferred (every Text.Insert produces a separate Item, none merged with same-client adjacent-clock neighbours). The fix is paired Apply-side partial-overlap handling + commit-time squash; in the Roadmap and scoped into the v1.0 grant work. Until then, prefer V2 for persistence/snapshot workloads where size matters.

A direct head-to-head harness against native yrs under identical hardware is on the roadmap but not yet run; the numbers above are honest absolute figures with hardware caveats.

Roadmap

Towards v1.0: Undo manager · Snapshots · Subdocs · GC merging · Y.Array.move · commit-time block squash · external security audit · documentation site.

Per-layer port notes live in docs/yrs-port-notes/. Items intentionally deferred or partial are tracked in docs/tech-debt.md. Detailed design decisions in DESIGN.md.

Documentation

• DESIGN.md — project design document
• BENCHMARKS.md — performance baseline + B1-B4 methodology
• gomobile/README.md — iOS xcframework + Android AAR build instructions
• docs/yrs-port-notes/ — per-layer port notes describing how each yrs subsystem maps to Go
• docs/tech-debt.md — deferred work and known limitations
• CONTRIBUTING.md — DCO sign-off, no CLA

License

MIT. See LICENSE.

Acknowledgements

• Kevin Jahns (dmonad) for Yjs and the YATA algorithm.
• Bartosz Sypytkowski for yrs and the architecture deep dive.

Documentation

Overview

Package ygo is a pure-Go port of the Yjs CRDT framework.

ygo is binary-protocol compatible with the npm yjs package (V1 update encoding), allowing JavaScript clients to synchronize seamlessly with Go servers and vice versa.

Pure-Go means no CGO, so gomobile bind works for iOS/Android targets.

Status: pre-alpha. Public API is unstable.

See https://github.com/Deln0r/ygo for documentation and examples.

Index

• Constants
• func ApplyUpdate(d *Doc, raw []byte) error
• func ApplyUpdateV2(d *Doc, raw []byte) error
• func EncodeDiff(d *Doc, remoteSVBytes []byte) ([]byte, error)
• func EncodeDiffV2(d *Doc, remoteSVBytes []byte) ([]byte, error)
• func EncodeStateAsUpdate(d *Doc) []byte
• func EncodeStateAsUpdateV2(d *Doc) []byte
• func EncodeStateVector(d *Doc) []byte
• func HasPending(d *Doc) bool
• func MergeUpdates(updates [][]byte) ([]byte, error)
• func MissingSV(d *Doc) []byte
• type Array
  • func NewArray(d *Doc, name string) *Array
• type Attrs
• type Awareness
  • func NewAwareness(clientID uint64) *Awareness
• type Branch
• type ChunkKind
• type DeltaOp
• type Doc
  • func NewDoc() *Doc
  • func NewDocWithOptions(opts Options) *Doc
• type Map
  • func NewMap(d *Doc, name string) *Map
• type Options
• type Text
  • func NewText(d *Doc, name string) *Text
• type Transaction
• type TransactionMut
• type XmlElement
  • func NewXmlElement(d *Doc, name string) *XmlElement
• type XmlFragment
  • func NewXmlFragment(d *Doc, name string) *XmlFragment
• type XmlText
  • func NewXmlText(d *Doc, name string) *XmlText

Constants

const (
	ChunkString = types.ChunkString
	ChunkEmbed  = types.ChunkEmbed
)

Text.Range emits chunks of either of these kinds.

const DefaultAwarenessTimeout = awareness.DefaultTimeout

DefaultAwarenessTimeout is the y-protocols convention for stale awareness-entry eviction (30 seconds). Pass to Awareness.SweepOutdated.

const MaxClientID = doc.MaxClientID

MaxClientID is the upper bound on Doc.ClientID values (2^53 - 1, matching JS Yjs's safe-integer range).

const Version = "0.0.0-dev"

Version is the current ygo version.

Functions

func ApplyUpdate

func ApplyUpdate(d *Doc, raw []byte) error

ApplyUpdate decodes raw and integrates it into d. Items whose dependencies the local store has not yet seen queue in the per-doc pending buffer and drain automatically on subsequent ApplyUpdate calls that satisfy them.

Use HasPending / MissingSV to inspect the queue.

func ApplyUpdateV2

func ApplyUpdateV2(d *Doc, raw []byte) error

ApplyUpdateV2 decodes V2 wire bytes and integrates them into d. Pending-buffer semantics identical to ApplyUpdate (V1) — items missing causal dependencies queue silently and drain on subsequent ApplyUpdate / ApplyUpdateV2 calls.

V1 and V2 are NOT wire-interchangeable. Calling ApplyUpdateV2 on V1 bytes (or ApplyUpdate on V2 bytes) is undefined behaviour — either errors loudly or yields a semantically-wrong Update that fails integrate. Per the docs there is no autodetect; the caller must know which version they have via the surrounding transport metadata.

func EncodeDiff

func EncodeDiff(d *Doc, remoteSVBytes []byte) ([]byte, error)

EncodeDiff returns the wire-encoded V1 update covering the blocks d has that the remote (per remoteSVBytes) does not. A nil remoteSVBytes is treated as the empty SV — emit everything.

remoteSVBytes is the V1 wire-encoded form of the remote's state vector (the same shape EncodeStateVector produces).

func EncodeDiffV2

func EncodeDiffV2(d *Doc, remoteSVBytes []byte) ([]byte, error)

EncodeDiffV2 is the V2 analogue of EncodeDiff. State-vector argument shape is identical (still V1 wire-encoded SV); only the outgoing update bytes use the V2 column layout.

func EncodeStateAsUpdate

func EncodeStateAsUpdate(d *Doc) []byte

EncodeStateAsUpdate returns the wire-encoded V1 update carrying the doc's full state. Apply to a fresh peer doc to bring it up to speed in one shot; same bytes interoperate with JS Yjs's Y.encodeStateAsUpdate.

func EncodeStateAsUpdateV2

func EncodeStateAsUpdateV2(d *Doc) []byte

EncodeStateAsUpdateV2 returns the wire-encoded V2 update carrying the doc's full state. V2 is the column-oriented alternative wire format used by Y.encodeStateAsUpdateV2 / Hocuspocus-V2 paths and some adopters' on-disk persistence layers (y-leveldb, y-indexeddb, SQLite/Postgres Hocuspocus adapters).

V1 and V2 are NOT wire-interchangeable — see ApplyUpdateV2.

func EncodeStateVector

func EncodeStateVector(d *Doc) []byte

EncodeStateVector returns the wire-encoded V1 state vector of d. Sync-protocol callers send this to peers as "here's what I have; send me everything else."

func HasPending

func HasPending(d *Doc) bool

HasPending reports whether d has any queued items awaiting causal dependencies.

func MergeUpdates

func MergeUpdates(updates [][]byte) ([]byte, error)

MergeUpdates decodes every blob in updates in order, applies them to a fresh Doc, and returns a single V1 update blob equivalent to the merged state. Returns nil for an empty input.

Used by persistence layers for compaction (Flush) and by transports that want to batch-coalesce updates before sending.

func MissingSV

func MissingSV(d *Doc) []byte

MissingSV returns the wire-encoded V1 state vector identifying the clocks d needs to receive in order to drain its pending buffer. An empty result means the queue is empty.

Sync-protocol callers send this to peers as a re-fetch request: "I am stuck on items that need updates past this clock."

Types

type Array

type Array = types.Array

Shared-type wrappers — re-exported from internal/types.

func NewArray

func NewArray(d *Doc, name string) *Array

type Attrs

type Attrs = types.Attrs

Attrs is the format-attribute map used by Text's rich-text API and DeltaOp. Keys are arbitrary strings; values are JSON-serializable scalars. A nil value clears the attribute on the affected range.

type Awareness

type Awareness = awareness.Awareness

Awareness tracks per-client ephemeral state (cursors, names, selections). Independent of any Doc; the local clientID is passed at construction. Embedders typically pair an Awareness with a Doc and use d.ClientID() as the awareness clientID.

func NewAwareness

func NewAwareness(clientID uint64) *Awareness

NewAwareness returns a fresh Awareness for the given local clientID. Use d.ClientID() to keep the awareness layer in sync with the doc.

type Branch

type Branch = block.Branch

Branch is the low-level shared-data container the types layer wraps. Most callers should use the typed constructors (NewMap, NewArray, NewText, NewXmlFragment) rather than building Branches directly. Re-exported here for advanced cases (custom shared-type implementations, observability code).

type ChunkKind

type ChunkKind = types.ChunkKind

ChunkKind discriminates the variants emitted by Text.Range.

type DeltaOp

type DeltaOp = types.DeltaOp

DeltaOp is one Quill-style delta operation produced by Text.ToDelta and (future) consumed by Text.ApplyDelta.

type Doc

type Doc = doc.Doc

Doc is a single CRDT replica — the local view of a collaborative document. Construct with NewDoc; mutate via WriteTxn; read via ReadTxn. See the doc-comment on the underlying type for the full concurrency contract.

func NewDoc

func NewDoc() *Doc

NewDoc returns a fresh Doc with default options and a random client identifier.

func NewDocWithOptions

func NewDocWithOptions(opts Options) *Doc

NewDocWithOptions returns a fresh Doc with the given options.

type Map

type Map = types.Map

Shared-type wrappers — re-exported from internal/types.

func NewMap

func NewMap(d *Doc, name string) *Map

NewMap, NewArray, NewText, NewXmlFragment return wrappers bound to the root branch with the given name in d. The branch is lazily created on first call; subsequent calls with the same name return wrappers pointing at the same underlying state.

Per-branch type discipline: a branch should be used as ONE type (Map OR Array OR Text OR XML). Mixing types on the same root branch produces undefined behaviour.

type Options

type Options = doc.Options

Options bundles per-Doc settings (deterministic ClientID, GC disable). The zero value is the recommended configuration.

type Text

type Text = types.Text

Shared-type wrappers — re-exported from internal/types.

func NewText

func NewText(d *Doc, name string) *Text

type Transaction

type Transaction = doc.Transaction

Transaction is a read-only transaction holding the doc's read lock for its lifetime. Created by Doc.ReadTxn; released by Close.

type TransactionMut

type TransactionMut = doc.TransactionMut

TransactionMut is a write transaction holding the doc's write lock. Created by Doc.WriteTxn; released by Commit.

type XmlElement

type XmlElement = types.XmlElement

Shared-type wrappers — re-exported from internal/types.

func NewXmlElement

func NewXmlElement(d *Doc, name string) *XmlElement

NewXmlElement wraps a branch as an XmlElement. Typically used for root-level XML where the branch was constructed via d.Branch(name); nested elements should be constructed via XmlFragment.InsertXmlElement / XmlElement.InsertXmlElement which set TypeRef and Name automatically.

type XmlFragment

type XmlFragment = types.XmlFragment

Shared-type wrappers — re-exported from internal/types.

func NewXmlFragment

func NewXmlFragment(d *Doc, name string) *XmlFragment

type XmlText

type XmlText = types.XmlText

Shared-type wrappers — re-exported from internal/types.

func NewXmlText

func NewXmlText(d *Doc, name string) *XmlText