rdoc

README

rdoc

Build better decentralized and offline-first application in Go

rdoc is a native go implementation of a conflict-free replicated JSON data structure, as introduced by Martin Kleppmann and Alastair R. Beresford in their seminal work [1]. A JSON CRDT is "[...] an algorithm and formal semantics for a JSON data structure that automatically resolves concurrent modifications such that no updates are lost, and such that all replicas converge towards the same state (a conflict-free replicated datatype or CRDT)." [1];

Do you want to learn more about the JSON CRDT data type? This youtube video is a good introduction to the original paper [1] by Martin Kleppmann and Alastair R. Beresford.

Features

• Simple API; One API call allows the application logic to update and manage coverging JSON replicas in decentralized settings;
• Supports JSON Patch notation as defined in RFC6902;
• Supports cbor serialization [WIP; v1.1.0 milestone];

Examples

// starts a new replicated JSON document with an unique ID (in the context of the replicas sample)
doc := Init("doc_replica_1")

// updates the document state with a JSON patch operation:
patch := []byte(`{"op": "add", "path": "/", "value": "user"`)
err := doc.Apply(patch)
if err != nil {
    panic(err)
}

// update the document state with remote operations (i.e. operations executed by a remote replica); 
// remote operations will update the state of the document iif all its dependencies have been applied.  
remotePath := []byte(`[
 {"op": "add", "path": "/", "value": "user", "id":"1.380503024", "deps": [] },
 {"op": "add", "path": "/name", "value": "Jane", "id":"2.1", "deps": ["1.1"] },
 {"op": "add", "path": "/name", "value": "Jane", "id":"2.380503024", "deps": ["1.380503024"] }
]`)

err := doc.Apply(remotePath)
if err != nil {
    panic(err)
}

// Get Doc operations to sent over the wire to merge into other replicas
doc1Operations := doc.Operations()

// ... apply state from doc1 into doc2, in order for replicas to converge

doc2.Apply(doc1Operations)

// ...

// Native Go marshaling/unmarshaling supported 
buffer, err := json.Marshal(*doc)
if err != nil {
    panic(err)
}

References

1. A Conflict-Free Replicated JSON Datatype (Martin Kleppmann, Alastair R. Beresford)

Documentation

Overview

Package rdoc is a native go implementation of a conflict-free replicated JSON data structure (JSON CRDT). A JSON CRDT is a data structure that automatically resolves concurrent modifications such that no updates are lost, and such that all replicas converge towards the same state.

Index

• type Doc
  • func Init(id string) *Doc
  • func (doc *Doc) Apply(rawPatch []byte) error
  • func (doc Doc) MarshalFullJSON() ([]byte, error)
  • func (doc Doc) MarshalJSON() ([]byte, error)
  • func (doc Doc) Operations() ([]byte, error)
• type Operation

Examples

• Doc.Apply

Types

type Doc

type Doc struct {
	// contains filtered or unexported fields
}

Doc represents a JSON CRDT document; It maintains the metadata necessary to guarantee that the state of the document replicas converge over time without losing data.

func Init

func Init(id string) *Doc

Init initiates and returns a new JSON CRDT document. The input is a string encoding an unique ID of the replica. The application logic *must* ensure that the replica IDs are unique within its network

func (*Doc) Apply

func (doc *Doc) Apply(rawPatch []byte) error

Apply applies a valid operation represented as a JSON patch (https://tools.ietf.org/html/rfc6902) on the document. Apply handles both local and remote operations.

Example

package main

import (
	"encoding/json"
	"fmt"

	"github.com/gpestana/rdoc"
)

func main() {
	// Initiates a new document with an unique ID (not enforced by underlying library)
	doc := rdoc.Init("document_id")

	remotePatch := []byte(`[
{"op": "add", "path": "/", "value": "user", "id":"1.380503024", "deps": [] },
{"op": "add", "path": "/name", "value": "Jane", "id":"2.1", "deps": ["1.1"] },
{"op": "add", "path": "/name", "value": "Jane", "id":"2.380503024", "deps": ["1.380503024"] }
]`)

	localPatch := []byte(`[
{"op": "add", "path": "/", "value": "user" },
{"op": "add", "path": "/age", "value": 20 }
]`)

	err := doc.Apply(remotePatch)
	if err != nil {
		// handle error, most likely reason is malformed patch
	}

	err = doc.Apply(localPatch)
	if err != nil {
		// handle error, most likely reason is malformed patch
	}

	// Marshals applied document into buffer
	buffer, err := json.Marshal(*doc)
	if err != nil {
		// handle error
	}

	fmt.Println(buffer)
}

func (Doc) MarshalFullJSON

func (doc Doc) MarshalFullJSON() ([]byte, error)

MarshalFullJSON marshals a Doc into a buffer, including the dependencies field of each operation.

func (Doc) MarshalJSON

func (doc Doc) MarshalJSON() ([]byte, error)

MarshalJSON marshals a Doc into a buffer, excluding the deps field of each operation. The returned buffer *contains only the applied operations* that mutated the document state. The buffered operations as not included in the document serialization.

func (Doc) Operations

func (doc Doc) Operations() ([]byte, error)

Operations returns the encoded operations applied to the document. The output of this function can be sent over the wire to other replicas, in order to achieve convergence.

type Operation

type Operation struct {
	// contains filtered or unexported fields
}

Operation the metadata of a valid operation to perform a mutation on the JSON CRDT's document state.