sync

README

Sync

Sync is a synchronization library for distributed systems.

Overview

Distributed systems by their very nature are decoupled and independent. In most cases they must honour 2 out of 3 letters of the CAP theorem e.g Availability and Partitional tolerance but sacrificing consistency. In the case of microservices we often offload this concern to an external database or eventing system. Go Sync provides a framework for synchronization which can be used in the application by the developer.

Getting Started

• Data - simple distributed data storage
• Leader - leadership election for group coordination
• Lock - distributed locking for exclusive resource access
• Task - distributed job execution
• Time - provides synchronized time

Lock

The Lock interface provides distributed locking. Multiple instances attempting to lock the same id will block until available.

import "github.com/micro/go-micro/sync/lock/consul"

lock := consul.NewLock()

// acquire lock
err := lock.Acquire("id")
// handle err

// release lock
err = lock.Release("id")
// handle err

Leader

Leader provides leadership election. Useful where one node needs to coordinate some action.

import (
	"github.com/micro/go-micro/sync/leader"
	"github.com/micro/go-micro/sync/leader/consul"
)

l := consul.NewLeader(
	leader.Group("name"),
)

// elect leader
e, err := l.Elect("id")
// handle err


// operate while leader
revoked := e.Revoked()

for {
	select {
	case <-revoked:
		// re-elect
		e.Elect("id")
	default:
		// leader operation
	}
}

// resign leadership
e.Resign() 

Data

Data provides a simple interface for distributed data storage.

import (
	"github.com/micro/go-micro/sync/data"
	"github.com/micro/go-micro/sync/data/consul"
)

keyval := consul.NewData()

err := keyval.Write(&data.Record{
	Key: "foo",
	Value: []byte(`bar`),
})
// handle err

v, err := keyval.Read("foo")
// handle err

err = keyval.Delete("foo")

Task

Task provides distributed job execution. It's a simple way to distribute work across a coordinated pool of workers.

import (
	"github.com/micro/go-micro/sync/task"
	"github.com/micro/go-micro/sync/task/local"
)

t := local.NewTask(
	task.WithPool(10),
)

err := t.Run(task.Command{
	Name: "atask",
	Func: func() error {
		// exec some work
		return nil
	},
})

if err != nil {
	// do something
}

Time

Time provides synchronized time. Local machines may have clock skew and time cannot be guaranteed to be the same everywhere. Synchronized Time allows you to decide how time is defined for your applications.

import (
	"github.com/micro/go-micro/sync/time/ntp"
)


t := ntp.NewTime()
time, err := t.Now()

TODO

• Event package - strongly consistent event stream e.g kafka

Documentation

Overview

Package sync is a distributed synchronization framework

Index

• type Cron
  • func NewCron(opts ...Option) Cron
• type DB
  • func NewDB(opts ...Option) DB
• type Option
  • func WithData(s data.Data) Option
  • func WithLeader(l leader.Leader) Option
  • func WithLock(l lock.Lock) Option
  • func WithTime(t time.Time) Option
• type Options

Types

type Cron

type Cron interface {
	Schedule(task.Schedule, task.Command) error
}

Cron is a distributed scheduler using leader election and distributed task runners. It uses the leader and task interfaces.

func NewCron

func NewCron(opts ...Option) Cron

type DB

type DB interface {
	// Read value with given key
	Read(key, val interface{}) error
	// Write value with given key
	Write(key, val interface{}) error
	// Delete value with given key
	Delete(key interface{}) error
	// Iterate over all key/vals. Value changes are saved
	Iterate(func(key, val interface{}) error) error
}

DB provides synchronized access to key-value storage. It uses the data interface and lock interface to provide a consistent storage mechanism.

func NewDB

func NewDB(opts ...Option) DB

type Option

type Option func(o *Options)

func WithData

func WithData(s data.Data) Option

WithData sets the data implementation option

func WithLeader

func WithLeader(l leader.Leader) Option

WithLeader sets the leader election implementation opton

func WithLock

func WithLock(l lock.Lock) Option

WithLock sets the locking implementation option

func WithTime

func WithTime(t time.Time) Option

WithTime sets the time implementation option

type Options

type Options struct {
	Leader leader.Leader
	Lock   lock.Lock
	Data   data.Data
	Task   task.Task
	Time   time.Time
}