Documentation
¶
Index ¶
- Constants
- Variables
- func AddNames(ctx *context.T, ss interfaces.SyncServerMethods, svr rpc.Server) error
- func Close(ss interfaces.SyncServerMethods)
- func DefaultBlobFetcherFunc(ctx *context.T, blobRef wire.BlobRef, clientData interface{}) error
- func New(ctx *context.T, sv interfaces.Service, blobStEngine, blobRootDir string, ...) (*syncService, error)
- func NewBlobFetcher(ctx *context.T, maxThreads int) *blobFetcher
- func NewSyncDatabase(db interfaces.Database) *syncDatabase
- func ServerBlobFetcher(ctx *context.T, ssm interfaces.SyncServerMethods, done *sync.WaitGroup)
- func SgIdToGid(dbId, id wire.Id) interfaces.GroupId
- type BatchInfo
- type BlobFetcherFunc
- type DagNode
- type DbSyncState
- type LocalLogRec
- type SgLocalState
- type SyncData
Constants ¶
const ( NoVersion = "" NoBatchId = uint64(0) )
Variables ¶
var ( // NeighborConnectionTimeout is the time duration we wait for a connection to be // established with a peer discovered from the neighborhood. // // TODO(suharshs): Make the timeouts below more dynamic based on network. (i.e. // bt vs tcp). Currently bt connection establishment takes ~3 seconds, so // neighborhood connections get more time than cloud connections. NeighborConnectionTimeout = 5 * time.Second )
Functions ¶
func AddNames ¶
func AddNames(ctx *context.T, ss interfaces.SyncServerMethods, svr rpc.Server) error
AddNames publishes all the names for this Syncbase instance gathered from all the syncgroups it is currently participating in. This is needed when syncbased is restarted so that it can republish itself at the names being used in the syncgroups.
func Close ¶
func Close(ss interfaces.SyncServerMethods)
Close waits for spawned sync threads to shut down, and closes the local blob store handle.
func DefaultBlobFetcherFunc ¶
DefaultBlobFetcherFunc() is a BlobFetcherFunc that fetches blob blobRef in the normal way.
func New ¶
func New(ctx *context.T, sv interfaces.Service, blobStEngine, blobRootDir string, cl *vclock.VClock, publishInNh bool) (*syncService, error)
New creates a new sync module.
Concurrency: sync initializes 3 goroutines at startup: a "syncer", a "neighborhood scanner", and a "peer manager". In addition, one "watcher" thread is started for each database to watch its store for changes to its objects. The "syncer" thread is responsible for periodically contacting peers to fetch changes from them. The "neighborhood scanner" thread continuously scans the neighborhood to learn of other Syncbases and syncgroups in its neighborhood. The "peer manager" thread continuously maintains viable peers that the syncer can pick from. In addition, the sync module responds to incoming RPCs from remote sync modules and local clients.
func NewBlobFetcher ¶
NewBlobFetcher() returns a new blobFetcher that can use maxThreads fetcher threads.
func NewSyncDatabase ¶
func NewSyncDatabase(db interfaces.Database) *syncDatabase
func ServerBlobFetcher ¶
func ServerBlobFetcher(ctx *context.T, ssm interfaces.SyncServerMethods, done *sync.WaitGroup)
ServerBlobFetcher() calls serverBlobScan() repeatedly on ssm (which must contain a *syncService), with gaps specified by parameters in parameters.go before scanning them all again. It stops only if the context *ctx is cancelled. The function fetchFunc() will be used to fetch blobs, and passed the argument clientData on each call. If done!=nil, done.Done() is called just before the routine returns.
Types ¶
type BatchInfo ¶
BatchInfo holds the information on a write batch:
- The map of syncable (versioned) objects: {oid: version}
- The map of linked objects {oid: version} that were not explicitly written as part of the batch but were reaffirmed during conflict resolution along with other objects written in this batch by the app by choosing "pickLocal" or "pickRemote". NOTE: this map is non empty only for batches created during conflict resolution. Unlike the Objects map, the collection of oid:version present in this map do not point back to this batch. They point to the batches that there were originally created in.
- The total count of batch objects, including non-syncable ones.
TODO(rdaoud): add support to track the read and scan sets.
func (BatchInfo) VDLReflect ¶
type BlobFetcherFunc ¶
A BlobFetcherFunc can be passed into the public calls of this abstraction to customize how to fetch a blob.
type DagNode ¶
type DagNode struct {
Level uint64 // node distance from root
Parents []string // references to parent versions
Logrec string // reference to log record
BatchId uint64 // ID of a write batch
Deleted bool // true if the change was a delete
PermId string // ID of the permissions controlling this version
PermVers string // current version of the permissions object
}
DagNode holds the information on an object mutation in the DAG. The node information is extracted from the log records exchanged between Syncbases. They are also stored in the DAG node to improve DAG traversal for conflict resolution and pruning without having to fetch the full log record.
func (DagNode) VDLReflect ¶
type DbSyncState ¶
type DbSyncState struct {
GenVecs interfaces.Knowledge // knowledge capturing the locally-known generations of remote peers for data in Database.
SgGenVecs interfaces.Knowledge // knowledge capturing the locally-known generations of remote peers for syncgroups in Database.
IsPaused bool // tracks whether sync is paused by client.
}
DbSyncState represents the persistent sync state of a Database.
func (DbSyncState) VDLIsZero ¶
func (x DbSyncState) VDLIsZero() bool
func (DbSyncState) VDLReflect ¶
func (DbSyncState) VDLReflect(struct { Name string `vdl:"v.io/x/ref/services/syncbase/vsync.DbSyncState"` })
type LocalLogRec ¶
type LocalLogRec struct {
Metadata interfaces.LogRecMetadata
Pos uint64 // position in the Database log.
}
LocalLogRec represents the persistent local state of a log record. Metadata is synced across peers, while pos is local-only.
func (LocalLogRec) VDLIsZero ¶
func (x LocalLogRec) VDLIsZero() bool
func (LocalLogRec) VDLReflect ¶
func (LocalLogRec) VDLReflect(struct { Name string `vdl:"v.io/x/ref/services/syncbase/vsync.LocalLogRec"` })
type SgLocalState ¶
type SgLocalState struct {
// The count of local joiners to the same syncgroup.
NumLocalJoiners uint32
// The syncgroup is watched when the sync Watcher starts processing the
// syncgroup data. When a syncgroup is created or joined, an entry is
// added to the Watcher queue (log) to inform it from which point to
// start accepting store mutations, an asynchronous notification similar
// to regular store mutations. When the Watcher processes that queue
// entry, it sets this bit to true. When Syncbase restarts, the value
// of this bit allows the new sync Watcher to recreate its in-memory
// state by resuming to watch only the prefixes of syncgroups that were
// previously being watched.
Watched bool
// The syncgroup was published here by this remote peer (if non-empty
// string), typically the syncgroup creator. In this case the syncgroup
// cannot be GCed locally even if it has no local joiners.
RemotePublisher string
// The syncgroup is in pending state on a device that learns the current
// state of the syncgroup from another device but has not yet received
// through peer-to-peer sync the history of the changes (DAG and logs).
// This happens in two cases:
// 1- A joiner was accepted into a syncgroup by a syncgroup admin and
// only given the current syncgroup info synchronously and will
// receive the full history later via p2p sync.
// 2- A remote server where the syncgroup is published was told by the
// syncgroup publisher the current syncgroup info synchronously and
// will receive the full history later via p2p sync.
// The pending state is over when the device reaches or exceeds the
// knowledge level indicated in the pending genvec. While SyncPending
// is true, no local syncgroup mutations are allowed (i.e. no join or
// set-spec requests).
SyncPending bool
PendingGenVec interfaces.GenVector
}
SgLocalState holds the syncgroup local state, only relevant to this member (i.e. the local Syncbase). This is needed for crash recovery of the internal state transitions of the syncgroup.
func (SgLocalState) VDLIsZero ¶
func (x SgLocalState) VDLIsZero() bool
func (SgLocalState) VDLReflect ¶
func (SgLocalState) VDLReflect(struct { Name string `vdl:"v.io/x/ref/services/syncbase/vsync.SgLocalState"` })
type SyncData ¶
type SyncData struct {
Id uint64
}
SyncData represents the persistent state of the sync module.
func (SyncData) VDLReflect ¶
Source Files
¶
- blob.go
- clock.go
- conflict_resolution.go
- constants.go
- cr_app_resolves.go
- cr_closure.go
- cr_last_wins.go
- cr_permissions.go
- cr_policy_filter.go
- cr_syncgroup.go
- dag.go
- initiator.go
- parameters.go
- peer_manager.go
- responder.go
- server_blob_fetcher.go
- sync.go
- sync_state.go
- syncer.go
- syncgroup.go
- util.go
- vsync.vdl.go
- watcher.go