sync

package
v0.0.0-...-c5655c4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 4, 2017 License: Apache-2.0 Imports: 10 Imported by: 3

Documentation

Overview

Package sync provides utilities necessary to synchronize metadata across dataplanes.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CheckSnapConflict 

func CheckSnapConflict(snapTgt, snapCur, snapInit *snapshot.Snapshot) metastore.ResolveStatus

CheckSnapConflict ...

func CheckVSConflict 

func CheckVSConflict(vsTgt, vsCur, vsInit *volumeset.VolumeSet) metastore.ResolveStatus

CheckVSConflict checks and returns action code for sync based on the set of volume set objects Conflicts for snapshots and volumeset metadata are resolved as the following: init == target current == init target == current state action true||false true||false true tgt won't change none true false false update use current false false false conflict use tgt, report conflict false true false tgt data changed use tgt 

but not current's

for the purpose of one-way sync no action is the same as 'use tgt'

func IsStopIteration 

func IsStopIteration(err error) bool

IsStopIteration is the preferred way to detect the StopIteration error case.

func NewObjects 

func NewObjects(source metastore.Syncable, target metastore.Syncable, vsid volumeset.ID) error

NewObjects is a function that ensures that the target storage contains a volumeset with the same ID as sourceVolumeSet and that all metadata in sourceVolumeSet is also present in the target volumeset. In other words, after PushMetadata() completes without an error the target metadata should be a subset of the source metadata (or possibly exactly the same).

The synchronization is done in one direction.

The metadata on either side is not locked, so the synchronization property should hold for the source metadata that was present before PushMetadata is called. There are no guarantees for any metadata that is being added concurrently with PushMetadata.

Some open questions:

Q1. Should this function have any filters, so that only selected branches are synchronized?

Q2. At present there is only basic handling of renamed branchs: a renamed branch is pushed as if it was a new branch with history that contains all of the history of the original branch. So, after the push there will two branches on the target side, one with the old name and the other with the new one. Should we be smarter about that? Should there be a method to remove a branch? TODO: This is exported for test only, not export may be?

func PullDataForAllSnapshots 

func PullDataForAllSnapshots(source BlobSpewer, mds metastore.Client, vsid volumeset.ID,
	receiver dataplane.BlobDownloader) error

PullDataForAllSnapshots retrieves all blobs missing on target which source is willing to provide.

func PullDataForCertainSnapshots 

func PullDataForCertainSnapshots(source BlobSpewer, mds metastore.Client, receiver dataplane.BlobDownloader,
	pullSnapshots []snapshot.ID) error

PullDataForCertainSnapshots retrieves the blobs associated with any of the specified snapshots which are missing on target and which source is willing to provide.

func PullDataForQualifyingSnapshots 

func PullDataForQualifyingSnapshots(source BlobSpewer, mds metastore.Client, vsid volumeset.ID,
	receiver dataplane.BlobDownloader, shouldPull SnapshotPredicate) error

PullDataForQualifyingSnapshots downloads all blobs associated with snapshots allowed by a given predicate and which the source is willing to send.

func PushDataForAllSnapshots 

func PushDataForAllSnapshots(mds metastore.Store, vsid volumeset.ID, sender dataplane.BlobUploader, target BlobAccepter) error

PushDataForAllSnapshots sends all blobs available in source which target is willing to take.

func PushDataForCertainSnapshots 

func PushDataForCertainSnapshots(mds metastore.Store, sender dataplane.BlobUploader, target BlobAccepter, pushSnapshots []snapshot.ID) error

PushDataForCertainSnapshots sends the blobs associated any of the specified snapshots which the target is willing to take.

Note: The snapshots should be ordered from oldest to newest (wherever there is an ancestor/descendant relationship) to avoid unnecessary blob transfer.

func PushDataForQualifyingSnapshots 

func PushDataForQualifyingSnapshots(mds metastore.Store, vsid volumeset.ID, sender dataplane.BlobUploader, target BlobAccepter, shouldPush SnapshotPredicate) error

PushDataForQualifyingSnapshots sends all blobs associated with snapshots allowed by a given predicate and which the target is willing to take.

func ToSlice 

func ToSlice(i SnapshotIterator) ([]*snapshot.Snapshot, error)

ToSlice returns a slice of snapshots containing all of the snapshots produced by a SnapshotIterator, in the order they are produced.

Types

type BlobAccepter 

type BlobAccepter interface {
	// OfferBlobDiff suggests to a peer that we might have some data they
	// want.  If the peer wants it, they can return a base snapshot ID to
	// use and a token which can be used to perform the upload.  Otherwise
	// they an decline with an empty token or an error.
	OfferBlobDiff(vsid volumeset.ID, targetID snapshot.ID, baseCandidateIDs []snapshot.ID) (*snapshot.ID, string, string, error)
}

BlobAccepter is an entity which can negotiate about which blob diffs it would like to receive.

type BlobSpewer 

type BlobSpewer interface {
	RequestBlobDiff(vsid volumeset.ID, targetID snapshot.ID, baseCandidateIDs []snapshot.ID) (*snapshot.ID, string, string, error)
}

BlobSpewer is an entity which can negotiate about which blob diffs it is willing to transmit.

type Error 

type Error interface {
	error
	GetBranch() string
}

Error is an interface for synchronization errors that can be returned by the functions in this package.

type FilterIterator 

type FilterIterator struct {
	Snapshots SnapshotIterator
	Predicate SnapshotPredicate
}

FilterIterator is a SnapshotIterator which loads snapshots from another SnapshotIterator but only passes through values which satisfy a predicate.

func (*FilterIterator) Next 

func (i *FilterIterator) Next() (*snapshot.Snapshot, error)

Next loads values from the wrapped iterator until one satisfies the predicate, then returns that one.

type HistoryDivergedError 

type HistoryDivergedError struct {
	// contains filtered or unexported fields
}

HistoryDivergedError is an error reported when a branch can not be synchronized because its histories (snapshot chains) in repositories has diverged.

func (*HistoryDivergedError) Error 

func (e *HistoryDivergedError) Error() string

func (*HistoryDivergedError) GetBranch 

func (e *HistoryDivergedError) GetBranch() string

GetBranch returns a name of the failed branch.

type LazySnapshotLoader 

type LazySnapshotLoader struct {
	// contains filtered or unexported fields
}

LazySnapshotLoader is a SnapshotIterator which only loads snapshots by ID when a Next() call demands them.

Note: IDs is used as a stack and so snapshots are produced in reverse order.

func (*LazySnapshotLoader) Next 

func (i *LazySnapshotLoader) Next() (*snapshot.Snapshot, error)

Next loads the next snapshot by ID and returns it.

func (*LazySnapshotLoader) VolumeSetID 

func (i *LazySnapshotLoader) VolumeSetID() volumeset.ID

VolumeSetID ...

type MetaConflicts 

type MetaConflicts struct {
	VsC []metastore.VSMetaConflict
	SnC []metastore.SnapMetaConflict
	BrC []metastore.BranchMetaConflict
}

MetaConflicts - list of conflicts for vs, snaps, branches. Used for reporting to the user.

func Do 

func Do(
	storeTgt, storeCur, storeInit metastore.Syncable,
	vsid volumeset.ID,
	pullOnly bool,
) (MetaConflicts, error)

Do syncs the volumeset between the metadata stores. In a two way sync mode:

  1. Push new snapshots from current to target
  2. Pull new snapshots from target to current
  3. Pull new snapshots from current to initial(including locally newly created and pulled from target)
  4. Sync meta data including both existing and new among all three stores. This is done after new snapshots are synced first because during sync, target might change some of the meta fields, for example, creator, owner, etc.

In one way sync mode:

  1. Pull new snapshots from target to current
  2. Pull new snapshots from current to initial(including locally newly created and pulled from target)
  3. Sync meta data (new and old) from target to current and initial. Local changes will be overwritten with data from target when there are conflicts.

func (*MetaConflicts) HasConflicts 

func (c *MetaConflicts) HasConflicts() bool

HasConflicts returns true if there are conflicts. COnflicts if current == initial && target != current

func (*MetaConflicts) Report 

func (c *MetaConflicts) Report()

Report prints out conflicts.

type SnapshotIterator 

type SnapshotIterator interface {
	Next() (*snapshot.Snapshot, error)
}

SnapshotIterator is an interface that allows simpler iteration over some collection of snapshots.

func NewBranchSnapshotIterator 

func NewBranchSnapshotIterator(mds metastore.Store, vsid volumeset.ID, tip *snapshot.Snapshot) (SnapshotIterator, error)

NewBranchSnapshotIterator returns an iterator over all of the snapshots which are in the history of a specified branch, starting at the tip and proceeding to the root.

func NewFilterSnapshotIterator 

func NewFilterSnapshotIterator(snapshots SnapshotIterator, predicate SnapshotPredicate) SnapshotIterator

NewFilterSnapshotIterator returns an iterator over all of the snapshots which are in the history of a specified branch, starting at the tip and proceeding to the root.

func NewLazyLoadSnapshotIterator 

func NewLazyLoadSnapshotIterator(mds metastore.Store, vsid volumeset.ID, snapshotIDs []snapshot.ID) SnapshotIterator

NewLazyLoadSnapshotIterator returns an iterator over all of the snapshots which are in the history of a specified branch, starting at the tip and proceeding to the root.

func NewSnapshotIterator 

func NewSnapshotIterator(mds metastore.Store, vsid volumeset.ID) (SnapshotIterator, error)

NewSnapshotIterator returns an iterator over all of the snapshots in a volumeset, starting at the root and proceeding depth-first out to the branch tips. Snapshots which are reachable in the history of more than branch tip are only returned by the iterator once. Branches are visited in alphabetic order.

type SnapshotPredicate 

type SnapshotPredicate func(snapshot *snapshot.Snapshot) bool

SnapshotPredicate is a function which makes a true/false decision about an individual snapshot - for example, whether to push its blob or not.

type StopIteration 

type StopIteration struct {
	// contains filtered or unexported fields
}

StopIteration is the error returned by SnapshotIterator.Next when all snapshots have been seen.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.