strata

package
v0.0.0-...-ae6cc27 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 5, 2018 License: BSD-3-Clause Imports: 21 Imported by: 0

Documentation

Index

Constants

View Source 
const MockSnapshotNumFiles int = 3

MockSnapshotNumFiles is the number of files in a mock snapshot

View Source 
const MockSnapshotSize int64 = 6

MockSnapshotSize is the "total size" of the files in mock snapshot. This is just the sum of the size fields, which are made up.

Variables

This section is empty.

Functions

func GetInfoFromMetadataPath 

func GetInfoFromMetadataPath(metadataPath string) (replicaID, snapshotID, time string, err error)

GetInfoFromMetadataPath returns replica ID, snapshot ID, and snapshot time based on metadataPath

func HelpTestStorage 

func HelpTestStorage(t *testing.T, s Storage)

HelpTestStorage runs basic tests on the given Storage

func HelpTestStorageManyFiles 

func HelpTestStorageManyFiles(t *testing.T, s Storage)

HelpTestStorageManyFiles uses the given Storage to Put and List many files

func IsErrNoSnapshotMetadata 

func IsErrNoSnapshotMetadata(e error) bool

IsErrNoSnapshotMetadata returns true if the error is an ErrNoSnapshotMetadata

func IsErrNotFound 

func IsErrNotFound(e error) bool

IsErrNotFound returns true if the error is an ErrNotFound

func Log 

func Log(msg string)

Log prints logging info

func PartialChecksum 

func PartialChecksum(reader ReaderReaderAt, filesize int64) ([]byte, error)

PartialChecksum computes a checksum based on three 8KB chunks from the beginning, middle, and end of the file

func RunCLI 

func RunCLI(factory DriverFactory)

RunCLI parses command-line arguments and runs the corresponding command

func ToGB 

func ToGB(bytes int64) float64

ToGB converts bytes to Gibibytes

func ToMB 

func ToMB(bytes int64) float64

ToMB converts bytes to mebibytes

func Try 

func Try(f func() error, header string) error

Try provides strata's retry strategy. It calls the given function up to five times or until the given function returns nil, whichever comes first.

Types

type BackupCommand 

type BackupCommand struct {
	ReplicaID string `short:"r" long:"replica-id" description:"An arbitrary string representing this node in the metadata. Defaults to hostname."`
	// contains filtered or unexported fields
}

BackupCommand defines "backup"

func (*BackupCommand) Execute 

func (c *BackupCommand) Execute(args []string) error

Execute validates input to the backup command, then calls backup to implement "backup

type ByTime 

type ByTime []LazySnapshotMetadata

ByTime is used to sort a slice of LazySnapshotMetadata by time. Implements sort.Interface.

func (ByTime) Len 

func (lz ByTime) Len() int

Len is the number of elements in the collection

func (ByTime) Less 

func (lz ByTime) Less(i, j int) bool

Less reports whether the element with index i should sort before the element with index j

func (ByTime) Swap 

func (lz ByTime) Swap(i, j int)

Swap swaps the elements with indexes i and j

type ChecksummingReader 

type ChecksummingReader struct {
	// contains filtered or unexported fields
}

ChecksummingReader computes an MD5 sum as it reads. If expected is not nil, then the calculated MD5 sum is compared to expected when EOF is read, and an error is returned if the sums do not match. ChecksummingReader is used to guard against network corruption. The checksum calculated happens to be compatible with the checksum that Amazon S3 uses in Content-MD5 header (see http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html)

func NewChecksummingReader 

func NewChecksummingReader(reader io.Reader, expected []byte) *ChecksummingReader

NewChecksummingReader constructs a ChecksummingReader

func (*ChecksummingReader) Close 

func (cr *ChecksummingReader) Close() error

Close calls Close() on the underlying Reader, if the underling Reader is a ReadCloser

func (ChecksummingReader) Read 

func (cr ChecksummingReader) Read(buf []byte) (n int, err error)

Read calls the Read() function of the underlying Reader and computes an MD5 sum on the result. If EOF is read and expected is not nil, then the calculated MD5 sum is compared to expected, and an error is returned if the sums do not match.

func (*ChecksummingReader) Sum 

func (cr *ChecksummingReader) Sum() []byte

Sum returns the MD5 sum computed by the ChecksummingReader so far

type DeleteCommand 

type DeleteCommand struct {
	ReplicaID string `short:"r" long:"replica-id" description:"An arbitrary string representing this node in the metadata" required:"true"`
	BackupID  string `short:"d" long:"delete-backup-id" description:"Optional. If set, delete this backup id." optional:"true" default:""`
	Age       string `` /* 182-byte string literal not displayed */
	// contains filtered or unexported fields
}

DeleteCommand defines "delete"

func (*DeleteCommand) Execute 

func (c *DeleteCommand) Execute(args []string) error

Execute validates input to "delete" and calls the appropriate delete operation

type Driver 

type Driver struct {
	Manager *SnapshotManager
}

Driver holds a SnapshotManager and provides functions that correspond closely to command-line arguments

func (*Driver) Backup 

func (driver *Driver) Backup(replicaID string) error

Backup triggers a backup

func (*Driver) CollectGarbage 

func (driver *Driver) CollectGarbage(replicaID string) error

CollectGarbage calls SnapshotManager.CollectGarbage

func (*Driver) DeleteBackup 

func (driver *Driver) DeleteBackup(replicaID, backupID string) error

DeleteBackup calls SnapshtoManager.DeleteForReplicaByID

func (*Driver) DeleteOlderThan 

func (driver *Driver) DeleteOlderThan(replicaID, age string) error

DeleteOlderThan calls SnapshotManager.DeleteEarlierThan

func (*Driver) LastBackupTime 

func (driver *Driver) LastBackupTime(replicaID string) error

LastBackupTime prints the time of the last backup, in seconds since Unix epoch, to stdout. Nothing else is printed to stdout.

func (*Driver) ListBackups 

func (driver *Driver) ListBackups(replicaID string, showBackupSize bool) error

ListBackups prints a list of backups for a given replicaID If showBackupSize is true, then attempt to calculate backup size even if a stats file is missing.

func (*Driver) ListReplicaIDs 

func (driver *Driver) ListReplicaIDs() error

ListReplicaIDs prints the replica IDs of backups that are on remote storage

func (*Driver) PrintLazyMetadatas 

func (driver *Driver) PrintLazyMetadatas(lazyMetadatas []LazySnapshotMetadata, showBackupSize bool) error

PrintLazyMetadatas prints backup information given a list of LazySnapshotMetadata

func (*Driver) PrintMetadataPaths 

func (driver *Driver) PrintMetadataPaths(mdpaths []string) error

PrintMetadataPaths prints backup information given a list of metadata paths

func (*Driver) Restore 

func (driver *Driver) Restore(replicaID, backupID, targetPath string) error

Restore calls SnapshotManager.RestoreSnapshot

func (*Driver) RestoreReadOnly 

func (driver *Driver) RestoreReadOnly(replicaID, backupID, mountPath, targetPath string) error

RestoreReadOnly calls SnapshotManager.RestoreReadOnly

type DriverFactory 

type DriverFactory interface {
	// GetOptions should return a pointer suitable for use as the first
	// argument of flags.NewParser()
	GetOptions() interface{}
	// Driver returns an initialized Driver
	Driver() (*Driver, error)
}

DriverFactory holds options relevant to SnapshotManagers/Drivers, and provides a function to construct Drivers from these options.

type ErrNoSnapshotMetadata 

type ErrNoSnapshotMetadata string

ErrNoSnapshotMetadata may be returned if there is no snapshot metadata found for a replica ID. It is intended for use with drivers. Suggestion: The error string should be the replica ID with an optional message appended

func (ErrNoSnapshotMetadata) Error 

func (e ErrNoSnapshotMetadata) Error() string

Error returns the error string for ErrNoSnapshotMetadata

type ErrNotFound 

type ErrNotFound string

ErrNotFound should be returned by Replica or Storage if a given key is not found, or by Storage if no object exists at the given path This is to help differentiate unknown keys from other errors

func (ErrNotFound) Error 

func (e ErrNotFound) Error() string

Error returns the error string for ErrNotFound

type File 

type File struct {
	// The original name of the file
	Name     string `json:"name"`
	Size     int64  `json:"size"`
	Checksum string `json:"checksum"` // This is a partial checksum
}

File contains snapshot file attributes All of these attributes can be determined without looking through the whole file

type FileMetadata 

type FileMetadata struct {
	CompleteChecksum []byte `json:"completechecksum"`
}

FileMetadata stores the complete checksum for a database file. Unlike File, which are persisted as part of SnapshotMetadata, each FileMetadata are persisted individually. Additional attributes may be added in the future.

type GCCommand 

type GCCommand struct {
	ReplicaID string `short:"r" long:"replica-id" description:"An arbitrary string representing this node in the metadata" required:"true"`
	// contains filtered or unexported fields
}

GCCommand defines "gc"

func (*GCCommand) Execute 

func (c *GCCommand) Execute(args []string) error

Execute calls CollectGarbage to implement "gc"

type GCStats 

type GCStats struct {
	// NumNeeded and NumGarbage include only DB files
	NumNeeded  int
	NumGarbage int
	// FM means "file metadata"
	NumFMNeeded     int
	NumFMGarbage    int
	NumStatsNeeded  int
	NumStatsGarbage int
	NumErrsDeleting int
	// Sizes exclude file metadata and stats files, since those do not include
	// their sizes in their names
	SizeNeeded            int64
	SizeGarbage           int64
	SizeGarbageNotDeleted int64
	Duration              time.Duration
}

GCStats holds statistics about a call to CollectGarbage(). SizeGarbage and SizeGarbageNotDeleted are technically lower bounds because it is possible that sizes cannot be extracted from garbage file names. However, these values should usually be exact. Sizes exclude file metadata and stats files, since those do not store their

type LazySnapshotMetadata 

type LazySnapshotMetadata struct {
	// The path of the persistent-storage file for the SnapshotMetadata
	// Snapshot ID and time should be embedded in path name
	MetadataPath string

	// Indicate that the SnapshotMetadata should be removed from persistent storage
	DeleteMark bool

	// Indicate that the SnapshotMetadata should be saved to persistent storage
	// Overridden by DeleteMark
	SaveMark bool
	// contains filtered or unexported fields
}

LazySnapshotMetadata is used to lazily handle access to a SnapshotMetadata, touching persistent storage only when necessary. A LazySnapshotMetadata is in a usable state if MetadataPath is set and readerFunc or snapshotMetadata is set.

func NewLazySMFromM 

func NewLazySMFromM(s *SnapshotMetadata) *LazySnapshotMetadata

NewLazySMFromM constructs a LazySnapshotMetadata from a SnapshotMetadata

func NewLazySMFromPath 

func NewLazySMFromPath(metadataPath string, readerFunc ReaderFunc) *LazySnapshotMetadata

NewLazySMFromPath constructs a LazySnapshotMetadata from a metadata path and a ReaderFunc. The ReaderFunc should return a Reader for the metadata path.

func (*LazySnapshotMetadata) Get 

func (z *LazySnapshotMetadata) Get() (*SnapshotMetadata, error)

Get returns SnapshotMetadata, loading it from persistent storage if necessary

func (*LazySnapshotMetadata) Reset 

func (z *LazySnapshotMetadata) Reset()

Reset sets snapshotMetadata and err to nil. It does not change DeleteMark or SaveMark.

type MetadataStore 

type MetadataStore struct {
	// contains filtered or unexported fields
}

MetadataStore uses a map to store one slice of LazySnapshotMetadatas for each replica ID

func NewMetadataStore 

func NewMetadataStore() MetadataStore

NewMetadataStore constructs an empty MetadataStore

func (*MetadataStore) Add 

func (m *MetadataStore) Add(lazy *LazySnapshotMetadata) error

Add inserts a LazySnapshotMetadata into the appropriate slice of MetadataStore

func (*MetadataStore) Delete 

func (m *MetadataStore) Delete(metadata SnapshotMetadata) error

Delete marks metadata to be deleted from persistent storage the next time that SaveMetadataForReplica() is called.

func (*MetadataStore) DeleteEarlierThan 

func (m *MetadataStore) DeleteEarlierThan(replicaID string, t time.Time) ([]string, error)

DeleteEarlierThan finds snapshot metadata from earlier than the specified time and marks them to be deleted from persistent storage the next time that SaveMetadataForReplica() is called. The first return value is a slice of the metadata paths that were marked for deletion. Currently, this return value is only used as a statistic.

func (*MetadataStore) DeleteForReplicaByID 

func (m *MetadataStore) DeleteForReplicaByID(replicaID string, id string) error

DeleteForReplicaByID finds the metadata that corresponds to replicaID and id, and marks the metadata to be deleted from persistent storage the next time that SaveMetadataForReplica() is called.

func (*MetadataStore) GetReplicaIDs 

func (m *MetadataStore) GetReplicaIDs() []string

GetReplicaIDs returns all replica IDs in MetadataStore

type MockReplica 

type MockReplica struct {
	// contains filtered or unexported fields
}

MockReplica is a mocked implementation of ReplicaManager for use in tests

func NewMockReplica 

func NewMockReplica() *MockReplica

NewMockReplica returns a mocked implementation of ReplicaManager

func (*MockReplica) CreateSnapshot 

func (m *MockReplica) CreateSnapshot(replicaID, snapshotID string) (*Snapshot, error)

CreateSnapshot generates a MockSnapshot with a static set of files

func (*MockReplica) Delete 

func (*MockReplica) Delete(string) error

Delete does nothing

func (*MockReplica) DeleteSnapshot 

func (m *MockReplica) DeleteSnapshot(lz LazySnapshotMetadata) error

DeleteSnapshot removes the specified snapshot from the local metadata

func (*MockReplica) GetSnapshot 

func (m *MockReplica) GetSnapshot(s SnapshotMetadata) (*Snapshot, error)

GetSnapshot returns a Snapshot given the specified metadata

func (*MockReplica) List 

func (*MockReplica) List(string, int) ([]string, error)

List does nothing

func (*MockReplica) MaxBackgroundCopies 

func (m *MockReplica) MaxBackgroundCopies() int

MaxBackgroundCopies returns 24

func (*MockReplica) PrepareToRestoreSnapshot 

func (*MockReplica) PrepareToRestoreSnapshot(string, string, *Snapshot) (string, error)

PrepareToRestoreSnapshot does nothing

func (*MockReplica) PutReader 

func (m *MockReplica) PutReader(path string, reader io.Reader) error

PutReader consumes reader and does nothing else 

func (m *MockReplica) PutSoftlink(string, string) error

PutSoftlink does nothing

func (*MockReplica) Same 

func (*MockReplica) Same(string, File) (bool, error)

Same returns false, nil

type MockStorage 

type MockStorage struct {
	// contains filtered or unexported fields
}

MockStorage implements PersistenceManager for mocking

func NewMockStorage 

func NewMockStorage(saveDelay time.Duration) *MockStorage

NewMockStorage instantiates a MockPersistenceManager

func (*MockStorage) Corrupt 

func (m *MockStorage) Corrupt(key string) error

Corrupt corrupts the data stored at |key|

func (*MockStorage) Delete 

func (m *MockStorage) Delete(key string) error

Delete an item from the mock store

func (*MockStorage) Get 

func (m *MockStorage) Get(key string) (io.ReadCloser, error)

Get returns the data stored at key

func (*MockStorage) List 

func (m *MockStorage) List(prefix string, maxSize int) ([]string, error)

List returns items matching the given prefix from the mocked store, up to maxSize items

func (*MockStorage) Lock 

func (m *MockStorage) Lock(key string) error

Lock is not implemented

func (*MockStorage) Put 

func (m *MockStorage) Put(key string, data []byte) error

Put places the byte slice in the mocked store

func (*MockStorage) PutReader 

func (m *MockStorage) PutReader(key string, reader io.Reader) error

PutReader consumes the reader and places it in the mocked store

func (*MockStorage) Unlock 

func (m *MockStorage) Unlock(key string) error

Unlock is not implemented

type ReaderFunc 

type ReaderFunc func() (io.ReadCloser, error)

ReaderFunc are intended to retrieve Readers that can consume snapshot files or metadata files

type ReaderReaderAt 

type ReaderReaderAt interface {
	io.Reader
	io.ReaderAt
}

ReaderReaderAt has the methods of an io.Reader and an io.ReaderAt

type Replica 

type Replica interface {
	// CreateSnapshot is called to trigger Snapshot creation on the target source
	CreateSnapshot(replicaID, snapshotID string) (*Snapshot, error)

	// DeleteSnapshot is used to clean up the snapshot files on the target host
	DeleteSnapshot(lz LazySnapshotMetadata) error

	// PrepareToRestoreSnapshot() is called by a SnapshotManager when the
	// SnapshotManager is about to restore a snapshot on the replica.
	// If targetPath is the empty string, then the database is restored to the directory in Snapshot s.
	// The first return value is the path/prefix on replica for SST files. For example, /data/mongodb/db.
	// Note that this is slightly different from MongoDB's --dbpath argument, which would be /data/mongodb.
	PrepareToRestoreSnapshot(replicaID, targetPath string, s *Snapshot) (dbpath string, err error)

	// PutReader lays down a file.
	PutReader(path string, reader io.Reader) error

	// PutSoftlink creates a softlink
	PutSoftlink(pointingTo, path string) error

	// Delete the contents stored at the specified string
	Delete(string) error

	// List returns up to |n| files that have path/prefix |prefix| on replica
	// This list does not include any directories
	List(prefix string, n int) ([]string, error)

	// Same returns true if the replica object at |path| has the same fingerprint as |file|.
	// This fingerprint might be a partial checksum. It should be computed quickly.
	Same(path string, file File) (bool, error)

	// GetSnapshot returns a Snapshot given the specified SnapshotMetadata
	GetSnapshot(s SnapshotMetadata) (*Snapshot, error)

	// MaxBackgroundCopies returns the maximum number of copies that CreateSnapshot
	// and RestoreSnapshot may perform in parallel
	MaxBackgroundCopies() int
}

Replica defines the interface for creating, deleting, and restoring snapshots on mongo replicas. Ugly details of triggering snapshots and retrieving them from disk should be handled here

type RestoreCommand 

type RestoreCommand struct {
	ReplicaID  string `short:"r" long:"replica-id" description:"An arbitrary string representing this node in the metadata" required:"true"`
	BackupID   string `short:"i" long:"backup-id" description:"The backup id to restore from" required:"true"`
	TargetPath string `short:"t" long:"target-path" description:"If not specified, restore to the path in snapshot metadata."`
	// contains filtered or unexported fields
}

RestoreCommand defines "restore"

func (*RestoreCommand) Execute 

func (c *RestoreCommand) Execute(args []string) error

Execute calls restore to implement "restore"

type ShowBackupsCommand 

type ShowBackupsCommand struct {
	ReplicaID string `short:"r" long:"replica-id" description:"The replica ID to show backups for" required:"true"`
	ShowSize  bool   `short:"s" long:"show-size" description:"Print backup size with backup ids"`
	// contains filtered or unexported fields
}

ShowBackupsCommand defines "show backups"

func (*ShowBackupsCommand) Execute 

func (c *ShowBackupsCommand) Execute(args []string) error

Execute calls ListBackups to implement "show backups"

type ShowCommand 

type ShowCommand struct{}

ShowCommand defines "show", which is split into subcommands

type ShowLastBackupTimeCommand 

type ShowLastBackupTimeCommand struct {
	ReplicaID string `short:"r" long:"replica-id" description:"The replica ID to show backups for" required:"true"`
	// contains filtered or unexported fields
}

ShowLastBackupTimeCommand defines "show last-backup-time"

func (*ShowLastBackupTimeCommand) Execute 

func (c *ShowLastBackupTimeCommand) Execute(args []string) error

Execute calls LastBackupTime to implement "show last-backup-time"

type ShowReplicaIDsCommand 

type ShowReplicaIDsCommand struct {
	// contains filtered or unexported fields
}

ShowReplicaIDsCommand defines "show replica-ids"

func (*ShowReplicaIDsCommand) Execute 

func (c *ShowReplicaIDsCommand) Execute(args []string) error

Execute calls ListReplicaIDs to implement "show replica-ids"

type Snapshot 

type Snapshot struct {
	Metadata *SnapshotMetadata
	// contains filtered or unexported fields
}

Snapshot represents a readable snapshot. It is the combination of snapshot data and a number of ReaderFuncs which are used to retrieve Readers that can consume the actual snapshot files

func NewSnapshot 

func NewSnapshot(metadata *SnapshotMetadata) *Snapshot

NewSnapshot creates a Snapshot from the given metadata.

func (*Snapshot) AddReaderFunc 

func (s *Snapshot) AddReaderFunc(name string, readerFunc func() (io.ReadCloser, error))

AddReaderFunc is used to tell the Snapshot how to retrieve a Reader for a given file.

func (*Snapshot) GetReader 

func (s *Snapshot) GetReader(name string) (io.ReadCloser, error)

GetReader calls the ReaderFunc for the given file name, if it exists, and returns the resulting Reader and error information. GetReader uses strata's retry strategy.

type SnapshotManager 

type SnapshotManager struct {
	// contains filtered or unexported fields
}

SnapshotManager orchestrates snapshots by integrating the Replica and Storage interfaces

func NewSnapshotManager 

func NewSnapshotManager(r Replica, p Storage) (*SnapshotManager, error)

NewSnapshotManager builds a new SnapshotManager with the specified Replica and Storage implementations

func (*SnapshotManager) CollectGarbage 

func (s *SnapshotManager) CollectGarbage(replicaID string) (*GCStats, error)

CollectGarbage examines the persisted files related to the specified replica and deletes files that are not needed by current backups. This is mostly intended to remove database files that were once needed, but are no longer needed by existing backups. CollectGarbage also has the following important property: It examines all the paths where SnapshotManager might write objects associated with replicaID, and it deletes all .tmp files that it finds.

func (*SnapshotManager) CreateSnapshot 

func (s *SnapshotManager) CreateSnapshot(replicaID string) (*SnapshotStats, error)

CreateSnapshot generates a snapshot on the host specified by replicaID and persists the snapshot. The snapshot metadata will be persisted when SaveMetadataForReplica() is called.

func (*SnapshotManager) DeleteEarlierThan 

func (s *SnapshotManager) DeleteEarlierThan(replicaID string, t time.Time) ([]string, error)

DeleteEarlierThan finds snapshot metadata from earlier than the specified time and marks them to be deleted from persistent storage the next time that SaveMetadataForReplica() is called. The first return value is a slice of the metadata paths that were marked for deletion. Currently, this return value is only used as a statistic.

func (*SnapshotManager) DeleteForReplicaByID 

func (s *SnapshotManager) DeleteForReplicaByID(replicaID string, id string) error

DeleteForReplicaByID finds the metadata that corresponds to replicaID and id, and marks the metadata to be deleted from persistent storage the next time that SaveMetadataForReplica() is called.

func (*SnapshotManager) DeleteSnapshot 

func (s *SnapshotManager) DeleteSnapshot(metadata SnapshotMetadata) error

DeleteSnapshot marks metadata to be deleted from persistent storage the next time that SaveMetadataForReplica() is called.

func (*SnapshotManager) GetBackupStats 

func (s *SnapshotManager) GetBackupStats(lazy *LazySnapshotMetadata, force bool) (*SnapshotStats, error)

GetBackupStats returns a SnapshotStats struct for the backup described by |lazy|. It tries to do this by looking up a persisted stats file. If it can't find a stats file and |force| is false, then stats fields are set to -1. If it can't find a stats file and |force| is true, then SnapshotStats.NumFiles and SnapshotStats.SizeFiles are computed from lazy's metadata, and the other fields are set to -1. Even if GetSnapshotStats returns an error, all the fields of the returned stats should be populated.

func (*SnapshotManager) GetLazyMetadata 

func (s *SnapshotManager) GetLazyMetadata(replicaID string) ([]LazySnapshotMetadata, error)

GetLazyMetadata returns all LazySnapshotMetadata for the given replicaID

func (*SnapshotManager) GetReplicaIDs 

func (s *SnapshotManager) GetReplicaIDs() []string

GetReplicaIDs returns all known replica ids in the metadata store

func (*SnapshotManager) GetSnapshot 

func (s *SnapshotManager) GetSnapshot(metadata SnapshotMetadata) (*Snapshot, error)

GetSnapshot builds a Snapshot struct from the given metadata, which can be used to access the actual snapshot data

func (*SnapshotManager) GetSnapshotMetadata 

func (s *SnapshotManager) GetSnapshotMetadata(replicaID, snapshotID string) (*SnapshotMetadata, error)

GetSnapshotMetadata returns the SnapshotMetadata specified by replicaID and snapshotID

func (*SnapshotManager) RefreshMetadata 

func (s *SnapshotManager) RefreshMetadata() error

RefreshMetadata instructs the SnapshotManager to retrieve the snapshot metadata from persisted storage

func (*SnapshotManager) RestoreReadOnly 

func (s *SnapshotManager) RestoreReadOnly(replicaID, mountPath, targetPath string, metadata SnapshotMetadata) (*SnapshotStats, error)

RestoreReadOnly aims to perform a fast restore upon which the user can perform read-only queries. mountPath should be a mountpoint on replicaID that provides read-only access to storage. For example, remote storage might be Amazon S3 and mountPath might be created with `yas3fs s3:://bucket-name/bucket-prefix mountPath --mkdir --no-metadata --read-only`. See https://github.com/danilop/yas3fs. Allowing write access to mountPath is unsafe.

func (*SnapshotManager) RestoreSnapshot 

func (s *SnapshotManager) RestoreSnapshot(replicaID string, targetPath string, metadata SnapshotMetadata) (*SnapshotStats, error)

RestoreSnapshot retrieves a Snapshot specified by the given metadata and triggers a restore. The Replica implementation determines the physical machine on which to write the restored files. The Replica implementation may or may not consider |replicaID| when making this determination. The restored files go to the database path specified by |targetPath|.

func (*SnapshotManager) SaveMetadataForReplica 

func (s *SnapshotManager) SaveMetadataForReplica(replicaID string) error

SaveMetadataForReplica instructs the SnapshotManager to persist a single replica ID's metadata to storage. After SaveMetadataForReplica is called, the SnapshotManager is in an undefined state until RefreshMetadata is called. If metadata is marked for deletion, then then SaveMetadataForReplica also calls replica.DeleteSnapshot()

type SnapshotMetadata 

type SnapshotMetadata struct {
	Files []File `json:"files"`
	// ReplicaID should uniquely identify the replica in the environment
	// An example would be combination of replica set and hostname
	ReplicaID string `json:"source_id"`
	// ID should uniquely identify this snapshot within the context
	// of a ReplicaID. It can be as simple as an incrementing counter
	ID string `json:"id"`
	// Path is where the snapshot originated from on disk
	Path string    `json:"path"`
	Tags []string  `json:"tags"`
	Time time.Time `json:"time"`
}

SnapshotMetadata stores all information related to the snapshot.

type SnapshotStats 

type SnapshotStats struct {
	NumFiles             int           `json:"num_files"`
	NumIncrementalFiles  int           `json:"num_ifiles"`
	SizeFiles            int64         `json:"size_files"`
	SizeIncrementalFiles int64         `json:"size_ifiles"`
	Duration             time.Duration `json:"duration"`
}

SnapshotStats are for CreateSnapshot() and RestoreSnapshot() operations

func (*SnapshotStats) Throughput 

func (s *SnapshotStats) Throughput() float64

Throughput is in MB/s

type Storage 

type Storage interface {
	// Get returns a Reader at the specified string
	Get(string) (reader io.ReadCloser, err error)

	// Put stores the bytes at the specified location.
	// Implementations should see comments on PutReader.
	Put(path string, data []byte) error

	// PutReader stores the contents of Reader at the specified location
	// Put and PutReader should not leave any incomplete files in the event of
	// a failure, unless those files end in ".tmp". Garbage collection will
	// eventually remove .tmp files.
	// Suggestion: If the storage system does not support atomic Puts, consider
	// implementing Put and PutReader by first writing to a .tmp file, and then
	// atomically renaming the file to remove the .tmp suffix.
	PutReader(path string, reader io.Reader) error

	// Delete the contents stored at the specified string.
	// Should return nil if there is no object at the specified string.
	Delete(string) error

	// List up to n items satisfying the specified prefix
	List(prefix string, n int) ([]string, error)

	// Lock is optional. If implemented, provide exclusive access to the specified item
	// Lock is not currently used by SnapshotManager anyway
	Lock(string) error

	// Unlock
	Unlock(string) error
}

Storage defines a generic interface for persisting snapshots and snapshot metadata

Source Files

View all Source files

Directories

Path Synopsis
cmd
mongo/lreplica_drivers/lrazureblobdriver
mongo/lreplica_drivers/lrldriver
mongo/lreplica_drivers/lrminiodriver
mongo/lreplica_drivers/lrs3driver
mongo/lreplica_drivers/mongoq
mongo/lreplica_drivers/strata
lreplica
tests
lrs3test

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.