repository

package
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 22, 2017 License: BSD-2-Clause Imports: 23 Imported by: 0

Documentation

Overview

Package repository implements a restic repository on top of a backend. In the following the abstractions used for this package are listed. More information can be found in the restic design document.

File

A file is a named handle for some data saved in the backend. For the local backend, this corresponds to actual files saved to disk. Usually, the SHA256 hash of the content is used for a file's name (hexadecimal, in lower-case ASCII characters). An exception is the file `config`. Most files are encrypted before being saved in a backend. This means that the name is the hash of the ciphertext.

Blob

A blob is a number of bytes that has a type (data or tree). Blobs are identified by an ID, which is the SHA256 hash of the blobs' contents. One or more blobs are bundled together in a Pack and then saved to the backend. Blobs are always encrypted before being bundled in a Pack.

Pack

A Pack is a File in the backend that contains one or more (encrypted) blobs, followed by a header at the end of the Pack. The header is encrypted and contains the ID, type, length and offset for each blob contained in the Pack.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNoKeyFound is returned when no key for the repository could be decrypted.
	ErrNoKeyFound = errors.New("wrong password or no key found")

	// ErrMaxKeysReached is returned when the maximum number of keys was checked and no key could be found.
	ErrMaxKeysReached = errors.New("maximum number of keys reached")
)
View Source 
var (
	// KDFTimeout specifies the maximum runtime for the KDF.
	KDFTimeout = 500 * time.Millisecond

	// KDFMemory limits the memory the KDF is allowed to use.
	KDFMemory = 60
)
View Source 
var ErrOldIndexFormat = errors.New("index has old format")

ErrOldIndexFormat means an index with the old format was detected.

View Source 
var IndexFull = func(idx *Index) bool {
	idx.m.Lock()
	defer idx.m.Unlock()

	debug.Log("checking whether index %p is full", idx)

	packs := len(idx.pack)
	age := time.Now().Sub(idx.created)

	if age > indexMaxAge {
		debug.Log("index %p is old enough", idx, age)
		return true
	}

	if packs < indexMinBlobs || age < indexMinAge {
		debug.Log("index %p only has %d packs or is too young (%v)", idx, packs, age)
		return false
	}

	if packs > indexMaxBlobs {
		debug.Log("index %p has %d packs", idx, packs)
		return true
	}

	debug.Log("index %p is not full", idx)
	return false
}

IndexFull returns true iff the index is "full enough" to be saved as a preliminary index.

View Source 
var KDFParams *crypto.KDFParams

KDFParams tracks the parameters used for the KDF. If not set, it will be calibrated on the first run of AddKey().

Functions

func FilesInParallel 

func FilesInParallel(ctx context.Context, repo restic.Lister, t restic.FileType, n uint, f ParallelWorkFunc) error

FilesInParallel runs n workers of f in parallel, on the IDs that repo.List(t) yield. If f returns an error, the process is aborted and the first error is returned.

func Repack added in v0.3.0 

func Repack(ctx context.Context, repo restic.Repository, packs restic.IDSet, keepBlobs restic.BlobSet, p *restic.Progress) (obsoletePacks restic.IDSet, err error)

Repack takes a list of packs together with a list of blobs contained in these packs. Each pack is loaded and the blobs listed in keepBlobs is saved into a new pack. Returned is the list of obsolete packs which can then be removed.

func SaveIndex 

func SaveIndex(ctx context.Context, repo restic.Repository, index *Index) (restic.ID, error)

SaveIndex saves an index in the repository.

func TestBackend 

func TestBackend(t testing.TB) (be restic.Backend, cleanup func())

TestBackend returns a fully configured in-memory backend.

func TestOpenLocal added in v0.3.0 

func TestOpenLocal(t testing.TB, dir string) (r restic.Repository)

TestOpenLocal opens a local repository.

func TestRepository 

func TestRepository(t testing.TB) (r restic.Repository, cleanup func())

TestRepository returns a repository initialized with a test password on an in-memory backend. When the environment variable RESTIC_TEST_REPO is set to a non-existing directory, a local backend is created there and this is used instead. The directory is not removed, but left there for inspection.

func TestRepositoryWithBackend 

func TestRepositoryWithBackend(t testing.TB, be restic.Backend) (r restic.Repository, cleanup func())

TestRepositoryWithBackend returns a repository initialized with a test password. If be is nil, an in-memory backend is used. A constant polynomial is used for the chunker and low-security test parameters.

func TestUseLowSecurityKDFParameters added in v0.3.0 

func TestUseLowSecurityKDFParameters(t logger)

TestUseLowSecurityKDFParameters configures low-security KDF parameters for testing.

Types

type Index 

type Index struct {
	// contains filtered or unexported fields
}

Index holds a lookup table for id -> pack.

func DecodeIndex 

func DecodeIndex(buf []byte) (idx *Index, err error)

DecodeIndex loads and unserializes an index from rd.

func DecodeOldIndex 

func DecodeOldIndex(buf []byte) (idx *Index, err error)

DecodeOldIndex loads and unserializes an index in the old format from rd.

func LoadIndex 

func LoadIndex(ctx context.Context, repo restic.Repository, id restic.ID) (*Index, error)

LoadIndex loads the index id from backend and returns it.

func LoadIndexWithDecoder 

func LoadIndexWithDecoder(ctx context.Context, repo restic.Repository, id restic.ID, fn func([]byte) (*Index, error)) (idx *Index, err error)

LoadIndexWithDecoder loads the index and decodes it with fn.

func NewIndex 

func NewIndex() *Index

NewIndex returns a new index.

func (*Index) AddToSupersedes 

func (idx *Index) AddToSupersedes(ids ...restic.ID) error

AddToSupersedes adds the ids to the list of indexes superseded by this index. If the index has already been finalized, an error is returned.

func (*Index) Count 

func (idx *Index) Count(t restic.BlobType) (n uint)

Count returns the number of blobs of type t in the index.

func (*Index) Dump 

func (idx *Index) Dump(w io.Writer) error

Dump writes the pretty-printed JSON representation of the index to w.

func (*Index) Each 

func (idx *Index) Each(ctx context.Context) <-chan restic.PackedBlob

Each returns a channel that yields all blobs known to the index. When the context is cancelled, the background goroutine terminates. This blocks any modification of the index.

func (*Index) Encode 

func (idx *Index) Encode(w io.Writer) error

Encode writes the JSON serialization of the index to the writer w.

func (*Index) Final 

func (idx *Index) Final() bool

Final returns true iff the index is already written to the repository, it is finalized.

func (*Index) Finalize 

func (idx *Index) Finalize(w io.Writer) error

Finalize sets the index to final and writes the JSON serialization to w.

func (*Index) Has 

func (idx *Index) Has(id restic.ID, tpe restic.BlobType) bool

Has returns true iff the id is listed in the index.

func (*Index) ID 

func (idx *Index) ID() (restic.ID, error)

ID returns the ID of the index, if available. If the index is not yet finalized, an error is returned.

func (*Index) ListPack 

func (idx *Index) ListPack(id restic.ID) (list []restic.PackedBlob)

ListPack returns a list of blobs contained in a pack.

func (*Index) Lookup 

func (idx *Index) Lookup(id restic.ID, tpe restic.BlobType) (blobs []restic.PackedBlob, err error)

Lookup queries the index for the blob ID and returns a restic.PackedBlob.

func (*Index) LookupSize 

func (idx *Index) LookupSize(id restic.ID, tpe restic.BlobType) (plaintextLength uint, err error)

LookupSize returns the length of the plaintext content of the blob with the given id.

func (*Index) Packs 

func (idx *Index) Packs() restic.IDSet

Packs returns all packs in this index

func (*Index) SetID 

func (idx *Index) SetID(id restic.ID) error

SetID sets the ID the index has been written to. This requires that Finalize() has been called before, otherwise an error is returned.

func (*Index) Store 

func (idx *Index) Store(blob restic.PackedBlob)

Store remembers the id and pack in the index. An existing entry will be silently overwritten.

func (*Index) Supersedes 

func (idx *Index) Supersedes() restic.IDs

Supersedes returns the list of indexes this index supersedes, if any.

type Key 

type Key struct {
	Created  time.Time `json:"created"`
	Username string    `json:"username"`
	Hostname string    `json:"hostname"`

	KDF  string `json:"kdf"`
	N    int    `json:"N"`
	R    int    `json:"r"`
	P    int    `json:"p"`
	Salt []byte `json:"salt"`
	Data []byte `json:"data"`
	// contains filtered or unexported fields
}

Key represents an encrypted master key for a repository.

func AddKey 

func AddKey(ctx context.Context, s *Repository, password string, template *crypto.Key) (*Key, error)

AddKey adds a new key to an already existing repository.

func LoadKey 

func LoadKey(ctx context.Context, s *Repository, name string) (k *Key, err error)

LoadKey loads a key from the backend.

func OpenKey 

func OpenKey(ctx context.Context, s *Repository, name string, password string) (*Key, error)

OpenKey tries do decrypt the key specified by name with the given password.

func SearchKey 

func SearchKey(ctx context.Context, s *Repository, password string, maxKeys int) (*Key, error)

SearchKey tries to decrypt at most maxKeys keys in the backend with the given password. If none could be found, ErrNoKeyFound is returned. When maxKeys is reached, ErrMaxKeysReached is returned. When setting maxKeys to zero, all keys in the repo are checked.

func (Key) Name 

func (k Key) Name() string

Name returns an identifier for the key.

func (*Key) String 

func (k *Key) String() string

func (*Key) Valid 

func (k *Key) Valid() bool

Valid tests whether the mac and encryption keys are valid (i.e. not zero)

type MasterIndex 

type MasterIndex struct {
	// contains filtered or unexported fields
}

MasterIndex is a collection of indexes and IDs of chunks that are in the process of being saved.

func NewMasterIndex 

func NewMasterIndex() *MasterIndex

NewMasterIndex creates a new master index.

func (*MasterIndex) All 

func (mi *MasterIndex) All() []*Index

All returns all indexes.

func (*MasterIndex) Count 

func (mi *MasterIndex) Count(t restic.BlobType) (n uint)

Count returns the number of blobs of type t in the index.

func (*MasterIndex) Each added in v0.7.0 

func (mi *MasterIndex) Each(ctx context.Context) <-chan restic.PackedBlob

Each returns a channel that yields all blobs known to the index. When the context is cancelled, the background goroutine terminates. This blocks any modification of the index.

func (*MasterIndex) FullIndexes 

func (mi *MasterIndex) FullIndexes() []*Index

FullIndexes returns all indexes that are full.

func (*MasterIndex) Has 

func (mi *MasterIndex) Has(id restic.ID, tpe restic.BlobType) bool

Has queries all known Indexes for the ID and returns the first match.

func (*MasterIndex) Insert 

func (mi *MasterIndex) Insert(idx *Index)

Insert adds a new index to the MasterIndex.

func (*MasterIndex) ListPack 

func (mi *MasterIndex) ListPack(id restic.ID) (list []restic.PackedBlob)

ListPack returns the list of blobs in a pack. The first matching index is returned, or nil if no index contains information about the pack id.

func (*MasterIndex) Lookup 

func (mi *MasterIndex) Lookup(id restic.ID, tpe restic.BlobType) (blobs []restic.PackedBlob, err error)

Lookup queries all known Indexes for the ID and returns the first match.

func (*MasterIndex) LookupSize 

func (mi *MasterIndex) LookupSize(id restic.ID, tpe restic.BlobType) (uint, error)

LookupSize queries all known Indexes for the ID and returns the first match.

func (*MasterIndex) NotFinalIndexes 

func (mi *MasterIndex) NotFinalIndexes() []*Index

NotFinalIndexes returns all indexes that have not yet been saved.

func (*MasterIndex) RebuildIndex 

func (mi *MasterIndex) RebuildIndex(packBlacklist restic.IDSet) (*Index, error)

RebuildIndex combines all known indexes to a new index, leaving out any packs whose ID is contained in packBlacklist. The new index contains the IDs of all known indexes in the "supersedes" field.

func (*MasterIndex) Remove 

func (mi *MasterIndex) Remove(index *Index)

Remove deletes an index from the MasterIndex.

func (*MasterIndex) Store added in v0.3.3 

func (mi *MasterIndex) Store(pb restic.PackedBlob)

Store remembers the id and pack in the index.

type Packer added in v0.4.0 

type Packer struct {
	*pack.Packer
	// contains filtered or unexported fields
}

Packer holds a pack.Packer together with a hash writer.

type ParallelIDWorkFunc 

type ParallelIDWorkFunc func(ctx context.Context, id restic.ID) error

ParallelIDWorkFunc gets one restic.ID to work on. If an error is returned, processing stops. When the context is cancelled the function should return.

type ParallelWorkFunc 

type ParallelWorkFunc func(ctx context.Context, id string) error

ParallelWorkFunc gets one file ID to work on. If an error is returned, processing stops. When the contect is cancelled the function should return.

func ParallelWorkFuncParseID 

func ParallelWorkFuncParseID(f ParallelIDWorkFunc) ParallelWorkFunc

ParallelWorkFuncParseID converts a function that takes a restic.ID to a function that takes a string. Filenames that do not parse as a restic.ID are ignored.

type Repository 

type Repository struct {
	// contains filtered or unexported fields
}

Repository is used to access a repository in a backend.

func New 

func New(be restic.Backend) *Repository

New returns a new repository with backend be.

func (*Repository) Backend 

func (r *Repository) Backend() restic.Backend

Backend returns the backend for the repository.

func (*Repository) Close 

func (r *Repository) Close() error

Close closes the repository by closing the backend.

func (*Repository) Config 

func (r *Repository) Config() restic.Config

Config returns the repository configuration.

func (*Repository) Delete 

func (r *Repository) Delete(ctx context.Context) error

Delete calls backend.Delete() if implemented, and returns an error otherwise.

func (*Repository) Encrypt 

func (r *Repository) Encrypt(ciphertext, plaintext []byte) ([]byte, error)

Encrypt encrypts and authenticates the plaintext and saves the result in ciphertext.

func (*Repository) Flush 

func (r *Repository) Flush() error

Flush saves all remaining packs.

func (*Repository) Index 

func (r *Repository) Index() restic.Index

Index returns the currently used MasterIndex.

func (*Repository) Init 

func (r *Repository) Init(ctx context.Context, password string) error

Init creates a new master key with the supplied password, initializes and saves the repository config.

func (*Repository) Key 

func (r *Repository) Key() *crypto.Key

Key returns the current master key.

func (*Repository) KeyName 

func (r *Repository) KeyName() string

KeyName returns the name of the current key in the backend.

func (*Repository) List 

func (r *Repository) List(ctx context.Context, t restic.FileType) <-chan restic.ID

List returns a channel that yields all IDs of type t in the backend.

func (*Repository) ListPack 

func (r *Repository) ListPack(ctx context.Context, id restic.ID) ([]restic.Blob, int64, error)

ListPack returns the list of blobs saved in the pack id and the length of the file as stored in the backend.

func (*Repository) LoadAndDecrypt 

func (r *Repository) LoadAndDecrypt(ctx context.Context, t restic.FileType, id restic.ID) ([]byte, error)

LoadAndDecrypt loads and decrypts data identified by t and id from the backend.

func (*Repository) LoadBlob 

func (r *Repository) LoadBlob(ctx context.Context, t restic.BlobType, id restic.ID, buf []byte) (int, error)

LoadBlob loads a blob of type t from the repository to the buffer. buf must be large enough to hold the encrypted blob, since it is used as scratch space.

func (*Repository) LoadIndex 

func (r *Repository) LoadIndex(ctx context.Context) error

LoadIndex loads all index files from the backend in parallel and stores them in the master index. The first error that occurred is returned.

func (*Repository) LoadJSONUnpacked 

func (r *Repository) LoadJSONUnpacked(ctx context.Context, t restic.FileType, id restic.ID, item interface{}) (err error)

LoadJSONUnpacked decrypts the data and afterwards calls json.Unmarshal on the item.

func (*Repository) LoadTree added in v0.3.0 

func (r *Repository) LoadTree(ctx context.Context, id restic.ID) (*restic.Tree, error)

LoadTree loads a tree from the repository.

func (*Repository) LookupBlobSize 

func (r *Repository) LookupBlobSize(id restic.ID, tpe restic.BlobType) (uint, error)

LookupBlobSize returns the size of blob id.

func (*Repository) PrefixLength 

func (r *Repository) PrefixLength(t restic.FileType) (int, error)

PrefixLength returns the number of bytes required so that all prefixes of all IDs of type t are unique.

func (*Repository) SaveAndEncrypt 

func (r *Repository) SaveAndEncrypt(ctx context.Context, t restic.BlobType, data []byte, id *restic.ID) (restic.ID, error)

SaveAndEncrypt encrypts data and stores it to the backend as type t. If data is small enough, it will be packed together with other small blobs.

func (*Repository) SaveBlob added in v0.3.0 

func (r *Repository) SaveBlob(ctx context.Context, t restic.BlobType, buf []byte, id restic.ID) (restic.ID, error)

SaveBlob saves a blob of type t into the repository. If id is the null id, it will be computed and returned.

func (*Repository) SaveFullIndex 

func (r *Repository) SaveFullIndex(ctx context.Context) error

SaveFullIndex saves all full indexes in the backend.

func (*Repository) SaveIndex 

func (r *Repository) SaveIndex(ctx context.Context) error

SaveIndex saves all new indexes in the backend.

func (*Repository) SaveJSONUnpacked 

func (r *Repository) SaveJSONUnpacked(ctx context.Context, t restic.FileType, item interface{}) (restic.ID, error)

SaveJSONUnpacked serialises item as JSON and encrypts and saves it in the backend as type t, without a pack. It returns the storage hash.

func (*Repository) SaveTree added in v0.3.0 

func (r *Repository) SaveTree(ctx context.Context, t *restic.Tree) (restic.ID, error)

SaveTree stores a tree into the repository and returns the ID. The ID is checked against the index. The tree is only stored when the index does not contain the ID.

func (*Repository) SaveUnpacked 

func (r *Repository) SaveUnpacked(ctx context.Context, t restic.FileType, p []byte) (id restic.ID, err error)

SaveUnpacked encrypts data and stores it in the backend. Returned is the storage hash.

func (*Repository) SearchKey 

func (r *Repository) SearchKey(ctx context.Context, password string, maxKeys int) error

SearchKey finds a key with the supplied password, afterwards the config is read and parsed. It tries at most maxKeys key files in the repo.

func (*Repository) SetIndex 

func (r *Repository) SetIndex(i restic.Index)

SetIndex instructs the repository to use the given index.

type Saver 

type Saver interface {
	Save(context.Context, restic.Handle, io.Reader) error
}

Saver implements saving data in a backend.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.