index

package
v0.16.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 4, 2024 License: BSD-2-Clause Imports: 17 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var IndexFull = func(idx *Index, compress bool) bool {
	idx.m.Lock()
	defer idx.m.Unlock()

	debug.Log("checking whether index %p is full", idx)

	var blobs uint
	for typ := range idx.byType {
		blobs += idx.byType[typ].len()
	}
	age := time.Since(idx.created)
	var maxBlobs uint
	if compress {
		maxBlobs = indexMaxBlobsCompressed
	} else {
		maxBlobs = indexMaxBlobs
	}

	switch {
	case age >= indexMaxAge:
		debug.Log("index %p is old enough", idx, age)
		return true
	case blobs >= maxBlobs:
		debug.Log("index %p has %d blobs", idx, blobs)
		return true
	}

	debug.Log("index %p only has %d blobs and is too young (%v)", idx, blobs, age)
	return false

}

IndexFull returns true iff the index is "full enough" to be saved as a preliminary index.

Functions

func ForAllIndexes added in v0.15.0 

func ForAllIndexes(ctx context.Context, lister restic.Lister, repo restic.Repository,
	fn func(id restic.ID, index *Index, oldFormat bool, err error) error) error

ForAllIndexes loads all index files in parallel and calls the given callback. It is guaranteed that the function is not run concurrently. If the callback returns an error, this function is cancelled and also returns that error.

func SaveIndex added in v0.15.0 

func SaveIndex(ctx context.Context, repo restic.SaverUnpacked, index *Index) (restic.ID, error)

SaveIndex saves an index in the repository.

Types

type EachByPackResult added in v0.15.0 

type EachByPackResult struct {
	PackID restic.ID
	Blobs  []restic.Blob
}

type Index 

type Index struct {
	// contains filtered or unexported fields
}

Index holds lookup tables for id -> pack.

func DecodeIndex added in v0.15.0 

func DecodeIndex(buf []byte, id restic.ID) (idx *Index, oldFormat bool, err error)

DecodeIndex unserializes an index from buf.

func NewIndex added in v0.15.0 

func NewIndex() *Index

NewIndex returns a new index.

func TestMergeIndex added in v0.15.0 

func TestMergeIndex(t testing.TB, mi *MasterIndex) ([]*Index, int)

func (*Index) AddToSupersedes added in v0.15.0 

func (idx *Index) AddToSupersedes(ids ...restic.ID) error

AddToSupersedes adds the ids to the list of indexes superseded by this index. If the index has already been finalized, an error is returned.

func (*Index) Dump added in v0.15.0 

func (idx *Index) Dump(w io.Writer) error

Dump writes the pretty-printed JSON representation of the index to w.

func (*Index) Each added in v0.15.0 

func (idx *Index) Each(ctx context.Context, fn func(restic.PackedBlob))

Each passes all blobs known to the index to the callback fn. This blocks any modification of the index.

func (*Index) EachByPack added in v0.15.0 

func (idx *Index) EachByPack(ctx context.Context, packBlacklist restic.IDSet) <-chan EachByPackResult

EachByPack returns a channel that yields all blobs known to the index grouped by packID but ignoring blobs with a packID in packPlacklist for finalized indexes. This filtering is used when rebuilding the index where we need to ignore packs from the finalized index which have been re-read into a non-finalized index. When the context is cancelled, the background goroutine terminates. This blocks any modification of the index.

func (*Index) Encode added in v0.15.0 

func (idx *Index) Encode(w io.Writer) error

Encode writes the JSON serialization of the index to the writer w.

func (*Index) Final added in v0.15.0 

func (idx *Index) Final() bool

Final returns true iff the index is already written to the repository, it is finalized.

func (*Index) Finalize added in v0.15.0 

func (idx *Index) Finalize()

Finalize sets the index to final.

func (*Index) Has added in v0.15.0 

func (idx *Index) Has(bh restic.BlobHandle) bool

Has returns true iff the id is listed in the index.

func (*Index) IDs added in v0.15.0 

func (idx *Index) IDs() (restic.IDs, error)

IDs returns the IDs of the index, if available. If the index is not yet finalized, an error is returned.

func (*Index) Lookup added in v0.15.0 

func (idx *Index) Lookup(bh restic.BlobHandle, pbs []restic.PackedBlob) []restic.PackedBlob

Lookup queries the index for the blob ID and returns all entries including duplicates. Adds found entries to blobs and returns the result.

func (*Index) LookupSize added in v0.15.0 

func (idx *Index) LookupSize(bh restic.BlobHandle) (plaintextLength uint, found bool)

LookupSize returns the length of the plaintext content of the blob with the given id.

func (*Index) Packs 

func (idx *Index) Packs() restic.IDSet

Packs returns all packs in this index

func (*Index) SetID added in v0.15.0 

func (idx *Index) SetID(id restic.ID) error

SetID sets the ID the index has been written to. This requires that Finalize() has been called before, otherwise an error is returned.

func (*Index) StorePack added in v0.15.0 

func (idx *Index) StorePack(id restic.ID, blobs []restic.Blob)

StorePack remembers the ids of all blobs of a given pack in the index

func (*Index) Supersedes added in v0.15.0 

func (idx *Index) Supersedes() restic.IDs

Supersedes returns the list of indexes this index supersedes, if any.

type MasterIndex added in v0.15.0 

type MasterIndex struct {
	// contains filtered or unexported fields
}

MasterIndex is a collection of indexes and IDs of chunks that are in the process of being saved.

func NewMasterIndex added in v0.15.0 

func NewMasterIndex() *MasterIndex

NewMasterIndex creates a new master index.

func (*MasterIndex) AddPending added in v0.15.0 

func (mi *MasterIndex) AddPending(bh restic.BlobHandle) bool

AddPending adds a given blob to list of pending Blobs Before doing so it checks if this blob is already known. Returns true if adding was successful and false if the blob was already known

func (*MasterIndex) Each added in v0.15.0 

func (mi *MasterIndex) Each(ctx context.Context, fn func(restic.PackedBlob))

Each runs fn on all blobs known to the index. When the context is cancelled, the index iteration return immediately. This blocks any modification of the index.

func (*MasterIndex) Has added in v0.15.0 

func (mi *MasterIndex) Has(bh restic.BlobHandle) bool

Has queries all known Indexes for the ID and returns the first match. Also returns true if the ID is pending.

func (*MasterIndex) IDs added in v0.15.0 

func (mi *MasterIndex) IDs() restic.IDSet

IDs returns the IDs of all indexes contained in the index.

func (*MasterIndex) Insert added in v0.15.0 

func (mi *MasterIndex) Insert(idx *Index)

Insert adds a new index to the MasterIndex.

func (*MasterIndex) ListPacks added in v0.15.0 

func (mi *MasterIndex) ListPacks(ctx context.Context, packs restic.IDSet) <-chan restic.PackBlobs

ListPacks returns the blobs of the specified pack files grouped by pack file.

func (*MasterIndex) Lookup added in v0.15.0 

func (mi *MasterIndex) Lookup(bh restic.BlobHandle) (pbs []restic.PackedBlob)

Lookup queries all known Indexes for the ID and returns all matches.

func (*MasterIndex) LookupSize added in v0.15.0 

func (mi *MasterIndex) LookupSize(bh restic.BlobHandle) (uint, bool)

LookupSize queries all known Indexes for the ID and returns the first match.

func (*MasterIndex) MarkCompressed added in v0.15.0 

func (mi *MasterIndex) MarkCompressed()

func (*MasterIndex) MergeFinalIndexes added in v0.15.0 

func (mi *MasterIndex) MergeFinalIndexes() error

MergeFinalIndexes merges all final indexes together. After calling, there will be only one big final index in MasterIndex containing all final index contents. Indexes that are not final are left untouched. This merging can only be called after all index files are loaded - as removing of superseded index contents is only possible for unmerged indexes.

func (*MasterIndex) Packs added in v0.15.0 

func (mi *MasterIndex) Packs(packBlacklist restic.IDSet) restic.IDSet

Packs returns all packs that are covered by the index. If packBlacklist is given, those packs are only contained in the resulting IDSet if they are contained in a non-final (newly written) index.

func (*MasterIndex) Save added in v0.15.0 

func (mi *MasterIndex) Save(ctx context.Context, repo restic.SaverUnpacked, packBlacklist restic.IDSet, extraObsolete restic.IDs, p *progress.Counter) (obsolete restic.IDSet, err error)

Save saves all known indexes to index files, leaving out any packs whose ID is contained in packBlacklist from finalized indexes. The new index contains the IDs of all known indexes in the "supersedes" field. The IDs are also returned in the IDSet obsolete. After calling this function, you should remove the obsolete index files.

func (*MasterIndex) SaveFullIndex added in v0.15.0 

func (mi *MasterIndex) SaveFullIndex(ctx context.Context, r restic.SaverUnpacked) error

SaveFullIndex saves all full indexes in the backend.

func (*MasterIndex) SaveIndex added in v0.15.0 

func (mi *MasterIndex) SaveIndex(ctx context.Context, r restic.SaverUnpacked) error

SaveIndex saves all new indexes in the backend.

func (*MasterIndex) StorePack added in v0.15.0 

func (mi *MasterIndex) StorePack(id restic.ID, blobs []restic.Blob)

StorePack remembers the id and pack in the index.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.