par1

package
v0.0.0-...-a776223 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 24, 2021 License: BSD-3-Clause Imports: 13 Imported by: 0

Documentation

Index

Constants

View Source 
const NumParityFilesDefault = 3

NumParityFilesDefault is the default value used for CreateOptions.NumParityFiles if the latter is <= 0.

Variables

This section is empty.

Functions

func Create 

func Create(parPath string, filePaths []string, options CreateOptions) error

Create a par file for the given file paths at parPath with the given options.

func RepairErrorMeansRepairNecessaryButNotPossible 

func RepairErrorMeansRepairNecessaryButNotPossible(err error) bool

RepairErrorMeansRepairNecessaryButNotPossible returns true if the error returned by Repair means that repair is necessary but not possible.

Types

type CreateDelegate 

type CreateDelegate interface {
	EncoderDelegate
	OnFilesNotAllInSameDir()
}

CreateDelegate extends EncoderDelegate with another delegate function.

type CreateOptions 

type CreateOptions struct {
	// The number of parity files to create. If <= 0,
	// NumParityFilesDefault is used.
	NumParityFiles int
	// The CreateDelegate to use. If nil, DoNothingCreateDelegate
	// is used.
	CreateDelegate CreateDelegate
}

CreateOptions holds all the options for Create.

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

A Decoder keeps track of all information needed to check the integrity of a set of data files, and possibly repair any missing/corrupt data files from the parity files (.P00, .P01, etc.).

func NewDecoder 

func NewDecoder(delegate DecoderDelegate, indexFile string) (*Decoder, error)

NewDecoder reads the given index file, which usually has a .PAR extension.

func (*Decoder) FileCounts 

func (d *Decoder) FileCounts() FileCounts

FileCounts returns a FileCounts object for the current file set.

func (*Decoder) LoadFileData 

func (d *Decoder) LoadFileData() error

LoadFileData loads existing file data into memory.

func (*Decoder) LoadParityData 

func (d *Decoder) LoadParityData() error

LoadParityData searches for parity volumes and loads them into memory.

func (*Decoder) Repair 

func (d *Decoder) Repair(checkParity bool) ([]string, error)

Repair tries to repair any missing or corrupt data, using the parity volumes. Returns a list of paths to files that were successfully repaired (relative to the indexFile passed to NewDecoder) in no particular order, which is present even if an error is returned. If checkParity is true, extra checking is done of the reconstructed parity data.

func (*Decoder) VerifyAllData 

func (d *Decoder) VerifyAllData() (ok bool, err error)

VerifyAllData checks and returns whether all data and parity files contain correct data. Note that this is worth calling only when there are no unusable data or parity files, since if there are, then false is guaranteed to be returned for ok.

type DecoderDelegate 

type DecoderDelegate interface {
	OnHeaderLoad(headerInfo string)
	OnFileEntryLoad(i, n int, filename, entryInfo string)
	OnCommentLoad(comment []byte)
	OnDataFileLoad(i, n int, path string, byteCount int, corrupt bool, err error)
	OnDataFileWrite(i, n int, path string, byteCount int, err error)
	OnVolumeFileLoad(i uint64, path string, storedSetHash, computedSetHash [16]byte, dataByteCount int, err error)
}

DecoderDelegate holds methods that are called during the decode process.

type DoNothingCreateDelegate 

type DoNothingCreateDelegate struct{}

DoNothingCreateDelegate is an implementation of CreateDelegate that does nothing for all methods.

func (DoNothingCreateDelegate) OnDataFileLoad 

func (DoNothingCreateDelegate) OnDataFileLoad(i, n int, path string, byteCount int, err error)

OnDataFileLoad implements the CreateDelegate interface.

func (DoNothingCreateDelegate) OnFilesNotAllInSameDir 

func (DoNothingCreateDelegate) OnFilesNotAllInSameDir()

OnFilesNotAllInSameDir implements the CreateDelegate interface.

func (DoNothingCreateDelegate) OnVolumeFileWrite 

func (DoNothingCreateDelegate) OnVolumeFileWrite(i, n int, path string, dataByteCount, byteCount int, err error)

OnVolumeFileWrite implements the CreateDelegate interface.

type DoNothingDecoderDelegate 

type DoNothingDecoderDelegate struct{}

DoNothingDecoderDelegate is an implementation of DecoderDelegate that does nothing for all methods.

func (DoNothingDecoderDelegate) OnCommentLoad 

func (DoNothingDecoderDelegate) OnCommentLoad(comment []byte)

OnCommentLoad implements the DecoderDelegate interface.

func (DoNothingDecoderDelegate) OnDataFileLoad 

func (DoNothingDecoderDelegate) OnDataFileLoad(i, n int, path string, byteCount int, corrupt bool, err error)

OnDataFileLoad implements the DecoderDelegate interface.

func (DoNothingDecoderDelegate) OnDataFileWrite 

func (DoNothingDecoderDelegate) OnDataFileWrite(i, n int, path string, byteCount int, err error)

OnDataFileWrite implements the DecoderDelegate interface.

func (DoNothingDecoderDelegate) OnFileEntryLoad 

func (DoNothingDecoderDelegate) OnFileEntryLoad(i, n int, filename, entryInfo string)

OnFileEntryLoad implements the DecoderDelegate interface.

func (DoNothingDecoderDelegate) OnHeaderLoad 

func (DoNothingDecoderDelegate) OnHeaderLoad(headerInfo string)

OnHeaderLoad implements the DecoderDelegate interface.

func (DoNothingDecoderDelegate) OnVolumeFileLoad 

func (DoNothingDecoderDelegate) OnVolumeFileLoad(i uint64, path string, storedSetHash, computedSetHash [16]byte, dataByteCount int, err error)

OnVolumeFileLoad implements the DecoderDelegate interface.

type DoNothingRepairDelegate 

type DoNothingRepairDelegate struct {
	DoNothingDecoderDelegate
}

DoNothingRepairDelegate is an implementation of RepairDelegate that does nothing for all methods.

type DoNothingVerifyDelegate 

type DoNothingVerifyDelegate struct {
	DoNothingDecoderDelegate
}

DoNothingVerifyDelegate is an implementation of VerifyDelegate that does nothing for all methods.

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

An Encoder keeps track of all information needed to create parity volumes for a set of data files, and write them out to parity files (.PAR, .P00, .P01, etc.).

func NewEncoder 

func NewEncoder(delegate EncoderDelegate, filePaths []string, volumeCount int) (*Encoder, error)

NewEncoder creates an encoder with the given list of file paths, and with the given number of intended parity volumes.

func (*Encoder) ComputeParityData 

func (e *Encoder) ComputeParityData() error

ComputeParityData computes the parity data for the files.

func (*Encoder) LoadFileData 

func (e *Encoder) LoadFileData() error

LoadFileData loads the file data into memory.

func (*Encoder) Write 

func (e *Encoder) Write(indexPath string) error

type EncoderDelegate 

type EncoderDelegate interface {
	OnDataFileLoad(i, n int, path string, byteCount int, err error)
	OnVolumeFileWrite(i, n int, path string, dataByteCount, byteCount int, err error)
}

EncoderDelegate holds methods that are called during the encode process.

type FileCounts 

type FileCounts struct {
	// UsableDataFileCount is the number of data files that are
	// usable, i.e. not missing and not corrupt.
	UsableDataFileCount int
	// UnusableDataFileCount is the number of data files that are
	// unusable, i.e. missing or corrupt.
	UnusableDataFileCount int

	// UsableParityFileCount is the number of parity files that
	// exist, i.e. not missing and not corrupt.
	UsableParityFileCount int
	// UnusableDataFileCount is the number of parity files that
	// are unusable, i.e. missing or corrupt.
	//
	// Note that we can only infer missing parity files from gaps
	// in the counts, e.g. if we only have foo.p01 and foo.p03,
	// then we can infer that foo.p02 is missing, but if we have
	// foo.p01 and foo.p02, then foo.p03 might be missing or it
	// might not have existed in the first place.
	UnusableParityFileCount int
}

FileCounts contains file counts which can be used to deduce whether repair is necessary and/or possible.

func (FileCounts) AllFilesUsable 

func (fc FileCounts) AllFilesUsable() bool

AllFilesUsable returns whether or not all files are usable, i.e. whether UnusableDataFileCount == 0 and UnusableParityFileCount == 0.

func (FileCounts) RepairNeeded 

func (fc FileCounts) RepairNeeded() bool

RepairNeeded returns whether repair is needed, i.e. whether UnusableDataFileCount is non-zero.

func (FileCounts) RepairPossible 

func (fc FileCounts) RepairPossible() bool

RepairPossible returns whether repair is possible i.e. whether UsableParityFileCount >= UnusableDataFileCount.

type RepairDelegate 

type RepairDelegate interface {
	DecoderDelegate
}

RepairDelegate is just DecoderDelegate for now.

type RepairOptions 

type RepairOptions struct {
	// If DoubleCheck is true, then extra checking is done after
	// the repair to verify that the repaired shards are correct.
	DoubleCheck bool
	// The RepairDelegate to use. If nil, DoNothingRepairDelegate
	// is used.
	RepairDelegate RepairDelegate
}

RepairOptions holds all the options for Repair.

type RepairResult 

type RepairResult struct {
	// RepairedPaths contains the paths of the files that were
	// repaired.
	RepairedPaths []string
}

RepairResult holds the result of a Repair call.

func Repair 

func Repair(parPath string, options RepairOptions) (RepairResult, error)

Repair a par file at parPath with the given options. The returned RepairResult may be partially or not filled in if an error is returned.

type VerifyDelegate 

type VerifyDelegate interface {
	DecoderDelegate
}

VerifyDelegate is just DecoderDelegate for now.

type VerifyOptions 

type VerifyOptions struct {
	// If VerifyAllData is true, then check whether all data and
	// parity files contain correct data even if no missing or
	// corrupt files are detected.
	VerifyAllData bool
	// The VerifyDelegate to use. If nil, DoNothingVerifyDelegate
	// is used.
	VerifyDelegate VerifyDelegate
}

VerifyOptions holds all the options for Verify.

type VerifyResult 

type VerifyResult struct {
	// FileCounts contains file counts which can be used to deduce
	// whether repair is necessary and/or possible.
	FileCounts FileCounts
	// AllDataOk contains the result of calling VerifyAllData(),
	// when VerifyAllData is set to true in VerifyOptions and
	// FileCounts.AllFilesUsable() returns true, and contains
	// false otherwise.
	AllDataOk bool
}

VerifyResult holds the result of a Verify call.

func Verify 

func Verify(parPath string, options VerifyOptions) (VerifyResult, error)

Verify a par file at parPath with the given options. The returned VerifyResult is not filled in if an error is returned.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.