vfs

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 14, 2021 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrInjected = errors.New("injected error")

ErrInjected is an error artifically injected for testing fs error paths.

Functions

func Clone 

func Clone(srcFS, dstFS FS, srcPath, dstPath string) (bool, error)

Clone recursively copies a directory structure from srcFS to dstFS. srcPath specifies the path in srcFS to copy from and must be compatible with the srcFS path format. dstDir is the target directory in dstFS and must be compatible with the dstFS path format. Returns (true,nil) on a successful copy, (false,nil) if srcPath does not exist, and (false,err) if an error occurred.

func Copy 

func Copy(fs FS, oldname, newname string) error

Copy copies the contents of oldname to newname. If newname exists, it will be overwritten.

func IsNoSpaceError added in v1.3.0 

func IsNoSpaceError(err error) bool

IsNoSpaceError returns true if the given error indicates that the disk is out of space.

func LimitedCopy 

func LimitedCopy(fs FS, oldname, newname string, maxBytes int64) error

LimitedCopy copies up to maxBytes from oldname to newname. If newname exists, it will be overwritten.

func LinkOrCopy 

func LinkOrCopy(fs FS, oldname, newname string) error

LinkOrCopy creates newname as a hard link to the oldname file. If creating the hard link fails, LinkOrCopy falls back to copying the file (which may also fail if newname doesn't exist or oldname already exists).

func Prefetch added in v1.3.0 

func Prefetch(file File, offset uint64, size uint64) error

Prefetch signals the OS (on supported platforms) to fetch the next size bytes in file after offset into cache. Any subsequent reads in that range will not issue disk IO.

func ReportLeakedFD 

func ReportLeakedFD(fs FS, t *testing.T)

Types

type ErrorFS added in v1.2.0 

type ErrorFS struct {
	// contains filtered or unexported fields
}

FS implements vfs.FS, injecting errors into its operations.

func Wrap added in v1.2.0 

func Wrap(fs FS, inj Injector) *ErrorFS

Wrap wraps an existing vfs.FS implementation, returning a new vfs.FS implementation that shadows operations to the provided FS. It uses the provided Injector for deciding when to inject errors. If an error is injected, FS propagates the error instead of shadowing the operation.

func (*ErrorFS) Create added in v1.2.0 

func (fs *ErrorFS) Create(name string) (File, error)

Create implements FS.Create.

func (*ErrorFS) GetFreeSpace added in v1.3.0 

func (fs *ErrorFS) GetFreeSpace(path string) (uint64, error)

GetFreeSpace implements FS.GetFreeSpace. 

func (fs *ErrorFS) Link(oldname, newname string) error

Link implements FS.Link.

func (*ErrorFS) List added in v1.2.0 

func (fs *ErrorFS) List(dir string) ([]string, error)

List implements FS.List.

func (*ErrorFS) Lock added in v1.2.0 

func (fs *ErrorFS) Lock(name string) (io.Closer, error)

Lock implements FS.Lock.

func (*ErrorFS) MkdirAll added in v1.2.0 

func (fs *ErrorFS) MkdirAll(dir string, perm os.FileMode) error

MkdirAll implements FS.MkdirAll.

func (*ErrorFS) Open added in v1.2.0 

func (fs *ErrorFS) Open(name string, opts ...OpenOption) (File, error)

Open implements FS.Open.

func (*ErrorFS) OpenDir added in v1.2.0 

func (fs *ErrorFS) OpenDir(name string) (File, error)

OpenDir implements FS.OpenDir.

func (*ErrorFS) OpenForAppend added in v1.2.0 

func (fs *ErrorFS) OpenForAppend(name string) (File, error)

OpenForAppend implements FS.OpenForAppend.

func (*ErrorFS) PathBase added in v1.2.0 

func (fs *ErrorFS) PathBase(p string) string

PathBase implements FS.PathBase.

func (*ErrorFS) PathDir added in v1.2.0 

func (fs *ErrorFS) PathDir(p string) string

PathDir implements FS.PathDir.

func (*ErrorFS) PathJoin added in v1.2.0 

func (fs *ErrorFS) PathJoin(elem ...string) string

PathJoin implements FS.PathJoin.

func (*ErrorFS) Remove added in v1.2.0 

func (fs *ErrorFS) Remove(name string) error

Remove implements FS.Remove.

func (*ErrorFS) RemoveAll added in v1.2.0 

func (fs *ErrorFS) RemoveAll(fullname string) error

RemoveAll implements FS.RemoveAll.

func (*ErrorFS) Rename added in v1.2.0 

func (fs *ErrorFS) Rename(oldname, newname string) error

Rename implements FS.Rename.

func (*ErrorFS) ReuseForWrite added in v1.2.0 

func (fs *ErrorFS) ReuseForWrite(oldname, newname string) (File, error)

ReuseForWrite implements FS.ReuseForWrite.

func (*ErrorFS) Stat added in v1.2.0 

func (fs *ErrorFS) Stat(name string) (os.FileInfo, error)

Stat implements FS.Stat.

func (*ErrorFS) Unwrap added in v1.2.0 

func (fs *ErrorFS) Unwrap() FS

Unwrap returns the FS implementation underlying fs. See pebble/vfs.Root.

type FS 

type FS interface {
	// Create creates the named file for writing, truncating it if it already
	// exists.
	Create(name string) (File, error)

	// Link creates newname as a hard link to the oldname file.
	Link(oldname, newname string) error

	// Open opens the named file for reading. openOptions provides
	Open(name string, opts ...OpenOption) (File, error)

	// OpenDir opens the named directory for syncing.
	OpenDir(name string) (File, error)

	// OpenForAppend opens the named file for appending.
	OpenForAppend(name string) (File, error)

	// Remove removes the named file or directory.
	Remove(name string) error

	// Remove removes the named file or directory and any children it
	// contains. It removes everything it can but returns the first error it
	// encounters.
	RemoveAll(name string) error

	// Rename renames a file. It overwrites the file at newname if one exists,
	// the same as os.Rename.
	Rename(oldname, newname string) error

	// ReuseForWrite attempts to reuse the file with oldname by renaming it to newname and opening
	// it for writing without truncation. It is acceptable for the implementation to choose not
	// to reuse oldname, and simply create the file with newname -- in this case the implementation
	// should delete oldname. If the caller calls this function with an oldname that does not exist,
	// the implementation may return an error.
	ReuseForWrite(oldname, newname string) (File, error)

	// MkdirAll creates a directory and all necessary parents. The permission
	// bits perm have the same semantics as in os.MkdirAll. If the directory
	// already exists, MkdirAll does nothing and returns nil.
	MkdirAll(dir string, perm os.FileMode) error

	// Lock locks the given file, creating the file if necessary, and
	// truncating the file if it already exists. The lock is an exclusive lock
	// (a write lock), but locked files should neither be read from nor written
	// to. Such files should have zero size and only exist to co-ordinate
	// ownership across processes.
	//
	// A nil Closer is returned if an error occurred. Otherwise, close that
	// Closer to release the lock.
	//
	// On Linux and OSX, a lock has the same semantics as fcntl(2)'s advisory
	// locks. In particular, closing any other file descriptor for the same
	// file will release the lock prematurely.
	//
	// Attempting to lock a file that is already locked by the current process
	// has undefined behavior.
	//
	// Lock is not yet implemented on other operating systems, and calling it
	// will return an error.
	Lock(name string) (io.Closer, error)

	// List returns a listing of the given directory. The names returned are
	// relative to dir.
	List(dir string) ([]string, error)

	// Stat returns an os.FileInfo describing the named file.
	Stat(name string) (os.FileInfo, error)

	// PathBase returns the last element of path. Trailing path separators are
	// removed before extracting the last element. If the path is empty, PathBase
	// returns ".".  If the path consists entirely of separators, PathBase returns a
	// single separator.
	PathBase(path string) string

	// PathJoin joins any number of path elements into a single path, adding a
	// separator if necessary.
	PathJoin(elem ...string) string

	// PathDir returns all but the last element of path, typically the path's directory.
	PathDir(path string) string

	// GetFreeSpace returns the amount of free disk space for the filesystem
	// where path is any file or directory within that filesystem.
	GetFreeSpace(path string) (uint64, error)
}

FS is a namespace for files.

The names are filepath names: they may be / separated or \ separated, depending on the underlying operating system. 

var Default FS = defaultFS{}

Default is a FS implementation backed by the underlying operating system's file system.

type File 

type File interface {
	io.Closer
	io.Reader
	io.ReaderAt
	io.Writer
	io.WriterAt
	Stat() (os.FileInfo, error)
	Sync() error
}

File is a readable, writable sequence of bytes.

Typically, it will be an *os.File, but test code may choose to substitute memory-backed implementations.

func NewMemFile 

func NewMemFile(data []byte) File

NewMemFile returns a memory-backed File implementation. The memory-backed file takes ownership of data.

func NewSyncingFile 

func NewSyncingFile(f File, opts SyncingFileOptions) File

NewSyncingFile wraps a writable file and ensures that data is synced periodically as it is written. The syncing does not provide persistency guarantees for these periodic syncs, but is used to avoid latency spikes if the OS automatically decides to write out a large chunk of dirty filesystem buffers. The underlying file is fully synced upon close.

func WrapFile added in v1.2.0 

func WrapFile(f File, inj Injector) File

WrapFile wraps an existing vfs.File, returning a new vfs.File that shadows operations to the provided vfs.File. It uses the provided Injector for deciding when to inject errors. If an error is injected, the file propagates the error instead of shadowing the operation.

type InjectIndex added in v1.2.0 

type InjectIndex struct {
	// contains filtered or unexported fields
}

InjectIndex implements Injector, injecting an error at a specific index.

func OnIndex added in v1.2.0 

func OnIndex(index int32, op Op) *InjectIndex

OnIndex constructs an injector that returns an error on the (n+1)-th invocation of its MaybeError function. It may be passed to Wrap to inject an error into an FS.

func (*InjectIndex) Index added in v1.2.0 

func (ii *InjectIndex) Index() int32

Index returns the index at which the error will be injected.

func (*InjectIndex) MaybeError added in v1.2.0 

func (ii *InjectIndex) MaybeError(op Op) error

MaybeError implements the Injector interface.

func (*InjectIndex) SetIndex added in v1.2.0 

func (ii *InjectIndex) SetIndex(v int32)

SetIndex sets the index at which the error will be injected.

type Injector added in v1.2.0 

type Injector interface {
	MaybeError(Op) error
}

Injector injects errors into FS operations.

func WithProbability added in v1.2.0 

func WithProbability(op Op, p float64) Injector

WithProbability returns a function that returns an error with the provided probability when passed op. It may be passed to Wrap to inject an error into an ErrFS with the provided probability. p should be within the range [0.0,1.0].

type MemFS 

type MemFS struct {
	// contains filtered or unexported fields
}

MemFS implements FS.

func NewMem 

func NewMem() *MemFS

NewMem returns a new memory-backed FS implementation.

func NewStrictMem 

func NewStrictMem() *MemFS

NewStrictMem returns a "strict" memory-backed FS implementation. The behaviour is strict wrt needing a Sync() call on files or directories for the state changes to be finalized. Any changes that are not finalized are visible to reads until MemFS.ResetToSyncedState() is called, at which point they are discarded and no longer visible.

Expected usage: 

strictFS := NewStrictMem()
db := Open(..., &Options{FS: strictFS})
// Do and commit various operations.
...
// Prevent any more changes to finalized state.
strictFS.SetIgnoreSyncs(true)
// This will finish any ongoing background flushes, compactions but none of these writes will
// be finalized since syncs are being ignored.
db.Close()
// Discard unsynced state.
strictFS.ResetToSyncedState()
// Allow changes to finalized state.
strictFS.SetIgnoreSyncs(false)
// Open the DB. This DB should have the same state as if the earlier strictFS operations and
// db.Close() were not called.
db := Open(..., &Options{FS: strictFS})

func (*MemFS) Create 

func (y *MemFS) Create(fullname string) (File, error)

Create implements FS.Create.

func (*MemFS) GetFreeSpace added in v1.3.0 

func (*MemFS) GetFreeSpace(string) (uint64, error)

GetFreeSpace implements FS.GetFreeSpace.

func (*MemFS) Iterate 

func (y *MemFS) Iterate(f func(path string, isDir bool, refs int32) error) error
func (y *MemFS) Link(oldname, newname string) error

Link implements FS.Link.

func (*MemFS) List 

func (y *MemFS) List(dirname string) ([]string, error)

List implements FS.List.

func (*MemFS) Lock 

func (y *MemFS) Lock(fullname string) (io.Closer, error)

Lock implements FS.Lock.

func (*MemFS) MkdirAll 

func (y *MemFS) MkdirAll(dirname string, perm os.FileMode) error

MkdirAll implements FS.MkdirAll.

func (*MemFS) Open 

func (y *MemFS) Open(fullname string, opts ...OpenOption) (File, error)

Open implements FS.Open.

func (*MemFS) OpenDir 

func (y *MemFS) OpenDir(fullname string) (File, error)

OpenDir implements FS.OpenDir.

func (*MemFS) OpenForAppend 

func (y *MemFS) OpenForAppend(fullname string) (File, error)

OpenForAppend implements FS.OpenForAppend.

func (*MemFS) PathBase 

func (*MemFS) PathBase(p string) string

PathBase implements FS.PathBase.

func (*MemFS) PathDir 

func (*MemFS) PathDir(p string) string

PathDir implements FS.PathDir.

func (*MemFS) PathJoin 

func (*MemFS) PathJoin(elem ...string) string

PathJoin implements FS.PathJoin.

func (*MemFS) Remove 

func (y *MemFS) Remove(fullname string) error

Remove implements FS.Remove.

func (*MemFS) RemoveAll 

func (y *MemFS) RemoveAll(fullname string) error

RemoveAll implements FS.RemoveAll.

func (*MemFS) Rename 

func (y *MemFS) Rename(oldname, newname string) error

Rename implements FS.Rename.

func (*MemFS) ResetToSyncedState 

func (y *MemFS) ResetToSyncedState()

ResetToSyncedState discards state in the FS that is not synced. See the usage comment with NewStrictMem() for details.

func (*MemFS) ReuseForWrite 

func (y *MemFS) ReuseForWrite(oldname, newname string) (File, error)

ReuseForWrite implements FS.ReuseForWrite.

func (*MemFS) SetIgnoreSyncs 

func (y *MemFS) SetIgnoreSyncs(ignoreSyncs bool)

SetIgnoreSyncs sets the MemFS.ignoreSyncs field. See the usage comment with NewStrictMem() for details.

func (*MemFS) Stat 

func (y *MemFS) Stat(name string) (os.FileInfo, error)

Stat implements FS.Stat.

func (*MemFS) String 

func (y *MemFS) String() string

String dumps the contents of the MemFS.

type Op added in v1.2.0 

type Op int

Op is an enum describing the type of operation performed. 

const (
	// OpRead describes read operations.
	OpRead Op = iota
	// OpWrite describes write operations.
	OpWrite
	// OpSync describes the fsync operation.
	OpSync
)

type OpenOption 

type OpenOption interface {
	// Apply is called on the file handle after it's opened.
	Apply(File)
}

OpenOption provide an interface to do work on file handles in the Open() call. 

var RandomReadsOption OpenOption = &randomReadsOption{}

RandomReadsOption is an OpenOption that optimizes opened file handle for random reads, by calling fadvise() with POSIX_FADV_RANDOM on Linux systems to disable readahead. 

var SequentialReadsOption OpenOption = &sequentialReadsOption{}

SequentialReadsOption is an OpenOption that optimizes opened file handle for sequential reads, by calling fadvise() with POSIX_FADV_SEQUENTIAL on Linux systems to enable readahead.

type SyncingFileOptions 

type SyncingFileOptions struct {
	BytesPerSync    int
	PreallocateSize int
}

SyncingFileOptions holds the options for a syncingFile.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.