vfscache

package
v1.52.1-0...-9b21a15 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 7, 2020 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package vfscache deals with caching of files locally for the VFS layer

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AddVirtualFn

type AddVirtualFn func(remote string, size int64, isDir bool) error

AddVirtualFn if registered by the WithAddVirtual method, can be called to register the object or directory at remote as a virtual entry in directory listings.

This is used when reloading the Cache and uploading items need to go into the directory tree.

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache opened files

func New

func New(ctx context.Context, fremote fs.Fs, opt *vfscommon.Options, avFn AddVirtualFn) (*Cache, error)

New creates a new cache heirachy for fremote

This starts background goroutines which can be cancelled with the context passed in.

func (*Cache) AddVirtual

func (c *Cache) AddVirtual(remote string, size int64, isDir bool) error

AddVirtual adds a virtual directory entry by calling the addVirtual callback if one has been registered.

func (*Cache) CleanUp

func (c *Cache) CleanUp() error

CleanUp empties the cache of everything

func (*Cache) DirtyItem

func (c *Cache) DirtyItem(name string) (item *Item)

DirtyItem the Item if it exists in the cache and is Dirty

name should be a remote path not an osPath

func (*Cache) Dump

func (c *Cache) Dump() string

Dump the cache into a string for debugging purposes

func (*Cache) Exists

func (c *Cache) Exists(name string) bool

Exists checks to see if the file exists in the cache or not.

This is done by bringing the item into the cache which will validate the backing file and metadata and then asking if the Item exists or not.

func (*Cache) InUse

func (c *Cache) InUse(name string) bool

InUse returns whether the name is in use in the cache

name should be a remote path not an osPath

func (*Cache) Item

func (c *Cache) Item(name string) (item *Item)

Item gets a cache item for name

To use it item.Open will need to be called

name should be a remote path not an osPath

func (*Cache) Remove

func (c *Cache) Remove(name string) (wasWriting bool)

Remove should be called if name is deleted

This returns true if the file was in the transfer queue so may not have completedly uploaded yet.

func (*Cache) Rename

func (c *Cache) Rename(name string, newName string, newObj fs.Object) (err error)

Rename the item in cache

func (*Cache) SetModTime

func (c *Cache) SetModTime(name string, modTime time.Time)

SetModTime should be called to set the modification time of the cache file

func (*Cache) TotalInUse

func (c *Cache) TotalInUse() (n int)

TotalInUse returns the number of items in the cache which are InUse

type Info

type Info struct {
	ModTime     time.Time     // last time file was modified
	ATime       time.Time     // last time file was accessed
	Size        int64         // size of the file
	Rs          ranges.Ranges // which parts of the file are present
	Fingerprint string        // fingerprint of remote object
	Dirty       bool          // set if the backing file has been modified
}

Info is persisted to backing store

type Item

type Item struct {
	// contains filtered or unexported fields
}

Item is stored in the item map

The Info field is written to the backing store to store status

func (*Item) Close

func (item *Item) Close(storeFn StoreFn) (err error)

Close the cache file

func (*Item) Dirty

func (item *Item) Dirty()

Dirty marks the item as changed and needing writeback

func (*Item) Exists

func (item *Item) Exists() bool

Exists returns whether the backing file for the item exists or not

func (*Item) FindMissing

func (item *Item) FindMissing(r ranges.Range) (outr ranges.Range)

FindMissing adjusts r returning a new ranges.Range which only contains the range which needs to be downloaded. This could be empty - check with IsEmpty. It also adjust this to make sure it is not larger than the file.

func (*Item) GetSize

func (item *Item) GetSize() (size int64, err error)

GetSize gets the current size of the item

func (*Item) HasRange

func (item *Item) HasRange(r ranges.Range) bool

HasRange returns true if the current ranges entirely include range

func (*Item) IsDirty

func (item *Item) IsDirty() bool

IsDirty returns true if the item is dirty

func (*Item) Open

func (item *Item) Open(o fs.Object) (err error)

Open the local file from the object passed in (which may be nil) which implies we are about to create the file

func (*Item) ReadAt

func (item *Item) ReadAt(b []byte, off int64) (n int, err error)

ReadAt bytes from the file at off

func (*Item) Sync

func (item *Item) Sync() (err error)

Sync commits the current contents of the file to stable storage. Typically, this means flushing the file system's in-memory copy of recently written data to disk.

func (*Item) Truncate

func (item *Item) Truncate(size int64) (err error)

Truncate the item to the given size, creating it if necessary

If the new size is shorter than the existing size then the object will be shortened and marked as dirty.

If the new size is longer than the old size then the object will be extended and the extended data will be filled with zeros. The object will be marked as dirty in this case also.

func (*Item) WriteAt

func (item *Item) WriteAt(b []byte, off int64) (n int, err error)

WriteAt bytes to the file at off

func (*Item) WriteAtNoOverwrite

func (item *Item) WriteAtNoOverwrite(b []byte, off int64) (n int, skipped int, err error)

WriteAtNoOverwrite writes b to the file, but will not overwrite already present ranges.

This is used by the downloader to write bytes to the file

It returns n the total bytes processed and skipped the number of bytes which were processed but not actually written to the file.

type Items

type Items []*Item

Items are a slice of *Item ordered by ATime

func (Items) Len

func (v Items) Len() int

func (Items) Less

func (v Items) Less(i, j int) bool

func (Items) Swap

func (v Items) Swap(i, j int)

type StoreFn

type StoreFn func(fs.Object)

StoreFn is called back with an object after it has been uploaded

Directories

Path Synopsis
Package writeback keeps track of the files which need to be written back to storage
Package writeback keeps track of the files which need to be written back to storage

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL