storage

package
v0.0.0-...-4d67cfe Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 22, 2018 License: BSD-3-Clause, BSD-3-Clause Imports: 10 Imported by: 10

README

This is a goinstall-able mirror of modified code already published at:
https://git.nic.cz/redmine/projects/gofileutil/repository/show/storage

Install: $go get github.com/cznic/fileutil/storage
Godocs: http://gopkgdoc.appspot.com/pkg/github.com/cznic/fileutil/storage

Documentation

Overview

WIP: Package storage defines and implements storage providers and store accessors.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func LockedMutate

func LockedMutate(a Accessor, l sync.Locker, f func() error) (err error)

LockedMutate wraps Mutate in yet another layer consisting of a l.Lock/l.Unlock pair. All other limitations apply as in Mutate, e.g. no panics are allowed to happen - otherwise no guarantees can be made about Unlock matching the Lock.

func Mutate

func Mutate(a Accessor, f func() error) (err error)

Mutate is a helper/wrapper for executing f in between a.BeginUpdate and a.EndUpdate. Any parameters and/or return values except an error should be captured by a function literal passed as f. The returned err is either nil or the first non nil error returned from the sequence of execution: BeginUpdate, [f,] EndUpdate. The pair BeginUpdate/EndUpdate *is* invoked always regardles of any possible errors produced. Mutate doesn't handle panic, it should be used only with a function [literal] which doesn't panic. Otherwise the pairing of BeginUpdate/EndUpdate is not guaranteed.

NOTE: If BeginUpdate, which is invoked before f, returns a non-nil error, then f is not invoked at all (but EndUpdate still is).

Types

type Accessor

type Accessor interface {

	// Close closes the store, rendering it unusable for I/O. It returns an
	// error, if any.
	Close() error

	// Name returns the name of the file as presented to Open.
	Name() string

	// ReadAt reads len(b) bytes from the store starting at byte offset off.
	// It returns the number of bytes read and the error, if any.
	// EOF is signaled by a zero count with err set to os.EOF.
	// ReadAt always returns a non-nil Error when n != len(b).
	ReadAt(b []byte, off int64) (n int, err error)

	// Stat returns the FileInfo structure describing the store. It returns
	// the os.FileInfo and an error, if any.
	Stat() (fi os.FileInfo, err error)

	// Sync commits the current contents of the store to stable storage.
	// Typically, this means flushing the file system's in-memory copy of
	// recently written data to disk.
	Sync() (err error)

	// Truncate changes the size of the store. It does not change the I/O
	// offset.
	Truncate(size int64) error

	// WriteAt writes len(b) bytes to the store starting at byte offset off.
	// It returns the number of bytes written and an error, if any.
	// WriteAt returns a non-nil Error when n != len(b).
	WriteAt(b []byte, off int64) (n int, err error)

	// Before every [structural] change of a store the BeginUpdate is to be
	// called and paired with EndUpdate after the change makes the store's
	// state consistent again. Invocations of BeginUpdate may nest. On
	// invoking the last non nested EndUpdate an implicit "commit" should
	// be performed by the store/provider. The concrete mechanism is
	// unspecified. It could be for example a write-ahead log.  Stores may
	// implement BeginUpdate and EndUpdate as a (documented) no op.
	BeginUpdate() error
	EndUpdate() error
}

Accessor provides I/O methods to access a store.

func NewFile

func NewFile(name string, flag int, perm os.FileMode) (store Accessor, err error)

NewFile returns an Accessor backed by an os.File named name, It opens the named file with specified flag (os.O_RDWR etc.) and perm, (0666 etc.) if applicable. If successful, methods on the returned Accessor can be used for I/O. It returns the Accessor and an Error, if any.

NOTE: The returned Accessor implements BeginUpdate and EndUpdate as a no op.

func NewMem

func NewMem(f *os.File) (store Accessor, err error)

NewMem returns a new Accessor backed by an os.File. The returned Accessor keeps all of the store content in memory. The memory and file images are synced only by Sync and Close. Recomended for small amounts of data only and content which may be lost on process kill/crash. NewMem return the Accessor or an error of any.

NOTE: The returned Accessor implements BeginUpdate and EndUpdate as a no op.

func OpenFile

func OpenFile(name string, flag int, perm os.FileMode) (store Accessor, err error)

OpenFile returns an Accessor backed by an existing os.File named name, It opens the named file with specified flag (os.O_RDWR etc.) and perm, (0666 etc.) if applicable. If successful, methods on the returned Accessor can be used for I/O. It returns the Accessor and an Error, if any.

NOTE: The returned Accessor implements BeginUpdate and EndUpdate as a no op.

func OpenMem

func OpenMem(f *os.File) (store Accessor, err error)

OpenMem return a new Accessor backed by an os.File. The store content is loaded from f. The returned Accessor keeps all of the store content in memory. The memory and file images are synced only Sync and Close. Recomended for small amounts of data only and content which may be lost on process kill/crash. OpenMem return the Accessor or an error of any.

NOTE: The returned Accessor implements BeginUpdate and EndUpdate as a no op.

type Cache

type Cache struct {
	Rq    int64 // Pages requested from cache
	Load  int64 // Pages loaded (cache miss)
	Purge int64 // Pages purged
	Top   int   // "High water" pages
	// contains filtered or unexported fields
}

Cache provides caching support for another store Accessor.

func NewCache

func NewCache(store Accessor, maxcache int64, advise func(int64, int, bool)) (c *Cache, err error)

NewCache creates a caching Accessor from store with total of maxcache bytes. NewCache returns the new Cache, implementing Accessor or an error if any.

The LRU mechanism is used, so the cache tries to keep often accessed pages cached.

func (*Cache) Accessor

func (c *Cache) Accessor() Accessor

func (*Cache) BeginUpdate

func (c *Cache) BeginUpdate() error

Implementation of Accessor.

func (*Cache) Close

func (c *Cache) Close() (err error)

func (*Cache) EndUpdate

func (c *Cache) EndUpdate() error

Implementation of Accessor.

func (*Cache) Name

func (c *Cache) Name() (s string)

func (*Cache) ReadAt

func (c *Cache) ReadAt(b []byte, off int64) (n int, err error)

func (*Cache) Stat

func (c *Cache) Stat() (fi os.FileInfo, err error)

func (*Cache) Sync

func (c *Cache) Sync() (err error)

func (*Cache) Truncate

func (c *Cache) Truncate(size int64) (err error)

func (*Cache) WriteAt

func (c *Cache) WriteAt(b []byte, off int64) (n int, err error)

type FileAccessor

type FileAccessor struct {
	*os.File
}

FileAccessor is the concrete type returned by NewFile and OpenFile.

func (*FileAccessor) BeginUpdate

func (f *FileAccessor) BeginUpdate() error

Implementation of Accessor.

func (*FileAccessor) EndUpdate

func (f *FileAccessor) EndUpdate() error

Implementation of Accessor.

type FileInfo

type FileInfo struct {
	FName    string      // base name of the file
	FSize    int64       // length in bytes
	FMode    os.FileMode // file mode bits
	FModTime time.Time   // modification time
	FIsDir   bool        // abbreviation for Mode().IsDir()
	// contains filtered or unexported fields
}

FileInfo is a type implementing os.FileInfo which has setable fields, like the older os.FileInfo used to have. It is used wehere e.g. the Size is needed to be faked (encapsulated/memory only file, file cache, etc.).

func NewFileInfo

func NewFileInfo(fi os.FileInfo, sys interface{}) *FileInfo

NewFileInfo creates FileInfo from os.FileInfo fi.

func (*FileInfo) IsDir

func (fi *FileInfo) IsDir() bool

Implementation of os.FileInfo

func (*FileInfo) ModTime

func (fi *FileInfo) ModTime() time.Time

Implementation of os.FileInfo

func (*FileInfo) Mode

func (fi *FileInfo) Mode() os.FileMode

Implementation of os.FileInfo

func (*FileInfo) Name

func (fi *FileInfo) Name() string

Implementation of os.FileInfo

func (*FileInfo) Size

func (fi *FileInfo) Size() int64

Implementation of os.FileInfo

func (*FileInfo) Sys

func (fi *FileInfo) Sys() interface{}

type Probe

type Probe struct {
	Accessor
	Chain     *Probe
	OpsRd     int64
	OpsWr     int64
	BytesRd   int64
	BytesWr   int64
	SectorsRd int64 // Assuming 512 byte sector size
	SectorsWr int64
}

Probe collects usage statistics of the embeded Accessor. Probe itself IS an Accessor.

func NewProbe

func NewProbe(src Accessor, chain *Probe) *Probe

NewProbe returns a newly created probe which embedes the src Accessor. The retuned *Probe satisfies Accessor. if chain != nil then Reset() is cascaded down the chained Probes.

func (*Probe) ReadAt

func (p *Probe) ReadAt(b []byte, off int64) (n int, err error)

func (*Probe) Reset

func (p *Probe) Reset()

Reset zeroes the collected statistics of p.

func (*Probe) WriteAt

func (p *Probe) WriteAt(b []byte, off int64) (n int, err error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL