svfs

package
v0.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2016 License: BSD-3-Clause Imports: 11 Imported by: 6

Documentation

Index

Constants

View Source 
const (
	DirContentType = "application/directory"
	ObjContentType = "application/octet-stream"
)
View Source 
const (
	ManifestHeader = "X-Object-Manifest"
)
View Source 
const (
	SegmentContainerSuffix = "_segments"
)

Variables

View Source 
var (
	FolderRegex      = regexp.MustCompile("^.+/$")
	SubdirRegex      = regexp.MustCompile(".*/.*$")
	SegmentPathRegex = regexp.MustCompile("^([^/]+)/(.*)$")
	DirectoryCache   = new(Cache)
	DirectoryLister  = new(DirLister)
)
View Source 
var (
	SwiftConnection *swift.Connection
	DefaultUID      uint64 = 0
	DefaultGID      uint64 = 0
	DefaultMode     uint64 = 0700
)
View Source 
var SegmentRegex = regexp.MustCompile("^.+_segments$")
View Source 
var (
	SegmentSize uint64
)

Functions

This section is empty.

Types

type Cache added in v0.2.6 

type Cache struct {
	// contains filtered or unexported fields
}

Cache holds a map of cache entries. Its size can be configured as well as cache entries access limit and expiration time.

func NewCache added in v0.2.6 

func NewCache(cconf *CacheConfig) *Cache

NewCache creates a new cache

func (*Cache) AddAll added in v0.3.0 

func (c *Cache) AddAll(container, path string, node Node, nodes map[string]Node)

AddAll creates a new cache entry with the key container:path and a map of nodes as a value. Node represent the parent node type. If the cache entry count limit is reached, it will be marked as temporary thus evicted after one read.

func (*Cache) Delete added in v0.2.6 

func (c *Cache) Delete(container, path, name string)

Delete removes a node from cache.

func (*Cache) DeleteAll added in v0.3.0 

func (c *Cache) DeleteAll(container, path string)

DeleteAll removes all nodes for the cache key container:path.

func (*Cache) Get added in v0.2.6 

func (c *Cache) Get(container, path, name string) Node

Get retrieves a specific node from the cache. It returns nil if the cache key container:path is missing.

func (*Cache) GetAll added in v0.3.0 

func (c *Cache) GetAll(container, path string) (Node, map[string]Node)

GetAll retrieves all nodes for the cache key container:path. It returns the parent node and its children nodes. If the cache entry is not found or expired or access count exceeds the limit, both values will be nil.

func (*Cache) Peek added in v0.3.0 

func (c *Cache) Peek(container, path string) (Node, bool)

Peek checks if a valid cache entry belongs to container:path key without changing cache access count for this entry. Returns the parent node with the result.

func (*Cache) Set added in v0.2.6 

func (c *Cache) Set(container, path, name string, node Node)

Set adds a specific node in cache, given a previous peek operation succeeded.

type CacheConfig added in v0.2.6 

type CacheConfig struct {
	Timeout    time.Duration
	MaxEntries int64
	MaxAccess  int64
}

CacheConfig is the cache configuration.

type CacheValue added in v0.3.0 

type CacheValue struct {
	// contains filtered or unexported fields
}

CacheValue is the representation of a cache entry. It tracks expiration date, access count and holds a parent node with its children. It can be set as temporary, meaning that it will be stored within the cache but evicted on first access.

type Config added in v0.2.6 

type Config struct {
	Container             string
	ConnectTimeout        time.Duration
	ReadAheadSize         uint
	SegmentSizeMB         uint64
	MaxReaddirConcurrency uint64
	MaxUploadConcurrency  uint64
}

Config represents SVFS configuration settings.

type Container 

type Container struct {
	*Directory
}

Container is a node representing a directory entry bound to a Swift container.

func (*Container) Attr 

func (c *Container) Attr(ctx context.Context, a *fuse.Attr) error

Attr fills the container file attributes within the current context.

type DirLister added in v0.2.6 

type DirLister struct {
	// contains filtered or unexported fields
}

DirLister is a concurrent processor for segmented objects. Its job is to get information about manifests stored within directories.

func (*DirLister) AddTask added in v0.2.6 

func (dl *DirLister) AddTask(o *Object, c chan<- *Object)

AddTask asynchronously adds a new task to be processed. It returns immediately with no guarantee that the task has been added to the channel nor retrieved by a worker.

func (*DirLister) Start added in v0.2.6 

func (dl *DirLister) Start()

Start spawns workers waiting for tasks. Once a task comes in the task channel, one worker will process it by opening a connection to swift and asking information about the current manifest. The real size of the object is modified then it sends the modified object into the task result channel.

type DirListerTask added in v0.2.6 

type DirListerTask struct {
	// contains filtered or unexported fields
}

DirListerTask represents a manifest ready to be processed by the DirLister. Every task must provide a manifest object and a result channel to which retrieved information will be send.

type Directory 

type Directory struct {
	// contains filtered or unexported fields
}

Directory represents a standard directory entry.

func (*Directory) Attr 

func (d *Directory) Attr(ctx context.Context, a *fuse.Attr) error

Attr fills file attributes of a directory within the current context.

func (*Directory) Create 

func (d *Directory) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.CreateResponse) (fs.Node, fs.Handle, error)

Create makes a new object node represented by a file. It returns an object node and an opened file handle.

func (*Directory) Export 

func (d *Directory) Export() fuse.Dirent

Export gives a direntry for the current directory node.

func (*Directory) Lookup 

func (d *Directory) Lookup(ctx context.Context, req *fuse.LookupRequest, resp *fuse.LookupResponse) (fs.Node, error)

Lookup gets a children node if its name matches the requested direntry name. If the cache is empty for the current directory, it will fill it and try to match the requested direnty after this operation. It returns ENOENT if not found.

func (*Directory) Mkdir added in v0.2.0 

func (d *Directory) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error)

Mkdir creates a new directory node within the current directory. It is represented by an empty object ending with a slash in the Swift container.

func (*Directory) Name 

func (d *Directory) Name() string

Name gets the direntry name

func (*Directory) ReadDirAll 

func (d *Directory) ReadDirAll(ctx context.Context) (direntries []fuse.Dirent, err error)

ReadDirAll reads the content of a directory and returns a list of children nodes as direntries, using/filling the cache of nodes.

func (*Directory) Remove 

func (d *Directory) Remove(ctx context.Context, req *fuse.RemoveRequest) error

Remove deletes a direntry and relevant node. It is not supported on container nodes. It handles standard and segmented object deletion.

func (*Directory) Rename 

func (d *Directory) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error

Rename moves a node from its current directory node to a new directory node and updates the cache.

type Node 

type Node interface {
	Name() string
	Export() fuse.Dirent
}

Node is the generic interface of an SVFS node.

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object is a node representing a swift object. It belongs to a container and segmented objects are bound to a container of segments.

func (*Object) Attr 

func (o *Object) Attr(ctx context.Context, a *fuse.Attr) error

Attr fills the file attributes for an object node.

func (*Object) Export 

func (o *Object) Export() fuse.Dirent

Export converts this object node as a direntry.

func (*Object) Name 

func (o *Object) Name() string

Name gets the name of the underlying swift object.

func (*Object) Open 

func (o *Object) Open(ctx context.Context, req *fuse.OpenRequest, resp *fuse.OpenResponse) (fs.Handle, error)

Open returns the file handle associated with this object node.

type ObjectHandle 

type ObjectHandle struct {
	// contains filtered or unexported fields
}

ObjectHandle represents an open object handle, similarly to file handles.

func (*ObjectHandle) Read 

func (fh *ObjectHandle) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error

Read gets a swift object data for a request within the current context. The request size is always honored.

func (*ObjectHandle) Release 

func (fh *ObjectHandle) Release(ctx context.Context, req *fuse.ReleaseRequest) error

Release frees the file handle, closing all readers/writers in use. In case we used this file handle to write a large object, it creates the manifest file. Cache is refreshed if the writer was used during the lifetime of this handle.

func (*ObjectHandle) Write 

func (fh *ObjectHandle) Write(ctx context.Context, req *fuse.WriteRequest, resp *fuse.WriteResponse) (err error)

Write pushes data to a swift object. If we detect that we are writing more data than the configured segment size, then the first object we were writing to is moved to the segment container and named accordingly to DLO conventions. Remaining data will be split into segments sequentially until file handle release is called.

type Root 

type Root struct {
	*Directory
}

Root is a fake root node used to hold a list of container nodes.

func (*Root) Create 

func (r *Root) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.CreateResponse) (fs.Node, fs.Handle, error)

Create is not supported on a root node.

func (*Root) Lookup added in v0.2.5 

func (r *Root) Lookup(ctx context.Context, req *fuse.LookupRequest, resp *fuse.LookupResponse) (fs.Node, error)

Lookup gets a container node if its name matches the request name within the current context.

func (*Root) Mkdir added in v0.2.0 

func (r *Root) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error)

Mkdir is not supported on a root node.

func (*Root) ReadDirAll 

func (r *Root) ReadDirAll(ctx context.Context) (direntries []fuse.Dirent, err error)

ReadDirAll retrieves all containers within the current Openstack tenant, as direntries. Segment containers are not shown and created if missing.

func (*Root) Remove 

func (r *Root) Remove(ctx context.Context, req *fuse.RemoveRequest) error

Remove is not supported on a root node.

func (*Root) Rename 

func (r *Root) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error

Rename is not supported on a root node.

type SVFS 

type SVFS struct {
	// contains filtered or unexported fields
}

SVFS implements the Swift Virtual File System.

func (*SVFS) Init 

func (s *SVFS) Init(sc *swift.Connection, conf *Config, cconf *CacheConfig) error

Init sets up the filesystem. It sets configuration settings, starts mandatory services and make sure authentication in Swift has succeeded.

func (*SVFS) Root 

func (s *SVFS) Root() (fs.Node, error)

Root gets the root node of the filesystem. It can either be a fake root node filled with all the containers found for the given Openstack tenant or a container node if a container name have been specified in mount options.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.