svfs

package
v0.5.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 18, 2016 License: BSD-3-Clause Imports: 15 Imported by: 6

Documentation

Index

Constants

View Source 
const (
	DirContentType = "application/directory"
	ObjContentType = "application/octet-stream"
	AutoContent    = "X-Detect-Content-Type"
)
View Source 
const (
	ManifestHeader    = "X-Object-Manifest"
	ObjectMetaHeader  = "X-Object-Meta-"
	ObjectMtimeHeader = ObjectMetaHeader + "Mtime"
)
View Source 
const (
	HubicEndpoint = "https://api.hubic.com"
)
View Source 
const (
	SegmentContainerSuffix = "_segments"
)
View Source 
const Version = "0.5.2"

Variables

View Source 
var (
	CacheTimeout    time.Duration
	CacheMaxEntries int64
	CacheMaxAccess  int64
)

CacheConfig is the cache configuration.

View Source 
var (
	FolderRegex      = regexp.MustCompile("^.+/$")
	SubdirRegex      = regexp.MustCompile(".*/.*$")
	SegmentPathRegex = regexp.MustCompile("^([^/]+)/(.*)$")
	DirectoryCache   = NewCache()
	ChangeCache      = NewSimpleCache()
	DirectoryLister  = new(Lister)
)
View Source 
var (
	// Swift
	SwiftConnection = new(swift.Connection)
	TargetContainer string
	ExtraAttr       bool
	SegmentSize     uint64

	// FS
	AllowRoot          bool
	AllowOther         bool
	DefaultGID         uint64
	DefaultUID         uint64
	DefaultMode        uint64
	DefaultPermissions bool
	ReadAheadSize      uint
)
View Source 
var (
	HubicRefreshToken  string
	HubicAuthorization string
)
View Source 
var (
	ListerConcurrency uint64
)
View Source 
var SegmentRegex = regexp.MustCompile("^.+_segments$")

Functions

This section is empty.

Types

type Cache added in v0.2.6 

type Cache struct {
	// contains filtered or unexported fields
}

Cache holds a map of cache entries. Its size can be configured as well as cache entries access limit and expiration time.

func NewCache added in v0.2.6 

func NewCache() *Cache

NewCache creates a new cache

func (*Cache) AddAll added in v0.3.0 

func (c *Cache) AddAll(container, path string, node Node, nodes map[string]Node)

AddAll creates a new cache entry with the key container:path and a map of nodes as a value. Node represents the parent node type. If the cache entry count limit is reached, it will be marked as temporary thus evicted after one read.

func (*Cache) Delete added in v0.2.6 

func (c *Cache) Delete(container, path, name string)

Delete removes a node from cache.

func (*Cache) DeleteAll added in v0.3.0 

func (c *Cache) DeleteAll(container, path string)

DeleteAll removes all nodes for the cache key container:path.

func (*Cache) Get added in v0.2.6 

func (c *Cache) Get(container, path, name string) Node

Get retrieves a specific node from the cache. It returns nil if the cache key container:path is missing.

func (*Cache) GetAll added in v0.3.0 

func (c *Cache) GetAll(container, path string) (Node, map[string]Node)

GetAll retrieves all nodes for the cache key container:path. It returns the parent node and its children nodes. If the cache entry is not found or expired or access count exceeds the limit, both values will be nil.

func (*Cache) Peek added in v0.3.0 

func (c *Cache) Peek(container, path string) (Node, bool)

Peek checks if a valid cache entry belongs to container:path key without changing cache access count for this entry. Returns the parent node with the result.

func (*Cache) Set added in v0.2.6 

func (c *Cache) Set(container, path, name string, node Node)

Set adds a specific node in cache, given a previous peek operation succeeded.

type CacheValue added in v0.3.0 

type CacheValue struct {
	// contains filtered or unexported fields
}

CacheValue is the representation of a cache entry. It tracks expiration date, access count and holds a parent node with its children. It can be set as temporary, meaning that it will be stored within the cache but evicted on first access.

type Container 

type Container struct {
	*Directory
}

Container is a node representing a directory entry bound to a Swift container.

func (*Container) Attr 

func (c *Container) Attr(ctx context.Context, a *fuse.Attr) error

Attr fills the container file attributes within the current context.

type Directory 

type Directory struct {
	// contains filtered or unexported fields
}

Directory represents a standard directory entry.

func (*Directory) Attr 

func (d *Directory) Attr(ctx context.Context, a *fuse.Attr) error

Attr fills file attributes of a directory within the current context.

func (*Directory) Create 

func (d *Directory) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.CreateResponse) (fs.Node, fs.Handle, error)

Create makes a new object node represented by a file. It returns an object node and an opened file handle.

func (*Directory) Export 

func (d *Directory) Export() fuse.Dirent

Export gives a direntry for the current directory node.

func (*Directory) Lookup 

func (d *Directory) Lookup(ctx context.Context, req *fuse.LookupRequest, resp *fuse.LookupResponse) (fs.Node, error)

Lookup gets a children node if its name matches the requested direntry name. If the cache is empty for the current directory, it will fill it and try to match the requested direnty after this operation. It returns ENOENT if not found.

func (*Directory) Mkdir added in v0.2.0 

func (d *Directory) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error)

Mkdir creates a new directory node within the current directory. It is represented by an empty object ending with a slash in the Swift container.

func (*Directory) Name 

func (d *Directory) Name() string

Name gets the direntry name

func (*Directory) ReadDirAll 

func (d *Directory) ReadDirAll(ctx context.Context) (direntries []fuse.Dirent, err error)

ReadDirAll reads the content of a directory and returns a list of children nodes as direntries, using/filling the cache of nodes.

func (*Directory) Remove 

func (d *Directory) Remove(ctx context.Context, req *fuse.RemoveRequest) error

Remove deletes a direntry and relevant node. It is not supported on container nodes. It handles standard and segmented object deletion.

func (*Directory) Rename 

func (d *Directory) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error

Rename moves a node from its current directory node to a new directory node and updates the cache.

func (*Directory) Setattr added in v0.5.2 

func (d *Directory) Setattr(ctx context.Context, req *fuse.SetattrRequest, resp *fuse.SetattrResponse) error

type HubicAuth added in v0.5.0 

type HubicAuth struct {
	// contains filtered or unexported fields
}

HubicAuth is a swift-compliant authenticatior for hubic.

func (*HubicAuth) CdnUrl added in v0.5.0 

func (h *HubicAuth) CdnUrl() string

CdnUrl retrieves the CDN URL from the authentication response.

func (*HubicAuth) Request added in v0.5.0 

func (h *HubicAuth) Request(*swift.Connection) (*http.Request, error)

Request constructs the authentication request.

func (*HubicAuth) Response added in v0.5.0 

func (h *HubicAuth) Response(*http.Response) error

Response reads the authentication response.

func (*HubicAuth) StorageUrl added in v0.5.0 

func (h *HubicAuth) StorageUrl(Internal bool) string

StorageUrl retrieves the swift's storage URL from the authentication response.

func (*HubicAuth) Token added in v0.5.0 

func (h *HubicAuth) Token() string

Token retrieves keystone token from the authentication response.

type Lister added in v0.5.2 

type Lister struct {
	// contains filtered or unexported fields
}

Lister is a concurrent processor of direntries. Its job is to get extra information about files.

func (*Lister) AddTask added in v0.5.2 

func (dl *Lister) AddTask(n Node, rc chan Node)

AddTask asynchronously adds a new task to be processed. It returns immediately with no guarantee that the task has been added to the channel nor retrieved by a worker.

func (*Lister) Start added in v0.5.2 

func (dl *Lister) Start()

Start spawns workers waiting for tasks. Once a task comes in the task channel, one worker will process it by opening a connection to swift and asking information about the current object.

type ListerTask added in v0.5.2 

type ListerTask struct {
	// contains filtered or unexported fields
}

ListerTask represents a manifest ready to be processed by the Lister. Every task must provide a manifest object and a result channel to which retrieved information will be sent.

type Node 

type Node interface {
	Name() string
	Export() fuse.Dirent
}

Node is the generic interface of an SVFS node.

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object is a node representing a swift object. It belongs to a container and segmented objects are bound to a container of segments.

func (*Object) Attr 

func (o *Object) Attr(ctx context.Context, a *fuse.Attr) (err error)

Attr fills the file attributes for an object node.

func (*Object) Export 

func (o *Object) Export() fuse.Dirent

Export converts this object node as a direntry.

func (*Object) Name 

func (o *Object) Name() string

Name gets the name of the underlying swift object.

func (*Object) Open 

func (o *Object) Open(ctx context.Context, req *fuse.OpenRequest, resp *fuse.OpenResponse) (fs.Handle, error)

Open returns the file handle associated with this object node.

func (*Object) Setattr added in v0.5.2 

func (o *Object) Setattr(ctx context.Context, req *fuse.SetattrRequest, resp *fuse.SetattrResponse) error

type ObjectHandle 

type ObjectHandle struct {
	// contains filtered or unexported fields
}

ObjectHandle represents an open object handle, similarly to file handles.

func (*ObjectHandle) Read 

func (fh *ObjectHandle) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) (err error)

Read gets a swift object data for a request within the current context. The request size is always honored. We open the file on the first write.

func (*ObjectHandle) Release 

func (fh *ObjectHandle) Release(ctx context.Context, req *fuse.ReleaseRequest) error

Release frees the file handle, closing all readers/writers in use.

func (*ObjectHandle) Write 

func (fh *ObjectHandle) Write(ctx context.Context, req *fuse.WriteRequest, resp *fuse.WriteResponse) (err error)

Write pushes data to a swift object. If we detect that we are writing more data than the configured segment size, then the first object we were writing to is moved to the segment container and named accordingly to DLO conventions. Remaining data will be split into segments sequentially until file handle release is called. If we are overwriting an object we handle segment deletion, and object creation.

type Root 

type Root struct {
	*Directory
}

Root is a fake root node used to hold a list of container nodes.

func (*Root) Create 

func (r *Root) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.CreateResponse) (fs.Node, fs.Handle, error)

Create is not supported on a root node.

func (*Root) Lookup added in v0.2.5 

func (r *Root) Lookup(ctx context.Context, req *fuse.LookupRequest, resp *fuse.LookupResponse) (fs.Node, error)

Lookup gets a container node if its name matches the request name within the current context.

func (*Root) Mkdir added in v0.2.0 

func (r *Root) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error)

Mkdir is not supported on a root node.

func (*Root) ReadDirAll 

func (r *Root) ReadDirAll(ctx context.Context) (direntries []fuse.Dirent, err error)

ReadDirAll retrieves all containers within the current Openstack tenant, as direntries. Segment containers are not shown and created if missing.

func (*Root) Remove 

func (r *Root) Remove(ctx context.Context, req *fuse.RemoveRequest) error

Remove is not supported on a root node.

func (*Root) Rename 

func (r *Root) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error

Rename is not supported on a root node.

type SVFS 

type SVFS struct{}

SVFS implements the Swift Virtual File System.

func (*SVFS) Init 

func (s *SVFS) Init() (err error)

Init sets up the filesystem. It sets configuration settings, starts mandatory services and make sure authentication in Swift has succeeded.

func (*SVFS) Root 

func (s *SVFS) Root() (fs.Node, error)

Root gets the root node of the filesystem. It can either be a fake root node filled with all the containers found for the given Openstack tenant or a container node if a container name have been specified in mount options.

type SimpleCache added in v0.5.2 

type SimpleCache struct {
	// contains filtered or unexported fields
}

SimpleCache is a simplistic caching implementation only relying on a hashmap with basic functions.

func NewSimpleCache added in v0.5.2 

func NewSimpleCache() *SimpleCache

NewSimpleCache creates a new simplistic cache.

func (*SimpleCache) Add added in v0.5.2 

func (c *SimpleCache) Add(container, path string, node Node)

Add pushes a new cache entry.

func (*SimpleCache) Exist added in v0.5.2 

func (c *SimpleCache) Exist(container, path string) bool

Exist checks whether a cache key exist or not.

func (*SimpleCache) Get added in v0.5.2 

func (c *SimpleCache) Get(container, path string) Node

Get retrieves a cache entry for the given key.

func (*SimpleCache) Remove added in v0.5.2 

func (c *SimpleCache) Remove(container, path string)

Remove pops the cache entry at this key.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.