vfs

package
v0.0.0-...-9964dcd Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 7, 2020 License: GPL-3.0 Imports: 29 Imported by: 0

Documentation

Overview

Package vfs provides local and remote filesystems support

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetSFTPError 

func GetSFTPError(fs Fs, err error) error

GetSFTPError returns an sftp error from a filesystem error

func IsDirectory 

func IsDirectory(fs Fs, path string) (bool, error)

IsDirectory checks if a path exists and is a directory

func IsLocalOsFs 

func IsLocalOsFs(fs Fs) bool

IsLocalOsFs returns true if fs is the local filesystem implementation

func SetPathPermissions 

func SetPathPermissions(fs Fs, path string, uid int, gid int)

SetPathPermissions calls fs.Chown. It does nothing for local filesystem on windows

func ValidateGCSFsConfig 

func ValidateGCSFsConfig(config *GCSFsConfig, credentialsFilePath string) error

ValidateGCSFsConfig returns nil if the specified GCS config is valid, otherwise an error

func ValidateS3FsConfig 

func ValidateS3FsConfig(config *S3FsConfig) error

ValidateS3FsConfig returns nil if the specified s3 config is valid, otherwise an error

Types

type FileInfo 

type FileInfo struct {
	// contains filtered or unexported fields
}

FileInfo implements os.FileInfo for a file in S3.

func NewFileInfo 

func NewFileInfo(name string, isDirectory bool, sizeInBytes int64, modTime time.Time) FileInfo

NewFileInfo creates file info.

func (FileInfo) IsDir 

func (fi FileInfo) IsDir() bool

IsDir provides the abbreviation for Mode().IsDir()

func (FileInfo) ModTime 

func (fi FileInfo) ModTime() time.Time

ModTime provides the last modification time.

func (FileInfo) Mode 

func (fi FileInfo) Mode() os.FileMode

Mode provides the file mode bits

func (FileInfo) Name 

func (fi FileInfo) Name() string

Name provides the base name of the file.

func (FileInfo) Size 

func (fi FileInfo) Size() int64

Size provides the length in bytes for a file.

func (FileInfo) Sys 

func (fi FileInfo) Sys() interface{}

Sys provides the underlying data source (can return nil)

type Fs 

type Fs interface {
	Name() string
	ConnectionID() string
	Stat(name string) (os.FileInfo, error)
	Lstat(name string) (os.FileInfo, error)
	Open(name string) (*os.File, *pipeat.PipeReaderAt, func(), error)
	Create(name string, flag int) (*os.File, *pipeat.PipeWriterAt, func(), error)
	Rename(source, target string) error
	Remove(name string, isDir bool) error
	Mkdir(name string) error
	Symlink(source, target string) error
	Chown(name string, uid int, gid int) error
	Chmod(name string, mode os.FileMode) error
	Chtimes(name string, atime, mtime time.Time) error
	ReadDir(dirname string) ([]os.FileInfo, error)
	IsUploadResumeSupported() bool
	IsAtomicUploadSupported() bool
	CheckRootPath(username string, uid int, gid int) bool
	ResolvePath(sftpPath string) (string, error)
	IsNotExist(err error) bool
	IsPermission(err error) bool
	ScanRootDirContents() (int, int64, error)
	GetAtomicUploadPath(name string) string
	GetRelativePath(name string) string
	Join(elem ...string) string
}

Fs defines the interface for filesystem backends

func NewGCSFs 

func NewGCSFs(connectionID, localTempDir string, config GCSFsConfig) (Fs, error)

NewGCSFs returns an GCSFs object that allows to interact with Google Cloud Storage

func NewOsFs 

func NewOsFs(connectionID, rootDir string) Fs

NewOsFs returns an OsFs object that allows to interact with local Os filesystem

func NewS3Fs 

func NewS3Fs(connectionID, localTempDir string, config S3FsConfig) (Fs, error)

NewS3Fs returns an S3Fs object that allows to interact with an s3 compatible object storage

type GCSFs 

type GCSFs struct {
	// contains filtered or unexported fields
}

GCSFs is a Fs implementation for Google Cloud Storage.

func (GCSFs) CheckRootPath 

func (fs GCSFs) CheckRootPath(username string, uid int, gid int) bool

CheckRootPath creates the specified root directory if it does not exists

func (GCSFs) Chmod 

func (GCSFs) Chmod(name string, mode os.FileMode) error

Chmod changes the mode of the named file to mode. Silently ignored.

func (GCSFs) Chown 

func (GCSFs) Chown(name string, uid int, gid int) error

Chown changes the numeric uid and gid of the named file. Silently ignored.

func (GCSFs) Chtimes 

func (GCSFs) Chtimes(name string, atime, mtime time.Time) error

Chtimes changes the access and modification times of the named file. Silently ignored.

func (GCSFs) ConnectionID 

func (fs GCSFs) ConnectionID() string

ConnectionID returns the SSH connection ID associated to this Fs implementation

func (GCSFs) Create 

func (fs GCSFs) Create(name string, flag int) (*os.File, *pipeat.PipeWriterAt, func(), error)

Create creates or opens the named file for writing

func (GCSFs) GetAtomicUploadPath 

func (GCSFs) GetAtomicUploadPath(name string) string

GetAtomicUploadPath returns the path to use for an atomic upload. S3 uploads are already atomic, we never call this method for S3

func (GCSFs) GetRelativePath 

func (fs GCSFs) GetRelativePath(name string) string

GetRelativePath returns the path for a file relative to the user's home dir. This is the path as seen by SFTP users

func (GCSFs) IsAtomicUploadSupported 

func (GCSFs) IsAtomicUploadSupported() bool

IsAtomicUploadSupported returns true if atomic upload is supported. S3 uploads are already atomic, we don't need to upload to a temporary file

func (GCSFs) IsNotExist 

func (GCSFs) IsNotExist(err error) bool

IsNotExist returns a boolean indicating whether the error is known to report that a file or directory does not exist

func (GCSFs) IsPermission 

func (GCSFs) IsPermission(err error) bool

IsPermission returns a boolean indicating whether the error is known to report that permission is denied.

func (GCSFs) IsUploadResumeSupported 

func (GCSFs) IsUploadResumeSupported() bool

IsUploadResumeSupported returns true if upload resume is supported. SFTP Resume is not supported on S3

func (GCSFs) Join 

func (GCSFs) Join(elem ...string) string

Join joins any number of path elements into a single path

func (GCSFs) Lstat 

func (fs GCSFs) Lstat(name string) (os.FileInfo, error)

Lstat returns a FileInfo describing the named file

func (GCSFs) Mkdir 

func (fs GCSFs) Mkdir(name string) error

Mkdir creates a new directory with the specified name and default permissions

func (GCSFs) Name 

func (fs GCSFs) Name() string

Name returns the name for the Fs implementation

func (GCSFs) Open 

func (fs GCSFs) Open(name string) (*os.File, *pipeat.PipeReaderAt, func(), error)

Open opens the named file for reading

func (GCSFs) ReadDir 

func (fs GCSFs) ReadDir(dirname string) ([]os.FileInfo, error)

ReadDir reads the directory named by dirname and returns a list of directory entries.

func (GCSFs) Remove 

func (fs GCSFs) Remove(name string, isDir bool) error

Remove removes the named file or (empty) directory.

func (GCSFs) Rename 

func (fs GCSFs) Rename(source, target string) error

Rename renames (moves) source to target. We don't support renaming non empty directories since we should rename all the contents too and this could take long time: think about directories with thousands of files, for each file we should execute a CopyObject call.

func (GCSFs) ResolvePath 

func (fs GCSFs) ResolvePath(sftpPath string) (string, error)

ResolvePath returns the matching filesystem path for the specified sftp path

func (GCSFs) ScanRootDirContents 

func (fs GCSFs) ScanRootDirContents() (int, int64, error)

ScanRootDirContents returns the number of files contained in the bucket, and their size

func (GCSFs) Stat 

func (fs GCSFs) Stat(name string) (os.FileInfo, error)

Stat returns a FileInfo describing the named file 

func (GCSFs) Symlink(source, target string) error

Symlink creates source as a symbolic link to target.

type GCSFsConfig 

type GCSFsConfig struct {
	Bucket string `json:"bucket,omitempty"`
	// KeyPrefix is similar to a chroot directory for local filesystem.
	// If specified the SFTP user will only see objects that starts with
	// this prefix and so you can restrict access to a specific virtual
	// folder. The prefix, if not empty, must not start with "/" and must
	// end with "/".
	// If empty the whole bucket contents will be available
	KeyPrefix      string `json:"key_prefix,omitempty"`
	CredentialFile string `json:"-"`
	Credentials    string `json:"credentials,omitempty"`
	StorageClass   string `json:"storage_class,omitempty"`
}

GCSFsConfig defines the configuration for Google Cloud Storage based filesystem

type OsFs 

type OsFs struct {
	// contains filtered or unexported fields
}

OsFs is a Fs implementation that uses functions provided by the os package.

func (OsFs) CheckRootPath 

func (fs OsFs) CheckRootPath(username string, uid int, gid int) bool

CheckRootPath creates the root directory if it does not exists

func (OsFs) Chmod 

func (OsFs) Chmod(name string, mode os.FileMode) error

Chmod changes the mode of the named file to mode

func (OsFs) Chown 

func (OsFs) Chown(name string, uid int, gid int) error

Chown changes the numeric uid and gid of the named file.

func (OsFs) Chtimes 

func (OsFs) Chtimes(name string, atime, mtime time.Time) error

Chtimes changes the access and modification times of the named file

func (OsFs) ConnectionID 

func (fs OsFs) ConnectionID() string

ConnectionID returns the SSH connection ID associated to this Fs implementation

func (OsFs) Create 

func (OsFs) Create(name string, flag int) (*os.File, *pipeat.PipeWriterAt, func(), error)

Create creates or opens the named file for writing

func (OsFs) GetAtomicUploadPath 

func (OsFs) GetAtomicUploadPath(name string) string

GetAtomicUploadPath returns the path to use for an atomic upload

func (OsFs) GetRelativePath 

func (fs OsFs) GetRelativePath(name string) string

GetRelativePath returns the path for a file relative to the user's home dir. This is the path as seen by SFTP users

func (OsFs) IsAtomicUploadSupported 

func (OsFs) IsAtomicUploadSupported() bool

IsAtomicUploadSupported returns true if atomic upload is supported

func (OsFs) IsNotExist 

func (OsFs) IsNotExist(err error) bool

IsNotExist returns a boolean indicating whether the error is known to report that a file or directory does not exist

func (OsFs) IsPermission 

func (OsFs) IsPermission(err error) bool

IsPermission returns a boolean indicating whether the error is known to report that permission is denied.

func (OsFs) IsUploadResumeSupported 

func (OsFs) IsUploadResumeSupported() bool

IsUploadResumeSupported returns true if upload resume is supported

func (OsFs) Join 

func (OsFs) Join(elem ...string) string

Join joins any number of path elements into a single path

func (OsFs) Lstat 

func (OsFs) Lstat(name string) (os.FileInfo, error)

Lstat returns a FileInfo describing the named file

func (OsFs) Mkdir 

func (OsFs) Mkdir(name string) error

Mkdir creates a new directory with the specified name and default permissions

func (OsFs) Name 

func (fs OsFs) Name() string

Name returns the name for the Fs implementation

func (OsFs) Open 

func (OsFs) Open(name string) (*os.File, *pipeat.PipeReaderAt, func(), error)

Open opens the named file for reading

func (OsFs) ReadDir 

func (OsFs) ReadDir(dirname string) ([]os.FileInfo, error)

ReadDir reads the directory named by dirname and returns a list of directory entries.

func (OsFs) Remove 

func (OsFs) Remove(name string, isDir bool) error

Remove removes the named file or (empty) directory.

func (OsFs) Rename 

func (OsFs) Rename(source, target string) error

Rename renames (moves) source to target

func (OsFs) ResolvePath 

func (fs OsFs) ResolvePath(sftpPath string) (string, error)

ResolvePath returns the matching filesystem path for the specified sftp path

func (OsFs) ScanRootDirContents 

func (fs OsFs) ScanRootDirContents() (int, int64, error)

ScanRootDirContents returns the number of files contained in a directory and their size

func (OsFs) Stat 

func (OsFs) Stat(name string) (os.FileInfo, error)

Stat returns a FileInfo describing the named file 

func (OsFs) Symlink(source, target string) error

Symlink creates source as a symbolic link to target.

type S3Fs 

type S3Fs struct {
	// contains filtered or unexported fields
}

S3Fs is a Fs implementation for Amazon S3 compatible object storage.

func (S3Fs) CheckRootPath 

func (fs S3Fs) CheckRootPath(username string, uid int, gid int) bool

CheckRootPath creates the specified root directory if it does not exists

func (S3Fs) Chmod 

func (S3Fs) Chmod(name string, mode os.FileMode) error

Chmod changes the mode of the named file to mode. Silently ignored.

func (S3Fs) Chown 

func (S3Fs) Chown(name string, uid int, gid int) error

Chown changes the numeric uid and gid of the named file. Silently ignored.

func (S3Fs) Chtimes 

func (S3Fs) Chtimes(name string, atime, mtime time.Time) error

Chtimes changes the access and modification times of the named file. Silently ignored.

func (S3Fs) ConnectionID 

func (fs S3Fs) ConnectionID() string

ConnectionID returns the SSH connection ID associated to this Fs implementation

func (S3Fs) Create 

func (fs S3Fs) Create(name string, flag int) (*os.File, *pipeat.PipeWriterAt, func(), error)

Create creates or opens the named file for writing

func (S3Fs) GetAtomicUploadPath 

func (S3Fs) GetAtomicUploadPath(name string) string

GetAtomicUploadPath returns the path to use for an atomic upload. S3 uploads are already atomic, we never call this method for S3

func (S3Fs) GetRelativePath 

func (fs S3Fs) GetRelativePath(name string) string

GetRelativePath returns the path for a file relative to the user's home dir. This is the path as seen by SFTP users

func (S3Fs) IsAtomicUploadSupported 

func (S3Fs) IsAtomicUploadSupported() bool

IsAtomicUploadSupported returns true if atomic upload is supported. S3 uploads are already atomic, we don't need to upload to a temporary file

func (S3Fs) IsNotExist 

func (S3Fs) IsNotExist(err error) bool

IsNotExist returns a boolean indicating whether the error is known to report that a file or directory does not exist

func (S3Fs) IsPermission 

func (S3Fs) IsPermission(err error) bool

IsPermission returns a boolean indicating whether the error is known to report that permission is denied.

func (S3Fs) IsUploadResumeSupported 

func (S3Fs) IsUploadResumeSupported() bool

IsUploadResumeSupported returns true if upload resume is supported. SFTP Resume is not supported on S3

func (S3Fs) Join 

func (S3Fs) Join(elem ...string) string

Join joins any number of path elements into a single path

func (S3Fs) Lstat 

func (fs S3Fs) Lstat(name string) (os.FileInfo, error)

Lstat returns a FileInfo describing the named file

func (S3Fs) Mkdir 

func (fs S3Fs) Mkdir(name string) error

Mkdir creates a new directory with the specified name and default permissions

func (S3Fs) Name 

func (fs S3Fs) Name() string

Name returns the name for the Fs implementation

func (S3Fs) Open 

func (fs S3Fs) Open(name string) (*os.File, *pipeat.PipeReaderAt, func(), error)

Open opens the named file for reading

func (S3Fs) ReadDir 

func (fs S3Fs) ReadDir(dirname string) ([]os.FileInfo, error)

ReadDir reads the directory named by dirname and returns a list of directory entries.

func (S3Fs) Remove 

func (fs S3Fs) Remove(name string, isDir bool) error

Remove removes the named file or (empty) directory.

func (S3Fs) Rename 

func (fs S3Fs) Rename(source, target string) error

Rename renames (moves) source to target. We don't support renaming non empty directories since we should rename all the contents too and this could take long time: think about directories with thousands of files, for each file we should execute a CopyObject call. TODO: rename does not work for files bigger than 5GB, implement multipart copy or wait for this pull request to be merged:

https://github.com/aws/aws-sdk-go/pull/2653

func (S3Fs) ResolvePath 

func (fs S3Fs) ResolvePath(sftpPath string) (string, error)

ResolvePath returns the matching filesystem path for the specified sftp path

func (S3Fs) ScanRootDirContents 

func (fs S3Fs) ScanRootDirContents() (int, int64, error)

ScanRootDirContents returns the number of files contained in the bucket, and their size

func (S3Fs) Stat 

func (fs S3Fs) Stat(name string) (os.FileInfo, error)

Stat returns a FileInfo describing the named file 

func (S3Fs) Symlink(source, target string) error

Symlink creates source as a symbolic link to target.

type S3FsConfig 

type S3FsConfig struct {
	Bucket string `json:"bucket,omitempty"`
	// KeyPrefix is similar to a chroot directory for local filesystem.
	// If specified the SFTP user will only see objects that starts with
	// this prefix and so you can restrict access to a specific virtual
	// folder. The prefix, if not empty, must not start with "/" and must
	// end with "/".
	// If empty the whole bucket contents will be available
	KeyPrefix    string `json:"key_prefix,omitempty"`
	Region       string `json:"region,omitempty"`
	AccessKey    string `json:"access_key,omitempty"`
	AccessSecret string `json:"access_secret,omitempty"`
	Endpoint     string `json:"endpoint,omitempty"`
	StorageClass string `json:"storage_class,omitempty"`
}

S3FsConfig defines the configuration for S3 based filesystem

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.