README
¶
VFS for GoLang
vfs is GO library to support Virtual Filesystems. It provides the basic
abstractions of filesystems and implementations, like:
OSaccessing the file system of the underlying OS,memfsa full filesystem in-memory, anddummywhich does nothing other than outputting what file operation was called without actually modifiying the underlying file system.
Usage
$ go get github.com/lordofscripts/vfs
Note: Always vendor your dependencies or fix on a specific version tag.
import github.com/lordofscripts/vfs
// Create a vfs accessing the filesystem of the underlying OS
var osfs vfs.Filesystem = vfs.OS()
osfs.Mkdir("/tmp", 0777)
// Make the filesystem read-only:
osfs = vfs.ReadOnly(osfs) // Simply wrap filesystems to change its behaviour
// os.O_CREATE will fail and return vfs.ErrReadOnly
// os.O_RDWR is supported but Write(..) on the file is disabled
f, _ := osfs.OpenFile("/tmp/example.txt", os.O_RDWR, 0)
// Return vfs.ErrReadOnly
_, err := f.Write([]byte("Write on readonly fs?"))
if err != nil {
fmt.Errorf("Filesystem is read only!\n")
}
// Create a fully writable filesystem in memory
mfs := memfs.Create()
mfs.Mkdir("/root", 0777)
// Create a vfs supporting mounts
// The root fs is accessing the filesystem of the underlying OS
fs := mountfs.Create(osfs)
// Mount a memfs inside /memfs
// /memfs may not exist
fs.Mount(mfs, "/memfs")
// This will create /testdir inside the memfs
fs.Mkdir("/memfs/testdir", 0777)
// This would create /tmp/testdir inside your OS fs
// But the rootfs `osfs` is read-only
fs.Mkdir("/tmp/testdir", 0777)
// Now use a BitBucket Filesystem in Silent mode
fsb1 := bucketfs.Create()
fsb1.Mkdir("/bucket/testdir", 0777))
// Or an extended BitBucket Filesystem
ErrCustom := errors.New("A BitBucket error"))
fsb2 := bucketfs.CreateWithError(ErrCustom).
WithFakeDirs([]string{"/bucket/Dir1", "/bucket/Dir2"}).
WithFakeFiles([]string{"/bucket/Dir1/test.doc", "/bucket/Dir2/test.pdf"})
entries, err := fsb2.ReadDir("/bucket")
Check detailed examples below. Also check the GoDocs. The BucketFS can also be made read-only in the same manner as the first example above. See Detailed documentation about BucketVFS.)
Why should I use this package?
- Pure unadulterated GO
- (Nearly) Fully tested
- Easy to create your own filesystem
- Mock a full filesystem for testing (or use included
memfsorbucketfs) - Compose/Wrap Filesystems
ReadOnly(OS())and write simple Wrappers - Flexible BitBucket filesystem
Features and Examples
Many features, see GoDocs and examples below
- OS Filesystem support
- ReadOnly Wrapper
- DummyFS for quick mocking
- MemFS - full in-memory filesystem
- MountFS - support mounts across filesystems
Current state: RELEASE
While the functionality is quite stable and heavily tested, interfaces are subject to change.
You need more/less abstraction? Let me know by creating a Issue, thank you.
Motivation
The original author Benedikt Lang wrote:
I simply couldn't find any lib supporting this wide range of variation and adaptability.
And I (LordOfScripts) share his thoughts. In fact, I evaluated several similar GO libraries but many were too bloated and included other appendages I was not interested in. I loved this VFS version because it had no other dependencies.
What's Different (History)?
The original GO-VFS repository was github.com/blang/vfs by
Benedikt Lang). Which stopped at v1.0.0 about
nine (9) years ago. It was a quite mature implementation which I was actually
using in one of my projects. But it came short in emulating a Dry Run. I ended
up writing my own quick & grey Dry Run
for that purpose, and it worked quite well.
Since I was still using blang/vfs in the Unit Tests (only), I figured I might
as well invest some time and integrate it fully to replace my Dry Run as a
mere exercise. I started looking for better-fitting alternatives. There were none,
but there were about 50+ forks of the original blang/vfs but all where just
abandoned forks with no extra commits (people's fake trophies?). In that long list
I came across the single fork that had actually extra commits and extra
functionality.
So, I discovered 3JoB's fork tagged wrongly v1.0.0 when it should have been
v1.1.0 because it built on the original with added functionality:
- Support for Symbolic Links
- Support for Walk function
- Minor changes like using
anyinstead ofinterface{}
But 3JoB/vfs's version didn't quite meet my requirements. While the original version dated from 9 years ago, 3JoB's last commit was a year ago, but it seemed to have ended there too. I decided to fork that and enhance it:
Is this a YAUF (Yet-Another-Useless-Fork)? well, no! I plan on making certain enhancements that would make it suitable for my application out-of-the-box without the need for glue structures. So, Keep tuned! But to start with:
- Updated it to use
mainas branch instead of the deprecatedmaster - Added
go.mod - Included a GO workflow for building.
- Has a flexible BitBucket Filesystem
bucketfsmore suitable for testing - Bumped version to
v1.2.0to properly reflect new functionality beyond Benedikt's & 3JoB's.
In due time more changes will come.
License
See LICENSE file.
Documentation
¶
Overview ¶
Package vfs defines an abstract file system and various implementations.
----------------------------------------------------------------- * L o r d O f S c r i p t s (tm) * Copyright (C)2024 Dídimo Grimaldo T. * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * *----------------------------------------------------------------- ----------------------------------------------------------------- * L o r d O f S c r i p t s (tm) * Copyright (C)2024 Dídimo Grimaldo T. * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * *-----------------------------------------------------------------
Example ¶
package main
import (
"errors"
"os"
"github.com/lordofscripts/vfs"
"github.com/lordofscripts/vfs/memfs"
"github.com/lordofscripts/vfs/mountfs"
)
func main() {
// Create a vfs accessing the filesystem of the underlying OS
var osfs vfs.Filesystem = vfs.OS()
osfs.Mkdir("/tmp", 0777)
// Make the filesystem read-only:
osfs = vfs.ReadOnly(osfs) // Simply wrap filesystems to change its behaviour
// os.O_CREATE will fail and return vfs.ErrReadOnly
// os.O_RDWR is supported but Write(..) on the file is disabled
f, _ := osfs.OpenFile("/tmp/example.txt", os.O_RDWR, 0)
// Return vfs.ErrReadOnly
_, err := f.Write([]byte("Write on readonly fs?"))
if err != nil {
panic(errors.New("filesystem is read only!\n"))
}
// Create a fully writable filesystem in memory
mfs := memfs.Create()
mfs.Mkdir("/root", 0777)
// Create a vfs supporting mounts
// The root fs is accessing the filesystem of the underlying OS
fs := mountfs.Create(osfs)
// Mount a memfs inside /memfs
// /memfs may not exist
fs.Mount(mfs, "/memfs")
// This will create /testdir inside the memfs
fs.Mkdir("/memfs/testdir", 0777)
// This would create /tmp/testdir inside your OS fs
// But the rootfs `osfs` is read-only
fs.Mkdir("/tmp/testdir", 0777)
}
Index ¶
- Variables
- func CleanPath(path string) string
- func HasFileModeFlag(flag os.FileMode, flags os.FileMode) bool
- func HasModeFlag(flag int, flags int) bool
- func MkdirAll(fs Filesystem, path string, perm os.FileMode) error
- func ReadFile(fs Filesystem, filename string) ([]byte, error)
- func RemoveAll(fs Filesystem, path string) error
- func SplitPath(path string, sep string) []string
- func Walk(fs Filesystem, root string, walkFunc filepath.WalkFunc) error
- func WriteFile(fs Filesystem, filename string, data []byte, perm os.FileMode) error
- type DumFile
- func (f DumFile) Close() error
- func (f DumFile) Name() string
- func (f DumFile) Read(p []byte) (n int, err error)
- func (f DumFile) ReadAt(p []byte, off int64) (n int, err error)
- func (f DumFile) Seek(offset int64, whence int) (int64, error)
- func (f DumFile) Sync() error
- func (f DumFile) Truncate(size int64) error
- func (f DumFile) Write(p []byte) (n int, err error)
- type DumFileInfo
- type DummyFS
- func (fs DummyFS) Lstat(name string) (os.FileInfo, error)
- func (fs DummyFS) Mkdir(name string, perm os.FileMode) error
- func (fs DummyFS) MkdirAll(path string, perm os.FileMode) error
- func (fs DummyFS) Open(name string) (File, error)
- func (fs DummyFS) OpenFile(name string, flag int, perm os.FileMode) (File, error)
- func (fs DummyFS) PathSeparator() uint8
- func (fs DummyFS) ReadDir(path string) ([]os.FileInfo, error)
- func (fs DummyFS) Remove(name string) error
- func (fs DummyFS) RemoveAll(path string) error
- func (fs DummyFS) Rename(oldpath, newpath string) error
- func (fs DummyFS) Stat(name string) (os.FileInfo, error)
- func (fs DummyFS) Symlink(oldname, newname string) error
- type File
- type Filesystem
- type OsFS
- func (fs OsFS) Lstat(name string) (os.FileInfo, error)
- func (fs OsFS) Mkdir(name string, perm os.FileMode) error
- func (fs OsFS) MkdirAll(path string, perm os.FileMode) error
- func (fs OsFS) Open(name string) (File, error)
- func (fs OsFS) OpenFile(name string, flag int, perm os.FileMode) (File, error)
- func (fs OsFS) PathSeparator() uint8
- func (fs OsFS) ReadDir(path string) ([]os.FileInfo, error)
- func (fs OsFS) Remove(name string) error
- func (fs OsFS) RemoveAll(name string) error
- func (fs OsFS) Rename(oldpath, newpath string) error
- func (fs OsFS) Stat(name string) (os.FileInfo, error)
- func (fs OsFS) Symlink(oldname, newname string) error
- type Permission
- type RoFS
- func (fs RoFS) Mkdir(name string, perm os.FileMode) error
- func (fs RoFS) Open(name string) (File, error)
- func (fs RoFS) OpenFile(name string, flag int, perm os.FileMode) (File, error)
- func (fs RoFS) Remove(name string) error
- func (fs RoFS) Rename(oldpath, newpath string) error
- func (fs RoFS) Symlink(oldname, newname string) error
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrIsDirectory is returned if a file is a directory ErrIsDirectory = errors.New("is directory") // ErrNotDirectory is returned if a file is not a directory ErrNotDirectory = errors.New("is not a directory") )
var ErrReadOnly = errors.New("Filesystem is read-only")
ErrorReadOnly is returned on every disabled operation.
Functions ¶
func CleanPath ¶ added in v1.2.0
CleanPath trims leading/trailing white space, does filepath.Clean() and expands ~/ to the Home Directory of the current user.
func HasFileModeFlag ¶ added in v1.2.0
HasFileModeFlag checks if the 'flag' is set in the 'flags' value. Example: HasFileModeFlag(os.ModeSymlink, os.ModeSymLink|0666) returns true
func HasModeFlag ¶ added in v1.2.0
HasFileModeFlag checks if the 'flag' is set in the 'flags' value.
func MkdirAll ¶
func MkdirAll(fs Filesystem, path string, perm os.FileMode) error
MkdirAll creates a directory named path on the given Filesystem, along with any necessary parents, and returns nil, or else returns an error. The permission bits perm are used for all directories that MkdirAll creates. If path is already a directory, MkdirAll does nothing and returns nil.
func ReadFile ¶
func ReadFile(fs Filesystem, filename string) ([]byte, error)
ReadFile reads the file named by filename and returns the contents. A successful call returns err == nil, not err == EOF. Because ReadFile reads the whole file, it does not treat an EOF from Read as an error to be reported.
This is a port of the stdlib ioutil.ReadFile function.
func RemoveAll ¶
func RemoveAll(fs Filesystem, path string) error
RemoveAll removes path and any children it contains. It removes everything it can but returns the first error it encounters. If the path does not exist, RemoveAll returns nil.
func SplitPath ¶
SplitPath splits the given path in segments:
"/" -> []string{""}
"./file" -> []string{".", "file"}
"file" -> []string{".", "file"}
"/usr/src/linux/" -> []string{"", "usr", "src", "linux"}
The returned slice of path segments consists of one more more segments.
func Walk ¶
func Walk(fs Filesystem, root string, walkFunc filepath.WalkFunc) error
Walk walks the file tree rooted at root, calling walkFunc for each file or directory in the tree, including root. All errors that arise visiting files and directories are filtered by walkFn. The files are walked in lexical order, which makes the output deterministic but means that for very large directories Walk can be inefficient. Walk does not follow symbolic links.
Types ¶
type DumFile ¶
type DumFile struct {
// contains filtered or unexported fields
}
DumFile represents a dummy File
func DummyFile ¶
DummyFile mocks a File returning an error on every operation To create a DummyFS returning a dummyFile instead of an error you can your own DummyFS:
type writeDummyFS struct {
Filesystem
}
func (fs writeDummyFS) OpenFile(name string, flag int, perm os.FileMode) (File, error) {
return DummyFile(dummyError), nil
}
type DumFileInfo ¶
type DumFileInfo struct {
IName string
ISize int64
IMode os.FileMode
IModTime time.Time
IDir bool
ISys any
}
DumFileInfo mocks a os.FileInfo returning default values on every operation Struct fields can be set.
func (DumFileInfo) ModTime ¶
func (fi DumFileInfo) ModTime() time.Time
ModTime returns the field IModTime
type DummyFS ¶
type DummyFS struct {
// contains filtered or unexported fields
}
DummyFS is dummy filesystem which returns an error on every operation. It can be used to mock a full filesystem for testing or fs creation.
Example ¶
package main
import (
"errors"
"fmt"
"os"
"github.com/lordofscripts/vfs"
)
type myFS struct {
vfs.Filesystem // Embed the Filesystem interface and fill it with vfs.Dummy on creation
}
func MyFS() *myFS {
return &myFS{
Filesystem: vfs.Dummy(errors.New("Not implemented yet!")),
}
}
func (fs myFS) Mkdir(name string, perm os.FileMode) error {
// Create a directory
// ...
return nil
}
func main() {
// Simply bootstrap your filesystem
var fs vfs.Filesystem = MyFS()
// Your mkdir implementation
fs.Mkdir("/tmp", 0777)
// All necessary methods like OpenFile (therefor Create) are stubbed
// and return the dummys error
_, err := vfs.Create(fs, "/tmp/vfs/example.txt")
if err != nil {
fmt.Print("Error will be: Not implemented yet!\n")
}
}
func (DummyFS) PathSeparator ¶
PathSeparator returns the path separator
type File ¶
type File interface {
Name() string
Sync() error
// Truncate shrinks or extends the size of the File to the specified size.
Truncate(int64) error
io.Reader
io.ReaderAt
io.Writer
io.Seeker
io.Closer
}
File represents a File with common operations. It differs from os.File so e.g. Stat() needs to be called from the Filesystem instead.
osfile.Stat() -> filesystem.Stat(file.Name())
func Create ¶
func Create(fs Filesystem, name string) (File, error)
Create creates the named file mode 0666 (before umask) on the given Filesystem, truncating it if it already exists. The associated file descriptor has mode os.O_RDWR. If there is an error, it will be of type *os.PathError.
func Open ¶
func Open(fs Filesystem, name string) (File, error)
Open opens the named file on the given Filesystem for reading. If successful, methods on the returned file can be used for reading. The associated file descriptor has mode os.O_RDONLY. If there is an error, it will be of type *PathError.
func ReadOnlyFile ¶
ReadOnlyFile wraps the given file and disables Write(..) operation.
type Filesystem ¶
type Filesystem interface {
PathSeparator() uint8
OpenFile(name string, flag int, perm os.FileMode) (File, error)
Remove(name string) error
RemoveAll(path string) error
Rename(oldpath, newpath string) error
Mkdir(name string, perm os.FileMode) error
MkdirAll(path string, perm os.FileMode) error
Symlink(oldname, newname string) error
// TempDir() string
// Chmod(name string, mode FileMode) error
// Chown(name string, uid, gid int) error
Stat(name string) (os.FileInfo, error)
Lstat(name string) (os.FileInfo, error)
ReadDir(path string) ([]os.FileInfo, error)
}
Filesystem represents an abstract filesystem
type OsFS ¶
type OsFS struct{}
OsFS represents a filesystem backed by the filesystem of the underlying OS.
Example ¶
package main
import (
"fmt"
"github.com/lordofscripts/vfs"
)
func main() {
// Create a vfs accessing the filesystem of the underlying OS
osFS := vfs.OS()
err := osFS.Mkdir("/tmp/vfs_example", 0777)
if err != nil {
fmt.Printf("Error creating directory: %s\n", err)
}
// Convenience method
f, err := vfs.Create(osFS, "/tmp/vfs_example/example.txt")
// f, err := osFS.OpenFile("/tmp/vfs/example.txt", os.O_CREATE|os.O_RDWR, 0666)
if err != nil {
fmt.Printf("Could not create file: %s\n", err)
}
defer f.Close()
if _, err := f.Write([]byte("VFS working on your filesystem")); err != nil {
fmt.Printf("Error writing to file: %s\n", err)
}
}
Example (MyWrapper) ¶
package main
import (
"errors"
"fmt"
"os"
"github.com/lordofscripts/vfs"
)
type noNewDirs struct {
vfs.Filesystem
}
func NoNewDirs(fs vfs.Filesystem) *noNewDirs {
return &noNewDirs{Filesystem: fs}
}
// Mkdir is disabled
func (fs *noNewDirs) Mkdir(name string, perm os.FileMode) error {
return errors.New("Mkdir disabled!")
}
func main() {
// Disable Mkdirs on the OS Filesystem
var fs vfs.Filesystem = NoNewDirs(vfs.OS())
err := fs.Mkdir("/tmp", 0777)
if err != nil {
fmt.Print("Mkdir disabled!\n")
}
}
func OS ¶
func OS() *OsFS
OS returns a filesystem backed by the filesystem of the os. It wraps os.* stdlib operations.
func (OsFS) PathSeparator ¶
PathSeparator returns the path separator
type Permission ¶ added in v1.2.0
type Permission int
func (Permission) Flags ¶ added in v1.2.0
func (pe Permission) Flags() int
Flags returns the permission value with its PrimaryMode bits set to zero.
func (Permission) PrimaryMode ¶ added in v1.2.0
func (pe Permission) PrimaryMode() int
PrimaryMode returns only the permission part corresponding to either of: os.O_RDWR, os.O_RDONLY or os.O_WRONLY
func (Permission) String ¶ added in v1.2.0
func (pe Permission) String() string
type RoFS ¶
type RoFS struct {
Filesystem
}
RoFS represents a read-only filesystem and works as a wrapper around existing filesystems.
Example ¶
Every vfs.Filesystem could be easily wrapped
package main
import (
"fmt"
"os"
"github.com/lordofscripts/vfs"
)
func main() {
// Create a readonly vfs accessing the filesystem of the underlying OS
roFS := vfs.ReadOnly(vfs.OS())
// Mkdir is disabled on ReadOnly vfs, will return vfs.ErrReadOnly
// See vfs.ReadOnly for all disabled operations
err := roFS.Mkdir("/tmp/vfs_example", 0777)
if err != nil {
fmt.Printf("Error creating directory: %s\n", err)
return
}
// OpenFile is controlled to support read-only functionality. os.O_CREATE or os.O_APPEND will fail.
// Flags like os.O_RDWR are supported but the returned file is protected e.g. from Write(..).
f, err := roFS.OpenFile("/tmp/vfs_example/example.txt", os.O_RDWR, 0)
if err != nil {
fmt.Printf("Could not create file: %s\n", err)
return
}
defer f.Close()
// Will fail and return vfs.ErrReadOnly
_, err = f.Write([]byte("VFS working on your filesystem"))
if err != nil {
fmt.Printf("Could not write file on read only filesystem: %s", err)
return
}
}
func ReadOnly ¶
func ReadOnly(fs Filesystem) *RoFS
ReadOnly creates a readonly wrapper around the given filesystem. It disables the following operations:
- Create
- Remove
- Rename
- Mkdir
And disables OpenFile flags: os.O_CREATE, os.O_APPEND, os.O_WRONLY
OpenFile returns a File with disabled Write() method otherwise.
func (RoFS) Open ¶
Open opens the named file on the given Filesystem for reading. If successful, methods on the returned file can be used for reading. The associated file descriptor has mode os.O_RDONLY. If there is an error, it will be of type *PathError.
func (RoFS) OpenFile ¶
OpenFile returns ErrorReadOnly if flag contains os.O_CREATE, os.O_APPEND, os.O_WRONLY. Otherwise it returns a read-only File with disabled Write(..) operation.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package memfs defines an in-memory filesystem
|
Package memfs defines an in-memory filesystem |
|
Package mountfs defines a filesystem supporting the composition of multiple filesystems by mountpoints.
|
Package mountfs defines a filesystem supporting the composition of multiple filesystems by mountpoints. |
|
----------------------------------------------------------------- * L o r d O f S c r i p t s (tm) * Copyright (C)2024 Dídimo Grimaldo T. * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * Library of Test Utilities *-----------------------------------------------------------------
|
----------------------------------------------------------------- * L o r d O f S c r i p t s (tm) * Copyright (C)2024 Dídimo Grimaldo T. * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * Library of Test Utilities *----------------------------------------------------------------- |