fs

package
v0.0.0-...-4a11b79 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 2, 2020 License: Apache-2.0 Imports: 17 Imported by: 0

Documentation

Overview

Package fs is file-system related utilities used internally by CIPD.

Index

Constants

View Source 
const (
	// SiteServiceDir is a name of the directory inside an installation root
	// reserved for cipd stuff.
	SiteServiceDir = ".cipd"
)

Variables

This section is empty.

Functions

func EnsureFile 

func EnsureFile(ctx context.Context, fs FileSystem, path string, content io.Reader) error

EnsureFile creates a file with the given content. It will create full directory path to the file if necessary.

func IsAccessDenied 

func IsAccessDenied(err error) bool

IsAccessDenied is true if err represents Windows' ERROR_ACCESS_DENIED.

Always false on Posix.

func IsCleanSlashPath 

func IsCleanSlashPath(p string) bool

IsCleanSlashPath returns true if path is a relative slash-separated path with no '..' or '.' entries and no '\\'. Basically "a/b/c/d".

func IsNotDir 

func IsNotDir(err error) bool

IsNotDir if true if err represents ENOTDIR or equivalent.

func IsNotEmpty 

func IsNotEmpty(err error) bool

IsNotEmpty is true if err represents ENOTEMPTY or equivalent.

func IsSubpath 

func IsSubpath(path, root string) bool

IsSubpath returns true if 'path' is 'root' or is inside a subdirectory of 'root'. Both 'path' and 'root' should be given as a native paths. If any of paths can't be converted to an absolute path returns false.

func TempDir 

func TempDir(dir string, prefix string, mode os.FileMode) (name string, err error)

TempDir is like ioutil.TempDir(dir, ""), but uses shorter path suffixes.

Path length is constraint resource of Windows.

Supposed to be used only in cases when the probability of a conflict is low (e.g. when 'dir' is some "private" directory, not global /tmp or something like that).

Additionally, this allows you to pass mode (which will respect the process umask). To get ioutils.TempDir behavior, pass 0700 for the mode.

Types

type CreateFileOptions 

type CreateFileOptions struct {
	// Executable makes the file executable.
	Executable bool
	// Writable makes the file user-writable.
	Writable bool
	// ModTime, when non-zero, sets the mtime of the file.
	ModTime time.Time
	// WinAttrs is used on Windows.
	WinAttrs WinAttrs
}

CreateFileOptions provides arguments to Destination.CreateFile().

type Destination 

type Destination interface {
	// CreateFile opens a writer to extract some package file to.
	CreateFile(ctx context.Context, name string, opts CreateFileOptions) (io.WriteCloser, error)
	// CreateSymlink creates a symlink (with absolute or relative target).
	CreateSymlink(ctx context.Context, name string, target string) error
}

Destination knows how to create files and symlink when extracting a package.

All paths are slash separated and relative to the destination root.

'CreateFile' and 'CreateSymlink' can be called concurrently.

func ExistingDestination 

func ExistingDestination(dest string, fs FileSystem) Destination

ExistingDestination returns an object that knows how to create files in an existing directory in the file system.

Will use a provided FileSystem object to operate on files if given, otherwise uses a default one. If FileSystem is provided, dest must be in a subdirectory of the given FileSystem root.

Note that the returned object doesn't support transactional semantics, since transactionally writing a bunch of files into an existing directory is hard.

See NewDestination for writing files into a completely new directory. This method supports transactions.

type File 

type File interface {
	// Name returns slash separated file path relative to a package root
	//
	// For example "dir/dir/file".
	Name() string

	// Size returns size of the file. 0 for symlinks.
	Size() uint64

	// Executable returns true if the file is executable.
	//
	// Only used for Linux\Mac archives. false for symlinks.
	Executable() bool

	// Writable returns true if the file is user-writable.
	Writable() bool

	// ModTime returns modification time of the file. Zero value means no mtime is
	// recorded.
	ModTime() time.Time

	// Symlink returns true if the file is a symlink.
	Symlink() bool

	// SymlinkTarget return a path the symlink is pointing to.
	SymlinkTarget() (string, error)

	// WinAttrs returns the windows attributes, if any.
	WinAttrs() WinAttrs

	// Open opens the regular file for reading.
	//
	// Returns error for symlink files.
	Open() (io.ReadCloser, error)
}

File defines a single file to be added or extracted from a package.

All paths are slash separated (including symlink targets).

func NewTestFile 

func NewTestFile(name string, data string, opts TestFileOpts) File

NewTestFile returns File implementation (Symlink == false) backed by a fake in-memory data. It is useful in unit tests. 

func NewTestSymlink(name string, target string) File

NewTestSymlink returns File implementation (Symlink == true) backed by a fake in-memory data. It is useful in unit tests.

func NewWinTestFile 

func NewWinTestFile(name string, data string, attrs WinAttrs) File

NewWinTestFile returns a File implementation (Symlink == false, Executable == false) backed by a fake in-memory data with windows attributes. It is useful in unit tests.

func ScanFileSystem 

func ScanFileSystem(dir string, root string, exclude ScanFilter, scanOpts ScanOptions) ([]File, error)

ScanFileSystem returns all files in some file system directory in alphabetical order.

It scans 'dir' path, returning File objects that have paths relative to 'root'. Returns only files, skipping directory entries (i.e. empty directories are completely "invisible").

It also skips files and directories for which 'exclude(path relative to dir)' returns true. It also always skips <root>/<SiteServiceDir>, but follows symlinks that point there.

Symlinks are preserved (see Symlink() method of File interface), but the following rules apply:

  • Relative symlinks pointing outside of the root are forbidden.
  • An absolute symlink with the target outside the root is kept as such.
  • An absolute symlink with the target within the root becomes relative.

If 'dir' is a symlink itself, it is emitted as such (with all above rules applying), except when 'dir == root' (lexicographically, as cleaned absolute paths). In that case the symlink is followed and whatever it points to is used as new 'dir' and 'root'. This is completely transparent in the output though, since File uses only relative paths. Without this exception using a symlink as a package root leads to very surprising errors.

func WrapFile 

func WrapFile(abs string, root string, fileInfo *os.FileInfo, scanOpts ScanOptions) (File, error)

WrapFile constructs File object for some file in the file system, specified by its native absolute path 'abs' (subpath of 'root', also specified as a native absolute path). Returned File object has path relative to 'root'. If fileInfo is given, it will be used to grab file mode and size, otherwise os.Lstat will be used to get it. Recognizes symlinks.

type FileSystem 

type FileSystem interface {
	// Root returns absolute path to a directory FileSystem operates in.
	//
	// All FS actions are restricted to this directory.
	Root() string

	// CaseSensitive returns true if the file system that has the root is
	// case-sensitive.
	CaseSensitive() (bool, error)

	// CwdRelToAbs converts a path relative to cwd to an absolute one.
	//
	// If also verifies the path is under the root path of the FileSystem object.
	// If passed path is already absolute, just checks that it's under the root.
	CwdRelToAbs(path string) (string, error)

	// RootRelToAbs converts a path relative to Root() to an absolute one.
	//
	// It verifies the path is under the root path of the FileSystem object.
	// If passed path is already absolute, just checks that it's under the root.
	RootRelToAbs(path string) (string, error)

	// OpenFile opens a file and returns its file handle.
	//
	// Files opened with OpenFile can be safely manipulated by other
	// FileSystem functions.
	//
	// This differs from os.Open notably on Windows, where OpenFile ensures that
	// files are open with FILE_SHARE_DELETE permission to enable them to be
	// atomically renamed without contention.
	OpenFile(path string) (*os.File, error)

	// Stat returns a FileInfo describing the named file, following symlinks.
	Stat(ctx context.Context, path string) (os.FileInfo, error)

	// Lstat returns a FileInfo describing the named file, not following symlinks.
	Lstat(ctx context.Context, path string) (os.FileInfo, error)

	// EnsureDirectory creates a directory at given native path.
	//
	// Follows symlinks. Does nothing it the path already exists and it is a
	// directory (or a symlink pointing to a directory). Replaces an existing file
	// (or a symlink to a file, or a broken symlink) with a directory.
	//
	// It takes an absolute path or a path relative to the current working
	// directory and always returns absolute path.
	EnsureDirectory(ctx context.Context, path string) (string, error)

	// EnsureSymlink creates a symlink pointing to a target.
	//
	// It will create full directory path to the symlink if necessary.
	EnsureSymlink(ctx context.Context, path string, target string) error

	// EnsureFile creates a file and calls the function to write file content.
	//
	// It will create full directory path to the file if necessary.
	EnsureFile(ctx context.Context, path string, write func(*os.File) error) error

	// EnsureFileGone removes a file or an empty directory, logging the errors.
	//
	// Missing file is not an error.
	//
	// Fails if 'path' is a non-empty directory. Use EnsureDirectoryGone for this
	// case. Treats an empty directory as a file though (deletes it), since it is
	// difficult to distinguish the two without an extra syscall.
	EnsureFileGone(ctx context.Context, path string) error

	// EnsureDirectoryGone recursively removes a directory.
	EnsureDirectoryGone(ctx context.Context, path string) error

	// Renames oldpath to newpath as "atomically" as possible.
	//
	// If newpath already exists (be it a file or a directory), removes it first.
	// If oldpath is a symlink, it's moved as is (e.g. as a symlink).
	//
	// Not really atomic if newpath exists and it is a directory. There's a small
	// interval of time when 'oldpath' still exists, former 'newpath' is deleted,
	// but 'newpath' is not yet created. If someone creates 'newpath' during this
	// time, we retry the whole operation from scratch, and keep retrying like
	// that for 10 sec.
	Replace(ctx context.Context, oldpath, newpath string) error

	// CleanupTrash attempts to remove all files that ended up in the trash.
	//
	// This is a best effort operation. Errors are logged (either at Debug or
	// Warning level, depending on severity of the trash state).
	CleanupTrash(ctx context.Context)
}

FileSystem abstracts operations that touch single file system subpath.

All functions operate in terms of native file paths. It exists mostly to hide differences between file system semantic on Windows and Linux\Mac.

func NewFileSystem 

func NewFileSystem(root, trash string) FileSystem

NewFileSystem returns default FileSystem implementation.

It operates with files under a given path. All methods accept absolute paths or paths relative to current working directory. FileSystem will ensure they are under 'root' directory.

It can also accept a path to a directory to put "trash" into: files that can't be removed because there are some processes keeping lock on them. This is useful on Windows when replacing running executables. The trash directory must be on the same disk as the root directory.

It 'trash' is empty string, the trash directory will be created under 'root'.

type ScanFilter 

type ScanFilter func(rel string) bool

ScanFilter is predicate used by ScanFileSystem to decide what files to skip.

It receives a path being judged and returns true if it should be skipped. The path is a filepath relative to the starting scanning directory (i.e. 'dir' in ScanFileSystem).

type ScanOptions 

type ScanOptions struct {
	// PreserveModTime when true saves the file's modification time.
	PreserveModTime bool

	// PreserveWritable when true saves the file's writable bit for either user,
	// group or world (mask 0222), and converts it to user-only writable bit (mask
	// 0200).
	PreserveWritable bool
}

ScanOptions specify which file properties to preserve in the archive.

type TestFileOpts 

type TestFileOpts struct {
	Executable bool
	Writable   bool
	ModTime    time.Time
}

TestFileOpts holds options for NewTestFile method. Used in unittests.

type TransactionalDestination 

type TransactionalDestination interface {
	Destination

	// Begin initiates a new write transaction.
	Begin(ctx context.Context) error
	// End finalizes package extraction (commit or rollback, based on success).
	End(ctx context.Context, success bool) error
}

TransactionalDestination is a destination that supports transactions.

It provides 'Begin' and 'End' methods: all calls to 'CreateFile' and 'CreateSymlink' should happen between them. No changes are really applied until End(true) is called. A call to End(false) discards any pending changes.

func NewDestination 

func NewDestination(dest string, fs FileSystem) TransactionalDestination

NewDestination returns a destination in the file system (a new directory) to extract a package to.

Will use a provided FileSystem object to operate on files if given, otherwise use a default one. If FileSystem is provided, dir must be in a subdirectory of the given FileSystem root.

type WinAttrs 

type WinAttrs uint32

WinAttrs represents the extra file attributes for windows. 

const (
	WinAttrHidden WinAttrs = 0x2
	WinAttrSystem WinAttrs = 0x4

	WinAttrsAll WinAttrs = WinAttrHidden | WinAttrSystem
)

These are the valid WinAttrs values. They may be OR'd together to form a mask. These match the windows GetFileAttributes values.

func (WinAttrs) String 

func (w WinAttrs) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.