transfer

Documentation

Overview

Package transfer implements the copy engine for transports that move a file at a time: atomic temp-and-rename delivery, tree walking with deepest-first directory-mode restoration, symlink and special-file policy, per-file progress, and prompt cancellation. A provider plugs an FS into each side of Copy and inherits the semantics the invoketest transfer contracts specify, rather than re-implementing them per transport.

local copies HostFS to HostFS, ssh pairs HostFS with an SFTP adapter, and fake pairs it with an adapter over its virtual filesystem. docker does not use the engine: the daemon moves whole trees as tar archives, so there is no per-file write for an FS to perform. It decides each entry with the exported policy helpers in the invoke package instead — invoke.ClassifyEntry, invoke.EffectiveMode and invoke.CheckFollowTarget — which this engine applies too, so the two reach the same answers.

Index

• func ContainsPath(root, path, sep string) bool
• func Copy(ctx context.Context, src FS, srcPath string, dst FS, dstPath string, ...) error
• type ConcurrencyHinter
• type DataFS
• type FS
• type HostFS
  • func (HostFS) Abs(path string) (string, error)
  • func (HostFS) Base(path string) string
  • func (HostFS) Chmod(path string, mode fs.FileMode) error
  • func (HostFS) Contains(root, path string) bool
  • func (HostFS) CreateExclusive(path string) (WriteFile, error)
  • func (HostFS) Dir(path string) string
  • func (HostFS) Join(elem ...string) string
  • func (HostFS) Lstat(path string) (fs.FileInfo, error)
  • func (HostFS) Mkdir(path string) error
  • func (HostFS) MkdirAll(path string) error
  • func (HostFS) Open(path string) (ReadFile, error)
  • func (HostFS) ReadDir(path string) ([]fs.FileInfo, error)
  • func (HostFS) Readlink(path string) (string, error)
  • func (HostFS) Remove(path string) error
  • func (HostFS) Rename(oldPath, newPath string) error
  • func (HostFS) Resolve(path string) (string, error)
  • func (HostFS) SameFile(a, b fs.FileInfo) bool
  • func (HostFS) Stat(path string) (fs.FileInfo, error)
  • func (HostFS) Symlink(target, link string) error
• type LinkFS
• type MetaFS
• type PathFS
• type ReadFile
• type WriteFile

Functions

func ContainsPath

func ContainsPath(root, path, sep string) bool

ContainsPath reports whether path is root itself or lies under it, for clean paths whose elements sep separates.

The boundary is root followed by sep, so "/srv" does not contain "/srvx". A root that already ends in sep is its own boundary: that is the filesystem root, "/" or `C:\`, and appending another made "//", which nothing under it starts with.

func Copy

func Copy(ctx context.Context, src FS, srcPath string, dst FS, dstPath string, cfg invoke.TransferConfig) error

Copy copies srcPath on src to dstPath on dst after validating the transfer is safe: on a shared filesystem the paths must not alias each other, and a directory may not be copied into its own subtree.

The overlap guard applies only when both sides are the same filesystem, which is the only case where a path on one side means anything on the other. Two endpoints that happen to reach the same machine by different routes — an upload to the host the caller is running on, say — cannot be recognized as overlapping, and copying a directory into its own subtree that way will recurse into what it is writing.

Types

type ConcurrencyHinter

type ConcurrencyHinter interface {
	CopyConcurrency() int
}

ConcurrencyHinter is implemented by an FS that benefits from copying several files at once — a request-response transport whose per-file cost is round trips. CopyConcurrency is the preferred worker count for tree copies touching this side.

type DataFS

type DataFS interface {
	Open(path string) (ReadFile, error)

	// CreateExclusive creates path for writing, failing if it exists.
	CreateExclusive(path string) (WriteFile, error)

	// Rename moves oldPath over newPath, replacing any existing entry
	// atomically.
	Rename(oldPath, newPath string) error

	Remove(path string) error
}

DataFS moves file content and whole entries.

type FS

type FS interface {
	PathFS
	MetaFS
	DataFS
	LinkFS
}

FS is one side of a transfer. Implementations must be comparable: Copy compares its two sides to decide whether same-file overlap guards apply.

type HostFS

type HostFS struct{}

HostFS is the FS for the host's own filesystem — the local side of every provider's Upload and Download, and both sides of the local provider's.

func (HostFS) Abs

func (HostFS) Abs(path string) (string, error)

Abs makes path absolute against the working directory.

func (HostFS) Base

func (HostFS) Base(path string) string

Base returns the last element of path.

func (HostFS) Chmod

func (HostFS) Chmod(path string, mode fs.FileMode) error

Chmod sets a path's permission bits.

func (HostFS) Contains

func (HostFS) Contains(root, path string) bool

Contains reports whether path is root itself or lies under it.

func (HostFS) CreateExclusive

func (HostFS) CreateExclusive(path string) (WriteFile, error)

CreateExclusive creates path for writing, failing if it exists.

func (HostFS) Dir

func (HostFS) Dir(path string) string

Dir returns all but the last element of path.

func (HostFS) Join

func (HostFS) Join(elem ...string) string

Join joins path elements with the host separator.

func (HostFS) Lstat

func (HostFS) Lstat(path string) (fs.FileInfo, error)

Lstat stats path without following a trailing symlink.

func (HostFS) Mkdir

func (HostFS) Mkdir(path string) error

Mkdir creates one directory at the conventional default mode; walked directories are chmod'd to their source's real mode afterward.

func (HostFS) MkdirAll

func (HostFS) MkdirAll(path string) error

MkdirAll creates missing parent directories at the default mode.

func (HostFS) Open

func (HostFS) Open(path string) (ReadFile, error)

Open opens path for reading, behind a wrapper that exposes the ReadFile surface and nothing else.

Narrowed deliberately: *os.File implements io.WriterTo with a generic copy loop, and a copy that honored it would let the host side drive the transfer — over a destination that schedules its own writes, like the SFTP client, that replaces pipelined requests with one write per round trip. Driving the copy stays the transport side's claim.

func (HostFS) ReadDir

func (HostFS) ReadDir(path string) ([]fs.FileInfo, error)

ReadDir lists a directory with lstat-style entry info.

func (HostFS) Readlink

func (HostFS) Readlink(path string) (string, error)

Readlink returns the target of a symlink.

func (HostFS) Remove

func (HostFS) Remove(path string) error

Remove deletes one entry.

func (HostFS) Rename

func (HostFS) Rename(oldPath, newPath string) error

Rename moves oldPath over newPath atomically.

func (HostFS) Resolve

func (HostFS) Resolve(path string) (string, error)

Resolve canonicalizes path, following symbolic links.

func (HostFS) SameFile

func (HostFS) SameFile(a, b fs.FileInfo) bool

SameFile reports whether two stat results name the same file.

func (HostFS) Stat

func (HostFS) Stat(path string) (fs.FileInfo, error)

Stat stats path, following symlinks.

func (HostFS) Symlink

func (HostFS) Symlink(target, link string) error

Symlink creates link pointing at target.

type LinkFS

type LinkFS interface {
	Symlink(target, link string) error
	Readlink(path string) (string, error)

	// Resolve canonicalizes path, following symbolic links.
	Resolve(path string) (string, error)
}

LinkFS reads and writes symbolic links.

type MetaFS

type MetaFS interface {
	Lstat(path string) (fs.FileInfo, error)
	Stat(path string) (fs.FileInfo, error)
	ReadDir(path string) ([]fs.FileInfo, error)
	Mkdir(path string) error
	MkdirAll(path string) error
	Chmod(path string, mode fs.FileMode) error

	// SameFile reports whether two stat results name the same file, when
	// the filesystem can tell; a side that cannot may always report false.
	SameFile(a, b fs.FileInfo) bool
}

MetaFS reads and writes file metadata.

type PathFS

type PathFS interface {
	Abs(path string) (string, error)
	Join(elem ...string) string
	Dir(path string) string
	Base(path string) string

	// Contains reports whether path is root itself or lies under it.
	Contains(root, path string) bool
}

PathFS is the path algebra of one side of a transfer. Each side applies its own separator rules, so a transfer between unlike systems stays correct on both.

type ReadFile

type ReadFile interface {
	io.ReadCloser
	Stat() (fs.FileInfo, error)
}

ReadFile is an open source file. Stat re-checks the open handle, so a racing type replacement cannot stall the copy loop.

type WriteFile

type WriteFile interface {
	io.WriteCloser
	Chmod(mode fs.FileMode) error
	Sync() error
}

WriteFile is an exclusively created destination file being filled.