sftp

README

sftp

Pure-Go (CGO=0) SFTP server that exports any go-filesystems/interface Filesystem — so every driver in the family becomes reachable from any ordinary SFTP client, with nothing installed on the client side and nothing mounted.

No kernel extension. No FSKit entitlement. No cgo. No root, on either side.

Why SFTP

go-filesystems/nfs makes a driver mountable. Mounting is powerful and it has a price: it needs root on the client, a mount point, and an administrator's cooperation. SFTP asks for none of that.

An SFTP client is already everywhere, and it is already the thing people reach for:

sftp, scp	ships with OpenSSH — every Linux, every macOS, Windows 10 and later
Cyberduck, Transmit, FileZilla, WinSCP	ordinary desktop file browsers
VS Code, TRAMP	edit a file in the image in place
GNOME Files (gvfs)	mounts sftp:// as a plain user, no root

And the protocol brings what NFSv3 does not have at all: public-key authentication and encryption, built in rather than bolted on.

Why it makes per-client isolation cheap

"SFTP would let us isolate clients in their own file, in user space, with a dedicated server."

One disk image per client. One process per client. Its own keys.

The isolation boundary is the image file plus the process — not a chroot, not permissions, not a uid. There is nothing to escape to, because the server holds exactly one Filesystem and has no way to name anything else. It never touches a host path, and the only paths it ever resolves are the ones clean has already clamped inside the image.

That is a property, not an aspiration, so it is tested: TestNoPathEscapesTheExport and TestNoSymlinkEscapesTheExport do not merely assert that an escape fails — they assert that it resolves to the file the image actually contains at the clamped path, which is the much stronger statement.

Serve is cheap to instantiate and cheap to run N times on one machine, which is what makes "a server per tenant" a real deployment shape rather than a slogan. It is a library first (the target is weft, the fleet's microVM cloud) and a standalone binary second.

Install

go get github.com/go-filesystems/sftp

Serving one

fsys, err := fat32.Open("disk.img", -1)
if err != nil {
	return err
}
defer fsys.Close()

srv, err := sftp.New(fsys, sftp.ReadWrite()) // read-only by default
if err != nil {
	return err
}

d, err := sshd.New(srv, sshd.Config{
	HostKeys:       []ssh.Signer{hostKey},   // yours; nothing is read from a fixed path
	AuthorizedKeys: []ssh.PublicKey{clientKey},
})
if err != nil {
	return err
}
return d.ListenAndServe("127.0.0.1:2222")

Then, from anything:

sftp -P 2222 user@127.0.0.1

Host keys and authorised keys are always supplied by the caller. This module opens no key file, consults no ~/.ssh, and has no "allow any key" switch — not even in the demo, because such switches get copied into deployments.

Layout

package	depends on	why
wire/	nothing but the standard library	the v3 packet codec
.	interface, wire	the SFTP subsystem over a Filesystem
sshd/	., golang.org/x/crypto/ssh	the SSH front-end

The codec is deliberately isolated. wire imports neither the filesystem interface nor the server — it is checked, not merely intended. The reason is that an sftp:// client transport may one day live in go-streamkit, and it would share this encoding; keeping the dependency edge absent now means that extraction is a move rather than a rewrite. go-filesystems/nfs made the same split with its xdr/ and rpc/ packages.

Cryptography is not reimplemented: the transport is golang.org/x/crypto/ssh. The SFTP subsystem itself (draft-ietf-secsh-filexfer-02, version 3 — what every client actually speaks) is implemented here, because the protocol is the thing this repository is for.

The write path, measured

SSH_FXP_READ(handle, offset, len) lands exactly on filesystem.Opener / File.ReadAt, and SSH_FXP_WRITE(handle, offset, data) lands exactly on filesystem.WritableFile (io.WriterAt + Truncate + Sync), published in interface v0.3.0 and implemented by fat32 v0.3.0. Both are optional capabilities, probed with a type assertion.

When a driver has not got WritableFile, the only write the base contract offers is WriteFile(path, data, perm), which replaces the file whole. A write at an offset then costs read-all + splice + write-all, per request — so streaming a file in fixed-size chunks is O(n²).

That cost is measured, not asserted. 2 MiB through the real OpenSSH client in its own 32 KiB chunks, Apple M-series, task measure:

path	time	throughput
WritableFile (positional)	46 ms	44 066 kB/s
WriteFile fallback (splice)	1.021 s	2 007 kB/s

22× slower — and the gap grows with the file, which is the part that matters:

size	time	throughput	cost of doubling
1 MiB	182 ms	5 638 kB/s
2 MiB	1.109 s	1 846 kB/s	6.11×
4 MiB	8.114 s	505 kB/s	7.31×

Worse than the 4× a purely quadratic term predicts, because the driver's WriteFile also rewalks the cluster chain each time. go-filesystems/nfs measured the identical construction at 90 kB/s, with a soft-mounted client giving up with EIO partway through. This is why the fallback is documented as a wall and not as a slow path — and why it is deliberately not hidden behind a write-back cache, which would make the benchmark look fine while turning "the server acknowledged this write" into a durability claim it cannot honour.

Verification

"It compiles" is not evidence. The gate is 100% of statements, error branches included, on both modules — but the claim that matters is that a real client agrees, so fat32demo/ serves a genuine driver-formatted image to the sftp binary that ships with OpenSSH and compares on content:

• ls, then get of a 1 MiB file — sha256 of what the client received equals sha256 of what the driver returns from ReadFile, not what the test wrote.
• reget resuming at offset 700 000, the only step that exercises a non-zero READ offset: a plain get always starts at zero, so a server that ignored the offset entirely would pass everything else.
• put of 64 KiB, verified by reopening the image and reading it back through the driver, so the assertion is about what reached the medium rather than state the live server still holds.
• a put against a read-only export must fail and leave nothing behind.

The wire is big-endian, so CI runs the test binaries under QEMU on s390x as well as riscv64, loong64 and ppc64le — s390x being the arch on which a forgotten binary.BigEndian would still pass on the developer's machine.

Every key in the test suite is generated with ssh-keygen into t.TempDir(), outside every repository, and removed when the test ends. No key is read from or written to a fixed location, .gitignore refuses key shapes as a net, and CI fails the build if any key material or image survives in the work tree.

Security posture

• Bind to loopback unless you have decided otherwise.
• Read-only is the default; sftp.ReadWrite() is opt-in, because an accidental write to a forensic or build artefact is unrecoverable.
• Authentication is public-key only. There is no password path, and no way to authorise a key the caller did not name.
• A server can reach nothing but the one Filesystem it was given. No host path, no .. escape, no symlink out of the image.

Licence

BSD-3-Clause.

Documentation

Overview

Package sftp implements an SFTP version 3 server (draft-ietf-secsh-filexfer-02) that exports any github.com/go-filesystems/interface.Filesystem.

Every driver in the family — ext4, xfs, btrfs, zfs, ntfs, fat32, exfat, hfsplus, apfs, iso9660, squashfs, ufs, ffs, uefi, oci — becomes something a person can reach from a file manager, an editor or a shell, over one TCP port, with SSH keys and encryption, from a pure-Go binary with no cgo.

Why SFTP

The client already exists, everywhere, and nobody has to install it.

• `sftp` on the command line ships with OpenSSH, which is present on macOS, on every Linux distribution, on the BSDs, and on Windows 10 and later as an optional feature that is on by default in current builds.
• Graphical clients are abundant and free or cheap: Cyberduck, FileZilla, WinSCP, Transmit.
• Editors speak it natively: VS Code, the JetBrains IDEs, Emacs by way of TRAMP.
• GNOME mounts sftp:// through gvfs as an ordinary unprivileged user, with no root and no entry in the kernel's mount table.

And the security is not bolted on. Authentication is SSH public keys and the channel is encrypted, both from golang.org/x/crypto/ssh — see github.com/go-filesystems/sftp/sshd. NFSv3, the sibling of this module, has neither: its AUTH_UNIX credentials are claims a client makes about itself and the wire cannot disagree. It is one TCP port, so a firewall can reason about it, and nothing needs privilege on either side: no mount, no kernel extension, no FSKit entitlement, no root.

The isolation this is built for

One image file per client, one server process per client, each with its own keys. The isolation boundary is the image file plus the process — not a chroot, not uid mapping, not namespaces.

That is why Server takes exactly one github.com/go-filesystems/interface.Filesystem and holds no other capability. It never touches the host filesystem, never resolves a path against anything but the driver it was handed, and cannot be induced to: a client path is cleaned with ".." clamped at the root before it reaches the driver, so no sequence of components names anything outside the image, and a symbolic link inside the image resolves inside the image because the driver is the only thing doing the resolving. The tests assert this directly rather than leaving it to be inferred.

A Server is cheap: no goroutine until a session arrives, no listener of its own, no global state. Running a hundred of them on one machine, one per tenant, is the intended shape, and Server.Serve is deliberately handed a stream rather than a listener so that an embedding program — a microVM supervisor, say — can wire it into an SSH server it already runs.

Serving one

fsys, err := fat32.Open("tenant.img", -1)
if err != nil {
	return err
}
defer fsys.Close()

srv, err := sftp.New(fsys)          // read-only by default
if err != nil {
	return err
}
// Then either hand srv.Serve a stream you own, or let the sshd
// subpackage own the SSH side:
d, err := sshd.New(srv, sshd.Config{HostKeys: ..., AuthorizedKeys: ...})

Two honest gaps, both in the contract and not in this server

Reads. github.com/go-filesystems/interface.Opener is exactly the shape of SSH_FXP_READ: OpenFile(path) returns a File whose ReadAt(buf, off) answers the request as asked. This module uses it when a driver has it. At the time of writing NO driver in the fleet implements it, so in practice every read goes through the fallback, which calls ReadFile — materialising the WHOLE file — once per READ request. An OpenSSH client reads in 32 KiB chunks, so fetching an n-byte file costs n/32768 full-file reads: quadratic. That is measured in the README rather than described.

The tempting fix — read the file once when the handle is opened and serve reads from that copy — is not taken, because it trades a quadratic time cost for a linear MEMORY cost and a 4 GiB image file becomes 4 GiB of resident memory in a process meant to run a hundred times over. The real fix is a driver implementing Opener, and leaving the cost visible is what keeps that pressure where it belongs.

Writes. SSH_FXP_WRITE carries an offset, and the base contract has only WriteFile, which replaces a file whole. A write at an offset therefore becomes read-modify-write of the entire file: O(filesize) per request, and O(filesize²) for a client streaming a file in chunks. This is the same wall github.com/go-filesystems/nfs hit and measured at 90 kB/s. Exports are consequently READ-ONLY BY DEFAULT — see ReadWrite — and the number for this module is in the README.

The escape is WritableFile: a File that also has WriteAt, Truncate and Sync. When a driver's File satisfies it, writes go to the offset directly and the wall disappears. The probe is here and tested; the implementations belong in the drivers.

Index

• Variables
• type Option
  • func ReadWrite() Option
  • func WithMaxPacket(n int) Option
• type Server
  • func New(fsys filesystem.Filesystem, opts ...Option) (*Server, error)
  • func (s *Server) ReadOnly() bool
  • func (s *Server) Serve(rw io.ReadWriter) error
• type TimeStat
• type WritableFile

Variables

var ErrNilFilesystem = errors.New("sftp: nil filesystem")

ErrNilFilesystem reports New called with no filesystem.

It is caught at construction because the alternative is a nil dereference on the first client request — long after the mistake, in a per-session goroutine, with no recover, taking down every other tenant the process is serving.

var ErrNoInit = errors.New("sftp: first packet was not SSH_FXP_INIT")

ErrNoInit reports a peer whose first packet was not SSH_FXP_INIT.

The session cannot continue: version negotiation is the only thing that establishes what the following bytes mean, so guessing would be worse than stopping.

var ErrVersion = errors.New("sftp: unsupported client protocol version")

ErrVersion reports a peer demanding a protocol version this server cannot speak. See wire.Version.

Types

type Option

type Option func(*Server)

Option configures a Server.

func ReadWrite

func ReadWrite() Option

ReadWrite makes the export writable.

Exports are READ-ONLY BY DEFAULT, for two reasons that both point the same way. Most of what this module is pointed at is a forensic or build artefact, and an accidental write to one is unrecoverable. And until a driver's File implements WritableFile, every write at a non-zero offset is a read-modify-write of the entire file — see the package documentation and the measurement in the README. Opting in should be a decision, not a default.

func WithMaxPacket

func WithMaxPacket(n int) Option

WithMaxPacket caps the size of a single SFTP packet, in bytes, counting the type byte and payload.

The default, wire.MaxPacket, accepts everything OpenSSH's client sends including its largest -B setting. Lowering it is how a host running many tenants bounds the memory any one session can make it hold. A non-positive value restores the default.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server exports one github.com/go-filesystems/interface.Filesystem over SFTP version 3.

One Server holds exactly one filesystem and no other capability. It has no listener, no goroutine and no global state until a session is handed to Server.Serve, which is what makes running one per tenant — the shape this module is built for — cheap rather than aspirational.

A Server is safe for concurrent use by any number of sessions.

func New

func New(fsys filesystem.Filesystem, opts ...Option) (*Server, error)

New returns a Server exporting fsys, read-only unless ReadWrite is given.

func (*Server) ReadOnly

func (s *Server) ReadOnly() bool

ReadOnly reports whether the export refuses mutating operations.

func (*Server) Serve

func (s *Server) Serve(rw io.ReadWriter) error

Serve runs one SFTP session over rw until the peer closes it or the stream fails, and returns nil on a clean close.

rw is one client's SFTP channel: in the ordinary case the SSH subsystem channel opened by github.com/go-filesystems/sftp/sshd, but deliberately only an io.ReadWriter, so a program that already runs an SSH server can hand this its own channel, and a test can hand it a pipe. Serve does not close rw — it did not open it.

Every handle the session opened is released before Serve returns, including when the client vanishes mid-transfer, which is what killing an `sftp` process does.

type TimeStat

type TimeStat interface {
	ModTime() int64 // seconds since the Unix epoch
}

TimeStat is the optional capability a driver's github.com/go-filesystems/interface.Stat may implement to report a real modification time.

No driver in the fleet does today, so every file this server exports currently carries the server's start time in atime and mtime. That is visibly wrong in a client's listing and is reported here rather than hidden: the fix is a timestamp accessor on interface.Stat, not a plausible guess in this module. The probe exists so the day a driver reports one, the listing gets it without a change here.

type WritableFile

type WritableFile = filesystem.WritableFile

WritableFile is the capability that removes this module's write wall.

It is an open file that can be written AT AN OFFSET, which is exactly what SSH_FXP_WRITE(handle, offset, data) asks for, and exactly what the base github.com/go-filesystems/interface.Filesystem contract cannot express: that contract's only write is WriteFile, which replaces a file WHOLE. So serving a 32 KiB write into the middle of a 100 MiB file without this capability means reading 100 MiB, splicing 32 KiB in, and writing 100 MiB back — per request. See [writeSplice] and the README for the measurement of both paths.

It is an alias for github.com/go-filesystems/interface.WritableFile, published in interface v0.3.0, rather than a local redeclaration. The alias exists purely so that this package's own documentation has somewhere to explain what the capability means TO AN SFTP SERVER — which is not the same question as what it means to the interface module — while the type itself stays the fleet's one canonical definition. A driver satisfies it or does not; there is no second contract here to drift out of sync.

This module never implements it. It probes for it on the File returned by github.com/go-filesystems/interface.Opener, uses it when present, and falls back to read-modify-write with a documented cost when absent.