configsftp

README

config-sftp

Read and write configuration on a remote host over SFTP through config

Part of the phpboyscout Go toolkit — small, framework-free Go modules extracted from go-tool-base. Documented with the parent module at config.go.phpboyscout.uk

---

config reads and writes files. When those files live on another machine reached over SSH, config-sftp bridges them: it adapts a connected *sftp.Client to the config.FS interface, so a remote configuration file takes part in loading, watching and writing exactly as a local one does.

SFTP is the least surprising remote adapter. Unlike an object store, it is not pretending to be a disk — it is a remote POSIX-shaped filesystem, with real directories, real metadata, and a real server-side rename. The six config.FS methods map almost directly onto the client.

Using it

You build the SSH transport — every host, port, user, key and host-key decision stays yours — build an *sftp.Client over it, and hand that to Wrap. The result is passed to WithFiles exactly as config.OS() is:

import (
	"github.com/pkg/sftp"
	"golang.org/x/crypto/ssh"
	"gitlab.com/phpboyscout/go/config"
	configsftp "gitlab.com/phpboyscout/go/config-sftp"
)

sshClient, _ := ssh.Dial("tcp", host, sshCfg) // your transport + auth
sftpClient, _ := sftp.NewClient(sshClient)     // your SFTP client

store, err := config.NewStore(ctx,
	config.WithFiles(configsftp.Wrap(sftpClient), "/etc/app/config.yaml"))

Wrap(client *sftp.Client) config.FS is the entire API. The adapter owns no connection and no credentials — it never re-decides host-key verification or auth. Building the SSH transport is yours; folding it in would drag the auth surface into the adapter, so config-sftp deliberately keeps to the file operations, exactly as config-afero takes an already-built afero.Fs.

Atomic writes require posix-rename@openssh.com

A config write never lets a reader see a half-written file: it stages the new content beside the target and renames over it. SFTP has two renames, and they differ exactly on overwrite:

• posix-rename@openssh.com (an OpenSSH extension) replaces the target atomically — POSIX rename(2) semantics, server-side. This is what makes the commit clean.
• Plain SFTP rename (SSH_FXP_RENAME) is specified not to overwrite an existing target, and OpenSSH refuses it.

config-sftp detects, per connection, whether the server advertises the extension and prefers it. Where it is not advertised, it falls back to plain rename, and where that fails because the target already exists, to remove-then-rename — which opens a narrow, non-atomic window during which a concurrent reader could momentarily find the target missing.

Atomic-overwrite commit therefore requires a server that supports posix-rename@openssh.com. OpenSSH and most other servers do. Note that even with the extension, the atomicity is the remote server's implementation of rename(2) over its own storage — a server backing SFTP with a non-POSIX store may not honour it. This assumption is not runtime-probed; it is stated here so it can be reasoned about.

Watching: polled, at a 15-second cadence

A remote SFTP path is not a local operating-system path, so the adapter does not claim config.RealPather — pointing native filesystem notification at it would watch the wrong local file. The Store therefore polls it. Because a poll is an SSH round-trip (though not a billed API call), the adapter hints a 15-second default cadence rather than the core's eager 2-second one, via config.PollIntervalHinter. config.WithPollInterval still overrides the hint.

Symlinks and directory modes

• Symlinks are genuinely resolved by SFTP, so the adapter implements config.LinkReader unconditionally: a configuration file reached through a remote symlink is written through the link rather than replacing it with a regular file.
• MkdirAll creates any missing parents, then applies the requested mode to each directory it creates with an explicit Chmod — because the SFTP mkdir request carries no mode field, a created directory would otherwise land at the server's umask. Directories that already exist keep their own mode.

What it costs

The config graph, plus the SFTP client and what it pulls: github.com/pkg/sftp, its filesystem walker github.com/kr/fs, and the SSH transport it is built on, golang.org/x/crypto. golang.org/x/sys already arrives with config, so net-new to your graph is just those three — far smaller than a cloud storage SDK. An allowlist test asserts exactly this set, so an unforeseen transitive addition through a version bump fails the build rather than arriving quietly in your go.sum.

Testing

The unit suite runs in ordinary go test with no Docker and no external sshd: it wires a pkg/sftp server to a client over in-process pipes — the true SFTP wire protocol, end to end, against a temp directory — and drives a real config Store through Wrap. A higher-fidelity SSH-loopback suite that exercises the real transport is gated on SFTP_TEST_LOOPBACK=1.

License

See LICENSE.

Documentation

Overview

Package configsftp adapts a connected sftp.Client to the config.FS interface, so a tool whose configuration lives on a remote host reached over SSH can hand that filesystem to a config Store unchanged.

sshClient, _ := ssh.Dial("tcp", host, sshCfg) // your transport + auth
sftpClient, _ := sftp.NewClient(sshClient)     // your SFTP client

store, err := config.NewStore(ctx,
	config.WithFiles(configsftp.Wrap(sftpClient), "/etc/app/config.yaml"))

SFTP is the least surprising remote adapter: it is not pretending to be a disk, it is a remote POSIX-shaped filesystem, so the six config.FS methods map almost directly onto the client. The adapter owns no connection and no credentials — every host, port, user, key and host-key decision lives in the SSH transport the consumer builds and hands in, exactly as afero's adapter takes an already-built afero.Fs.

What the adapter settles that a bare mapping does not

Three things are load-bearing and worth not rediscovering:

• Commit rename. A config write stages content beside the target and renames over it, so a reader never sees a half-written file. SFTP's plain rename is specified NOT to overwrite (OpenSSH refuses it), so the adapter prefers the posix-rename@openssh.com extension — an atomic replace of the destination — detecting per connection whether the server advertises it. Where it does not, the adapter falls back to plain Rename, and where that fails because the target exists, to Remove-then-Rename: a narrow non-atomic window. Atomic-overwrite commit therefore requires a server that supports posix-rename@openssh.com (OpenSSH and most others do); the atomicity is the remote server's, not the adapter's.

• Watching. A remote SFTP path is not a local operating-system path, so the adapter deliberately does not implement config.RealPather — claiming it would send native notification after the wrong local file. That routes the Store to its poll watcher; because an SFTP poll is a network round-trip (though not a billed call), the adapter implements config.PollIntervalHinter to hint a 15-second cadence rather than the core's eager 2-second default. config.WithPollInterval still overrides the hint.

• Symlinks. SFTP genuinely resolves symlinks, so the adapter implements config.LinkReader unconditionally — correct here precisely because it is always true for an SFTP server — and a configuration file reached through a remote link is written through it rather than replacing the link.

Index

• Variables
• func Wrap(client *sftp.Client) config.FS
• type Option
  • func WithClientOptions(o ...sftp.ClientOption) Option
• type OwnedFS
  • func FromSSH(conn *ssh.Client, opts ...Option) (*OwnedFS, error)
  • func (f *OwnedFS) Close() error
  • func (f *OwnedFS) PollInterval() time.Duration
  • func (f *OwnedFS) Readlink(name string) (string, error)

Examples

• Wrap

Variables

var ErrNoConn = errors.NewSentinel("configsftp.no_conn",
	"no SSH connection supplied; dial one with ssh.Dial and pass it")

ErrNoConn reports a nil SSH connection supplied to FromSSH.

Functions

func Wrap

func Wrap(client *sftp.Client) config.FS

Wrap adapts a connected SFTP client to config.FS.

The returned value additionally satisfies config.LinkReader (SFTP resolves symlinks) and config.PollIntervalHinter (a 15-second poll cadence), and deliberately does not satisfy config.RealPather (a remote path is not a local one). The Store discovers each by type assertion, so no capability flag is exported.

Whether the server advertises the posix-rename@openssh.com extension is detected once, here, from the live connection — so the commit rename picks the strongest primitive the server actually offers.

Example

ExampleWrap reads and writes configuration on a remote host over SFTP. You build the SSH transport — every host, port, user, key and host-key decision stays yours — build an *sftp.Client over it, and hand that to Wrap. The wrapped filesystem is passed to WithFiles exactly as config.OS() is.

package main

import (
	"context"
	"log"

	"github.com/pkg/sftp"
	"gitlab.com/phpboyscout/go/config"
	"golang.org/x/crypto/ssh"

	configsftp "gitlab.com/phpboyscout/go/config-sftp"
)

func main() {
	sshClient, err := ssh.Dial("tcp", "config.example.com:22", &ssh.ClientConfig{
		User:            "deploy",
		Auth:            []ssh.AuthMethod{ /* your keys */ },
		HostKeyCallback: ssh.FixedHostKey( /* your known host key */ nil),
	})
	if err != nil {
		log.Fatal(err)
	}

	sftpClient, err := sftp.NewClient(sshClient)
	if err != nil {
		log.Fatal(err)
	}

	store, err := config.NewStore(context.Background(),
		config.WithFiles(configsftp.Wrap(sftpClient), "/etc/app/config.yaml"),
	)
	if err != nil {
		log.Fatal(err)
	}

	_ = store.View().GetInt("server.port")
}

Types

type Option

type Option func(*settings)

Option configures the SFTP subsystem this rung opens. It exists so a caller can tune the client — packet size, concurrency — without reaching for sftp.NewClient themselves and losing the rung.

func WithClientOptions

func WithClientOptions(o ...sftp.ClientOption) Option

WithClientOptions passes the SFTP package's own options through to sftp.NewClient — MaxPacket, UseConcurrentReads and the rest.

type OwnedFS

type OwnedFS struct {
	config.FS
	// contains filtered or unexported fields
}

OwnedFS is a config.FS that opened its own SFTP subsystem and can close it.

It closes the SFTP client only, never the ssh.Client underneath: that connection is the caller's, may carry other subsystems, and closing somebody else's transport because this filesystem is finished with it would be presumptuous.

conn, err := ssh.Dial("tcp", host, sshCfg)   // yours; you close it
if err != nil { return err }
defer conn.Close()

fsys, err := configsftp.FromSSH(conn)
if err != nil { return err }

store, err := config.NewStore(ctx,
	config.WithFiles(fsys, "/etc/app/config.yaml"),
	config.WithCloser(fsys))                  // closing the Store closes the subsystem only
if err != nil { return err }
defer store.Close()                           // runs before conn.Close

Wrap returns a plain config.FS, because there the caller opened the subsystem and still owns it — the obligation follows who built it, not which rung (spec 0012 L-7, R2).

func FromSSH

func FromSSH(conn *ssh.Client, opts ...Option) (*OwnedFS, error)

FromSSH opens an SFTP subsystem over an SSH connection the caller dialled, and returns a filesystem over it.

This rung does I/O, unlike every other rung in the family

Elsewhere a rung 3 constructs a client without touching the network and defers reaching the service to the first read. There is no such split here: an SFTP client IS a subsystem channel plus a version handshake, so obtaining one is a round trip by definition. The connection is already established, so it is one exchange rather than a dial, and a failure names the subsystem rather than surfacing later as a confusing read error.

There is no ambient rung here, deliberately

Every remote adapter in this family that has an ambient convention to adopt offers a zero-conf Default(). SFTP has one — an agent, ~/.ssh/config, known_hosts — and deliberately does not take it, because using it would mean choosing a HOST-KEY VERIFICATION POLICY on the caller's behalf.

Every available choice is wrong for somebody. Trust-on-first-use silently accepts a man in the middle. Requiring known_hosts fails exactly the unattended machines most likely to want zero configuration. Skipping verification is not something a configuration library should ever ship.

So the ladder stops here, on the record rather than by omission: build the ssh.ClientConfig yourself, decide your own HostKeyCallback, and hand over the connection (spec 0012 L-6).

func (*OwnedFS) Close

func (f *OwnedFS) Close() error

Close releases the SFTP subsystem this filesystem opened. The SSH connection beneath is untouched. It is safe to call more than once.

func (*OwnedFS) PollInterval

func (f *OwnedFS) PollInterval() time.Duration

PollInterval forwards the owned filesystem's cadence hint, or reports none.

func (*OwnedFS) Readlink

func (f *OwnedFS) Readlink(name string) (string, error)

Readlink forwards to the filesystem this one owns. Without it the core cannot resolve a symlinked config file before its rename-over commit, and a write replaces the link instead of writing through it.