sftp

README

Golang FTP Server library

This library allows to easily build a simple and fully-featured FTP server using afero as the backend filesystem.

If you're interested in a fully featured FTP server, you should use sftpgo (fully featured SFTP/FTP server) or ftpserver (basic FTP server).

Current status of the project

Features

• Uploading and downloading files
• Directory listing (LIST + MLST)
• File and directory deletion and renaming
• TLS support (AUTH + PROT)
• File download/upload resume support (REST)
• Passive socket connections (PASV and EPSV commands)
• Active socket connections (PORT and EPRT commands)
• IPv6 support (EPSV + EPRT)
• Small memory footprint
• Clean code: No sleep, no panic, no global sync (only around control/transfer connection per client)
• Uses only the standard library except for:
  • afero for generic file systems handling
  • fclairamb/go-log for logging through your existing libraries go-kit/log, log15, zap, zerolog, logrus
• Supported extensions:
  • AUTH - Control session protection
  • AUTH TLS - TLS session
  • PROT - Transfer protection
  • EPRT/EPSV - IPv6 support
  • MDTM - File Modification Time
  • SIZE - Size of a file
  • REST - Restart of interrupted transfer
  • MLST - Simple file listing for machine processing
  • MLSD - Directory listing for machine processing
  • HASH - Hashing of files
  • AVLB - Available space
  • COMB - Combine files

Quick test

The easiest way to test this library is to use ftpserver.

The driver

The simplest way to get a good understanding of how the driver shall be implemented is to look at the tests driver.

The base API

The API is directly based on afero.

// MainDriver handles the authentication and ClientHandlingDriver selection
type MainDriver interface {
	// GetSettings returns some general settings around the server setup
	GetSettings() (*Settings, error)

	// ClientConnected is called to send the very first welcome message
	ClientConnected(cc ClientContext) (string, error)

	// ClientDisconnected is called when the user disconnects, even if he never authenticated
	ClientDisconnected(cc ClientContext)

	// AuthUser authenticates the user and selects an handling driver
	AuthUser(cc ClientContext, user, pass string) (ClientDriver, error)

	// GetTLSConfig returns a TLS Certificate to use
	// The certificate could frequently change if we use something like "let's encrypt"
	GetTLSConfig() (*tls.Config, error)
}


// ClientDriver is the base FS implementation that allows to manipulate files
type ClientDriver interface {
	afero.Fs
}

// ClientContext is implemented on the server side to provide some access to few data around the client
type ClientContext interface {
	// Path provides the path of the current connection
	Path() string

	// SetDebug activates the debugging of this connection commands
	SetDebug(debug bool)

	// Debug returns the current debugging status of this connection commands
	Debug() bool

	// Client's ID on the server
	ID() uint32

	// Client's address
	RemoteAddr() net.Addr

	// Servers's address
	LocalAddr() net.Addr

	// Client's version can be empty
	GetClientVersion() string

	// Close closes the connection and disconnects the client.
	Close() error

	// HasTLSForControl returns true if the control connection is over TLS
	HasTLSForControl() bool

	// HasTLSForTransfers returns true if the transfer connection is over TLS
	HasTLSForTransfers() bool

	// GetLastCommand returns the last received command
	GetLastCommand() string

	// GetLastDataChannel returns the last data channel mode
	GetLastDataChannel() DataChannel
}

// Settings define all the server settings
type Settings struct {
	Listener                 net.Listener     // (Optional) To provide an already initialized listener
	ListenAddr               string           // Listening address
	PublicHost               string           // Public IP to expose (only an IP address is accepted at this stage)
	PublicIPResolver         PublicIPResolver // (Optional) To fetch a public IP lookup
	PassiveTransferPortRange *PortRange       // (Optional) Port Range for data connections. Random if not specified
	ActiveTransferPortNon20  bool             // Do not impose the port 20 for active data transfer (#88, RFC 1579)
	IdleTimeout              int              // Maximum inactivity time before disconnecting (#58)
	ConnectionTimeout        int              // Maximum time to establish passive or active transfer connections
	DisableMLSD              bool             // Disable MLSD support
	DisableMLST              bool             // Disable MLST support
	DisableMFMT              bool             // Disable MFMT support (modify file mtime)
	Banner                   string           // Banner to use in server status response
	TLSRequired              TLSRequirement   // defines the TLS mode
	DisableLISTArgs          bool             // Disable ls like options (-a,-la etc.) for directory listing
	DisableSite              bool             // Disable SITE command
	DisableActiveMode        bool             // Disable Active FTP
	EnableHASH               bool             // Enable support for calculating hash value of files
	DisableSTAT              bool             // Disable Server STATUS, STAT on files and directories will still work
	DisableSYST              bool             // Disable SYST
	EnableCOMB               bool             // Enable COMB support
	DefaultTransferType      TransferType     // Transfer type to use if the client don't send the TYPE command
	// ActiveConnectionsCheck defines the security requirements for active connections
	ActiveConnectionsCheck DataConnectionRequirement
	// PasvConnectionsCheck defines the security requirements for passive connections
	PasvConnectionsCheck DataConnectionRequirement
}

Extensions

There are a few extensions to the base afero APIs so that you can perform some operations that aren't offered by afero.

Pre-allocate some space

// ClientDriverExtensionAllocate is an extension to support the "ALLO" - file allocation - command
type ClientDriverExtensionAllocate interface {

	// AllocateSpace reserves the space necessary to upload files
	AllocateSpace(size int) error
}

Get available space

// ClientDriverExtensionAvailableSpace is an extension to implement to support
// the AVBL ftp command
type ClientDriverExtensionAvailableSpace interface {
	GetAvailableSpace(dirName string) (int64, error)
}

Create symbolic link

// ClientDriverExtensionSymlink is an extension to support the "SITE SYMLINK" - symbolic link creation - command
type ClientDriverExtensionSymlink interface {

	// Symlink creates a symlink
	Symlink(oldname, newname string) error

	// SymlinkIfPossible allows to get the source of a symlink (but we don't need for now)
	// ReadlinkIfPossible(name string) (string, error)
}

Compute file hash

// ClientDriverExtensionHasher is an extension to implement if you want to handle file digests
// yourself. You have to set EnableHASH to true for this extension to be called
type ClientDriverExtensionHasher interface {
	ComputeHash(name string, algo HASHAlgo, startOffset, endOffset int64) (string, error)
}

History of the project

I wanted to make a system which would accept files through FTP and redirect them to something else. Go seemed like the obvious choice and it seemed there was a lot of libraries available but it turns out none of them were in a useable state.

• micahhausler/go-ftp is a minimalistic implementation
• shenfeng/ftpd.go is very basic and 4 years old.
• yob/graval is 3 years old and “experimental”.
• goftp/server seemed OK but I couldn't use it on both Filezilla and the MacOs ftp client.
• andrewarrow/paradise_ftp - Was the only one of the list I could test right away. This is the project I forked from.

Documentation

Index

• func NewFS(fs fs.FS, callback NotificationHandler) fs.FS
• func WithAddress(val string) func(server *Server)
• func WithBasePath(path string) func(server *Server)
• func WithCredentialValidator(...) func(server *Server)
• func WithNotificationCallback(callback NotificationHandler) func(srv *Server)
• func WithPort(val int) func(server *Server)
• func WithPrivateKey(val string) func(server *Server)
• func WithPublicKey(val string) func(server *Server)
• func WithSSHPath(val string) func(server *Server)
• func WithUserProvider(provider interfaces2.UserProvider) func(*Server)
• type FS
  • func (f *FS) Conn() *ssh.ServerConn
  • func (f *FS) Context() map[string]string
  • func (f *FS) Filecmd(request *sftp.Request) error
  • func (f *FS) Filelist(request *sftp.Request) (sftp.ListerAt, error)
  • func (f *FS) Fileread(request *sftp.Request) (io.ReaderAt, error)
  • func (f *FS) Filewrite(request *sftp.Request) (io.WriterAt, error)
  • func (f *FS) Logger() log.Logger
  • func (f *FS) Notify(request *sftp.Request, err error)
  • func (f *FS) Permissions() []string
  • func (f *FS) SetConn(sconn *ssh.ServerConn)
  • func (f *FS) SetContext(ctx map[string]string)
  • func (f *FS) SetID(p string)
  • func (f *FS) SetLogger(logger log.Logger)
  • func (f *FS) SetPermissions(p []string)
  • func (f *FS) Type() string
• type Notification
• type NotificationHandler
• type Server
  • func New(opts ...func(*Server)) *Server
  • func NewWithNotify(opts ...func(*Server)) *Server
  • func (c *Server) AcceptInboundConnection(conn net.Conn, config *ssh.ServerConfig)
  • func (c *Server) AddUser(user models.User)
  • func (c *Server) Initialize() error
  • func (c *Server) Validate(conn ssh.ConnMetadata, pass []byte) (*ssh.Permissions, error)

Functions

func NewFS

func NewFS(fs fs.FS, callback NotificationHandler) fs.FS

func WithAddress

func WithAddress(val string) func(server *Server)

func WithBasePath

func WithBasePath(path string) func(server *Server)

func WithCredentialValidator

func WithCredentialValidator(val func(server *Server, r fs.AuthenticationRequest) (*fs.AuthenticationResponse, error)) func(server *Server)

func WithNotificationCallback

func WithNotificationCallback(callback NotificationHandler) func(srv *Server)

func WithPort

func WithPort(val int) func(server *Server)

func WithPrivateKey

func WithPrivateKey(val string) func(server *Server)

func WithPublicKey

func WithPublicKey(val string) func(server *Server)

func WithSSHPath

func WithSSHPath(val string) func(server *Server)

func WithUserProvider

func WithUserProvider(provider interfaces2.UserProvider) func(*Server)

Types

type FS

type FS struct {
	// contains filtered or unexported fields
}

func (*FS) Conn

func (f *FS) Conn() *ssh.ServerConn

func (*FS) Context

func (f *FS) Context() map[string]string

func (*FS) Filecmd

func (f *FS) Filecmd(request *sftp.Request) error

func (*FS) Filelist

func (f *FS) Filelist(request *sftp.Request) (sftp.ListerAt, error)

func (*FS) Fileread

func (f *FS) Fileread(request *sftp.Request) (io.ReaderAt, error)

func (*FS) Filewrite

func (f *FS) Filewrite(request *sftp.Request) (io.WriterAt, error)

func (*FS) Logger

func (f *FS) Logger() log.Logger

func (*FS) Notify

func (f *FS) Notify(request *sftp.Request, err error)

func (*FS) Permissions

func (f *FS) Permissions() []string

func (*FS) SetConn

func (f *FS) SetConn(sconn *ssh.ServerConn)

func (*FS) SetContext

func (f *FS) SetContext(ctx map[string]string)

func (*FS) SetID

func (f *FS) SetID(p string)

func (*FS) SetLogger

func (f *FS) SetLogger(logger log.Logger)

func (*FS) SetPermissions

func (f *FS) SetPermissions(p []string)

func (*FS) Type

func (f *FS) Type() string

type Notification

type Notification struct {
	User          string    `json:"user"`
	FsType        string    `json:"fs_type"`
	ClientVersion string    `json:"client_version"`
	RemoteAddr    string    `json:"remote_addr"`
	Time          time.Time `json:"time"`
	Event         string    `json:"event"`
	Subject       string    `json:"subject"`
	Target        string    `json:"target"`
	Error         error     `json:"error"`
}

type NotificationHandler

type NotificationHandler func(notification Notification) error

type Server

type Server struct {
	// contains filtered or unexported fields
}

func New

func New(opts ...func(*Server)) *Server

func NewWithNotify

func NewWithNotify(opts ...func(*Server)) *Server

func (*Server) AcceptInboundConnection

func (c *Server) AcceptInboundConnection(conn net.Conn, config *ssh.ServerConfig)

AcceptInboundConnection ... Handles an inbound connection to the instance and determines if we should serve the request or not.

func (*Server) AddUser

func (c *Server) AddUser(user models.User)

func (*Server) Initialize

func (c *Server) Initialize() error

Initialize the SFTP server and add a persistent listener to handle inbound SFTP connections.

func (*Server) Validate

func (c *Server) Validate(conn ssh.ConnMetadata, pass []byte) (*ssh.Permissions, error)