goftp

package module
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 28, 2021 License: MIT Imports: 15 Imported by: 0

README

goftp - an FTP client for golang

Build Status GoDoc

goftp aims to be a high-level FTP client that takes advantage of useful FTP features when supported by the server.

Here are some notable package highlights:

  • Connection pooling for parallel transfers/traversal.
  • Automatic resumption of interruped file transfers.
  • Explicit and implicit FTPS support (TLS only, no SSL).
  • IPv6 support.
  • Reasonably good automated tests that run against pure-ftpd and proftpd.

Please see the godocs for details and examples.

Pull requests or feature requests are welcome, but in the case of the former, you better add tests.

Tests

How to run tests (windows not supported):

  • ./build_test_server.sh from root goftp directory (this downloads and compiles pure-ftpd and proftpd)
  • go test from the root goftp directory

Documentation

Overview

Package goftp provides a high-level FTP client for go.

Example 
package main

import (
	"os"

	"github.com/secsy/goftp"
)

func main() {
	// Create client object with default config
	client, err := goftp.Dial("ftp.example.com")
	if err != nil {
		panic(err)
	}

	// Download a file to disk
	readme, err := os.Create("readme")
	if err != nil {
		panic(err)
	}

	err = client.Retrieve("README", readme)
	if err != nil {
		panic(err)
	}

	// Upload a file from disk
	bigFile, err := os.Open("big_file")
	if err != nil {
		panic(err)
	}

	err = client.Store("big_file", bigFile)
	if err != nil {
		panic(err)
	}
}

Output:
Example (Config) 
package main

import (
	"bytes"
	"os"
	"time"

	"github.com/secsy/goftp"
)

func main() {
	config := goftp.Config{
		User:               "jlpicard",
		Password:           "beverly123",
		ConnectionsPerHost: 10,
		Timeout:            10 * time.Second,
		Logger:             os.Stderr,
	}

	client, err := goftp.DialConfig(config, "ftp.example.com")
	if err != nil {
		panic(err)
	}

	// download to a buffer instead of file
	buf := new(bytes.Buffer)
	err = client.Retrieve("pub/interesting_file.txt", buf)
	if err != nil {
		panic(err)
	}
}

Output:

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client maintains a connection pool to the FTP server(s), so you typically only need one Client object. Client methods are safe to call concurrently from different goroutines, but once you are using all ConnectionsPerHost connections per host, methods will block waiting for a free connection.

func Dial 

func Dial(hosts ...string) (*Client, error)

Dial creates an FTP client using the default config. See DialConfig for information about "hosts".

func DialConfig 

func DialConfig(config Config, hosts ...string) (*Client, error)

DialConfig creates an FTP client using the given config. "hosts" is a list of IP addresses or hostnames with an optional port (defaults to 21). Hostnames will be expanded to all the IP addresses they resolve to. The client's connection pool will pick from all the addresses in a round-robin fashion. If you specify multiple hosts, they should be identical mirrors of each other.

func (*Client) Close 

func (c *Client) Close() error

Close closes all open server connections. Currently this does not attempt to do any kind of polite FTP connection termination. It will interrupt all transfers in progress.

func (*Client) Delete 

func (c *Client) Delete(path string) error

Delete deletes the file "path".

func (*Client) Exists 

func (c *Client) Exists(filepath string) (os.FileInfo, error)

check file or directory exists, if exists return os.FileInfo

func (*Client) ExistsDir 

func (c *Client) ExistsDir(dirpath string) (os.FileInfo, error)

func (*Client) Getwd 

func (c *Client) Getwd() (string, error)

Getwd returns the current working directory.

func (*Client) Mkdir 

func (c *Client) Mkdir(path string) (string, error)

Mkdir creates directory "path". The returned string is how the client should refer to the created directory.

func (*Client) MkdirAll 

func (c *Client) MkdirAll(dirpath string) error

MkdirAll creates a directory named dirpath, along with any necessary parents, and returns nil, or else returns an error. If dirpath is already a directory, MkdirAll does nothing and returns nil.

func (*Client) OpenFile 

func (c *Client) OpenFile(path string, f OpenFileFlag) (*File, error)

func (*Client) OpenRawConn 

func (c *Client) OpenRawConn() (RawConn, error)

OpenRawConn opens a "raw" connection to the server which allows you to run any control or data command you want. See the RawConn interface for more details. The RawConn will not participate in the Client's pool (i.e. does not count against ConnectionsPerHost).

Example 
package main

import (
	"fmt"
	"io/ioutil"

	"github.com/secsy/goftp"
)

func main() {
	// ignore errors for brevity

	client, _ := goftp.Dial("ftp.hq.nasa.gov")

	rawConn, _ := client.OpenRawConn()

	code, msg, _ := rawConn.SendCommand("FEAT")
	fmt.Printf("FEAT: %d-%s\n", code, msg)

	// prepare data connection
	dcGetter, _ := rawConn.PrepareDataConn()

	// cause server to initiate data connection
	rawConn.SendCommand("LIST")

	// get actual data connection
	dc, _ := dcGetter()

	data, _ := ioutil.ReadAll(dc)
	fmt.Printf("LIST response: %s\n", data)

	// close data connection
	dc.Close()

	// read final response from server after data transfer
	code, msg, _ = rawConn.ReadResponse()
	fmt.Printf("Final response: %d-%s\n", code, msg)
}

Output:

func (*Client) ReadDir 

func (c *Client) ReadDir(path string) ([]os.FileInfo, error)

ReadDir fetches the contents of a directory, returning a list of os.FileInfo's which are relatively easy to work with programatically. It will not return entries corresponding to the current directory or parent directories. The os.FileInfo's fields may be incomplete depending on what the server supports. If the server does not support "MLSD", "LIST" will be used. You may have to set ServerLocation in your config to get (more) accurate ModTimes in this case.

Example (ParallelWalk)

Just for fun, walk an ftp server in parallel. I make no claim that this is correct or a good idea. 

package main

import (
	"fmt"
	"os"
	"path"
	"path/filepath"
	"sync/atomic"

	"github.com/secsy/goftp"
)

// Just for fun, walk an ftp server in parallel. I make no claim that this is
// correct or a good idea.
func main() {
	client, err := goftp.Dial("ftp.hq.nasa.gov")
	if err != nil {
		panic(err)
	}

	Walk(client, "", func(fullPath string, info os.FileInfo, err error) error {
		if err != nil {
			// no permissions is okay, keep walking
			if err.(goftp.Error).Code() == 550 {
				return nil
			}
			return err
		}

		fmt.Println(fullPath)

		return nil
	})
}

// Walk a FTP file tree in parallel with prunability and error handling.
// See http://golang.org/pkg/path/filepath/#Walk for interface details.
func Walk(client *goftp.Client, root string, walkFn filepath.WalkFunc) (ret error) {
	dirsToCheck := make(chan string, 100)

	var workCount int32 = 1
	dirsToCheck <- root

	for dir := range dirsToCheck {
		go func(dir string) {
			files, err := client.ReadDir(dir)

			if err != nil {
				if err = walkFn(dir, nil, err); err != nil && err != filepath.SkipDir {
					ret = err
					close(dirsToCheck)
					return
				}
			}

			for _, file := range files {
				if err = walkFn(path.Join(dir, file.Name()), file, nil); err != nil {
					if file.IsDir() && err == filepath.SkipDir {
						continue
					}
					ret = err
					close(dirsToCheck)
					return
				}

				if file.IsDir() {
					atomic.AddInt32(&workCount, 1)
					dirsToCheck <- path.Join(dir, file.Name())
				}
			}

			atomic.AddInt32(&workCount, -1)
			if workCount == 0 {
				close(dirsToCheck)
			}
		}(dir)
	}

	return ret
}

Output:

func (*Client) Rename 

func (c *Client) Rename(from, to string) error

Rename renames file "from" to "to".

func (*Client) Retrieve 

func (c *Client) Retrieve(path string, dest io.Writer) error

Retrieve file "path" from server and write bytes to "dest". If the server supports resuming stream transfers, Retrieve will continue resuming a failed download as long as it continues making progress. Retrieve will also verify the file's size after the transfer if the server supports the SIZE command.

func (*Client) Rmdir 

func (c *Client) Rmdir(path string) error

Rmdir removes directory "path".

func (*Client) Stat 

func (c *Client) Stat(path string) (os.FileInfo, error)

Stat fetches details for a particular file. The os.FileInfo's fields may be incomplete depending on what the server supports. If the server doesn't support "MLST", "LIST" will be attempted, but "LIST" will not work if path is a directory. You may have to set ServerLocation in your config to get (more) accurate ModTimes when using "LIST".

func (*Client) Store 

func (c *Client) Store(path string, src io.Reader) error

Store bytes read from "src" into file "path" on the server. If the server supports resuming stream transfers and "src" is an io.Seeker (*os.File is an io.Seeker), Store will continue resuming a failed upload as long as it continues making progress. Store will not attempt to resume an upload if the client is connected to multiple servers. Store will also verify the remote file's size after the transfer if the server supports the SIZE command.

type Config 

type Config struct {
	// User name. Defaults to "anonymous".
	User string

	// User password. Defaults to "anonymous" if required.
	Password string

	// Maximum number of FTP connections to open per-host. Defaults to 5. Keep in
	// mind that FTP servers typically limit how many connections a single user
	// may have open at once, so you may need to lower this if you are doing
	// concurrent transfers.
	ConnectionsPerHost int

	// Timeout for opening connections, sending control commands, and each read/write
	// of data transfers. Defaults to 5 seconds.
	Timeout time.Duration

	// TLS Config used for FTPS. If provided, it will be an error if the server
	// does not support TLS. Both the control and data connection will use TLS.
	TLSConfig *tls.Config

	// FTPS mode. TLSExplicit means connect non-TLS, then upgrade connection to
	// TLS via "AUTH TLS" command. TLSImplicit means open the connection using
	// TLS. Defaults to TLSExplicit.
	TLSMode TLSMode

	// This flag controls whether to use IPv6 addresses found when resolving
	// hostnames. Defaults to false to prevent failures when your computer can't
	// IPv6. If the hostname(s) only resolve to IPv6 addresses, Dial() will still
	// try to use them as a last ditch effort. You can still directly give an
	// IPv6 address to Dial() even with this flag off.
	IPv6Lookup bool

	// Logging destination for debugging messages. Set to os.Stderr to log to stderr.
	// Password value will not be logged.
	Logger io.Writer

	// Time zone of the FTP server. Used when parsing mtime from "LIST" output if
	// server does not support "MLST"/"MLSD". Defaults to UTC.
	ServerLocation *time.Location

	// Enable "active" FTP data connections where the server connects to the client to
	// establish data connections (does not work if client is behind NAT). If TLSConfig
	// is specified, it will be used when listening for active connections.
	ActiveTransfers bool

	// Set the host:port to listen on for active data connections. If the host and/or
	// port is empty, the local address/port of the control connection will be used. A
	// port of 0 will listen on a random port. If not specified, the default behavior is
	// ":0", i.e. listen on the local control connection host and a random port.
	ActiveListenAddr string

	// Disables EPSV in favour of PASV. This is useful in cases where EPSV connections
	// neither complete nor downgrade to PASV successfully by themselves, resulting in
	// hung connections.
	DisableEPSV bool

	// Use coustom net.Conn
	CustomDail func(netWrok, host string, timeout time.Duration) (net.Conn, error)
	// contains filtered or unexported fields
}

Config contains configuration for a Client object.

type Error 

type Error interface {
	error

	// Whether the error was transient and attempting the same operation
	// again may be succesful. This includes timeouts.
	Temporary() bool

	// If the error originated from an unexpected response from the server, this
	// will return the FTP response code. Otherwise it will return 0.
	Code() int

	// Similarly, this will return the text response from the server, or empty
	// string.
	Message() string
}

Error is an expanded error interface returned by all Client methods. It allows discerning callers to discover potentially actionable qualities of the error.

type File 

type File struct {
	Path string
	// contains filtered or unexported fields
}

func (*File) Close 

func (f *File) Close() error

func (*File) Read 

func (f *File) Read(bt []byte) (int, error)

func (*File) Write 

func (f *File) Write(bt []byte) (int, error)

type OpenFileFlag 

type OpenFileFlag = int
const (
	OPEN_FILE_READ  OpenFileFlag = os.O_RDONLY
	OPEN_FILE_WRITE OpenFileFlag = os.O_WRONLY
)

type RawConn 

type RawConn interface {
	// Sends command fmt.Sprintf(f, args...) to the server, returning the response code,
	// response message, and error if any.
	SendCommand(f string, args ...interface{}) (int, string, error)

	// Prepares a data connection to the server. PrepareDataConn returns a getter function
	// because in active transfer mode you must first call PrepareDataConn (to tell server
	// what port to connect to), then send a control command to tell the server to initiate
	// a connection, then finally you invoke the getter function to get the actual
	// net.Conn.
	PrepareDataConn() (func() (net.Conn, error), error)

	// Read a pending response from the server. This is necessary after completing a
	// data command since the server sends an unsolicited response you must read.
	ReadResponse() (int, string, error)

	// Close the control and data connection, if open.
	Close() error
}

type TLSMode 

type TLSMode int

TLSMode represents the FTPS connection strategy. Servers cannot support both modes on the same port. 

const (
	// TLSExplicit means the client first runs an explicit command ("AUTH TLS")
	// before switching to TLS.
	TLSExplicit TLSMode = 0

	// TLSImplicit means both sides already implicitly agree to use TLS, and the
	// client connects directly using TLS.
	TLSImplicit TLSMode = 1
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.