download

README

download

A library to implement fast downloading, with concurrency.

godoc

Documentation

Index

• Variables
• type DL
  • func New() *DL
  • func (dl *DL) Chan(data Interface, ch chan FileBuffer, opt *Options) error
  • func (dl *DL) Tar(data Interface, file *os.File, opt *Options) error
• type File
• type FileBuffer
• type Interface
• type Options

Variables

var (
	ErrNoData       = errors.New("nothing to download")
	ErrNoGoroutines = errors.New("MaxGoroutines should be positive")
)

var DefaultOptions = &Options{
	Separator:     "/",
	MaxAttempts:   3,
	MaxGoroutines: 64,
}

Default Options

Types

type DL

type DL struct {
	Client *http.Client
	Write  sync.Mutex
}

DL is the downloader.

func New

func New() *DL

New DL instance.

func (*DL) Chan

func (dl *DL) Chan(data Interface, ch chan FileBuffer, opt *Options) error

Chan will download data, and send results to a channel. The channel will be closed before returning.

func (*DL) Tar

func (dl *DL) Tar(data Interface, file *os.File, opt *Options) error

Tar downloads data to a tarball at file.

type File

type File struct {
	Path, Name, URL string
}

File to download. URL is where to get the data, path is where to store it.

type FileBuffer

type FileBuffer struct {
	File
	Buffer *bytes.Buffer
}

FileBuffer is a File with its data downloaded to Buffer.

type Interface

type Interface interface {
	URLs() []File
}

A type that satisfies download.Interface can be downloaded with download.Tar etc.

type Options

type Options struct {
	// Separator to use between files, eg you can use "_" for a flat
	// file structure, or leave the default "/" to have dirs.
	Separator string

	// Max attempts trying to download a file
	MaxAttempts int

	// Concurrent mode
	Concurrent bool
	// Max number of gorountines to spawn
	MaxGoroutines int
}

Options for the downloader.