dl

Documentation

Overview

Package dl contains abstractions for downloadingfiles and directories from various sources.

Index

• Variables
• func Download(ctx context.Context, opts Options) (err error)
• type Downloader
• type FileDownloader
  • func (FileDownloader) Download(opts Options) (Type, string, error)
  • func (FileDownloader) MatchURL(string) bool
  • func (FileDownloader) Name() string
• type GitDownloader
  • func (GitDownloader) Download(opts Options) (Type, string, error)
  • func (GitDownloader) MatchURL(u string) bool
  • func (GitDownloader) Name() string
  • func (GitDownloader) Update(opts Options) (bool, error)
• type Manifest
• type Options
  • func (opts Options) NewHash() (hash.Hash, error)
• type TorrentDownloader
  • func (TorrentDownloader) Download(opts Options) (Type, string, error)
  • func (TorrentDownloader) MatchURL(u string) bool
  • func (TorrentDownloader) Name() string
• type Type
  • func (t Type) String() string
• type UpdatingDownloader

Variables

var (
	ErrChecksumMismatch = errors.New("dl: checksums did not match")
	ErrNoSuchHashAlgo   = errors.New("dl: invalid hashing algorithm")
)

ErrChecksumMismatch occurs when the checksum of a downloaded file does not match the expected checksum provided in the Options struct.

var (
	ErrAria2NotFound    = errors.New("aria2 must be installed for torrent functionality")
	ErrDestinationEmpty = errors.New("the destination directory is empty")
)

var Downloaders = []Downloader{
	GitDownloader{},
	TorrentDownloader{},
	FileDownloader{},
}

Downloaders contains all the downloaders in the order in which they should be checked

Functions

func Download

func Download(ctx context.Context, opts Options) (err error)

Download downloads a file or directory using the specified options. It first gets the appropriate downloader for the URL, then checks if caching is enabled. If caching is enabled, it attempts to get the cache directory for the URL and update it if necessary. If the source is found in the cache, it links it to the destination using hard links. If the source is not found in the cache, it downloads the source to a new cache directory and links it to the destination.

Types

type Downloader

type Downloader interface {
	// Name returns the name of the downloader
	Name() string
	// MatchURL checks if the given URL matches
	// the downloader.
	MatchURL(string) bool
	// Download downloads the object at the URL
	// provided in the options, to the destination
	// given in the options. It returns a type,
	// a name for the downloaded object (this may be empty),
	// and an error.
	Download(Options) (Type, string, error)
}

type FileDownloader

type FileDownloader struct{}

FileDownloader downloads files using HTTP

func (FileDownloader) Download

func (FileDownloader) Download(opts Options) (Type, string, error)

Download downloads a file using HTTP. If the file is compressed using a supported format, it will be extracted

func (FileDownloader) MatchURL

func (FileDownloader) MatchURL(string) bool

MatchURL always returns true, as FileDownloader is used as a fallback if nothing else matches

func (FileDownloader) Name

func (FileDownloader) Name() string

Name always returns "file"

type GitDownloader

type GitDownloader struct{}

GitDownloader downloads Git repositories

func (GitDownloader) Download

func (GitDownloader) Download(opts Options) (Type, string, error)

Download uses git to clone the repository from the specified URL. It allows specifying the revision, depth and recursion options via query string

func (GitDownloader) MatchURL

func (GitDownloader) MatchURL(u string) bool

MatchURL matches any URLs that start with "git+"

func (GitDownloader) Name

func (GitDownloader) Name() string

Name always returns "git"

func (GitDownloader) Update

func (GitDownloader) Update(opts Options) (bool, error)

Update uses git to pull the repository and update it to the latest revision. It allows specifying the depth and recursion options via query string. It returns true if update was successful and false if the repository is already up-to-date

type Manifest

type Manifest struct {
	Type Type
	Name string
}

Manifest holds information about the type and name of a downloaded file or directory. It is stored inside each cache directory for later use.

type Options

type Options struct {
	Hash             []byte
	HashAlgorithm    string
	Name             string
	URL              string
	Destination      string
	CacheDisabled    bool
	PostprocDisabled bool
	Progress         io.Writer
	LocalDir         string
}

Options contains the options for downloading files and directories

func (Options) NewHash

func (opts Options) NewHash() (hash.Hash, error)

type TorrentDownloader

type TorrentDownloader struct{}

func (TorrentDownloader) Download

func (TorrentDownloader) Download(opts Options) (Type, string, error)

Download downloads a file over the BitTorrent protocol.

func (TorrentDownloader) MatchURL

func (TorrentDownloader) MatchURL(u string) bool

MatchURL returns true if the URL is a magnet link or an http(s) link with a "torrent+" prefix

func (TorrentDownloader) Name

func (TorrentDownloader) Name() string

Name always returns "file"

type Type

type Type uint8

Type represents the type of download (file or directory)

const (
	TypeFile Type = iota
	TypeDir
)

func (Type) String

func (t Type) String() string

type UpdatingDownloader

type UpdatingDownloader interface {
	Downloader
	// Update checks for and performs any
	// available updates for the object
	// described in the options. It returns
	// true if an update was performed, or
	// false if no update was required.
	Update(Options) (bool, error)
}

UpdatingDownloader extends the Downloader interface with an Update method for protocols such as git, which allow for incremental updates without changing the URL.