core

Documentation

Index

• Constants
• func RegisterProvider(p Provider) error
• type Account
  • func TryTemplate(p Provider) Account
• type Accountant
• type Client
  • func NewClient() *Client
  • func NewClientWith(retrievers int) *Client
  • func (d *Client) AddURLs(urls []*url.URL) Container
  • func (d *Client) DryRun()
  • func (d *Client) Finalize()
  • func (d *Client) OnDeadend(f func(*url.URL))
  • func (d *Client) OnDownload(f func(*Download))
  • func (d *Client) OnError(f func(File, error))
  • func (d *Client) OnResolve(f func(*url.URL, File, error))
  • func (d *Client) OnSkip(f func(File))
  • func (d *Client) Resolve()
  • func (d *Client) Start()
  • func (d *Client) Stop()
  • func (d *Client) Use(acc Account)
• type Config
• type Configured
• type Container
• type ContainerID
  • func (urls ContainerID) String() string
• type Download
  • func (d *Download) Canceled() bool
  • func (d *Download) Done() bool
  • func (d *Download) Err() error
  • func (d *Download) Progress() int64
  • func (d *Download) Size() int64
  • func (d *Download) Stop()
  • func (d *Download) Wait()
  • func (d *Download) Waiter() <-chan struct{}
• type Field
• type File
  • func Resolve(urls []*url.URL) []File
• type MultiResolver
• type Progress
• type Prompter
• type Provider
• type Providers
  • func RegisteredProviders() Providers
  • func (ps Providers) FindProvider(f func(Provider) bool) Provider
  • func (ps Providers) GetProvider(name string) Provider
• type Queue
• type ReadProgress
• type Retriever
• type SingleResolver

Constants

const Version = "0.3.0"

Version is the version of this library

Functions

func RegisterProvider

func RegisterProvider(p Provider) error

RegisterProvider is not thread-safe!!!!

Types

type Account

type Account = api.Account

Account represents a persistent record on a provider (useful e.g. to access restricted files)

func TryTemplate

func TryTemplate(p Provider) Account

TryTemplate creates a new account template for the given provider and returns that -- if provider implements `Accountant` interface. Otherwise, simply `nil` is returned

type Accountant

type Accountant = api.Accountant

Accountant is a provider that stores user accounts

type Client

type Client struct {
	Directory     string
	NoSkip        bool
	NoContinue    bool
	Providers     Providers
	Accounts      map[string][]Account
	ResolvedQueue *queue
	// contains filtered or unexported fields
}

Client manages downloads

func NewClient

func NewClient() *Client

NewClient creates a new Client with 3 retrievers and 1 resolver

func NewClientWith

func NewClientWith(retrievers int) *Client

NewClientWith creates a new Client with the amount of workers provided. If amount is 0, the Client works in resolve-only mode.

func (*Client) AddURLs

func (d *Client) AddURLs(urls []*url.URL) Container

AddURLs adds a list of URLs to the download queue. Returns a WaitGroup for when the downloads are complete.

func (*Client) DryRun

func (d *Client) DryRun()

DryRun starts this downloader in dryrun mode, printing to stdout instead of downloading.

func (*Client) Finalize

func (d *Client) Finalize()

Finalize gracefully stops this Client

func (*Client) OnDeadend

func (d *Client) OnDeadend(f func(*url.URL))

OnDeadend calls the given hook when a file is offline.

func (*Client) OnDownload

func (d *Client) OnDownload(f func(*Download))

OnDownload calls the given hook when a new Download is started. The download object is passed.

func (*Client) OnError

func (d *Client) OnError(f func(File, error))

OnError calls the given hook when an error occurred in `Download`

func (*Client) OnResolve

func (d *Client) OnResolve(f func(*url.URL, File, error))

OnResolve calls the given hook when a resolve job is finished. It passes the original URLs, the File if successful or the error if not.

func (*Client) OnSkip

func (d *Client) OnSkip(f func(File))

OnSkip calls the given hook when a download is skipped

func (*Client) Resolve

func (d *Client) Resolve()

Resolve starts this Client in 'Resolve' mode, meaning there are no retrievers, and the wait groups will not wait for the retrievers to do their job.

func (*Client) Start

func (d *Client) Start()

Start starts the Client asynchronously

func (*Client) Stop

func (d *Client) Stop()

Stop stops this Client immediately

func (*Client) Use

func (d *Client) Use(acc Account)

Use adds an account to this client's repertoire. The account will be passed to Resolvers upon start.

type Config

type Config = api.Config

Config object

type Configured

type Configured = api.Configured

Configured are providers that require some kind of configuration/initialization

type Container

type Container interface {
	ID() ContainerID
	Wait()
}

Container combines URLs that were added in the same context

type ContainerID

type ContainerID []*url.URL

ContainerID calculates the sha256 sum of the underlying URLs

func (ContainerID) String

func (urls ContainerID) String() string

type Download

type Download struct {
	Provider Provider
	File     File
	// contains filtered or unexported fields
}

Download is an object that fetches a single remote file and presents information on its progress and status

func (*Download) Canceled

func (d *Download) Canceled() bool

Canceled returns whether this download was canceled. Panics if download is still running.

func (*Download) Done

func (d *Download) Done() bool

Done returns true if this download is finished. False otherwise

func (*Download) Err

func (d *Download) Err() error

Err returns the error during this download if there was one. Panics if download is still running.

func (*Download) Progress

func (d *Download) Progress() int64

Progress returns the current progress in int64

func (*Download) Size

func (d *Download) Size() int64

func (*Download) Stop

func (d *Download) Stop()

Stop cancels this download

func (*Download) Wait

func (d *Download) Wait()

Wait blocks the caller until this download is finished

func (*Download) Waiter

func (d *Download) Waiter() <-chan struct{}

Waiter returns the channel which is closed on completion

type Field

type Field = api.Field

Field defines a question to ask the user

type File

type File interface {
	api.File

	// Err returns the error associated with this file, if there is any.
	//
	// Read the note on call order in the interface description..
	Err() error

	// Offline returns whether this file is offline.
	//
	// Read the note on call order in the interface description.
	Offline() bool

	// LengthUnknown returns whether this file's length is known
	// e.g. HEAD request without Content-Length.
	LengthUnknown() bool

	// ID returns the identifier for this file (sha256-sum of the URL string)
	ID() string

	// OriginalURL returns the original URL (passed to Client) that ultimately yielded this File.
	OriginalURL() *url.URL
	// contains filtered or unexported methods
}

File denotes a remote file object

For any given File, the order of method calls must be:

1. `Err()` - if this returns `nil`, continue with checking the file's availability:
2. `Offline()` - and if this also returns false, the file is valid and available.

If `Err()` returns an error, `Offline()` and all non-URL methods will panic. Same for when `Offline()` returns `true`.

func Resolve

func Resolve(urls []*url.URL) []File

Resolve returns meta information on the given URLs

type MultiResolver

type MultiResolver = api.MultiResolver

MultiResolver is a provider which can resolve multiple URLs at once

type Progress

type Progress interface {
	Progress() int64
	Length() int64
}

Progress is an object that represents a long operation that can track a progress

type Prompter

type Prompter = api.Prompter

Prompter asks for user input

type Provider

type Provider = api.Provider

Provider is the base interface, other interfaces will be dynamically infered

type Providers

type Providers []Provider

Providers represents an array of providers with some utility functions

func RegisteredProviders

func RegisteredProviders() Providers

RegisteredProviders returns a list of registered providers

func (Providers) FindProvider

func (ps Providers) FindProvider(f func(Provider) bool) Provider

FindProvider searches providers (in order). Returns the first to satisfy the predicate

func (Providers) GetProvider

func (ps Providers) GetProvider(name string) Provider

GetProvider returns the provider for the given string, or `nil` if there was none.

type Queue

type Queue interface {
	Dequeue() <-chan File
	List() []File
}

Queue is a buffered thread safe priority queue

type ReadProgress

type ReadProgress interface {
	io.Reader
	Progress
}

ReadProgress is an io.ReadCloser that tracks progress

type Retriever

type Retriever = api.Retriever

Retriever is a provider which can download specific URLs

type SingleResolver

type SingleResolver = api.SingleResolver

SingleResolver is a provider which can only resolve URLs one by one