download

README

Go Download

CLI and SDK to download multiple files asynchronously with advanced features like resume support, bandwidth limiting, and checksum verification.

Table of Contents

• Go Download
  • Features
  • Installation
  • Usage
    • CLI
      • Using a Configuration File
      • Using a File with URLs
    • SDK
  • Advanced Features
    • Resume Downloads
    • Bandwidth Limiting
    • Checksum Verification
    • Authentication
  • License

Features

• Download multiple files in parallel
• Resume interrupted downloads
• Bandwidth limiting
• Progress bars with ETA and speed
• Custom filenames
• Authentication support (Basic Auth and Bearer Token)
• Checksum verification (MD5, SHA1, SHA256)
• Configurable retries and timeouts
• JSON configuration file support
• File-based URL input

Installation

go get -u github.com/akshaybabloo/go-download

or use the CLI from the releases page.

Usage

CLI

The CLI tool gdl provides a convenient way to download files from the command line:

# Basic usage
gdl --url https://example.com/file1.txt --url https://example.com/file2.txt \
    --output /path/to/output/directory \
    --title "My Downloads" \
    --parallel 5

# With resume and bandwidth limiting
gdl --url https://example.com/large-file.zip \
    --resume \
    --bandwidth 500 \
    --progress

# With custom filename
gdl --url https://example.com/file-with-long-name.txt \
    --filename "https://example.com/file-with-long-name.txt:short-name.txt"

# With authentication
gdl --url https://protected-site.com/file.zip \
    --basic-auth "username:password"

# With checksum verification
gdl --url https://example.com/important-file.iso \
    --verify-checksum \
    --checksum "https://example.com/important-file.iso:sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"

Using a Configuration File

For more complex downloads, you can use a JSON configuration file:

gdl --config download-config.json

Example configuration file:

{
  "urls": [
    "https://example.com/file1.zip",
    "https://example.com/file2.pdf"
  ],
  "output": "./downloads",
  "title": "my-downloads",
  "parallel": 2,
  "show_progress": true,
  "resume_download": true,
  "bandwidth_limit": 500,
  "timeout": 30,
  "retry": 3,
  "headers": {
    "User-Agent": "Go-Download-Client/1.0"
  },
  "custom_filenames": {
    "https://example.com/file1.zip": "custom-name.zip"
  },
  "verify_checksum": true,
  "expected_checksums": {
    "https://example.com/file1.zip": "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
  }
}

Using a File with URLs

You can also use a file containing URLs to download:

gdl --file urls.txt --output /path/to/output/directory --parallel 5

Where urls.txt contains:

https://example.com/file1.txt
https://example.com/file2.txt
https://example.com/file3.txt

SDK

The Go Download SDK provides a flexible way to integrate download functionality into your Go applications:

package main

import (
    "fmt"
    "github.com/akshaybabloo/go-download"
)

func main() {
    // Create a new download with default options
    d := download.NewDownload()
    
    // Set URLs to download
    d.SetURLs([]string{
        "https://example.com/file1.txt",
        "https://example.com/file2.txt",
    })
    
    // Configure download options
    d.SetOutput("/path/to/output/directory")
    d.SetTitle("My Downloads")
    d.SetParallel(5)
    
    // Enable advanced features
    d.ShowProgress = true
    d.ResumeDownload = true
    d.BandwidthLimit = 500 // KB/s
    
    // Set custom filename for a specific URL
    d.SetCustomFilename(
        "https://example.com/file2.txt",
        "custom-name.txt",
    )
    
    // Set authentication if needed
    d.SetBasicAuth("username", "password")
    // or
    d.SetBearerToken("your-token-here")
    
    // Enable checksum verification
    d.VerifyChecksum = true
    d.SetExpectedChecksum(
        "https://example.com/file1.txt",
        "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
    )
    
    // Start the download
    err := d.Start()
    if err != nil {
        fmt.Printf("Error: %v\n", err)
    }
}

Advanced Features

Resume Downloads

The resume functionality allows interrupted downloads to be continued from where they left off, saving bandwidth and time:

d := download.NewDownload()
d.ResumeDownload = true

Bandwidth Limiting

Control the download speed to avoid saturating your network connection:

d := download.NewDownload()
d.BandwidthLimit = 500 // Limit to 500 KB/s

Checksum Verification

Verify the integrity of downloaded files using MD5, SHA1, or SHA256 checksums:

d := download.NewDownload()
d.VerifyChecksum = true
d.SetExpectedChecksum(
    "https://example.com/file.zip",
    "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
)

Authentication

Access protected resources with authentication:

d := download.NewDownload()
// Basic authentication
d.SetBasicAuth("username", "password")
// Or Bearer token
d.SetBearerToken("your-token-here")

License

This project is licensed under the MIT License - see the LICENSE file for details.

Documentation

Index

• type Options
  • func NewDownload(options ...*Options) *Options
  • func (o *Options) SetBasicAuth(username, password string)
  • func (o *Options) SetBearerToken(token string)
  • func (o *Options) SetCustomFilename(url, filename string)
  • func (o *Options) SetExpectedChecksum(url, checksum string)
  • func (o *Options) SetOutput(p string)
  • func (o *Options) SetParallel(i int)
  • func (o *Options) SetTitle(t string) error
  • func (o *Options) SetURLs(u []string)
  • func (o *Options) Start() error

Types

type Options

type Options struct {
	// Headers is the headers to be sent with the request
	Headers map[string]string

	// Retry is the number of times the request should be retried
	// If not provided, 3 will be used
	Retry int

	// ShowProgress is the flag to show the progress of the download
	// If not provided, false will be used
	ShowProgress bool

	// DownloadPath is the path to download the files
	// If not provided, the current directory will be used
	DownloadPath string

	// Parallel is the number of parallel downloads
	// If not provided, 1 will be used
	Parallel int

	// Timeout is the time to wait for the download
	// If not provided, 10 seconds will be used
	Timeout int

	// Title is the title of the download folder
	// If not provided, the title will be empty
	// If a URL is provided, it will try to get the title from the URL webpage, if not, the domain name will be used.
	Title string

	// Log is the logger to be used
	Log *log.Logger

	// ResumeDownload is the flag to enable resuming interrupted downloads
	// If not provided, false will be used
	ResumeDownload bool

	// BandwidthLimit is the maximum download speed in KB/s
	// If not provided or set to 0, no limit will be applied
	BandwidthLimit int

	// CustomFilenames is a map of URL to custom filename
	// If provided, the custom filename will be used instead of the one from the URL
	CustomFilenames map[string]string

	// VerifyChecksum is the flag to enable checksum verification
	// If not provided, false will be used
	VerifyChecksum bool

	// ExpectedChecksums is a map of URL to expected checksum
	// The format is "algorithm:checksum", e.g. "sha256:1234abcd..."
	// Supported algorithms: md5, sha1, sha256
	ExpectedChecksums map[string]string
	// contains filtered or unexported fields
}

func NewDownload

func NewDownload(options ...*Options) *Options

NewDownload creates a new download with some default options

func (*Options) SetBasicAuth

func (o *Options) SetBasicAuth(username, password string)

SetBasicAuth sets the basic authentication header

func (*Options) SetBearerToken

func (o *Options) SetBearerToken(token string)

SetBearerToken sets the bearer token authentication header

func (*Options) SetCustomFilename

func (o *Options) SetCustomFilename(url, filename string)

SetCustomFilename sets a custom filename for a URL

func (*Options) SetExpectedChecksum

func (o *Options) SetExpectedChecksum(url, checksum string)

SetExpectedChecksum sets the expected checksum for a URL The format is "algorithm:checksum", e.g. "sha256:1234abcd..." Supported algorithms: md5, sha1, sha256

func (*Options) SetOutput

func (o *Options) SetOutput(p string)

SetOutput sets the output path

func (*Options) SetParallel

func (o *Options) SetParallel(i int)

SetParallel sets the number of parallel downloads. Default is 1

func (*Options) SetTitle

func (o *Options) SetTitle(t string) error

SetTitle sets the title of the download If a URL is provided, it will try to get the title from the URL webpage, if not, the domain name will be used.

func (*Options) SetURLs

func (o *Options) SetURLs(u []string)

SetURLs sets the URLs to be downloaded

func (*Options) Start

func (o *Options) Start() error

Start starts the download