grab

README

grab

Overview

grab is a versatile, extensible media downloader written in Go. It supports downloading videos, audios, documents, and other resources from various platforms, with features like multi-threaded downloads, resumable downloads, playlist support, and customizable extraction.

Features

• Supports multiple platforms via plugin-like extractors
• Multi-threaded, resumable downloads with chunked HTTP range requests
• M3U8/HLS stream support with zero-copy and AES-128 decryption
• Playlist and batch download support
• Customizable output directory, filename, quality, and format (with ffmpeg integration)
• Progress bars for multiple downloads
• Robust error handling and retry logic
• Custom HTTP headers, cookies, and proxy support
• Download subtitles, video-only, or audio-only as needed
• Extensible: add new extractors easily

Getting Started

go install github.com/hydrz/grab/cmd/grab@latest

Usage

grab [OPTIONS] <URL>...

Common Options

• -o, --output-dir <dir>: Output directory (default: ./downloads)
• -O, --output-filename <name>: Output filename
• -q, --quality <quality>: Preferred quality (e.g., best, 720p)
• -f, --format <fmt>: Output format (e.g., mp4, mkv, mp3)
• -c, --cookies <file>: Cookie file path
• -H, --header <header>: Custom HTTP header (can be used multiple times)
• -u, --user-agent <ua>: Custom user agent
• -x, --proxy <url>: HTTP proxy URL
• -r, --retry <n>: Number of retry attempts
• -t, --timeout <duration>: Request timeout (e.g., 30s)
• -n, --threads <n>: Number of concurrent download threads
• --chunk-size <bytes>: Download chunk size in bytes
• -S, --no-skip: Do not skip existing files
• -i, --info: Only extract media info, do not download
• -p, --playlist: Download all videos in playlist
• --playlist-start <n>: Playlist start index
• --playlist-end <n>: Playlist end index
• --subtitle: Download subtitles
• --video-only: Download video only, no audio
• --audio-only: Download audio only
• --ignore-errors: Continue on errors
• -d, --debug: Enable debug logging
• -v, --verbose: Enable verbose output
• --silent: Suppress all output except errors

Example

grab -o ./videos -q best -n 8 "https://example.com/video/123"

To see all registered extractors:

grab --help

Changelog

Contributing

We welcome contributions! Please read the CONTRIBUTING.md for guidelines on how to contribute to this project.

License

This project is licensed under the terms of the MIT license. See the LICENSE file for details.

Documentation

Index

• Variables
• func ListExtractors() []string
• func Register(name string, f extractorFactory)
• type Context
  • func NewContext(ctx context.Context, option Option) *Context
  • func (c *Context) Client() *resty.Client
  • func (c *Context) Context() context.Context
  • func (c *Context) GetProgressCallback() ProgressCallback
  • func (c *Context) Logger() *slog.Logger
  • func (c *Context) Option() Option
  • func (c *Context) SetProgressCallback(callback ProgressCallback)
• type Downloader
  • func NewDownloader(ctx *Context) *Downloader
  • func (d *Downloader) Download(medias []Media) error
  • func (d *Downloader) Stop()
• type Extractor
  • func FindExtractor(ctx *Context, url string) (Extractor, error)
• type Filter
• type Media
  • func (m *Media) String() string
• type Option
  • func (o *Option) Combine(other Option)
• type PlaylistFilter
  • func (f *PlaylistFilter) Filter(stream Stream) bool
• type ProgressCallback
• type Stream
• type StreamType

Variables

var (
	ErrNoExtractorFound = errors.New("no extractor found for the given URL")
	ErrInvalidURL       = errors.New("invalid URL provided")
	ErrFFmpegNotFound   = errors.New("ffmpeg executable not found in PATH")
)

var DefaultOptions = &Option{
	OutputPath: "./downloads",
	RetryCount: 5,
	Timeout:    30 * time.Second,
	Threads:    max(4, runtime.NumCPU()),
	ChunkSize:  1024 * 1024,
	UserAgent:  defaultUserAgent,
}

Functions

func ListExtractors

func ListExtractors() []string

ListExtractors returns the names of all registered extractors.

func Register

func Register(name string, f extractorFactory)

Register registers an extractor factory for internal use.

Types

type Context

type Context struct {
	// contains filtered or unexported fields
}

Context implements the Context for internal use.

func NewContext

func NewContext(ctx context.Context, option Option) *Context

NewContext creates a new Context with the provided options.

func (*Context) Client

func (c *Context) Client() *resty.Client

Client returns the resty client associated with this Context.

func (*Context) Context

func (c *Context) Context() context.Context

Context returns the context associated with this Context.

func (*Context) GetProgressCallback

func (c *Context) GetProgressCallback() ProgressCallback

GetProgressCallback returns the progress callback

func (*Context) Logger

func (c *Context) Logger() *slog.Logger

Logger returns the logger associated with this Context.

func (*Context) Option

func (c *Context) Option() Option

Option returns the options associated with this Context.

func (*Context) SetProgressCallback

func (c *Context) SetProgressCallback(callback ProgressCallback)

SetProgressCallback sets the progress callback for the context

type Downloader

type Downloader struct {
	// contains filtered or unexported fields
}

Downloader manages high-level download logic with support for HTTP range requests, resumable downloads, and multi-threaded downloads.

func NewDownloader

func NewDownloader(ctx *Context) *Downloader

NewDownloader creates a new Downloader instance with the provided context.

func (*Downloader) Download

func (d *Downloader) Download(medias []Media) error

Download downloads all streams from the extracted media for the given URL. If ExtractOnly is set, only prints media info without downloading.

func (*Downloader) Stop

func (d *Downloader) Stop()

Stop gracefully cancels all ongoing downloads

type Extractor

type Extractor interface {
	CanExtract(url string) bool
	Extract(url string) ([]Media, error)
}

Extractor defines the interface for media extractors.

func FindExtractor

func FindExtractor(ctx *Context, url string) (Extractor, error)

FindExtractor finds a suitable extractor for the given URL.

type Filter

type Filter interface {
	Filter(stream Stream) bool
}

Filter defines the interface for stream filtering.

type Media

type Media struct {
	Title       string            // Media title or name
	Streams     []Stream          // All available streams (keyed by quality or id)
	Thumbnail   string            // URL to thumbnail image (optional)
	Description string            // Description of the media (optional)
	Extra       map[string]string // Additional info for extensibility
}

Media represents a downloadable media resource with multiple streams.

func (*Media) String

func (m *Media) String() string

type Option

type Option struct {
	// Output options
	OutputPath string // Output directory for downloaded files (--output-dir, -o)
	OutputName string // Output filename (--output-filename, -O)

	// Quality and format
	Quality string // Preferred video quality, e.g. "best", "worst", "720p" (--quality, -q)
	Format  string // Output format, e.g. "mp4", "mkv", "mp3" (--format, -f)

	// Network options
	Headers    http.Header   // Custom HTTP headers (--header, -H)
	UserAgent  string        // Custom user agent (--user-agent, -u)
	Proxy      string        // HTTP proxy URL (--proxy, -x)
	RetryCount int           // Number of retry attempts (--retry, -r)
	Timeout    time.Duration // Request timeout (--timeout, -t)

	// Rate limit (bytes per second), 0 means unlimited
	RateLimit int64 // Download speed limit (--rate-limit)

	// Advanced authentication
	AuthType   string // Authentication type: "", "basic", "bearer", "header" (--auth-type)
	AuthUser   string // Username for basic auth (--auth-user)
	AuthPass   string // Password for basic auth (--auth-pass)
	AuthToken  string // Token for bearer auth (--auth-token)
	AuthHeader string // Custom header for auth, e.g. "X-API-Key: ..." (--auth-header)
	Cookie     string // Cookie file path for authentication (--cookies, -c)

	// Download options
	Threads        int   // Number of concurrent download threads (--threads, -n)
	ChunkSize      int64 // Download chunk size in bytes
	NoSkipExisting bool  // Do not skip existing files (--no-skip, -S)

	// Behavior options
	ExtractOnly   bool // Only extract media info, do not download (--info, -i)
	Playlist      bool // Download all videos in playlist (--playlist, -p)
	PlaylistStart int  // Playlist start index (--playlist-start)
	PlaylistEnd   int  // Playlist end index (--playlist-end)

	// Content options
	Subtitle     bool // Download subtitles (--subtitle)
	VideoOnly    bool // Download video only, no audio (--video-only)
	AudioOnly    bool // Download audio only (--audio-only)
	IgnoreErrors bool // Continue on errors (--ignore-errors)

	// Error handling and logging
	Debug   bool // Enable debug logging (--debug, -d)
	Verbose bool // Enable verbose output (--verbose, -v)
	Silent  bool // Suppress all output except errors (--silent)
}

Option defines all configurable parameters for downloading and extraction. Each field corresponds to a command-line flag in main.go/setupFlags. Callers should ensure OutputPath exists or is creatable. Threads should be >=1; ChunkSize in bytes.

func (*Option) Combine

func (o *Option) Combine(other Option)

type PlaylistFilter

type PlaylistFilter struct {
	Start int
	End   int
}

PlaylistFilter filters playlist streams by index range.

func (*PlaylistFilter) Filter

func (f *PlaylistFilter) Filter(stream Stream) bool

type ProgressCallback

type ProgressCallback func(current, total int64, description string)

ProgressCallback defines the callback function for progress updates

type Stream

type Stream struct {
	ID       string            // Unique identifier for this stream
	Title    string            // Title or name of the stream
	Type     StreamType        // Type of the stream (video, audio, etc.)
	URL      string            // Direct URL to this stream
	Format   string            // Format (e.g., "mp4", "webm", "mp3")
	Quality  string            // Quality/bitrate info (e.g., "1080p", "320kbps")
	Size     int64             // Size in bytes (if known)
	Duration time.Duration     // Duration of the stream (if applicable)
	Header   http.Header       // Custom headers for this stream (optional)
	Extra    map[string]string // Extensible fields (e.g., codec info)
	SaveAs   string            // Suggested filename to save this stream
}

Stream represents a single media stream (e.g. one quality/format)

type StreamType

type StreamType string

StreamType defines the type of media resource.

const (
	StreamTypeVideo    StreamType = "video"
	StreamTypeAudio    StreamType = "audio"
	StreamTypeImage    StreamType = "image"
	StreamTypeSubtitle StreamType = "subtitle"
	StreamTypePlaylist StreamType = "playlist"
	StreamTypeM3u8     StreamType = "m3u8"
	StreamTypeDocument StreamType = "document"
	StreamTypeOther    StreamType = "other"
)