README
¶
JetGet
JetGet is an open-source, thread-safe Go library for high-speed concurrent file downloading over HTTP. It features resumable downloads, real-time progress tracking, and optimized disk I/O handling via pre-allocation and concurrent file writing.
Features
- Concurrent Downloading: Splits the file into chunks and downloads them in parallel, reducing overall download time.
- Resumable Downloads: Saves internal state periodically to a
.jetgetJSON file, allowing downloads to be resumed even after application crashes or network interruptions. - Pre-allocation: Truncates the file to the final size before downloading to prevent disk fragmentation.
- Zero File Merging: Uses
os.File.WriteAtto write chunks directly into the final file concurrently, avoiding expensive post-download merging. - Functional Options: Clean API design pattern for client configuration.
- HTTP/3 (QUIC) Support: Next-generation HTTP protocol support for faster downloads on unreliable networks.
- Real-time Progress: Channel-based, non-blocking progress updates for building progress bars or UI.
Installation
go get github.com/Locon213/jetget
Quick Start
package main
import (
"context"
"fmt"
"log"
"github.com/Locon213/jetget"
)
func main() {
// 1. Initialize the client with functional options
client := jetget.NewClient(
jetget.WithConcurrency(8),
jetget.WithOutputDir("./downloads"),
jetget.WithUserAgent("MyCustomDownloader/1.0"),
jetget.WithHTTP3(), // Enable HTTP/3 for faster connections
)
url := "https://example.com/large-file.zip"
ctx := context.Background()
// 2. Start the download in the background
task, err := client.Download(ctx, url)
if err != nil {
log.Fatalf("Download failed to start: %v", err)
}
// 3. (Optional) Track progress in real-time
go func() {
for stats := range task.Progress() {
if stats.TotalBytes > 0 {
percent := float64(stats.DownloadedBytes) / float64(stats.TotalBytes) * 100
fmt.Printf("\rDownloaded: %d / %d bytes (%.2f%%)", stats.DownloadedBytes, stats.TotalBytes, percent)
} else {
fmt.Printf("\rDownloaded: %d bytes (unknown total size)", stats.DownloadedBytes)
}
}
}()
// 4. Wait for the download to finish
if err := task.Wait(); err != nil {
log.Fatalf("\nDownload error: %v", err)
}
fmt.Printf("\nDownload completed successfully! File saved to: %s\n", task.FilePath)
}
How It Works
- Initialization: The client sends an HTTP
HEADrequest to query theContent-Lengthand checks if the server supportsAccept-Ranges: bytes. - Pre-allocation: An empty file of the exact
Content-Lengthis created viaos.Truncate. - Chunking: The file goes through mathematical chunking and is downloaded concurrently by
nconfigured goroutines. - Resumability: Every second, a background goroutine exports
ChunkStateto a sidecar JSON file (e.g.,file.zip.jetget). Upon success, this state file is pruned. If the application restarts, it restores the state from this file and continues from previous offsets.
Dependencies
JetGet strictly relies on the Go Standard Library with additions for robust concurrency and protocol handling:
golang.org/x/sync/errgroupgithub.com/quic-go/quic-go/http3
License
MIT License
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ChunkState ¶
type ChunkState struct {
Index int `json:"index"`
Start int64 `json:"start"`
End int64 `json:"end"`
Current int64 `json:"current"` // Текущее глобальное смещение в байтах
}
ChunkState содержит информацию о чанке для возможной докачки.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client реализует основной клиент для многопоточной загрузки.
func NewClient ¶
func NewClient(opts ...ClientOption) *Client
NewClient создает новый экземпляр Client с заданными опциями.
type ClientOption ¶
type ClientOption func(*Options)
ClientOption — функциональная опция для настройки клиента.
func WithBufferSize ¶
func WithBufferSize(size int) ClientOption
WithBufferSize задает размер буфера чтения (по умолчанию 512 KB).
func WithConcurrency ¶
func WithConcurrency(n int) ClientOption
WithConcurrency задает количество параллельных потоков загрузки.
func WithHTTP3 ¶
func WithHTTP3() ClientOption
WithHTTP3 включает поддержку протокола HTTP/3 (QUIC) для клиента.
func WithMirrors ¶
func WithMirrors(urls ...string) ClientOption
WithMirrors задает дополнительные зеркала для скачивания.
func WithOutputDir ¶
func WithOutputDir(dir string) ClientOption
WithOutputDir задает директорию для сохранения скачанных файлов.
func WithUserAgent ¶
func WithUserAgent(ua string) ClientOption
WithUserAgent задает заголовок User-Agent для HTTP-запросов.
type Options ¶
type Options struct {
Concurrency int
UserAgent string
OutputDir string
HTTP3 bool
BufferSize int
Mirrors []string
}
Options хранит настройки для клиента.
Source Files
Directories