download

README

Download extension

extension/download is the public, transport-neutral download engine. It owns range probing, multipart assembly, bounded retries, idle timeouts, response validation, cancellation, and exact-length checks.

The caller supplies a Transport. Authentication, URL trust policy, and HTTP error classification belong to that transport; storage belongs to the consumer of Stream.Body.

Choose the representation contract deliberately:

• download.MutableSource: safe default. Multipart assembly requires a strong ETag; otherwise the engine restarts with one full response.
• download.ImmutableSource: permits multipart assembly without an ETag because the caller guarantees the source identifier pins the bytes.

External commands normally use the higher-level command.Download, which wires an authenticated OpenAPI transport and invocation-scoped FileIO while retaining the same engine options:

target := command.FileTarget{Name: args.Output}
options := command.DownloadOptions{
    Representation: download.Immutable,
    Transfer: download.Options{
        PartSize: 8 << 20,
    },
}

// Dry-run reports the logical destination without opening a stream.
dryRun := command.NewDryRun(request).File(
    target.Intent("OpenAPI response body"),
)

// Execute streams and saves the file; use the host-resolved location.
artifact, err := command.Download(
    ctx,
    commandContext,
    request,
    target,
    options,
)

The zero command.DownloadOptions value selects download.Mutable and the production transfer defaults. Existing targets fail by default; overwriting requires FileTarget{IfExists: command.IfExistsOverwrite}.

For a pre-signed or CDN URL, use command.DownloadURL with the same target and options. It accepts HTTPS only and routes through the host's external-request, SSRF, DNS/IP pinning, and redirect policies. Dry-run should describe the file intent without echoing a signed URL:

dryRun := command.NewDryRun().
    Desc("Download an external HTTPS resource").
    File(target.Intent("external URL response body"))

artifact, err := command.DownloadURL(
    ctx,
    commandContext,
    args.URL,
    target,
    options,
)

Documentation

Overview

Package download provides validated full and ranged streams for extensions and built-in commands. It owns transfer bounds, retries, representation consistency, progress timeouts, and response-length checks; callers supply a Transport that owns authentication and source/URL policy, and they choose the destination that consumes Stream.Body.

Index

• Constants
• type ByteRange
  • func (r ByteRange) HeaderValue() string
• type Options
• type Representation
• type Request
  • func (r Request) Headers() http.Header
• type Source
  • func ImmutableSource(transport Transport) Source
  • func MutableSource(transport Transport) Source
• type Stream
  • func Open(ctx context.Context, source Source, opts Options) (*Stream, error)
• type Transport

Constants

const (
	// DefaultPartSize keeps a 100 MiB object near 13 requests while capping replay at 8 MiB.
	DefaultPartSize = int64(8 * 1024 * 1024)
	// DefaultPartRetries tolerates three transient failures without prolonged retrying.
	DefaultPartRetries = 3
	// DefaultRetryDelay keeps three local backoffs below one second before jitter.
	DefaultRetryDelay = 100 * time.Millisecond
	// DefaultRetryWaitBudget caps cumulative local sleep for interactive callers.
	DefaultRetryWaitBudget = 3 * time.Second
	// DefaultIdleTimeout detects a dead connection within a minute without limiting slow progress.
	DefaultIdleTimeout = 60 * time.Second
)

Types

type ByteRange

type ByteRange struct {
	Start int64
	End   int64
	// contains filtered or unexported fields
}

ByteRange is an inclusive HTTP byte range.

func (ByteRange) HeaderValue

func (r ByteRange) HeaderValue() string

HeaderValue returns the value for an HTTP Range header.

type Options

type Options struct {
	// PartSize is the maximum byte count requested per range. Zero selects
	// DefaultPartSize.
	PartSize int64
	// MaxResponses bounds the total responses in one logical stream. Zero derives
	// a bound from the declared object size and PartSize.
	MaxResponses int
	// MaxPartRetries bounds retries per range. Zero selects the default.
	MaxPartRetries int
	// RetryDelay is the base exponential backoff. Zero selects the default.
	RetryDelay time.Duration
	// RetryWaitBudget bounds cumulative retry sleeps. Zero selects the default.
	RetryWaitBudget time.Duration
	// IdleTimeout bounds waiting for response headers or one body read. Time
	// between caller reads does not count. Zero selects the default.
	IdleTimeout time.Duration
	// DisableMultipart forces one full response.
	DisableMultipart bool
	// contains filtered or unexported fields
}

Options controls multipart behavior. Zero values select production defaults.

type Representation

type Representation string

Representation declares whether repeated range requests are guaranteed to address the same bytes.

const (
	// Mutable requires a strong ETag before Open combines multiple responses.
	Mutable Representation = "mutable"
	// Immutable permits multipart reads without an ETag because the caller
	// guarantees that the source identifier pins one representation.
	Immutable Representation = "immutable"
)

type Request

type Request struct {
	Range   *ByteRange
	IfRange string
	// contains filtered or unexported fields
}

Request describes one full or ranged fetch.

func (Request) Headers

func (r Request) Headers() http.Header

Headers keeps byte offsets tied to the transferred representation.

type Source

type Source struct {
	// contains filtered or unexported fields
}

Source binds a transport to its representation stability.

func ImmutableSource

func ImmutableSource(transport Transport) Source

ImmutableSource allows multipart reads without a validator.

func MutableSource

func MutableSource(transport Transport) Source

MutableSource requires a strong ETag before combining responses.

type Stream

type Stream struct {
	// Body joins all validated parts and must be closed by the caller.
	Body io.ReadCloser
	// Header is a copy of the first successful response headers.
	Header http.Header
	// ContentLength is the validated total size, or -1 when unknown.
	ContentLength int64
	// contains filtered or unexported fields
}

Stream is one logical full or multipart response.

func Open

func Open(ctx context.Context, source Source, opts Options) (*Stream, error)

Open probes range support and returns one validated stream.

type Transport

type Transport func(context.Context, Request) (*http.Response, error)

Transport performs one replay-safe fetch and binds ctx to the response body. Non-successful HTTP responses must be returned as typed errors with retryability decided at this boundary.