getit

README

A Go library for fetching and unpacking archives from multiple sources.

This is a bit like hashicorp/go-getter, but much simpler, and with less supported protocols (so far).

Features

• Git repositories: Clone from git://, git+ssh://, or git+https:// URLs with optional ref and depth parameters
• TAR archives: Fetch and extract .tar, .tar.gz, .tar.bz2, .tar.xz, and other compressed tarballs
• ZIP archives: Download and unzip .zip files
• GitHub shortcuts: Map shorthand URLs like user/repo or github.com/user/repo to full git URLs
• Subdirectory support: Extract specific subdirectories using // delimiter (e.g., https://example.com/archive.tar.gz//subdir)

Usage

import "github.com/block/getit"

// Create a fetcher with resolvers and mappers
fetcher := getit.New(
    []getit.Resolver{
        getit.NewGit(),
        getit.NewTAR(),
        getit.NewZIP(),
    },
    []getit.Mapper{
        getit.GitHub,
        getit.GitHubOrgRepo,
        getit.SingleGitHubOrg("myorg"),
    },
)

// Fetch an archive
err := fetcher.Fetch(ctx, "user/repo?ref=main&depth=1", "./destination")

Documentation

Overview

Package getit provides a simple API for fetching archives.

Index

• Variables
• func Fetch(ctx context.Context, source, dest string) error
• func FetchIntoPipe(ctx context.Context, u *url.URL, cmd string, args ...string) error
• func FilePath(source string) (string, bool)
• func GitHub(source string) (string, bool)
• func GitHubOrgRepo(source string) (string, bool)
• func Resolve(source string) (Resolver, Source, error)
• type Fetcher
  • func New(resolvers []Resolver, mappers []Mapper) *Fetcher
  • func (f *Fetcher) Fetch(ctx context.Context, source, dest string) error
  • func (f *Fetcher) Resolve(source string) (Resolver, Source, error)
• type File
  • func NewFile() *File
  • func (f *File) Fetch(ctx context.Context, source Source, dest string) error
  • func (f *File) Match(source *url.URL) bool
• type Git
  • func NewGit() *Git
  • func (g *Git) Fetch(ctx context.Context, source Source, dest string) error
  • func (g *Git) Match(source *url.URL) bool
• type Mapper
  • func SingleGitHubOrg(org string) Mapper
• type Resolver
• type Source
• type TAR
  • func NewTAR() *TAR
  • func (t *TAR) Fetch(ctx context.Context, source Source, dest string) error
  • func (t *TAR) Match(source *url.URL) bool
• type ZIP
  • func NewZIP() *ZIP
  • func (z *ZIP) Fetch(ctx context.Context, source Source, dest string) error
  • func (z *ZIP) Match(source *url.URL) bool

Variables

var Default = New(
	[]Resolver{
		NewFile(),
		NewGit(),
		NewTAR(),
		NewZIP(),
	},
	[]Mapper{
		GitHub,
		GitHubOrgRepo,
		FilePath,
	},
)

Default Fetcher with built-in resolvers and mappers.

Functions

func Fetch

func Fetch(ctx context.Context, source, dest string) error

Fetch fetches an archive from a source and unpacks it to a destination.

func FetchIntoPipe

func FetchIntoPipe(ctx context.Context, u *url.URL, cmd string, args ...string) error

FetchIntoPipe retrieves the given URL using Go's HTTP library then pipes it into the input of the given command.

func FilePath

func FilePath(source string) (string, bool)

FilePath is a Mapper that maps filesystem paths to file:// URLs.

It handles absolute paths, relative paths (./..., ../...), home-relative paths (~/...), and bare directory names. The path must exist and be a directory.

func GitHub

func GitHub(source string) (string, bool)

GitHub is a Mapper that supports shorthand GitHub URLs with no scheme or org/repo.

Query parameters and anchors are preserved.

func GitHubOrgRepo

func GitHubOrgRepo(source string) (string, bool)

GitHubOrgRepo is a Mapper that supports shorthand GitHub URLs with org/repo.

Query parameters and anchors are preserved.

func Resolve

func Resolve(source string) (Resolver, Source, error)

Resolve a source string to a Source and URL.

Types

type Fetcher

type Fetcher struct {
	// contains filtered or unexported fields
}

Fetcher retrieves archives from a pluggable source.

All sources support an optional subdirectory, specified via appending a //<subdir> to the URL path:

git+ssh://host/path/to/repo.git//path/to/subdir
https://host/path/to/archive.tgz//path/to/subdir

func New

func New(resolvers []Resolver, mappers []Mapper) *Fetcher

func (*Fetcher) Fetch

func (f *Fetcher) Fetch(ctx context.Context, source, dest string) error

Fetch fetches an archive from a source and unpacks it to a destination.

func (*Fetcher) Resolve

func (f *Fetcher) Resolve(source string) (Resolver, Source, error)

Resolve a source string to a Source and URL.

type File

type File struct{}

File is a Resolver that copies local directories.

The URL format supported is:

file:///absolute/path/to/dir
file://relative/path/to/dir

func NewFile

func NewFile() *File

func (*File) Fetch

func (f *File) Fetch(ctx context.Context, source Source, dest string) error

func (*File) Match

func (f *File) Match(source *url.URL) bool

type Git

type Git struct{}

The Git Resolver uses Git repositories as archive sources, cloning directly.

The URL format supported is:

git://host/path/to/repo
git+ssh://host/path/to/repo
git+https://host/path/to/repo

All forms support the following query parameters that control cloning behaviour:

ref=<ref>
depth=<depth>

func NewGit

func NewGit() *Git

func (*Git) Fetch

func (g *Git) Fetch(ctx context.Context, source Source, dest string) error

func (*Git) Match

func (g *Git) Match(source *url.URL) bool

type Mapper

type Mapper func(source string) (string, bool)

Mapper maps one form of a source to another.

eg.

github.com/user/repo -> https://github.com/user/repo.git
user/repo -> https://github.com/user/repo.git

func SingleGitHubOrg

func SingleGitHubOrg(org string) Mapper

SingleGitHubOrg is a Mapper that supports shorthand GitHub URLs as just repo.

Query parameters and anchors are preserved.

type Resolver

type Resolver interface {
	// Match returns true if this Resolver can handle the given source URL.
	Match(source *url.URL) bool
	// Fetch an archive from a source and unpacks it to a destination.
	Fetch(ctx context.Context, source Source, dest string) error
}

type Source

type Source struct {
	URL    *url.URL
	SubDir string
}

Source is a resolved source with optional sub-directory.

type TAR

type TAR struct{}

The TAR Resolver knows how to unpack tarballs.

func NewTAR

func NewTAR() *TAR

func (*TAR) Fetch

func (t *TAR) Fetch(ctx context.Context, source Source, dest string) error

func (*TAR) Match

func (t *TAR) Match(source *url.URL) bool

type ZIP

type ZIP struct{}

func NewZIP

func NewZIP() *ZIP

func (*ZIP) Fetch

func (z *ZIP) Fetch(ctx context.Context, source Source, dest string) error

func (*ZIP) Match

func (z *ZIP) Match(source *url.URL) bool