binkit

README

binkit

Provision, verify, pin, and update external CLI binaries from Go.

Some tools you depend on aren't written in Go, so go tool can't manage them. binkit fetches them from GitHub releases, verifies them against a pinned SHA-256, caches them per version, and hands your program back a path.

Pin once, from the command line, and commit the result:

binkit pin typst        # writes tools.json — commit it

Then resolve it from your program:

path, err := resolver.Ensure(ctx, catalog.Typst())
if err != nil {
    return err
}
cmd := exec.CommandContext(ctx, path, "compile", "in.typ", "out.pdf")

Adding binkit to a Go application? INTEGRATION.md is the step-by-step guide.

CLI

binkit pin <tool>[@version]   resolve, install, and record the pin
binkit ensure <tool>          install if needed, print the path
binkit path <tool>            print an installed tool's path; never uses the network
binkit list                   list the pins in the lock file

Stdout carries only paths and the pin table, so command substitution is safe:

TYPST=$(binkit ensure typst) && "$TYPST" compile in.typ out.pdf

Install it with go install github.com/jroedel/binkit/cmd/binkit@latest, or manage it as a tool dependency with go get -tool — binkit is a Go program, so go tool handles it, which is exactly what it cannot do for Typst.

The one-dependency promise

go.mod has exactly one require line: github.com/ulikunitz/xz, which is itself stdlib-only with zero transitive dependencies.

That dependency exists for one reason — many projects (Typst among them) ship .tar.xz, and the Go standard library has archive/tar, archive/zip, and compress/gzip but no xz decoder. Shelling out to tar -xJf would trade a compile-time dependency for a runtime environment dependency and break on Windows and minimal containers, so it isn't done.

Everything else is stdlib or deliberately hand-rolled: semver comparison instead of golang.org/x/mod, TTY detection instead of mattn/go-isatty, atomic directory rename instead of a file-locking library.

Pinning

tools.json lives in your repo and is checked in:

{
  "typst": {
    "version": "0.15.1",
    "repo": "typst/typst",
    "digests": {
      "linux/amd64":   "sha256:...",
      "darwin/arm64":  "sha256:...",
      "windows/amd64": "sha256:..."
    }
  }
}

Digests are captured for every platform at pin time, so a colleague on macOS gets a verified install from a lockfile generated on Linux.

Ensure never resolves "latest": a pinned version downloads by direct URL, so provisioning makes no GitHub API call, needs no token, and cannot hit the API rate limit. The one API request Ensure can make is the advisory update check below, which changes nothing and is disableable. Only an explicit Update call changes the pin.

Update checks

Once every seven days, binkit checks whether a newer release exists and reports it — to stderr, and only when stderr is a terminal, so it never pollutes piped output or CI logs. When there is nowhere to show a notice the check is skipped entirely rather than performed and discarded.

binkit: typst 0.15.1 is pinned; 0.16.0 is available.
        run: massprep --update-tools typst

The second line appears only if you set Resolver.UpdateHint — binkit cannot know your CLI's flags.

A failed check backs off for an hour rather than consuming the seven-day window, so being offline for a week neither retries on every invocation nor silences the next real check. Nothing here can fail a build, and nothing is ever updated automatically. Set BINKIT_NO_UPDATE_CHECK=1, or Resolver.NoCheck, to disable.

Escape hatch

BINKIT_<TOOLNAME> (e.g. BINKIT_TYPST=/usr/bin/typst) short-circuits everything and uses that binary. Useful for CI images, Nix, and distro packages.

The tool name is upper-cased and every character outside A-Z0-9 becomes an underscore, so go-task reads BINKIT_GO_TASK. Don't name a tool cache or no-update-check — those collide with BINKIT_CACHE and BINKIT_NO_UPDATE_CHECK.

Stability

v0: the Go API may change in a minor release. The lock file format is held to a stricter standard, since it lives in your repository and a break there breaks builds. Changes are recorded in CHANGELOG.md.

License

MIT

Documentation

Overview

Package binkit provisions, verifies, pins, and updates external CLI binaries that a Go program depends on but cannot build itself.

The Go toolchain can manage Go-built tools via "go tool", but nothing in the toolchain helps with a Rust or C binary you shell out to. binkit fills that gap: it downloads a pinned release from GitHub, verifies it against a recorded SHA-256, caches it by version, and returns a path.

Pinning

Pins live in a lock file — tools.json by convention — which belongs in the consuming project's repository. Resolver.Ensure reads the pin, never resolves "latest", and fetches the pinned asset by direct download URL. Provisioning therefore makes no GitHub API call, needs no token, and is not subject to the API rate limit. Only Resolver.Update changes a pin.

Update checks

After a tool is in place, Ensure may report that a newer release exists. That check is the one part of Ensure that queries the GitHub API. It runs at most once per Resolver.CheckEvery, only when a notice could actually be displayed, never changes what is installed, and never returns an error. Resolver.NoCheck and EnvNoUpdateCheck disable it.

Stability

The module is v0: the Go API may still change in a minor release. The lock file format is treated more conservatively, because it lives in a consuming project's repository — a break there breaks builds rather than merely compilations. Any change to it is called out in CHANGELOG.md.

Design

binkit knows nothing about any particular tool. A Tool is plain data supplied by the caller; ready-made definitions live in the separate catalog subpackage, which this package deliberately does not import. Ensure returns a path and never executes anything — what to run is the caller's decision.

Example

The common case: resolve a pinned tool to a path, then run it. Ensure downloads and verifies against the pinned digest on a cache miss, and returns the cached path otherwise.

package main

import (
	"context"
	"log"
	"os/exec"

	"github.com/jroedel/binkit"
	"github.com/jroedel/binkit/catalog"
)

func main() {
	ctx := context.Background()
	var resolver binkit.Resolver

	path, err := resolver.Ensure(ctx, catalog.Typst())
	if err != nil {
		log.Fatal(err)
	}

	cmd := exec.CommandContext(ctx, path, "compile", "in.typ", "out.pdf")
	if err := cmd.Run(); err != nil {
		log.Fatal(err)
	}
}

Index

• Constants
• Variables
• type LockEntry
• type LockFile
• type Platform
  • func (p Platform) String() string
• type Resolver
  • func (r *Resolver) CachedPath(t Tool) (string, error)
  • func (r *Resolver) Ensure(ctx context.Context, t Tool) (string, error)
  • func (r *Resolver) Pins() (LockFile, error)
  • func (r *Resolver) Update(ctx context.Context, t Tool, version string) (string, error)
• type Tool
  • func (t Tool) EnvKey() string

Examples

• Package
• Resolver (UpdateNotices)
• Resolver.Ensure (Unpinned)
• Resolver.Pins
• Resolver.Update
• Tool
• Tool.EnvKey

Constants

const (
	// DefaultCheckEvery is the minimum interval between upstream update checks.
	DefaultCheckEvery = 7 * 24 * time.Hour

	// EnvNoUpdateCheck disables update checks when set to any non-empty value.
	EnvNoUpdateCheck = "BINKIT_NO_UPDATE_CHECK"
)

const (
	// EnvCacheDir overrides the cache location.
	EnvCacheDir = "BINKIT_CACHE"
)

Environment variables binkit consults.

Variables

var (
	ErrInvalidTool         = errors.New("binkit: invalid tool definition")
	ErrNotPinned           = errors.New("binkit: tool is not pinned")
	ErrNotCached           = errors.New("binkit: tool is pinned but not installed")
	ErrNoDigest            = errors.New("binkit: no pinned digest for this platform")
	ErrDigestMismatch      = errors.New("binkit: digest mismatch")
	ErrUnsupportedPlatform = errors.New("binkit: tool is not published for this platform")
	ErrUnsupportedArchive  = errors.New("binkit: unsupported archive format")
	ErrBinaryNotInArchive  = errors.New("binkit: binary not found in archive")
	ErrAssetNotFound       = errors.New("binkit: release has no such asset")
)

Errors reported by this package. All are wrapped, so match with errors.Is.

var DefaultPlatforms = []Platform{
	{OS: "linux", Arch: "amd64"},
	{OS: "linux", Arch: "arm64"},
	{OS: "darwin", Arch: "amd64"},
	{OS: "darwin", Arch: "arm64"},
	{OS: "windows", Arch: "amd64"},
	{OS: "windows", Arch: "arm64"},
}

DefaultPlatforms is the set Resolver.Update records digests for. A tool that does not publish for one of these is skipped, so an over-broad list costs nothing.

Types

type LockEntry

type LockEntry struct {
	Version string            `json:"version"`
	Repo    string            `json:"repo"`
	Digests map[string]string `json:"digests,omitzero"`
}

LockEntry is one tool's pinned state.

Digests are keyed by "GOOS/GOARCH" and hold the SHA-256 of the *release asset*, not of the extracted binary. They are captured for every platform the tool supports at pin time, so a lock file generated on Linux still yields a verified install on macOS.

type LockFile

type LockFile map[string]LockEntry

LockFile maps tool name to pin. It belongs in the consuming project's repository and is meant to be committed — it is what makes a build reproducible. Resolver.Pins reads one.

type Platform

type Platform struct {
	OS   string
	Arch string
}

Platform is a GOOS/GOARCH pair.

func (Platform) String

func (p Platform) String() string

type Resolver

type Resolver struct {
	// CacheDir overrides the cache location. Falls back to $BINKIT_CACHE, then to
	// <user cache dir>/binkit.
	CacheDir string

	// Lock is the path to the lock file. Defaults to "tools.json".
	Lock string

	// HTTP is the client used for all requests. Defaults to [http.DefaultClient].
	HTTP *http.Client

	// Platforms are the platforms Update records digests for. Defaults to
	// [DefaultPlatforms].
	Platforms []Platform

	// Now returns the current time. Defaults to [time.Now].
	Now func() time.Time

	// Stderr receives update notices. Defaults to [os.Stderr]. Notices never go to
	// stdout, and are suppressed entirely when the default stderr is not a terminal —
	// a CI log has no one to read them.
	Stderr io.Writer

	// CheckEvery is the minimum interval between upstream update checks. Defaults to
	// [DefaultCheckEvery].
	CheckEvery time.Duration

	// NoCheck disables update checks. The BINKIT_NO_UPDATE_CHECK environment variable
	// does the same for an end user.
	NoCheck bool

	// UpdateHint returns the command that updates the named tool, shown as a second
	// line of the update notice. binkit cannot know a consuming CLI's flags, so
	// without this the notice reports versions only.
	UpdateHint func(toolName string) string
	// contains filtered or unexported fields
}

Resolver installs tools. The zero value is usable: it caches under the user cache directory and reads tools.json from the working directory.

A Resolver is safe for concurrent use and must not be copied after first use.

Example (UpdateNotices)

Update notices are advisory. UpdateHint supplies the second line, because binkit cannot know a consuming CLI's flags.

package main

import (
	"context"

	"github.com/jroedel/binkit"
	"github.com/jroedel/binkit/catalog"
)

func main() {
	resolver := binkit.Resolver{
		UpdateHint: func(tool string) string {
			return "myapp --update-tools " + tool
		},
	}

	// binkit: typst 0.15.1 is pinned; 0.16.0 is available.
	//         run: myapp --update-tools typst
	_, _ = resolver.Ensure(context.Background(), catalog.Typst())
}

func (*Resolver) CachedPath

func (r *Resolver) CachedPath(t Tool) (string, error)

CachedPath returns the path to the pinned version of t if it is already installed.

Unlike Resolver.Ensure it never downloads, never verifies, and never touches the network — it answers "is this already provisioned?" and nothing else. A pinned tool that is not in the cache yields ErrNotCached rather than being fetched, which is what makes this usable on a machine that is deliberately offline.

The per-tool environment override is honoured first, exactly as Ensure honours it, so a caller that consults CachedPath before Ensure cannot reach a different conclusion about which binary is in play.

func (*Resolver) Ensure

func (r *Resolver) Ensure(ctx context.Context, t Tool) (string, error)

Ensure returns the path to the pinned version of t, downloading and verifying it if it is not already cached.

Provisioning downloads by direct URL and makes no GitHub API call, so it needs no token and is not subject to the API rate limit. A tool already in the cache is not downloaded again.

Ensure may still make one API request after the tool is in place: the advisory update check, which runs at most once per Resolver.CheckEvery, only when a notice could actually be displayed, and never changes what was installed or fails the call. Set Resolver.NoCheck or EnvNoUpdateCheck to suppress it, in which case Ensure touches the network only to download an uncached tool.

A tool with no pin is an error rather than an implicit "fetch latest": changing what a build runs should be a deliberate, reviewable act.

Example (Unpinned)

A tool with no pin is reported as binkit.ErrNotPinned rather than silently fetched, so a CLI can tell the user which command would establish the pin.

package main

import (
	"context"
	"errors"
	"fmt"
	"log"

	"github.com/jroedel/binkit"
	"github.com/jroedel/binkit/catalog"
)

func main() {
	var resolver binkit.Resolver

	_, err := resolver.Ensure(context.Background(), catalog.Typst())
	switch {
	case errors.Is(err, binkit.ErrNotPinned):
		fmt.Println("run: myapp --update-tools typst")
	case err != nil:
		log.Fatal(err)
	}
}

func (*Resolver) Pins

func (r *Resolver) Pins() (LockFile, error)

Pins returns every tool pinned in the lock file, keyed by tool name.

This is how a caller learns which version Resolver.Ensure will resolve, and with what digest: Ensure itself returns only a path. Pins reads a local file — it never contacts the network and never installs anything.

A missing lock file yields an empty LockFile rather than an error, matching Ensure, which reports an unpinned tool as ErrNotPinned rather than as an I/O failure. The returned map is freshly allocated, so mutating it is safe and affects nothing; Resolver.Update is the only thing that writes a pin.

Example

Pins reads the lock file without touching the network. It is how a CLI reports the versions it is pinned to, since Ensure returns only a path.

package main

import (
	"fmt"
	"log"
	"maps"
	"slices"

	"github.com/jroedel/binkit"
)

func main() {
	var resolver binkit.Resolver

	pins, err := resolver.Pins()
	if err != nil {
		log.Fatal(err)
	}
	for _, name := range slices.Sorted(maps.Keys(pins)) {
		fmt.Printf("%s %s (%s)\n", name, pins[name].Version, pins[name].Repo)
	}
}

func (*Resolver) Update

func (r *Resolver) Update(ctx context.Context, t Tool, version string) (string, error)

Update resolves a version — the latest release when version is empty — installs it, and rewrites the lock file.

Digests are recorded for every platform in Resolver.Platforms from the single release response, so one run on Linux produces a lock file that verifies correctly on macOS and Windows too.

Example

Update resolves a version — the latest release when version is empty — installs it, and rewrites the lock file with digests for every platform in Resolver.Platforms, so one run on Linux produces a lock file that verifies on macOS and Windows too.

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/jroedel/binkit"
	"github.com/jroedel/binkit/catalog"
)

func main() {
	resolver := binkit.Resolver{Lock: "tools.json"}

	path, err := resolver.Update(context.Background(), catalog.Typst(), "")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("installed", path)
}

type Tool

type Tool struct {
	// Name keys the cache and the lock file, and forms the per-tool environment
	// override. Two names are unavailable because they would collide with binkit's
	// own variables; see [Tool.EnvKey]. Required.
	Name string

	// Repo is the GitHub "owner/name" hosting the releases. Required.
	Repo string

	// Tag maps a version to its release tag. Optional; defaults to "v" + version.
	Tag func(version string) string

	// Asset returns the release asset filename for a version and platform. Returning
	// an error means the tool is not published for that platform, which Update treats
	// as "skip" rather than as a failure. Required.
	Asset func(version, goos, goarch string) (string, error)

	// BinaryPath returns the path of the executable inside the archive. Required.
	BinaryPath func(version, goos, goarch string) string
}

Tool describes an external binary and where to obtain it. It is data, not behaviour: the two function fields exist only because asset naming varies per project and cannot be expressed as a format string in general.

Example

A tool the catalog does not cover is an ordinary struct literal — nothing in the catalog is privileged. The asset naming below is illustrative; consult the project's own releases page for the real thing.

Returning an error from Asset means "not published for this platform", which Update treats as a platform to skip rather than as a failure.

package main

import (
	"fmt"

	"github.com/jroedel/binkit"
)

func main() {
	targets := map[string]string{
		"linux/amd64":  "linux-x64",
		"darwin/arm64": "macos-arm64",
	}

	widget := binkit.Tool{
		Name: "widget",
		Repo: "acme/widget",

		// Default is "v" + version; override when a project tags differently.
		Tag: func(version string) string { return "release-" + version },

		Asset: func(version, goos, goarch string) (string, error) {
			target, ok := targets[goos+"/"+goarch]
			if !ok {
				return "", fmt.Errorf("widget publishes no build for %s/%s", goos, goarch)
			}
			return fmt.Sprintf("widget-%s-%s.tar.gz", version, target), nil
		},

		BinaryPath: func(version, goos, goarch string) string {
			return fmt.Sprintf("widget-%s-%s/widget", version, targets[goos+"/"+goarch])
		},
	}

	fmt.Println(widget.EnvKey())
}

Output:
BINKIT_WIDGET

func (Tool) EnvKey

func (t Tool) EnvKey() string

EnvKey is the environment variable that overrides this tool entirely, e.g. BINKIT_TYPST. When set, Ensure returns its value untouched.

The name is upper-cased and every character outside A-Z and 0-9 becomes an underscore, so "go-task" yields BINKIT_GO_TASK. That mapping shares a namespace with binkit's own environment variables: a tool named "cache" would produce EnvCacheDir and one named "no-update-check" would produce EnvNoUpdateCheck. Those two names collide and must not be used — setting the variable to steer binkit would silently be read as a path override for the tool, and vice versa.

Example

EnvKey reports the variable that bypasses binkit entirely for one tool — worth printing in a CLI's help text so users can point at a distro or Nix build.

package main

import (
	"fmt"

	"github.com/jroedel/binkit"
)

func main() {
	fmt.Println(binkit.Tool{Name: "typst"}.EnvKey())
	fmt.Println(binkit.Tool{Name: "go-task"}.EnvKey())
}

Output:
BINKIT_TYPST
BINKIT_GO_TASK