machine

Documentation

Overview

Package machine is a cross-platform microVM lifecycle API.

The package defines a Spec that describes a VM and a Machine interface that backends implement. Backends register themselves via Register and callers look them up with Get. A backend wraps one hypervisor — firecracker on Linux, vz on macOS — and the Spec is intentionally the intersection of what they can all express.

Capabilities that not every backend can provide (snapshot, EFI boot, etc.) are reported via Backend.Capabilities and surfaced as optional interfaces (Snapshottable, ...). Callers feature-check before using them.

Example (BootOCI)

Example_bootOCI shows the simplest way to boot a VM from an OCI image on macOS (vz, Apple Virtualization.framework). The same Spec works on Linux with backend "firecracker" — only the Net variant changes (TAP for firecracker, UserMode for vz).

package main

import (
	"context"
	"log"

	"github.com/clawkwork/clawk/machine"

	// Side-effect imports register backends. Import only the ones whose
	// runtime you intend to support. Unused imports are free — they register
	// nothing on the wrong OS.
	_ "github.com/clawkwork/clawk/machine/firecracker"
	_ "github.com/clawkwork/clawk/machine/vz"
)

func main() {
	ctx := context.Background()

	b, err := machine.Get("vz")
	if err != nil {
		log.Fatal(err)
	}

	m, err := b.New(ctx, machine.Spec{
		ID:        "demo",
		VCPU:      2,
		MemoryMiB: 1024,
		Boot: machine.DirectKernel{
			Vmlinux: "/etc/vmlinux", // caller supplies a direct-boot kernel
			Cmdline: "console=hvc0 root=/dev/vda rw",
		},
		RootFS: machine.OCIImage{
			Ref:      "docker.io/library/alpine:3.20",
			CacheDir: "/var/cache/clawk/oci",
		},
		Net: []machine.Net{
			machine.UserMode{
				Forwards: []machine.PortForward{
					{HostPort: 2222, GuestPort: 22, Proto: machine.ProtoTCP},
				},
			},
		},
	}, "/var/lib/clawk/vms/demo")
	if err != nil {
		log.Fatal(err)
	}

	if err := m.Create(ctx); err != nil {
		log.Fatal(err)
	}
	if err := m.Start(ctx); err != nil {
		log.Fatal(err)
	}
	defer m.Destroy(ctx)
}

Example (Snapshot)

Example_snapshot demonstrates the Snapshottable optional capability. Only backends whose Caps.Snapshot is true satisfy this interface.

package main

import (
	"context"
	"log"

	"github.com/clawkwork/clawk/machine"

	// Side-effect imports register backends. Import only the ones whose
	// runtime you intend to support. Unused imports are free — they register
	// nothing on the wrong OS.
	_ "github.com/clawkwork/clawk/machine/firecracker"
	_ "github.com/clawkwork/clawk/machine/vz"
)

func main() {
	ctx := context.Background()
	b, err := machine.Get("firecracker")
	if err != nil {
		log.Fatal(err)
	}
	if !b.Capabilities().Snapshot {
		return
	}
	m, err := b.New(ctx, machine.Spec{ /* … */ }, "/tmp/vm")
	if err != nil {
		log.Fatal(err)
	}
	snap, ok := m.(machine.Snapshottable)
	if !ok {
		log.Fatal("backend claims snapshot capability but does not implement Snapshottable")
	}
	if err := snap.Snapshot(ctx, "/tmp/vm/snap"); err != nil {
		log.Fatal(err)
	}
}

Index

• Variables
• func Register(b Backend)
• func SuspendSpecFingerprint(s Spec) string
• func SuspendStateExists(dir string) bool
• func WriteSuspendMeta(dir string, m SuspendMeta) error
• type Backend
  • func Get(name string) (Backend, error)
• type Boot
• type Caps
• type DirectKernel
• type Disk
• type EFIBoot
• type Filter
• type InjectFile
• type Machine
• type Net
• type OCIImage
• type Pauseable
• type PortForward
• type Proto
• type RawDisk
• type RootFS
• type Serial
• type Share
• type Snapshottable
• type Spec
  • func (s Spec) Validate() error
• type State
• type SuspendMeta
  • func ReadSuspendMeta(dir string) (SuspendMeta, bool)
  • func (m SuspendMeta) IncompatibleWith(want SuspendMeta) string
• type Suspendable
• type TAP
• type Unixgram
• type UserMode
• type VSockListener

Examples

• Package (BootOCI)
• Package (Snapshot)

Variables

var (
	// ErrNoBackend is returned by Get when the named backend is not
	// registered in this binary (typically because the backend's package was
	// not imported, or it is compiled out on this GOOS).
	ErrNoBackend = errors.New("machine: no such backend")

	// ErrVSockUnsupported is returned by Machine.VSock when the backend does
	// not expose vsock or the Spec did not allocate a CID.
	ErrVSockUnsupported = errors.New("machine: vsock not supported")

	// ErrUnsupportedSpec is returned by Backend.New when the Spec requests a
	// feature the backend does not support (e.g. EFI boot on firecracker,
	// TAP net on vz). Wrapped errors explain which field.
	ErrUnsupportedSpec = errors.New("machine: spec not supported by backend")

	// ErrInvalidState is returned when a Machine method is called in an
	// incompatible lifecycle state (e.g. Start after Destroy).
	ErrInvalidState = errors.New("machine: invalid state transition")
)

Functions

func Register

func Register(b Backend)

Register adds b to the package-wide backend registry. Typically called from a backend package's init. Panics on duplicate names.

func SuspendSpecFingerprint

func SuspendSpecFingerprint(s Spec) string

SuspendSpecFingerprint reduces a Spec to the fields that shape the guest-visible virtual hardware. Deliberately a readable string, not a hash: when a restore is skipped over a mismatch, the log line shows what changed instead of two opaque digests.

func SuspendStateExists

func SuspendStateExists(dir string) bool

SuspendStateExists reports whether dir holds a suspend-to-disk state written by a Suspendable backend, i.e. whether a Restore from it could continue a guest. It deliberately does not say WHICH backend wrote it — a mismatched restore fails cleanly at Restore time.

func WriteSuspendMeta

func WriteSuspendMeta(dir string, m SuspendMeta) error

WriteSuspendMeta records m beside the state files in dir. Best-effort by contract — callers log a failure and move on, because the state itself is already safely written.

Types

type Backend

type Backend interface {
	// Name is a stable, lowercase identifier ("firecracker", "vz").
	Name() string

	// Capabilities describes what the backend supports. Stable for the life
	// of the process.
	Capabilities() Caps

	// New prepares a Machine for the given Spec. stateDir is an
	// already-created directory that the Machine owns for the duration of its
	// life; the backend may place disks, sockets, pidfiles, and logs inside.
	//
	// New validates the Spec against the backend's capabilities and returns
	// an error before any side effects if the Spec is unsupported. A
	// successful New does not boot the VM; callers must call Machine.Create
	// and then Machine.Start.
	New(ctx context.Context, spec Spec, stateDir string) (Machine, error)
}

Backend constructs Machines for a specific hypervisor. Backends register themselves at package-init via Register; callers look them up by name with Get.

func Get

func Get(name string) (Backend, error)

Get returns the backend registered under name. Returns an error wrapping ErrNoBackend if no such backend is registered in this binary.

type Boot

type Boot interface {
	// contains filtered or unexported methods
}

Boot describes how the guest kernel is loaded. Sealed union.

type Caps

type Caps struct {
	// Snapshot: Machine implements Snapshottable.
	Snapshot bool

	// OCIRootFS: Spec.RootFS of type OCIImage is accepted.
	OCIRootFS bool

	// DirectKernel: Spec.Boot of type DirectKernel is accepted.
	DirectKernel bool

	// EFIBoot: Spec.Boot of type EFIBoot is accepted.
	EFIBoot bool

	// VSock: Machine.VSock returns real connections (not ErrVSockUnsupported).
	VSock bool

	// VirtioFS: Spec.Shares is honored.
	VirtioFS bool

	// UserModeNet: Spec.Net may contain a UserMode entry.
	UserModeNet bool

	// TAPNet: Spec.Net may contain a TAP entry.
	TAPNet bool

	// UnixgramNet: Spec.Net may contain a Unixgram entry.
	UnixgramNet bool

	// NestedVirt: Spec.NestedVirt is honored. For vz this depends on the
	// host (macOS 15+ / M3+), so the capability is queried dynamically
	// rather than hard-coded in Backend.Capabilities().
	NestedVirt bool
}

Caps reports what a Backend supports. Returned by Backend.Capabilities.

Callers use Caps to decide whether a Spec is satisfiable before creating a Machine, and to branch on optional interfaces (e.g. only type-asserting Snapshottable when Caps.Snapshot is true).

type DirectKernel

type DirectKernel struct {
	Vmlinux string
	Initrd  string
	Cmdline string
}

DirectKernel loads an uncompressed vmlinux directly, optionally with an initrd, and passes Cmdline on the kernel command line. Supported by every backend in the module.

type Disk

type Disk struct {
	Path     string
	ReadOnly bool
}

Disk is a non-root block device attached to the guest.

type EFIBoot

type EFIBoot struct {
	// StorePath is the file that backs persistent EFI NVRAM. Created on
	// first use; read on subsequent boots so boot order and other UEFI
	// state survive restarts. Required.
	StorePath string
}

EFIBoot uses the platform's EFI firmware to read a kernel from the attached rootfs at a standard UEFI path. Use when the rootfs is a stock cloud image (Ubuntu, Fedora) that ships a GRUB binary. Firecracker does not support this mode; vz does.

type Filter

type Filter interface {
	// AllowTCP is called before the userspace TCP stack establishes an
	// outbound connection. addr is "host:port" where host is a literal IP.
	// A non-nil error drops the SYN and is logged by the backend.
	AllowTCP(addr string) error

	// AllowUDP is called before the userspace stack forwards a new outbound
	// UDP flow. addr is "host:port" where host is a literal IP. A non-nil
	// error drops the flow and is logged by the backend.
	AllowUDP(addr string) error

	// AllowICMP is called before the userspace stack forwards an outbound
	// ICMP echo (ping). addr is a literal destination IP (ICMP has no
	// port). A non-nil error drops the packet and is logged by the backend.
	AllowICMP(addr string) error

	// ObserveDNS is called after the userspace DNS responder returns an
	// answer to the guest. Implementations may use this to auto-allow IPs
	// resolved from allow-listed domains (wildcard support).
	ObserveDNS(name string, ip net.IP)
}

Filter is the policy hook consulted by UserMode networking. Implementations must be safe for concurrent use.

The shape matches gvisor-tap-vsock's forwarder and DNS hooks directly so bridging is a no-op.

type InjectFile

type InjectFile struct {
	// GuestPath is the absolute destination inside the image
	// (e.g. "/sbin/clawk-init"). Parent directories are created as needed.
	GuestPath string

	// HostPath is the file whose content is injected.
	HostPath string

	// Mode is the file mode inside the image (e.g. 0o755).
	Mode uint32
}

InjectFile is one host file copied into an OCIImage-built filesystem.

type Machine

type Machine interface {
	// Create provisions on-disk artifacts (disks, sockets, configs) but does
	// not boot. Idempotent: calling Create twice returns nil the second time.
	Create(ctx context.Context) error

	// Start boots the VM. Returns once the hypervisor process is running and
	// the guest has begun boot; it does not wait for guest userspace.
	Start(ctx context.Context) error

	// Stop halts the VM. If graceful, the backend asks the guest to shut down
	// and waits up to an implementation-defined deadline; otherwise it kills
	// the hypervisor process.
	Stop(ctx context.Context, graceful bool) error

	// Destroy releases all on-disk and in-kernel resources owned by the
	// Machine. After Destroy returns nil, the state directory may be removed.
	Destroy(ctx context.Context) error

	// State reports the current lifecycle phase.
	State(ctx context.Context) (State, error)

	// VSock dials the guest on the given AF_VSOCK port. Returns
	// ErrVSockUnsupported if the backend does not expose vsock or if the Spec
	// did not allocate a CID.
	VSock(ctx context.Context, port uint32) (net.Conn, error)
}

Machine is a single VM instance bound to a backend and a state directory.

Methods are safe to call in any state; they return an error if the transition is invalid (e.g. Start on a destroyed machine). Implementations must be safe for concurrent use.

type Net

type Net interface {
	// contains filtered or unexported methods
}

Net describes a guest network interface. Sealed union.

type OCIImage

type OCIImage struct {
	// Ref is a fully-qualified OCI image reference
	// (e.g. "ghcr.io/clawkwork/clawk-agent:latest").
	Ref string

	// CacheDir stores digest-keyed built images. Required.
	CacheDir string

	// SizeMiB is the minimum filesystem size in MiB; the gap between the
	// image content and this floor is free space the guest can write into
	// without growpart/resize2fs in the image. Sparse, so a generous floor
	// costs no physical disk. Zero means the builder's default (1 GiB).
	SizeMiB int

	// Platform forces a specific OCI platform ("linux/amd64",
	// "linux/arm64"). Empty picks the registry default for the host arch.
	Platform string

	// Inject is a list of host files written into the built filesystem on
	// top of the image content. Used to plant a guest init or agent into
	// arbitrary images that ship neither. The injected content is part of
	// the disk cache key, so two machines injecting different binaries
	// never share a built disk.
	Inject []InjectFile
}

OCIImage pulls an OCI image reference and materializes its merged layers as an ext4 image. The image digest is the cache key; a second machine using the same Ref reuses the built image.

type Pauseable

type Pauseable interface {
	// Pause stops the vCPUs without altering memory or device state.
	// Idempotent: pausing a paused VM returns nil.
	Pause(ctx context.Context) error

	// Resume restarts the vCPUs after a Pause.
	// Idempotent: resuming a running VM returns nil.
	Resume(ctx context.Context) error
}

Pauseable is an optional capability for backends that can suspend and resume a running guest without saving state to disk. Callers type-assert; backends without this capability run only on hosts whose power management is benign enough not to require it.

The motivating use case is host sleep/wake: when the Mac hibernates, the daemon can Pause the VM before sleep and Resume on wake to keep guest timers and in-flight network state synchronised with the host's view of time. The wallclock watchdog also calls these on detected time jumps when the OS sleep notifications are missed (macOS standby is unreliable about delivering them).

type PortForward

type PortForward struct {
	HostPort  uint16
	GuestIP   string
	GuestPort uint16
	Proto     Proto
}

PortForward exposes a guest port on the host. Only valid inside a UserMode.

type Proto

type Proto string

Proto is the L4 protocol for a PortForward.

const (
	ProtoTCP Proto = "tcp"
	ProtoUDP Proto = "udp"
)

type RawDisk

type RawDisk struct {
	Path     string
	ReadOnly bool
}

RawDisk boots from an existing block image (raw, ext4, squashfs, etc.). The backend decides whether to copy-on-write it or mount it directly.

type RootFS

type RootFS interface {
	// contains filtered or unexported methods
}

RootFS describes the guest's root filesystem. Sealed union.

type Serial

type Serial struct {
	// LogPath, if non-empty, captures serial output. Empty discards it.
	LogPath string
}

Serial configures the guest serial console.

type Share

type Share struct {
	// Tag is the mount tag the guest uses to address the share.
	Tag      string
	HostPath string
	ReadOnly bool
}

Share is a virtio-fs mount exposed to the guest.

type Snapshottable

type Snapshottable interface {
	// Snapshot pauses the VM, writes memory + device state into dir, and
	// resumes it. The VM must be running.
	Snapshot(ctx context.Context, dir string) error

	// Restore boots a new VM from a snapshot directory previously produced by
	// Snapshot. The Machine must be in StateCreated (not yet Started).
	Restore(ctx context.Context, dir string) error
}

Snapshottable is an optional capability implemented by backends whose Caps.Snapshot is true. Callers must type-assert.

A snapshot is written to a directory the backend owns for the duration of the call. The directory layout is backend-defined but stable across Snapshot/Restore pairs of the same backend.

type Spec

type Spec struct {
	// ID is a stable identifier used to name on-disk artifacts (sockets,
	// pidfiles, log files). Must be non-empty and filesystem-safe.
	ID string

	VCPU      uint
	MemoryMiB uint64

	// MemoryMaxMiB is the guest-visible memory ceiling. When it exceeds
	// MemoryMiB, a backend that supports ballooning (firecracker's
	// /balloon, virtio-balloon on VZ) configures the balloon to reclaim
	// (MemoryMaxMiB - MemoryMiB) back to the host at boot and deflate on
	// guest pressure. Zero means "same as MemoryMiB" — no ballooning.
	MemoryMaxMiB uint64

	Boot   Boot
	RootFS RootFS

	// Disks are additional block devices. RootFS handles the boot disk.
	Disks []Disk

	// Net lists network interfaces. Backends that do not support every Net
	// variant return an error from Backend.New.
	Net []Net

	// Shares are virtio-fs mounts exposed to the guest.
	Shares []Share

	// VSockCID is the guest's AF_VSOCK context ID. 0 means the backend picks
	// one. Must be >= 3 if set.
	VSockCID uint32

	Serial Serial

	// NestedVirt enables hardware-assisted nested virtualization, letting
	// the guest run its own VMs. Only honored by backends that report
	// Caps.NestedVirt == true; on vz this requires macOS 15+ and an M3 or
	// newer Apple Silicon chip.
	NestedVirt bool
}

Spec describes a VM. It is the input to Backend.New.

Spec is intentionally the intersection of what the supported backends can express. Fields that only make sense on one backend live in backend-specific option structs, not here.

func (Spec) Validate

func (s Spec) Validate() error

Validate performs common checks that every backend would otherwise repeat. Backends may still reject a Spec for backend-specific reasons.

type State

type State string

State is a Machine's lifecycle phase.

const (
	StateCreated State = "created"
	StateRunning State = "running"
	// StatePaused is a running VM whose vCPUs are suspended (Pauseable).
	// Memory and device state stay resident; Resume continues execution.
	StatePaused    State = "paused"
	StateStopped   State = "stopped"
	StateDestroyed State = "destroyed"
)

type SuspendMeta

type SuspendMeta struct {
	// Backend is the machine backend that wrote the state ("vz",
	// "firecracker"). Restoring across backends can never work.
	Backend string `json:"backend"`

	// SpecFingerprint captures the Spec fields that shape the
	// guest-visible virtual hardware (see SuspendSpecFingerprint). A
	// clawk release that changes how it builds the VM changes this, and
	// the state is then not restorable.
	SpecFingerprint string `json:"spec_fingerprint"`

	// ClawkVersion is the writer's version string. Informational only —
	// restores are never gated on it, because most upgrades don't touch
	// the VM shape — but it makes "which release wrote this?" a cat away.
	ClawkVersion string `json:"clawk_version,omitempty"`
}

SuspendMeta identifies what wrote a suspend-to-disk state, so a later boot can tell — before handing the bytes to the hypervisor — whether a restore even makes sense. Written beside the state files by the daemon at suspend, consulted by restoreOrStart at the next boot.

Absence is not an error: states written by earlier clawks have no meta, and the hypervisor's own validation (both vz and firecracker refuse state they can't load, and the caller cold-boots on failure) remains the backstop. The meta exists to turn that late, cryptic refusal into an early, readable log line.

func ReadSuspendMeta

func ReadSuspendMeta(dir string) (SuspendMeta, bool)

ReadSuspendMeta loads the meta written next to a suspend state. ok=false means absent or unreadable — a pre-meta state; callers proceed to the restore attempt and let the hypervisor decide.

func (SuspendMeta) IncompatibleWith

func (m SuspendMeta) IncompatibleWith(want SuspendMeta) string

IncompatibleWith returns a human-readable reason when a state written under m cannot be restored by a boot described by want, or "" when the restore should be attempted. An empty want field skips that check.

type Suspendable

type Suspendable interface {
	// Suspend pauses the VM (if it isn't already paused), writes memory +
	// device state into dir, and stops the VM without resuming it. On
	// success the Machine is in StateStopped. On save failure the backend
	// makes a best effort to resume the guest so the VM isn't left wedged.
	Suspend(ctx context.Context, dir string) error
}

Suspendable is an optional capability for backends that can hibernate a VM: save its memory + device state into a directory and stop it WITHOUT letting the guest execute again. That ordering is the whole contract — because the guest never runs past the save point, the rootfs on disk is frozen at exactly the saved moment, which is what makes a later Snapshottable.Restore from the same directory safe. (A live Snapshot that resumes the guest afterwards lets the disk drift ahead of the saved memory image; restoring such a pair corrupts the guest filesystem.)

The directory layout matches the backend's Snapshottable layout, so Suspend/Restore pair up the same way Snapshot/Restore do.

type TAP

type TAP struct {
	Device string
	MAC    string
}

TAP attaches to an existing host TAP device. Linux only. The caller is responsible for creating the device, assigning host-side IPs, and configuring packet forwarding.

type Unixgram

type Unixgram struct {
	Path string
	MAC  string
}

Unixgram is a unixgram-socket NIC as used by vz and firecracker. The caller brings their own network stack on the other end (typically gvproxy).

type UserMode

type UserMode struct {
	Forwards []PortForward

	// Filter, if non-nil, is consulted on every outbound TCP SYN and every DNS
	// answer. A nil Filter allows everything.
	Filter Filter

	// GuestTAP/HostTAP select the TAP-bridge mode (see above). Linux-only;
	// ignored by backends that attach the fd NIC.
	GuestTAP string
	HostTAP  string

	// HostTAPFile, when set, is an already-open fd for the gvproxy-side TAP,
	// and HostTAP is ignored. It exists so the TAP can live in a network
	// namespace the backend is not in: creating a TAP needs CAP_NET_ADMIN,
	// but a TAP fd is namespace-agnostic once open, so the caller creates it
	// wherever it has that capability (for clawk, an unprivileged user
	// namespace) and passes the fd here. Ownership transfers to the backend.
	//
	// The raw fd MUST be put in nonblocking mode before it is wrapped in the
	// os.File, so Go's poller owns it and closing the file interrupts an
	// in-flight read; backends reject a blocking fd rather than deadlock on
	// shutdown.
	HostTAPFile *os.File

	// NetNSExec, when set, is an argv prefix that re-execs its arguments
	// inside the VM's network namespace — e.g. {"/proc/self/exe",
	// "__in-netns", "/proc/1234/ns/net", "--"}. The backend prepends it when
	// spawning the hypervisor, so the VM's NIC lives in that namespace while
	// gvproxy keeps the backend's own (which is where its egress sockets have
	// to be). Empty leaves the hypervisor in the current namespace.
	NetNSExec []string
}

UserMode runs an in-process userspace TCP/IP stack (gvisor-tap-vsock). Works identically on macOS and Linux. No host root required.

Two attachment modes:

• fd NIC (GuestTAP and HostTAP empty): the backend exposes the stack's unixgram socket as the VM's NIC fd directly. Used by vz, whose file-handle NIC speaks the same datagram protocol gvproxy does.

• TAP bridge (GuestTAP and HostTAP set): the VM speaks only a host TAP (firecracker). The backend boots the VM's virtio-net on GuestTAP and runs gvproxy bridged to HostTAP — a daemon-owned TAP on the same L2 bridge — through a userspace frame pump. Both TAPs must be pre-created by the caller (firecracker can't drive gvproxy's socket transport, and gvproxy can't drive a TAP fd, so the two are joined at L2). The backend assigns the NIC the MAC gvproxy's DHCP lease expects.

type VSockListener

type VSockListener interface {
	// VSockListen returns a net.Listener that accepts AF_VSOCK
	// connections initiated by the guest on the given port. Caller
	// owns the listener and must Close it.
	VSockListen(ctx context.Context, port uint32) (net.Listener, error)
}

VSockListener is an optional capability for backends whose vsock transport allows the host to accept guest-initiated connections. Apple Virtualization.framework supports this; firecracker's implementation is one-way (guest listens, host dials) so it does not implement this interface.

Used by the host-side SSH-agent proxy: a guest process dials a per-VM vsock port, the host accepts and bridges bytes to the host's local $SSH_AUTH_SOCK so 1Password / launchd ssh-agent keys are reachable from inside the VM without an SSH session.