virtle

README

virtle 🐢🪐

Virtle is a VM manager for sandbox workflows.

Status: Beta. Used day-to-day by a few people, feedback appreciated.

Background: Originally designed to be used with agentspace, a NixOS-based sandbox builder, but virtle has been refined enough to act as a standalone tool.

How does it work?

virtle reads a manifest, starts the required host processes, launches QEMU, waits for guest SSH readiness, attaches an active session with --ssh.

It also handles teardown, QMP-based shutdown, disk-backed suspend/resume, runtime vsock CID allocation, QGA-based remote commands, and more.

Features

• Runs a QEMU microvm.
• Allocates block overlay images.
• Manages virtiofsd daemons for virtiofs mounts.
• Provisions SSH between host and guest.
• Connects over SSH upon boot via signaling.
• Writes files between guest and host on boot or shutdown.
• Suspend and resume.
• Notification execution hooks.
• Exposes a virtle.sock for RPC (also usable via virtle rpc sub-command).
• (Experimental) Balloon memory: Auto-adjust memory available to the VM based on internal memory pressure metrics.
• (Experimental) Hotplug: Attach/detach devices during runtime (requires full VM).

Usage

First, write a simple manifest and save it as manifest.toml. See: ./examples/manifest-*.toml.

Global flags:

• virtle [--manifest=MANIFEST] ... - When --manifest is omitted, ./manifest.toml is used, falling back to ./manifest.json. Manifests may be TOML or JSON.
• virtle -v ... - Show useful VM and SSH lifecycle information.
• virtle -vv ... - Show debugging details and output from background commands.
• virtle --version - Print the virtle version and exit.

Launch a VM:

• virtle launch [--ssh] [--resume=no|auto|force] [-- <remote-cmd...>]

Other advanced features:

• virtle suspend
• virtle status
• virtle rpc METHOD [JSON_ARGS]
• virtle hotplug ID (experimental)
• virtle hotplug --detach ID (experimental)

Manifest

There are some handy sub-commands for working with manifest files:

• virtle manifest defaults [--resolved]
• virtle manifest validate
• virtle manifest resolve
• virtle manifest schema

Manifest exec arrays render each argv element as a Go text/template. The host process environment is available as .Env on every surface.

Surface	Template values	Injected environment
qemu.exec	HostName, WorkingDir, StateDir, HostOS, HostArch, HostSystem, .Env	none
qemu.fwd_tunnel_exec	Host, Port, .Env	none; QEMU starts the command
ssh.exec	CID, User, Destination, .Env	CID, USER, DESTINATION
mounts[type=virtiofs].virtiofs	Socket, MountSource, MountTag, CID, StateDir, .Env	SOCKET, MOUNT_SOURCE, MOUNT_TAG, CID, STATE_DIR, VIRTIOFSD_SOCKET
run[].exec	CID, StateDir, Workspace.GuestPath, Workspace.HostPath, user vars, .Env	scalar top-level values only
notifications.exec	State, Message, notification context values, .Env	STATE, MESSAGE, normalized context values, VIRTLE_NOTIFY_STATE, VIRTLE_NOTIFY_MESSAGE, VIRTLE_NOTIFY_CONTEXT_<KEY>

Library

Virtle library docs: https://pkg.go.dev/github.com/shazow/virtle

Boot a VM, run a command, tear down:

spec := &vm.Spec{
	Kernel: vm.Kernel{Path: "vmlinuz", Initrd: "initrd.img"},
	Shares: []vm.Share{{Tag: "src", HostPath: ".", GuestPath: "/workspace"}},
	Memory: 2048 * units.Mebibyte,
}
b := &qemu.Backend{
	RemoteControl: qemu.QGA{},
	Logger:        slog.Default(),
}
m, err := b.Start(ctx, spec)
if err != nil {
	log.Fatal(err)
}
defer m.Shutdown(ctx)

g, err := m.RemoteControl()
if err != nil {
	log.Fatal(err) // this VM has no guest agent
}
err = g.Run(ctx, &vm.GuestCmd{Path: "make", Dir: "/workspace", Stdout: os.Stdout})

Optional functionality is discovered by type assertion, as in database/sql/driver:

if s, ok := m.(backend.Suspender); ok {
	err = s.Suspend(ctx)
} else {
	err = m.Shutdown(ctx)
}

Or drive it from a manifest, as the CLI does:

spec, b, err := manifest.Load(f)
m, err := b.Start(ctx, spec)

License

MIT

Documentation

Overview

Command virtle launches and controls QEMU sandbox VMs described by a manifest. Run virtle --help for the command list; see README.md for the manifest format.