threes

package module
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 27, 2025 License: BSD-3-Clause Imports: 36 Imported by: 0

README

threes

'cause threes a cloud innit

threes is a tool for bringing up tailscale connected virtual machines.

It comprises a server and a client powered with gRPC. Both components are managed via the binary provided by this repo.

Networking

threes needs a handful of things:

  1. An IP range for virtual machines to contact the threes metadata server (I tend to use 10.10.0.0/16 - you do what you want- you don't need to make this unique if you're running many threes hosts, they're not publically routablw)
  2. A bridge with the first IP of this range (10.10.0.1), and a sensible name (I tend to use threes0)
  3. An nbtables rule that stops VMs talking to one another locally (or not- I like this, but it doesn't matter if you don't do it; personally I want all traffic to route over tailscale)
  4. Various sysctl and iptables rules that allow VMs on this bridge access to the internet
$ sudo ip link add name threes0 type bridge
$ sudo ip addr add 10.10.0.1/16 dev threes0
$ sudo ip link set dev threes0 up
$ sudo ebtables -A FORWARD -i br0 -o threes0 -j DROP
$ sudo iptables -t nat -A POSTROUTING -s 10.10.0.0/16 -o wlan0 -j MASQUERADE
$ sudo iptables -A FORWARD -i threes0 -o wlan0 -j ACCEPT
$ sudo iptables -A FORWARD -i wlan0 -o threes0 -m state --state RELATED,ESTABLISHED -j ACCEPT
$ sudo sysctl -w net.ipv4.ip_forward=1

You will also need to ensure that /etc/qemu/bridge.conf contains the line

allow threes0

Database

threes uses rqlite; you'll want one of those running somewhere. Don't worry about configuration, starting the server will hook all that up.

Server

$ threes serve -t tskey-api-blah-blah -n mytailscale -c 10.10.0.0/16 -d http://database -i threes0 -l 10.10.0.1:9999

Where the arguments correspond to:

Flags:
  -c, --cidr string          CIDR to use for management addresses (default "10.10.0.0/16")
  -d, --database string      database to connect to (default "http://localhost")
  -h, --help                 help for server
  -i, --interface string     network interface to serve DHCP over (default "threes0")
  -l, --listen-addr string    (default "0.0.0.0:9999")
  -n, --name string          Name of the tailnet to manage
  -t, --token string         Tailscale auth token

This will start:

  1. A dhcp server listening on the specified interface, and serving IP addresses from the threes IPAM component
  2. An http server that returns configuration options for a VM, such as initial installation scripts, SSH keys, tailscale tokens, and so on
  3. a gRPC server for operations

Client

The threes command provides a lot of help text:

$ Manage a threes instance

Usage:
  threes [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  network     Manage the underlying threes tailnet
  server      Start a threes server
  vm          Manage virtual machines

Flags:
  -a, --address string   Address of threes server, ignored when starting a server (use --listen-addr instead) (default "localhost:9999")
  -h, --help             help for threes

Use "threes [command] --help" for more information about a command.

.... but below are some handy examples to get a virtual machine up and running

Creating a network

threes actually uses tailscale and ACLs to mimic stuff like VPCs and Security Groups.

$ threes network create cool-network

This will create a logical network called cool-network. Anything in this logical network will be able to talk to anything else in there.

Admins can ssh into it, and anyone can access port 443 on nodes in this network.

Creating a vm
$ threes vm create -i arch --network cool-network -k "$(cat ~/.ssh/id_rsa.pub)" -c example/startup toy0

2025/06/25 19:59:01 toy0
2025/06/25 19:59:01     ID: d1e4dplqs5f51296gtdg
2025/06/25 19:59:01     Management Address: 10.10.55.50

This will create a vm called toy0 in the cool-network. It'll add your local ssh key, and run the script provided in this repo.

Using this example script, you'll be able to login as per ssh threes@toy0 over your tailnet, using your local key.

This does assume there's a base image at /var/lib/libvirt/images/arch.qcow2 - for the travesty of a process to prepare this, visit https://code.fatlads.lol/threes/packer - it's as gnarly as you like

On interacting with VMs

Under the hood, threes interacts with VMs via their canonical ID and so a lot of commands for interacting with VMs require an ID.

While we thought about changing our philosophy and using VM names as unique identifiers (which they kind of are), the fact that we expect a specific ID format makes some operations a little nicer to protect against mistakes.

In order to determine the ID for a VM, there are two methods:

$ threes vm all
2025/08/26 10:34:44 prometheus
2025/08/26 10:34:44     ID: d1iopi72hcani0i9ov50
2025/08/26 10:34:44     Network: prometheus
2025/08/26 10:34:44     Type: teeny
2025/08/26 10:34:44     State: created
2025/08/26 10:34:44     Created At: 2025-07-02T19:47:20Z

Or

$ threes vm get-id prometheus
d1iopi72hcani0i9ov50

The first allows us to be more deliberate in ensuring we have the right name (but requires reading through the entire list), whereas the bottom can be used in shell scripts/ commands easier:

$ threes vm get $(threes vm get-id prometheus)
2025/08/26 10:36:39 prometheus
2025/08/26 10:36:39     ID: d1iopi72hcani0i9ov50
2025/08/26 10:36:39     Network: prometheus
2025/08/26 10:36:39     Type: teeny
2025/08/26 10:36:39     State: created
2025/08/26 10:36:39     Created At: 2025-07-02T19:47:20Z

As always, YMMV; choose the correct tool for the job.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	BaseImages = map[string]string{
		"base": filepath.Join(threesImagesDir, "base.qcow2"),
		"arch": filepath.Join(threesImagesDir, "arch.qcow2"),
	}
)
View Source 
var Migrations embed.FS

Functions

This section is empty.

Types

type Address 

type Address struct {
	MAC          string    `db:"mac_address"`
	ManagementIP string    `db:"address"`
	Hostname     string    `db:"hostname"`
	LeasedAt     time.Time `db:"leased_at"`
}

type Arper 

type Arper struct {
	// contains filtered or unexported fields
}

An Arper provides access to the host ARP table.

We use this to gate access to the Config endpoint; that endpoint contains passwords and tailscale keys and stuff.

By matching on source address for calls to the config endpoint, we can ensure the correct data is returned to the correct threes domain.

_However_ we also want to stop malicious threes domains from changing their IP address to be able to exfiltrate data.

Thus we want to lookup the MAC address of the IP Address making the call, and make sure it matches the MAC+IP pair as known by our IPAM.

func NewArper 

func NewArper(iface *net.Interface, i *IPAM) (a *Arper, err error)

NewArper accepts a network interface, and a pointer to the IPAM, and returns a new Arper instance

func (Arper) IsValid 

func (a Arper) IsValid(ipaddr netip.Addr) (mac string, v bool, err error)

IsValid accepts an IP Address, and returns the mac address it resolves to in the ARP table, a boolean signifying whether that MAC Address is the MAC Address our IPAM has associated with this IP Address, and an optional error, should one exist

type Config 

type Config struct {
	VM        string `json:"-" db:"vm"`
	Hostname  string `json:"hostname" db:"hostname"`
	Token     string `json:"token" db:"ts_token"`
	Script    string `json:"script" db:"script"`
	PublicKey string `json:"public_key" db:"public_key"`
}

Config holds various bits of data for the config http endpoint to return.

type DHCPServer 

type DHCPServer struct {
	// contains filtered or unexported fields
}

DHCPServer allows us to provide threes domains with a management IP address via the IPAM.

These management addresses are important, because that's how machines get access to the internet, and also in case tailscale crashes, or breaks, or goes down, and we need to SSH in to get access to fix stuff.

Providing threes domains IP addresses via DHCP allows us to avoid weird configurations or whatever

func NewDHCP 

func NewDHCP(iface *net.Interface, i *IPAM) (d *DHCPServer, err error)

NewDHCP accepts a network interface to respond on, and a pointer to the IPAM instance in order to allow us to configure the management interface of a thres domain

func (*DHCPServer) Serve 

func (d *DHCPServer) Serve() error

Serve starts the DHCP Server

type Database 

type Database struct {
	// contains filtered or unexported fields
}

func ConnectToDatabase 

func ConnectToDatabase(uri string) (d *Database, err error)

func (Database) AllocateIPAM 

func (d Database) AllocateIPAM(a *Address) (err error)

func (Database) DeleteKVs 

func (d Database) DeleteKVs(kvs *KVs) (err error)

func (Database) DeleteNetwork 

func (d Database) DeleteNetwork(n *Network) (err error)

func (Database) DeleteVM 

func (d Database) DeleteVM(vm *VM) (err error)

func (Database) GetConfig 

func (d Database) GetConfig(vmID string) (config *Config, err error)

func (Database) GetHostnameByMAC 

func (d Database) GetHostnameByMAC(mac string) (config *Config, err error)

func (Database) GetIPAddress 

func (d Database) GetIPAddress(a *Address) (err error)

func (Database) GetIPAddressAllocation 

func (d Database) GetIPAddressAllocation(a *Address) (err error)

func (Database) GetKVs 

func (d Database) GetKVs(vmID string) (kvs KVs, err error)

func (Database) GetNetwork 

func (d Database) GetNetwork(n *Network) (err error)

func (Database) GetVM 

func (d Database) GetVM(vm *VM) error

func (Database) GetVMFromMacAddress 

func (d Database) GetVMFromMacAddress(vm *VM) error

func (Database) GetVMs 

func (d Database) GetVMs() (vms []*VM, err error)

func (Database) InsertConfig 

func (d Database) InsertConfig(cfg *Config) (err error)

func (Database) InsertNetwork 

func (d Database) InsertNetwork(n *Network) (err error)

func (Database) InsertVM 

func (d Database) InsertVM(vm *VM) (err error)

func (Database) MarkVMAsDeleted 

func (d Database) MarkVMAsDeleted(vm *VM) (err error)

func (Database) UpdateAllocation 

func (d Database) UpdateAllocation(a *Address) (err error)

func (Database) UpdateKVs 

func (d Database) UpdateKVs(kvs *KVs) (err error)

type HTTPServer 

type HTTPServer struct {
	*gin.Engine
	// contains filtered or unexported fields
}

func NewHTTPServer 

func NewHTTPServer(a *Arper, db *Database) *HTTPServer

func (*HTTPServer) GetConfig 

func (s *HTTPServer) GetConfig(c *gin.Context)

func (*HTTPServer) GetHostname 

func (s *HTTPServer) GetHostname(c *gin.Context)

func (*HTTPServer) GetKV 

func (s *HTTPServer) GetKV(c *gin.Context)

func (*HTTPServer) GetPublicKey 

func (s *HTTPServer) GetPublicKey(c *gin.Context)

func (*HTTPServer) GetScript 

func (s *HTTPServer) GetScript(c *gin.Context)

func (*HTTPServer) GetToken 

func (s *HTTPServer) GetToken(c *gin.Context)

type Hypervisor 

type Hypervisor struct {
	// contains filtered or unexported fields
}

func NewHypervisor 

func NewHypervisor() (h *Hypervisor, err error)

func (*Hypervisor) CreateAndStart 

func (h *Hypervisor) CreateAndStart(dom libvirtxml.Domain) (err error)

func (*Hypervisor) Delete 

func (h *Hypervisor) Delete(id string) (err error)

func (*Hypervisor) IsRunning 

func (h *Hypervisor) IsRunning(id string) bool

func (*Hypervisor) Restart 

func (h *Hypervisor) Restart(id string) (err error)

func (*Hypervisor) StartExisting 

func (h *Hypervisor) StartExisting(id string) (err error)

func (*Hypervisor) Stop 

func (h *Hypervisor) Stop(id string) (err error)

type IPAM 

type IPAM struct {
	// contains filtered or unexported fields
}

func NewIPAM 

func NewIPAM(cidr string, d *Database) (i *IPAM, err error)

func (*IPAM) Allocate 

func (i *IPAM) Allocate(mac, hostname string) (mgmtAddr string, err error)

func (*IPAM) GetAllocation 

func (i *IPAM) GetAllocation(mac string) (hostname string, self, gateway, addr, broadcast netip.Addr, mask net.IPMask, err error)

func (*IPAM) GetMAC 

func (i *IPAM) GetMAC(ip string) (mac string, err error)

type KV 

type KV struct {
	VM     string `db:"vm" json:"-"`
	Key    string `db:"key"`
	Value  string `db:"value"`
	Secret bool   `db:"secret"`
}

type KVs 

type KVs []KV

func (*KVs) FromProtobuf 

func (k *KVs) FromProtobuf(in *server.KVs)

func (KVs) Map 

func (k KVs) Map() map[string]string

func (KVs) RedactedMap 

func (k KVs) RedactedMap() map[string]string

func (*KVs) ToProtobuf 

func (k *KVs) ToProtobuf() (out *server.KVs)

type Network 

type Network struct {
	Name      string    `db:"name"`
	CreatedAt time.Time `db:"created_at"`
	UpdatedAt time.Time `db:"updated_at"`
	DeletedAt time.Time `db:"deleted_at"`
}

func (*Network) NewFromProtobuf 

func (n *Network) NewFromProtobuf(in *server.NetworkRequest)

func (*Network) Reponse 

func (n *Network) Reponse() *server.NetworkResponse

type Server 

type Server struct {
	server.UnimplementedThreesServer
	// contains filtered or unexported fields
}

Server implements the gRPC interface to threes, and is where the magic lives

func New 

func New(ts *Tailscale, ipam *IPAM, db *Database, h *Hypervisor) (s *Server, err error)

New initialises a new threes server, and starts any VMs that need starting

func (*Server) All 

func (s *Server) All(_ *emptypb.Empty, ss grpc.ServerStreamingServer[server.VM]) (err error)

func (*Server) Create 

func (s *Server) Create(ctx context.Context, req *server.VM) (out *server.VM, err error)

func (*Server) CreateNetwork 

func (s *Server) CreateNetwork(ctx context.Context, req *server.NetworkRequest) (resp *server.NetworkResponse, err error)

CreateNetwork implements the Server.CreateNetwork gRPC call

func (*Server) Delete 

func (s *Server) Delete(ctx context.Context, vm *server.VM) (*server.VM, error)

func (*Server) DeleteKVs 

func (s *Server) DeleteKVs(ctx context.Context, in *server.KVs) (*server.KVs, error)

func (*Server) DeleteNetwork 

func (s *Server) DeleteNetwork(ctx context.Context, req *server.NetworkRequest) (_ *emptypb.Empty, err error)

func (*Server) Get 

func (s *Server) Get(ctx context.Context, req *server.VM) (out *server.VM, err error)

func (*Server) GetKVs 

func (s *Server) GetKVs(ctx context.Context, in *server.VM) (*server.KVs, error)

func (*Server) Purge 

func (s *Server) Purge(ctx context.Context, _ *emptypb.Empty) (*emptypb.Empty, error)

Purge should be run periodically; it will permanently delete any VMs marked as 'deleted', which frees up the hostname and disk space to be used again

func (*Server) Restart 

func (s *Server) Restart(ctx context.Context, vm *server.VM) (*server.VM, error)

func (*Server) Start 

func (s *Server) Start(ctx context.Context, vm *server.VM) (*server.VM, error)

func (*Server) Stop 

func (s *Server) Stop(ctx context.Context, vm *server.VM) (*server.VM, error)

func (*Server) UpsertKVs 

func (s *Server) UpsertKVs(ctx context.Context, in *server.KVs) (*server.KVs, error)

type Tailscale 

type Tailscale struct {
	// contains filtered or unexported fields
}

func NewTailscale 

func NewTailscale(tailnet, key string) (t *Tailscale, err error)

func (*Tailscale) CreateAuthToken 

func (t *Tailscale) CreateAuthToken(vm, network string) (token string, err error)

func (*Tailscale) CreateNetwork 

func (t *Tailscale) CreateNetwork(name string) (err error)

func (*Tailscale) DeleteNetwork 

func (t *Tailscale) DeleteNetwork(name string) (err error)

type VM 

type VM struct {
	ID          string    `db:"id"`
	Name        string    `db:"name" validate:"alphanum,min=3"`
	Description string    `db:"description" validate:"printascii,max=256"`
	Network     string    `db:"network" validate:"alphanum"`
	Image       string    `db:"image" validate:"isValidImage"`
	IPAddress   string    `db:"management_ip_address"`
	MACAddress  string    `db:"mac_address"`
	CreatedAt   time.Time `db:"created_at"`
	UpdatedAt   time.Time `db:"updated_at"`
	DeletedAt   time.Time `db:"deleted_at"`
	Type        *VMType   `db:"type"`
	// contains filtered or unexported fields
}

func (*VM) CopyImageToDomainDisk 

func (vm *VM) CopyImageToDomainDisk() error

func (*VM) DiskLocation 

func (vm *VM) DiskLocation() string

func (*VM) Domain 

func (vm *VM) Domain() libvirtxml.Domain

func (*VM) FromProtobuf 

func (vm *VM) FromProtobuf(in *server.VM) error

func (*VM) ToProtobuf 

func (vm *VM) ToProtobuf() (out *server.VM, err error)

type VMType 

type VMType int8
const (
	VMTypeUnknown VMType = iota
	VMTypeTeeny
	VMTypeSmall
	VMTypeMiddy
	VMTypeBiggun
	VMTypeFatLad
)

func (VMType) CPUs 

func (e VMType) CPUs() uint

func (*VMType) FromProto 

func (e *VMType) FromProto(s server.VMType)

func (*VMType) FromString 

func (e *VMType) FromString(s string)

func (*VMType) Proto 

func (e *VMType) Proto() server.VMType

func (VMType) RAM 

func (e VMType) RAM() uint

func (*VMType) Scan 

func (e *VMType) Scan(in any) error

func (VMType) Value 

func (e VMType) Value() (driver.Value, error)

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.