runtime

Documentation

Overview

Package runtime supervises a podman container that hosts this outpost's k3s-agent kubelet. The container's identity is THIS outpost's identity (NodeToken, AgentName, overlay credentials); from the cluster's POV there's one Node per outpost — the container is invisible.

Why a container (not a host subprocess): security isolation (kubelet + containerd run under cgroups managed by an outer runtime, not directly on the host); cross-platform Linux runtime (macOS hosts don't have a host kubelet but can run a privileged Linux container via Docker Desktop / Rancher Desktop / ycode-podman / Lima). One model, every OS.

Lifecycle: outpost daemon calls Up(ctx, opts). Up locates `podman` on PATH, pulls/builds the image if missing, starts a named container with the credentials threaded in via env, then streams its logs back to outpost's slog. Down(ctx, opts) stops + removes the container. Up is idempotent — repeated calls with the same AgentName reuse the existing container.

Index

• Constants
• Variables
• func BuildImage(ctx context.Context, opts BuildOptions) (string, error)
• func Down(ctx context.Context, opts Options) error
• func ExecInContainer(ctx context.Context, opts Options, args ...string) ([]byte, error)
• func PodmanAvailable() bool
• func TailLogs(ctx context.Context, opts Options) error
• func Up(ctx context.Context, opts Options) error
• type BuildOptions
• type Options
  • func (o Options) PodNetwork() PodNetwork
• type PodNetwork
  • func ClassifyPodNetwork(podCIDR string) PodNetwork
  • func (n PodNetwork) Log(node string)
  • func (n PodNetwork) Overlay() bool
• type PodNetworkMode

Constants

const DefaultImage = "outpost-runtime:dev"

DefaultImage is the runtime image tag the outpost daemon expects to find. Built by `outpost cluster build-runtime`; can be overridden via Options.Image.

const FallbackPodCIDR = "10.43.42.0/24"

FallbackPodCIDR mirrors the CNI_LOCAL_POD_CIDR default in image/entrypoint.sh — the fixed range the single-node fallback allocates from on EVERY node. Reported for operator legibility only; nothing here configures the container (the entrypoint owns that value, and an operator override rides in via ExtraEnv).

Variables

var ErrPodmanNotFound = errors.New("runtime: no `podman` or `docker` binary on PATH (install Docker Desktop / Rancher Desktop / podman to enable --cluster-mode=agent)")

ErrPodmanNotFound is returned by Up when neither `podman` nor `docker` is on PATH. The outpost daemon surfaces it as a clear "install Docker Desktop / Rancher Desktop / podman to enable --cluster-mode=agent" message; on macOS this is the expected gating.

Functions

func BuildImage

func BuildImage(ctx context.Context, opts BuildOptions) (string, error)

BuildImage materializes the embedded build context (Dockerfile + entrypoint.sh + cni/) to a tempdir and runs `podman build` against it. Idempotent: every invocation is a fresh tempdir, but podman's own layer cache means a second run with no source changes returns in seconds (each RUN line hashes its inputs; identical inputs reuse the cached layer).

Returns the image tag that was produced and any build error. The tempdir is cleaned up on return regardless of outcome.

func Down

func Down(ctx context.Context, opts Options) error

Down stops + removes the container. Used during outpost shutdown + when the operator flips --cluster-mode=off.

func ExecInContainer

func ExecInContainer(ctx context.Context, opts Options, args ...string) ([]byte, error)

ExecInContainer runs a command inside the outpost's runtime container and returns its combined output.

Used by the overlay refresher to talk to the tailscaled that lives in there — checking `tailscale status` and re-running `tailscale up` with a fresh key. Going through podman exec (rather than reaching into the container's socket) keeps the daemon free of any assumption about the container's internals beyond "the tailscale CLI is on its PATH".

A non-nil error covers both "podman could not run it" and "the command exited non-zero"; the output is returned either way so callers can log what actually happened.

func PodmanAvailable

func PodmanAvailable() bool

PodmanAvailable reports whether the runtime is usable on this host. Outpost CLI / admincore status surface uses this to render a clear "cluster-mode=agent unavailable — install podman" hint instead of failing silently at start time.

func TailLogs

func TailLogs(ctx context.Context, opts Options) error

TailLogs blocks and streams the container's logs to slog at info level. Returns when the container exits (or ctx is canceled). The caller typically runs this in a goroutine inside the errgroup.

func Up

func Up(ctx context.Context, opts Options) error

Up ensures the runtime container is running with the supplied credentials. Idempotent: if a container with the expected name already exists, Up restarts it (so credential changes take effect). Returns immediately after the container is started; container exit is observed through ctx + a follow-up goroutine the caller spins to tail logs.

Types

type BuildOptions

type BuildOptions struct {
	// Tag is the image reference to produce (e.g.
	// "outpost-runtime:dev" or a registry-qualified name for push).
	// Empty defaults to DefaultImage.
	Tag string

	// TargetArch is the linux architecture of the runtime image
	// ("amd64" or "arm64"). Empty defaults to the host's arch — the
	// usual case, since the container runs on the same machine the
	// outpost daemon does. Override only when cross-building (e.g.
	// from an arm64 dev machine for an amd64 production host).
	TargetArch string

	// PodmanBin overrides the autodetected `podman`/`docker` binary
	// used to drive the build. Empty triggers the same PATH lookup
	// the supervisor uses (pickPodmanBin).
	PodmanBin string

	// Stdout / Stderr receive the podman build's output. Defaults
	// (when nil) route to os.Stdout / os.Stderr so the operator
	// sees the build progress interactively.
	Stdout, Stderr *os.File
}

BuildOptions controls `outpost cluster build-runtime`. All fields are optional — sensible defaults match the supervisor in Up().

type Options

type Options struct {
	// AgentName is the outpost's identity. The container's k3s-agent
	// joins as Node <AgentName>; the container itself is named
	// <AgentName>-runtime.
	AgentName string

	// Image is the runtime container image (e.g. "outpost-runtime:dev").
	// Built once via `outpost cluster build-runtime` or pulled from a
	// registry. Empty defaults to DefaultImage.
	Image string

	// NodeToken is the k3s join token cloudbox handed out at pairing
	// (K10<ca-hash>::node:<secret>). Passed into the container via
	// the OUTPOST_NODE_TOKEN env var; never written to disk on the
	// host.
	NodeToken string

	// APIServer is the URL the container's k3s-agent dials. In the
	// cloudbox model this is the loopback STCP visitor inside the
	// container (see overlay package). Empty defaults to
	// "https://127.0.0.1:6443".
	APIServer string

	// CloudboxHost / CloudboxPort are where the container-side frpc
	// dials to establish the matrix-tunnel + STCP visitor. Required
	// for the kubelet-in-container model — entrypoint.sh runs frpc
	// to open 127.0.0.1:APIPort inside the container, tunneling to
	// cloudbox's embedded apiserver. e.g. "ai.dhnt.io" + 443.
	CloudboxHost string
	CloudboxPort int

	// STCPSecret authenticates the STCP visitor on the cloudbox side
	// (cluster.k3s-apiserver publisher). Cluster-wide secret minted
	// at pairing time; passed in via env.
	STCPSecret string

	// MatrixToken is the shared frp auth token (same value cloudbox
	// holds in MATRIX_TOKEN). Empty disables [auth] in frpc.toml.
	MatrixToken string

	// APIPort is the loopback port the STCP visitor binds inside the
	// container (must match cloudbox's ClusterAPIServerPort). Empty
	// defaults to 6443.
	APIPort int

	// KubeletPort is the per-outpost port cloudbox allocated at
	// pairing time (fc.Cluster.KubeletProxyPort). Three things ride
	// on this same number so the apiserver→kubelet hop terminates:
	//   - kubelet binds + advertises this port (so the Node's
	//     daemonEndpoint.Port matches what's reachable);
	//   - the in-container frpc publishes 127.0.0.1:<port> to
	//     cloudbox's loopback at the same port number;
	//   - cloudbox's apiserver dials 127.0.0.1:<port> for this Node.
	// Empty (0) leaves the kubelet on its default 10250 with no
	// outbound publish — `kubectl exec`/`logs`/`port-forward` won't
	// work against this outpost, but the rest of cluster-agent mode
	// keeps functioning. Old pairings without KubeletProxyPort
	// allocated land here.
	KubeletPort int

	// PodCIDR is the per-outpost /24 carved by cloudbox at Exchange
	// time. Empty disables the outpost-cni conflist; k3s falls back
	// to its own defaults (--flannel-backend=none means no pod
	// networking, fine for control-plane-only smoke tests).
	PodCIDR string

	// OverlayLoginServer / OverlayAuthKey turn on tailscaled inside
	// the container. Both must be non-empty; both empty leaves the
	// overlay off (single-node mode).
	OverlayLoginServer string
	OverlayAuthKey     string

	// PodmanBin overrides the autodetected `podman`/`docker` binary.
	// Empty triggers PATH lookup; tests set it.
	PodmanBin string

	// ExtraEnv is appended to the container's env in KEY=VALUE form.
	// Escape hatch for development.
	ExtraEnv []string
}

Options is the supervisor's input. All fields except ExtraEnv are required; LoginServer/AuthKey/PodCIDR may be empty for single-node (no-overlay) testing.

func (Options) PodNetwork

func (o Options) PodNetwork() PodNetwork

PodNetwork classifies the node these Options describe, honoring an ExtraEnv override of the fallback range so the reported CIDR matches what the container will really allocate from.

type PodNetwork

type PodNetwork struct {
	// Mode is the classification. Never empty.
	Mode PodNetworkMode

	// PodCIDR is the range pods on this node get IPs from. In overlay
	// mode it is the cloudbox-allocated per-node CIDR; in fallback mode
	// it is the fixed range shared with every other node.
	PodCIDR string
}

PodNetwork is the classified pod-network state of one node: which mode the container will come up in, and the pod CIDR that mode will actually allocate from.

func ClassifyPodNetwork

func ClassifyPodNetwork(podCIDR string) PodNetwork

ClassifyPodNetwork is the single source of truth for the mode. A non-empty pod CIDR means the overlay conflist; an empty one means the shared-range fallback.

func (PodNetwork) Log

func (n PodNetwork) Log(node string)

Log announces the pod-network mode at boot. The fallback is logged at WARN, not Info: a node with no pod network is a silent multi-node-cluster corruption, and this line is the only place it becomes visible before pods start colliding. The overlay case logs at Info with the CIDR so the two are trivially greppable.

func (PodNetwork) Overlay

func (n PodNetwork) Overlay() bool

Overlay reports whether this node has a real (per-node, routable) pod network.

type PodNetworkMode

type PodNetworkMode string

PodNetworkMode classifies which of the two CNI configurations the runtime container's entrypoint will write for this node. The two modes are NOT interchangeable and the difference is invisible from the outside — a node in either mode joins and reports Ready — so this type exists to make the distinction nameable, loggable, and reportable rather than implicit in "is OUTPOST_POD_CIDR set".

const (
	// PodNetworkOverlay means cloudbox carved a per-outpost pod CIDR
	// for this node and the entrypoint writes the outpost-cni conflist
	// over the tailscale overlay: unique pod IPs per node, cross-node
	// pod routing. This is the only mode that is correct in a
	// multi-node cluster.
	PodNetworkOverlay PodNetworkMode = "overlay"

	// PodNetworkSingleNodeFallback means no pod CIDR was allocated, so
	// the entrypoint falls back to a plain bridge + host-local IPAM out
	// of a FIXED range that is identical on every node. Correct for a
	// single-node cluster; catastrophic in a multi-node one — every
	// node hands out the same pod IPs, Service endpoint lists contain
	// duplicate addresses, and kube-proxy can DNAT a request to the
	// wrong local workload. Nothing errors and the node reports Ready,
	// which is precisely why it has to be announced.
	PodNetworkSingleNodeFallback PodNetworkMode = "single-node-fallback"
)