horchestra

README

horchestra

Beta. The shape is settled and the whole path is exercised on every change — a fleet is deployed in containers, workloads are placed, wired, policed and read back through the API. Not yet a promise of stability: pre-1.0, APIs and configuration formats still change, and they change outright rather than through a compatibility shim, because there is no installed base to carry.

horchestra drives systemd with kubectl. You declare an application with the tool you already have; on the host it becomes a hardened systemd service started from an image. Four things decide how: nothing runs with more privilege than it needs — including horchestra itself; an application is a hardened systemd service started from an image, with no container daemon underneath it; the setting it is built for is data services on machines dedicated to them; and the API is Kubernetes-shaped, so kubectl and what you already know about declaring objects both work. It is not a Kubernetes distribution: no Helm, no operators, no CRDs, and an Application is one program rather than a pod.

You describe what should run; horchestra makes it so. Applications are declared as objects, applied with kubectl, and end up as hardened systemd units started from OCI images. There is no cluster to bootstrap, no etcd, no container daemon, and nothing on a node that runs as root without a reason you can name.

Who it is for

Fleets in the awkward middle: more hosts than you want to configure by hand, fewer than justify Kubernetes and the team it needs. A dozen machines running an application and its dependencies. Edge sites. A product shipped as "some Linux boxes" into somebody else's datacentre.

If you already run Kubernetes and it fits, keep running it. horchestra is for when the operational surface costs more than the orchestration is worth.

What it does differently

systemd supervises workloads, not a daemon of ours. A workload is a transient systemd unit. It keeps running while the agent restarts, is upgraded, or crashes — the agent converges state, it does not babysit processes. There is no container runtime daemon on the node at all.

Nothing privileged runs by default. The agent holds no capabilities and enters its own user namespace unconditionally. Workloads run rootless, in namespaces of their own, from a read-only overlay root, under a seccomp filter with an empty capability bounding set. The one privileged component is a small network helper that exists precisely so the agent can hold nothing.

The API is Kubernetes-flavoured; the model is not. kubectl works — discovery, explain, tables, logs, RBAC — because it is a good client and everyone already has it. But there are no pods, no kubelet, no CNI plugins and no CRDs: an Application is the unit of work, and its schema is the Go type.

One flat network, no per-node address ranges. Workload addresses come from a single fleet-wide range, and an eBPF datapath answers "which host holds this address" from a map rather than from a routing table. Service addresses are translated at the socket, before a packet exists — so there is no per-flow state anywhere and no return path to rewrite. A workload still sees the service address it dialled rather than the backend it was sent to, and a UDP reply still arrives from it; two connections from one workload land on the same backend, chosen from a lookup table rather than remembered, so a backend joining or leaving moves only its own clients.

The tunnel between nodes is not authenticated, and this is the one thing the product cannot fix for you. Workload traffic between nodes rides vxlan or ipip. Neither carries a key: anything that can reach a node's tunnel port can inject a frame with any inner source and destination straight into the workload network. The mitigation is a firewall rule on your underlay — the port open to the other nodes and to nothing else — and it is configuration, not code, so nothing here can enforce it or warn you that you skipped it. Each node reports the encapsulation it is actually running, so intent can be compared against fact. Encryption between nodes is not shipped today.

A node decides nothing about what should run there. Desired state reaches a node one way — the mTLS session — and the agent listens on nothing at all: no port, no socket, no file that can add a workload. It keeps no record of what should be running, reads what IS running from the machine itself, and puts back what somebody changed underneath it, reporting the change to the control plane rather than to a log on the node that the same somebody could rewrite. Two fields suspend the putting-back while a machine is being repaired, and neither suspends the reporting. Written out as seven invariants in agent/README.md, with the honest limit beside them: this is enforcement against mistakes and half-finished manual work, not against the host's own root.

Secrets stay in memory. A resolved secret reaches a workload through RAM-backed carriers and its process environment. It is never written into the unit file, never to disk, and never readable over the message bus.

Security by design

Nothing privileged runs by default. The agent holds no capabilities and enters its own user namespace unconditionally. Workloads run rootless, in namespaces of their own, from a read-only overlay root, with every capability set emptied. The seccomp filter is what keeps it that way: it denies the namespace and mount syscalls, so a workload cannot unshare a namespace of its own — where it would hold a full capability set again, and could mount a writable path over the read-only root. The one privileged component is a small network helper that exists precisely so the agent can hold nothing.

Privilege boundaries are process boundaries, and the compiler holds them. The agent, the network helper and the workload sandbox are separate programs in separate Go modules, so a shortcut across a boundary is a build failure, not a code review comment. The helper's API is closed and typed — there is no "load this program" verb; it loads only what is compiled into its own binary and takes only data from the agent. The sandbox, the one program that becomes a workload's parent, has a single dependency and imports nothing else from the tree.

There is no way to turn authentication off. No flag, no constant, no bypass type left in the code to construct. Every identity is a certificate — the first command an operator runs creates the PKI — and a node is its certificate: what it may read and write is decided by the name it was signed for, and one certificate buys exactly one live session.

An image is checked before a byte of it lands. Every layer is digest-verified, the manifest's declared shape and each transfer's declared size are enforced up front, the unpacker refuses path and symlink traversal at write time, and a truncated layer is never adopted — it becomes visible whole or not at all.

The tunnel between nodes is not authenticated, and this is the one thing the product cannot fix for you. Workload traffic between nodes rides vxlan or ipip. Neither carries a key: anything that can reach a node's tunnel port can inject a frame with any inner source and destination straight into the workload network. The mitigation is a firewall rule on your underlay — the port open to the other nodes and to nothing else — and it is configuration, not code, so nothing here can enforce it or warn you that you skipped it. Each node reports the encapsulation it is actually running, so intent can be compared against fact. Encryption between nodes is not shipped today.

Secrets stay in memory. A resolved secret reaches a workload through RAM-backed carriers and its process environment. It is never written into the unit file, never to disk, and never readable over the message bus.

What it looks like

One file describes a fleet, two commands stand it up:

# node-tool.yaml
apiVersion: node-tool.horchestra.io/v1
kind: Inventory
ssh:
  user: deploy
nodes:
  - addr: 10.0.0.1
    role: controller
    binaries: [ ./bin/horchestra-controller ]
  - addr: 10.0.0.2
    role: agent
    netd: true
    binaries: [ ./bin/horchestra ]

$ node-tool init --local-pki --controller 10.0.0.1 --agent 10.0.0.2
$ node-tool apply -f node-tool.yaml

From there it is kubectl:

apiVersion: horchestra.io/v1
kind: Application
metadata:
  name: web
  namespace: default
spec:
  image: docker.io/library/nginx:alpine
  ports:
    - name: http
      port: 8080

$ kubectl apply -f web.yaml
$ kubectl get app -o wide
NAME   IMAGE                            STATUS    NODE       IP           AGE
web    docker.io/library/nginx:alpine   Running   10.0.0.2   10.244.0.1   12s
$ kubectl logs web

What ships

horchestra	the node — agent, network helper and workload sandbox in one binary: three units, three privileges
horchestra-controller	the control plane — API, scheduler, store
node-tool	the operator's CLI — the fleet's PKI, and apply -f for the fleet itself

Build with make; make help lists every target. The deployable binaries are Linux-only; the operator's tools build anywhere.

Each module says what it is and what it is made of:

core — the Kinds and contracts every other module speaks,

agent — the node half,

controller — the control plane,

netd — the privileged network helper,

sandbox — the workload's confinement.

Licence

AGPL-3.0-or-later (LICENSE) — with one deliberate exception: the eBPF datapath sources netd/bpf/*.bpf.c and the objects compiled from them are GPL-2.0. The kernel's verifier refuses gpl_only helpers to a program that declares anything else, so the licence there is a technical requirement rather than a preference. LICENSING.md has the reasoning, the scope, and the test that keeps it true.