testfleet

README

testfleet

testfleet is a single-binary CLI that dispatches a test directory to a remote machine over Tailscale SSH, runs it, and pulls the artifacts back so a calling agent (Claude, Codex, you) can reason about what happened.

It is intentionally thin: testfleet returns artifacts, not pass/fail verdicts. The mechanical layer (exit code, status) is the floor; the judgment layer (what do the screenshots / logs / JSON dumps actually mean) is the calling agent's job. See skills/testfleet/SKILL.md for the full two-layer model.

Install

go install github.com/Anro-Lab/testfleet/cmd/testfleet@latest
testfleet --version

@latest resolves to the newest release tag. Or build from this repo:

git clone https://github.com/Anro-Lab/testfleet     # clones the default branch, main
cd testfleet
make build              # or: go build -o testfleet ./cmd/testfleet

Branches & releases: main is the stable line and the default branch — clone it and go install @latest both land here. Active development happens on develop; tagged releases (v0.1.0, …) are cut from main. Requires Go 1.25+ (see docs/prerequisites.md).

Quick start

You'll need a Tailscale OAuth client, an SSH keypair deployed to one fleet machine, and gh authenticated. The full setup is in docs/prerequisites.md.

export TAILSCALE_OAUTH_CLIENT_ID=...
export TAILSCALE_OAUTH_CLIENT_SECRET=...
export TAILSCALE_TAILNET=example.com
export TESTFLEET_SSH_KEY=$HOME/.ssh/testfleet_ed25519
export TESTFLEET_SSH_USER=Administrator

# 1. See what's reachable, and grab one idle machine's name
testfleet machines list --json | jq '.data.machines[] | {name, os, status}'
MACHINE=$(testfleet machines list --json | jq -r '.data.machines[] | select(.status=="idle") | .name' | head -1)

# 2. Dispatch a test to it
testfleet run examples/test-sysinfo --machine "$MACHINE" --wait --json

# 3. Pull artifacts
RUN_ID=$(testfleet runs list --json | jq -r '.data.runs[0].id')
testfleet runs artifacts "$RUN_ID" --copy-to ./out --json
cat ./out/sysinfo.json | jq .

# 4. (Optional) File an issue from what you saw
testfleet issue open ./draft.json --repo Anro-Lab/testfleet

What's in the box

• cmd/testfleet — the CLI (cobra-based; machines, run, runs, issue subcommands).
• internal/ — Tailscale OAuth client, SSH/SFTP transport, runner orchestrator, JSON-file run store with flock, output envelope (schema_version: "v1"), gh issue wrapper.
• skills/testfleet/SKILL.md — the document a calling agent reads to drive the full Happy Path Step 1-8 loop. The doc test (cmd/testfleet/skill_doc_test.go) parses every CLI example in this file and asserts they match the live cobra commands; a renamed flag breaks the build.
• examples/test-sysinfo/ — minimal Windows + Linux smoke test you can dispatch immediately.
• docs/ — prerequisites, full JSON schemas, L3 manual walkthrough.
• test/e2e/ — L2 environment tests gated behind //go:build e2e and TESTFLEET_E2E=1.

Documentation

Doc	Audience
docs/user-flow.md	New colleague — one-page 白話 user flow（兩部分全貌 + 工程師驅動 AI / CI 兩條入口）+ 流程圖
skills/testfleet/SKILL.md	Calling agent — what commands to run and how to interpret the output
docs/prerequisites.md	Operator — SSH / Tailscale / Windows setup
docs/json-schemas.md	Anyone parsing --json output
docs/l3-walkthrough.md	PM / reviewer — manual end-to-end verification

Tests

go test ./...                         # L1 unit tests, always green
go test -tags e2e ./test/e2e/...      # L2; requires TESTFLEET_E2E=1 + Tailscale + a fleet machine

License

MIT — see LICENSE.