Documentation
¶
Overview ¶
Package upgrade carries the self-upgrade machinery shared by the CLI (`outpost upgrade`, `outpost rollback`) and the cloudbox-pushed daemon route (POST /admin/upgrade).
The wire shape (Envelope) is what cloudbox sends. The worker (Worker) owns serialized in-process state — one upgrade at a time, last-release-id dedup, a JSONL ledger of every attempt. The Stage / Probe helpers are the deterministic file-level steps both the CLI and the worker reuse.
Trust model: the daemon trusts cloudbox to push valid URLs and matching sha256s. The URL must be https, the sha256 must verify on download, and the candidate binary must self-identify as an outpost build via `version --json`. A future ArtifactVerifier hook (signed release manifests) is wired in but defaults to a no-op so the daemon can ship before signing infrastructure exists.
Index ¶
- Variables
- func CleanupStaleSwaps(_ string)
- func MountRoute(rg *gin.RouterGroup, w *Worker)
- func PendingPath(cacheDir string) string
- func Probe(path, expectedCommit string) (agent.BuildInfo, error)
- func RetainPrevious(binary, previous string) error
- func StageFromLocal(srcPath, dst, expectedSHA string) error
- func StageFromURL(ctx context.Context, dst, srcURL, expectedSHA string, client *http.Client) error
- func SwapAtomic(current, candidate string) error
- type ArtifactVerifier
- type Envelope
- type Ledger
- type LedgerEntry
- type NoopVerifier
- type Options
- type Result
- type RollbackResult
- type State
- type StateSnapshot
- type Status
- type Worker
Constants ¶
This section is empty.
Variables ¶
var ErrNoPrevious = errors.New("no outpost.previous on disk; nothing to roll back to")
ErrNoPrevious is returned when no rollback candidate is on disk (a fresh install or a daemon that hasn't been upgraded yet).
var ErrShortCommit = errors.New("candidate binary does not self-report the expected commit")
ErrShortCommit is returned by the Probe step when the candidate's self-reported commit doesn't match the envelope. This is the last defense against URL/SHA collisions or operator typos — we refuse to swap a binary that doesn't identify as the release we expected.
Functions ¶
func CleanupStaleSwaps ¶
func CleanupStaleSwaps(_ string)
CleanupStaleSwaps is a no-op on Unix — the atomic-rename path leaves no temp files behind. The function exists so cross-platform callers (main.go) don't need build tags.
func MountRoute ¶
func MountRoute(rg *gin.RouterGroup, w *Worker)
MountRoute attaches the cloudbox-pushed self-upgrade handler at `POST /admin/upgrade` on the given route group.
**Auth model: trust the tunnel.** No bearer check at this layer. The route lives on the daemon's matrix-tunnel-fronted main HTTP server, which binds 127.0.0.1 only — cloudbox is the only entity that can reach it (through the tunnel client's loopback proxy port on the cloudbox side). This is the same model the existing /apps and /healthz routes use; adding a per-host bearer here would require a shared secret cloudbox can present, and the obvious candidate (fc.Token aka cfg.MatrixToken) is empty in real production deployments where cloudbox runs without a matrix-tunnel auth secret. Defense-in-depth lives elsewhere: the AutoUpgrade toggle (operator opt-out), the sha256 + envelope.commit checks (artifact integrity), and the Probe step (commit-match self- verify) all gate what the worker will actually do.
The route is intentionally NOT under `/api/*` — cloudbox calls it through the matrix tunnel and the daemon's main HTTP server is loopback-only-via-tunnel, so the `/api/*` cookie-auth-protected surface (which lives on the separate admin listener at :17777) can't reach it anyway. Keeping it at `/admin/upgrade` is purely a naming choice — signals "this is a cloudbox→outpost control path," not "this is an admin-UI route."
Status codes: 202 accepted, 200 replay, 400 invalid envelope, 403 auto_upgrade off, 304 same commit, 412 min_from mismatch, 409 in-flight.
func PendingPath ¶
PendingPath returns the canonical path to upgrade.pending.json under the given cache dir. Public so main.go and the CLI's apply-pending command can both compute it without a magic string drifting between them.
func Probe ¶
Probe execs `<path> version --json` and decodes the BuildInfo. This is the single check that distinguishes "an outpost binary at the expected commit" from "anything else on disk." Bad exit code, bad JSON, missing go_version, or a commit mismatch all fail closed.
`expectedCommit` is the envelope's short commit; pass "" from the CLI (which doesn't pre-commit to a sha) to skip the commit check. The worker always passes the envelope's Commit so a man-in-the- middle that substitutes a same-named differently-built binary (different go.mod, different version of dependent packages, etc.) would still need to match the released sha to land.
func RetainPrevious ¶
RetainPrevious snapshots the running binary at "<binary>.previous" before rename. Hardlink first (instant, single-fs); fall back to a copy on cross-fs or filesystems without hardlink support. Exported so the CLI's `outpost upgrade --local|--from` path can call it before its own swap — both paths leave a rollback target on disk.
func StageFromLocal ¶
StageFromLocal copies `srcPath` into `dst`, verifying sha256 if `expectedSHA` is non-empty. CLI-only entry point; the daemon worker never reads from a local path (callers can't be trusted to deliver arbitrary paths over the wire).
func StageFromURL ¶
StageFromURL downloads the candidate binary at `srcURL` into `dst`, verifying the body against `expectedSHA` (hex) on the fly. Both caller-provided modes (cloudbox push and `outpost upgrade --from`) flow through here; the only difference is the daemon's worker always passes a non-empty expectedSHA while the CLI permits empty for ad-hoc test pushes from a local artifact server.
The destination is opened O_EXCL so a stale "<binary>.upgrading" from a crashed prior attempt surfaces as an error instead of silent clobber. Caller is responsible for cleaning up `dst` on downstream errors.
func SwapAtomic ¶
SwapAtomic puts `candidate` at `current`, replacing whatever was there. On Unix the kernel allows os.Rename over a file that's currently being executed — the running process keeps its file open via inode reference, the path is rebound to the new file, and the next exec from the path reads the new binary. Single rename, fully atomic.
The current file is replaced; callers that want to retain the prior generation for rollback must do so BEFORE calling SwapAtomic (the Worker uses RetainPrevious to hardlink outpost.previous next to outpost first).
Types ¶
type ArtifactVerifier ¶
type ArtifactVerifier interface {
Verify(env Envelope, candidatePath string, candidate agent.BuildInfo) error
}
ArtifactVerifier is the seam for adding signed-release-manifest verification later. Today the trust path is "HTTPS to cloudbox + sha256-in-envelope + Probe(commit-match)" — good enough while cloudbox itself is the root of trust for the fleet. When third-party release channels become a thing, an implementation will check a cryptographic signature (e.g. cosign / minisign / minimaCert) over the {commit, sha256, url} tuple before letting the worker swap the binary in.
Verify is called after Probe succeeds but before the os.Rename. A non-nil error aborts the swap (the candidate file is removed; one ledger entry is emitted with step="verify_failed" and the cause).
Default: NoopVerifier — always returns nil. The Worker.verifier field defaults to this when Options.Verifier is left zero, so today's cloudbox-as-root-of-trust model continues to work without callers having to opt in.
type Envelope ¶
type Envelope struct {
// ReleaseID is cloudbox's opaque identifier for this artifact
// (e.g. "v0.42.1-abc1234"). Used purely for dedup + ledger
// correlation — the daemon doesn't parse it.
ReleaseID string `json:"release_id"`
// URL must be HTTPS. The daemon refuses http://, file://, or
// anything else.
URL string `json:"url"`
// SHA256 of the artifact body (hex). Mandatory here even though
// the CLI variant treats it as optional — a cloudbox push without
// an integrity check is not acceptable.
SHA256 string `json:"sha256"`
// Commit is the 7-char short sha the candidate's BuildInfo must
// expose. Probe rejects the candidate if its self-reported
// commit doesn't match, defending against URL/SHA collisions
// against an unrelated build.
Commit string `json:"commit"`
// MinFrom optionally constrains "this upgrade only applies to
// hosts already running at least this commit." The daemon
// returns 412 if its current commit is older. Useful when a
// release requires migration state from a prior version.
MinFrom string `json:"min_from,omitempty"`
// Force bypasses the daemon's update_mode gate. Set by cloudbox
// (or the local `outpost upgrade apply` CLI) when the OPERATOR
// has explicitly chosen to apply this envelope right now — a
// "manual" host that gets Force=true behaves like an "auto"
// host for this one push. Never set by the GH-Action release
// webhook (those are advisory, not operator-blessed). "never"
// mode still refuses Force=true; the operator must flip the
// mode first.
Force bool `json:"force,omitempty"`
}
Envelope is what cloudbox POSTs to /admin/upgrade. Every field except MinFrom is required; the daemon returns 400 if any of {release_id, url, sha256, commit} is empty.
func ReadPending ¶
ReadPending is the public read variant — used by the apply-pending MCP tool / CLI to fetch the queued envelope before re-submitting it with Force=true.
type Ledger ¶
type Ledger struct {
// contains filtered or unexported fields
}
Ledger is an append-only JSONL writer + bounded tail-reader. Path is fixed at construction; concurrent appends serialize through mu.
func NewLedger ¶
NewLedger returns a Ledger backed by `path`. Doesn't touch the filesystem until the first Append.
func (*Ledger) Append ¶
func (l *Ledger) Append(entry LedgerEntry) error
Append writes one entry as a single JSON line. If `entry.At` is zero, it is filled with the current time. The file is opened O_APPEND so concurrent writers from a future fan-out don't have to coordinate seek positions — the OS handles atomic byte appends on POSIX up to PIPE_BUF, and a single JSON line never exceeds that for our shape.
Errors writing the ledger are NOT fatal to the upgrade — we'd rather complete an upgrade than abort it because we couldn't scribble a record. Callers log Append's error but continue.
func (*Ledger) Tail ¶
func (l *Ledger) Tail(n int) ([]LedgerEntry, error)
Tail returns up to the last `n` ledger entries, newest last. A missing ledger file returns an empty slice without error — a host that has never been upgraded simply has no history.
Implementation reads the whole file (the ledger is unbounded in principle but bounded in practice: one entry per upgrade attempt, which is rare enough that even a year of activity stays well under a megabyte). When we cross some real threshold we can rotate.
type LedgerEntry ¶
type LedgerEntry struct {
At time.Time `json:"at"`
ReleaseID string `json:"release_id,omitempty"`
Step string `json:"step"`
FromSHA string `json:"from_sha,omitempty"`
ToSHA string `json:"to_sha,omitempty"`
URL string `json:"url,omitempty"`
Detail string `json:"detail,omitempty"`
Error string `json:"error,omitempty"`
}
LedgerEntry is one JSON line in the upgrade history file. Each significant phase of an upgrade (received, stage_start, swap_done, restart, failed, rollback_used) emits one of these. The file is the source of truth for "what happened to this host" — surfaced via `outpost upgrade history` and the outpost://upgrade-history MCP resource.
type NoopVerifier ¶
type NoopVerifier struct{}
NoopVerifier is the zero-trust-cost default. Returns nil for every envelope. Replaced by a signature-checker when signed manifests land.
type Options ¶
type Options struct {
State State
Restart func()
Ledger *Ledger
Client *http.Client
Logger *slog.Logger
Verifier ArtifactVerifier // nil → NoopVerifier (today's cloudbox-as-root-of-trust)
}
Options configures a Worker. State and Restart are required; everything else has a sensible zero default.
type Result ¶
type Result struct {
Status Status `json:"status"`
Detail string `json:"detail,omitempty"`
ReleaseID string `json:"release_id,omitempty"`
Commit string `json:"commit,omitempty"`
}
Result is what Apply returns over the wire. Status carries the outcome; Detail is a short human-readable explanation; Commit (when set) lets cloudbox correlate the response against its release metadata.
type RollbackResult ¶
type RollbackResult struct {
Status string `json:"status"` // "" on success, "no_previous"/"in_flight" on refusal
Detail string `json:"detail,omitempty"`
Previous agent.BuildInfo `json:"previous,omitempty"` // build the rollback restored to
FromCommit string `json:"from_commit,omitempty"`
}
RollbackResult is what Rollback returns to its caller (the MCP tool and CLI alike). Empty Status + Detail when the rollback was applied; non-empty Status when refused.
type State ¶
type State func() StateSnapshot
State exposes the running daemon's knowledge that the upgrade worker needs to make decisions on each Apply call. Threaded as a closure so the worker doesn't have to import admincore/conf and pull in their construction graph for tests.
type StateSnapshot ¶
type StateSnapshot struct {
// UpdateMode is the per-host policy for incoming pushes. One of
// "auto", "manual", "never" (the conf.UpdateMode* constants;
// empty is treated as "auto"). See conf/file.go for the contract.
UpdateMode string
// CurrentCommit is the running daemon's short commit (e.g.
// "820e2e1"). Used for the same-commit short-circuit and the
// min_from precondition.
CurrentCommit string
// BinaryPath is the live binary's on-disk location (os.Executable
// of the daemon). The worker stages "<BinaryPath>.upgrading" next
// to it and hardlinks the current to "<BinaryPath>.previous"
// before rename for rollback.
BinaryPath string
// PendingPath is the path to upgrade.pending.json — where the
// worker persists envelopes received while in manual mode. The
// operator's `outpost upgrade apply` reads this file and re-POSTs
// the envelope with Force=true to consume it.
PendingPath string
}
StateSnapshot is what State returns each tick. The worker treats it as a momentary read — re-evaluates on each Apply call so a just-flipped update_mode toggle takes effect immediately.
type Status ¶
type Status string
Status enumerates the outcomes Apply can return. The HTTP route layer maps these to status codes — keep the mapping in one place (see HTTPStatus below) so adding a new outcome doesn't require editing the gin handler.
const ( StatusAccepted Status = "accepted" // upgrade queued; worker goroutine running StatusReplay Status = "replay" // same release_id we just handled; idempotent no-op StatusInFlight Status = "in_flight" // another upgrade is currently running StatusSameCommit Status = "same_commit" // current daemon is already on this commit StatusDisabled Status = "disabled" // operator turned update_mode to "never" StatusMinFrom Status = "min_from" // current commit is older than envelope.min_from StatusPendingManual Status = "pending_manual" // envelope persisted; operator must apply via UI/CLI )
func (Status) HTTPStatus ¶
HTTPStatus maps an Apply outcome to the wire HTTP status. 202 Accepted covers both "work is happening async" (StatusAccepted) and "envelope was persisted, awaiting operator" (StatusPendingManual); the body's status field disambiguates. The rest are terminal refusals with a human-readable reason.
type Worker ¶
type Worker struct {
// contains filtered or unexported fields
}
Worker drives the cloudbox-pushed upgrade flow on the daemon side. One Worker per daemon process; the route handler routes every /admin/upgrade POST through Worker.Apply.
Invariants:
- Only one upgrade goroutine runs at a time (enforced by inFlight).
- Replays of the same ReleaseID return StatusReplay without doing anything, even after a prior upgrade completed — defends against cloudbox retries during the restart window when the daemon may briefly appear unresponsive.
- All state changes funnel through Apply's lock; the worker goroutine never mutates Worker fields after it's spawned (it just appends to the ledger and calls restart).
func NewWorker ¶
NewWorker constructs a Worker. State and Restart are required — without State the worker can't decide anything; without Restart the upgrade can't take effect (the daemon would keep running the old binary even after the swap).
func (*Worker) Apply ¶
Apply is the single entry point. The route handler calls this after binding the JSON body. Returns the wire Result; the handler maps Status → HTTP code via HTTPStatus.
func (*Worker) LoadPending ¶
LoadPending returns the queued envelope for this worker, if any. The worker captures the pending path from its State closure on every Apply; LoadPending reuses the same source-of-truth so the MCP / CLI surfaces don't need to compute the path themselves.
func (*Worker) Rollback ¶
func (w *Worker) Rollback(ctx context.Context) (RollbackResult, error)
Rollback restores `<binary>.previous` over the live binary and triggers a restart. Refuses while another upgrade is in flight to avoid racing the inflight worker's own rename. After rollback the `.previous` file is gone; the operator must re-upgrade to climb forward again (we intentionally don't keep a "next" copy — that would require two-deep generation tracking we don't need today).
Source Files