Documentation
¶
Overview ¶
Package bgloop is fak's IN-KERNEL BACKGROUND-LOOP RUNTIME — the supervisor that keeps recurring work progressing while the kernel (`fak serve`) is up, and makes each loop observable.
Tier: foundation (1) — see internal/architest. It is stdlib-only and imports nothing internal, so it sits at the bottom of the layering DAG and any higher layer (the gateway, a demo command) may construct it. An upward import would fail the architest gate.
The gap it closes ¶
fak frames itself as "loops all the way down" (docs/explainers/ engineering-is-building-loops.md): the tool-call, turn, session, fleet, and RSI loops. But every one of those is driven from OUTSIDE the running kernel — an OS scheduled task on a 10/15/30-minute cadence (the dispatch fleet), a one-shot `cmd/rsiloop` invocation, or an agent harness re-invoking itself. When `fak serve` is the process that is actually up, there was no first-class notion of a loop the KERNEL itself owns and keeps ticking. bgloop is that notion: a registered Loop runs in its own supervised goroutine on the serve lifecycle context, restarts on a panic or error with capped exponential backoff (a misbehaving loop never takes the kernel down), and exposes a point-in-time Snapshot of its progress.
How it complements loopmgr (the ledger) and looprecover (the worklist) ¶
internal/loopmgr is the durable, hash-chained JSONL LEDGER of loop events, and its own doc is blunt that it "does not schedule, spawn, notify, or authorize anything by itself; those stay in the callers that produce events." bgloop is exactly that missing caller for in-kernel loops: it RUNS them. The two compose without coupling — bgloop stays dependency-free and exposes two seams a host wires to loopmgr at a higher tier:
- WithObserver(func(Status)) is the PUSH seam: a host can fold each tick into the loopmgr ledger (an armed/heartbeat/end event) so an in-kernel loop shows up in `fak loop status` next to the externally-scheduled ones. Metrics, by contrast, use the PULL model — the gateway reads Snapshot() at /metrics scrape time — so no observer is needed just to expose Prometheus state.
- WithAdmit(func(name) (ok, reason)) is the BACKPRESSURE seam: a host can gate each fire through loopmgr.Governor.Admit, giving an operator the existing pause/disable/cadence-floor knobs over a loop the kernel runs.
Pure, supervised, observable ¶
The runtime is stdlib-only and deterministic to test: production code reads the real clock, and the time-dependent witnesses drive it under testing/synctest's virtual time. The three invariants it ships:
- PROGRESS: an interval Loop keeps ticking while the kernel is up (Status.Ticks climbs, NextTickAt advances).
- CONTAINMENT: a Tick that panics or errors is recovered and counted (Panics / Errors / Restarts), the loop backs off and resumes, and neither the supervisor nor any sibling loop is affected.
- CLEAN SHUTDOWN: on context cancel (or Shutdown), every loop reaches Stopped and its goroutine is joined within the deadline; a Tick that ignores cancellation makes Shutdown return a timeout error rather than hang silently.
See AGENTS.md and internal/architest for the layering contract; the worked example is the `fak bgloop` verb (offline demo + live-server status) and the gateway wiring that registers the kernel's built-in heartbeat loop.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrUnauthenticatedEvent = errors.New("bgloop: unauthenticated wake event")
Functions ¶
This section is empty.
Types ¶
type Admission ¶ added in v0.45.0
Admission is the content-free result of a rehydration policy decision.
type Clock ¶ added in v0.45.0
Clock is the time boundary used by durable wake scheduling. Tests can advance it without sleeping; production uses WallClock.
type DurableScheduler ¶ added in v0.45.0
type DurableScheduler struct {
// contains filtered or unexported fields
}
DurableScheduler stores alarms in loopmgr's registry rather than holding a sleeping goroutine per loop. A new scheduler over the same registry re-arms every alarm after process restart.
func NewDurableScheduler ¶ added in v0.45.0
func NewDurableScheduler(path string, clock Clock, store RegistryStore, wake WakeFunc) (*DurableScheduler, error)
func (*DurableScheduler) Poll ¶ added in v0.45.0
func (s *DurableScheduler) Poll(ctx context.Context) (int, error)
Poll re-arms persisted alarms, firing every due alarm at most once. A due alarm outside its duty window is moved to the next on-window without firing.
func (*DurableScheduler) Run ¶ added in v0.45.0
func (s *DurableScheduler) Run(ctx context.Context) error
Run waits only until the nearest persisted deadline, then reloads the registry. Registering alarms in another process therefore survives death and is observed on the next startup/poll rather than requiring a held sleeper.
func (*DurableScheduler) SleepUntil ¶ added in v0.45.0
SleepUntil atomically records an absolute wake deadline on an existing loop.
type DutyCycle ¶ added in v0.45.0
DutyCycle is the scheduler's lower-tier view of a recurring on-window.
type EventScheduler ¶ added in v0.45.0
type EventScheduler struct {
RegistryPath string
Gate RehydrationGate
Store RegistryStore
Clock Clock
JitterWindow time.Duration
Admit func(context.Context, WakeRequest) (func(), error)
Fire func(context.Context, Job) error
}
func (EventScheduler) DeliverEvent ¶ added in v0.45.0
func (EventScheduler) ScheduleWakeCohort ¶ added in v0.45.0
type Job ¶ added in v0.45.0
type Job interface {
JobID() string
}
Job is the scheduler's lower-tier view of a persisted loop job.
type Loop ¶
type Loop struct {
// Name uniquely identifies the loop. It is the label in /metrics and the JSON
// snapshot, so keep it stable and metric-safe (lowercase, no spaces). Required.
Name string
// Interval is the wait between the END of one tick and the START of the next.
// A value <= 0 means CONTINUOUS: the next tick begins as soon as the last
// returns, so such a Tick must pace itself (block until there is work) or it
// will spin. Most kernel loops want a positive interval.
Interval time.Duration
// Tick is the work performed each iteration. It MUST honor ctx cancellation
// (return promptly when ctx is Done) so the kernel can shut down cleanly. A
// returned error is recorded and triggers backoff; a panic is recovered,
// recorded, and also triggers backoff — neither ever crashes the kernel.
// Required.
Tick func(ctx context.Context) error
}
Loop is one unit of recurring background work the kernel supervises while it is up. It is pure configuration; the live runtime state lives in the supervisor's private loopState and is read out as a Status.
type Option ¶
type Option func(*Supervisor)
Option configures a Supervisor at construction.
func WithAdmit ¶
WithAdmit installs an admission gate consulted before every fire. Returning false holds the fire (StatePaused) with the given reason and re-checks on the next interval — the BACKPRESSURE seam a host wires to loopmgr.Governor.Admit so an operator can pause, disable, or rate-floor a loop the kernel runs.
func WithBackoff ¶
WithBackoff sets the failure backoff bounds (first retry waits min, doubling up to max). Non-positive or inverted bounds are ignored in favor of the defaults.
func WithClock ¶ added in v0.38.0
WithClock installs the injectable clock seam — the `now func() time.Time` the dormancy clock and the rehydrate leaves also consume (epic #1178, #1192). Every instant the supervisor records — a loop's StartedAt, each tick's LastTickAt/duration, and the backoff / next-idle NextTickAt — is read through it, so a deterministic dormancy harness can fast-forward a loop's observable clock hours -> months with no real wait and no wall-clock read. Defaults to time.Now; a nil argument is ignored (the default stays in force) so production callers never lose their clock.
func WithObserver ¶
WithObserver installs a callback invoked with a loop's Status after each completed tick (and each refused fire). It is the PUSH seam a host uses to fold in-kernel loop activity into an external surface such as the loopmgr ledger. It must not block; it runs on the loop's own goroutine. Metrics do not need it — read Snapshot at scrape time instead.
type RegistryStore ¶ added in v0.45.0
type RegistryStore interface {
Validate(path string) error
LookupArmed(path, jobID string) (Job, bool, error)
SetWake(path, jobID string, at time.Time, duty DutyCycle, updatedAt time.Time) error
ArmedWakes(path string) ([]StoredWake, error)
MoveWake(path, jobID string, at, updatedAt time.Time) error
ClearWake(path, jobID string, updatedAt time.Time) error
}
RegistryStore isolates durable scheduling from the loop registry's storage representation. loopmgr provides the production adapter.
type RehydrationGate ¶ added in v0.45.0
RehydrationGate is the lower-tier policy seam used before firing a wake.
type State ¶
type State string
State is the observable lifecycle phase of a supervised loop.
const ( // StateIdle: the loop is waiting out its interval before the next tick. StateIdle State = "idle" // StateRunning: the loop is inside Tick right now. StateRunning State = "running" // StateBackoff: the last tick failed (error or panic); the loop is waiting out // the exponential backoff before retrying. StateBackoff State = "backoff" // StatePaused: the admit gate refused this fire (operator backpressure); the // loop is holding and will re-check on the next interval. StatePaused State = "paused" // StateStopped: the supervisor shut down or the lifecycle context was cancelled; // the loop's goroutine has exited. StateStopped State = "stopped" )
type Status ¶
type Status struct {
Name string `json:"name"`
State State `json:"state"`
Interval string `json:"interval"` // human-readable (e.g. "30s"); "continuous" when <= 0
StartedAt time.Time `json:"started_at,omitempty"` // when the loop's goroutine began
Ticks uint64 `json:"ticks"` // ticks that returned nil
Errors uint64 `json:"errors"` // ticks that returned a non-nil error
Panics uint64 `json:"panics"` // ticks that panicked (recovered)
Restarts uint64 `json:"restarts"` // backoff cycles entered (errors + panics)
Pauses uint64 `json:"pauses"` // fires the admit gate refused
LastTickAt time.Time `json:"last_tick_at,omitempty"`
LastErrAt time.Time `json:"last_err_at,omitempty"`
LastErr string `json:"last_err,omitempty"`
LastDurMS float64 `json:"last_dur_ms,omitempty"` // wall time of the last completed tick
NextTickAt time.Time `json:"next_tick_at,omitempty"`
}
Status is a point-in-time snapshot of one loop's progress — the observability surface a /metrics scrape, a /v1/fak/loops reader, or `fak bgloop status` renders. The JSON tags are the stable export schema; new fields are additive (omitempty).
type StoredWake ¶ added in v0.45.0
StoredWake carries one armed job and its durable alarm state.
type Supervisor ¶
type Supervisor struct {
// contains filtered or unexported fields
}
Supervisor runs a set of Loops on the kernel's lifecycle. Construct it with New, Register every loop BEFORE Start, then Start(ctx) with the serve lifecycle context. It spawns one goroutine per loop, supervises panics and errors with backoff, and exposes the live progress as a Snapshot. Shutdown cancels every loop and joins the goroutines within a deadline. All methods are safe for concurrent use.
func New ¶
func New(opts ...Option) *Supervisor
New returns an empty Supervisor with default backoff bounds.
func (*Supervisor) Get ¶
func (s *Supervisor) Get(name string) (Status, bool)
Get returns one loop's Status by name.
func (*Supervisor) Register ¶
func (s *Supervisor) Register(l Loop) error
Register adds a loop. It must be called before Start; a name must be non-empty and unique, and Tick must be non-nil. Returns an error otherwise (the loop is rejected, the supervisor is unchanged).
func (*Supervisor) Shutdown ¶
func (s *Supervisor) Shutdown(ctx context.Context) error
Shutdown cancels every loop and waits for the goroutines to exit, up to ctx's deadline. It returns nil once all loops are joined, or a timeout error naming the loops still running if a Tick ignored cancellation past the deadline. Safe to call before Start (a no-op) and more than once.
func (*Supervisor) Snapshot ¶
func (s *Supervisor) Snapshot() []Status
Snapshot returns the live Status of every loop, sorted by name. Safe to call any time (before Start it reports the registered loops as idle with zero counters).
func (*Supervisor) Start ¶
func (s *Supervisor) Start(ctx context.Context)
Start launches every registered loop on a context derived from ctx, then returns immediately (it does not block). It is idempotent — a second call is a no-op. After Start, Register is refused. Cancelling ctx, or calling Shutdown, stops every loop.
type WakeFunc ¶ added in v0.45.0
WakeFunc reconstitutes one persisted loop when its durable alarm becomes due.
Source Files