shepherd

README

shepherd-runtime-go

The runtime layer of the Shepherd kernel: the execution scope tree, the merge gate, settlement and recovery, the sandbox backends, the substrate/materialize layer, the task runtime, and the supervision rules engine.

It composes github.com/buchenberg/shepherd-kernel-go — the trace kernel (record model, canonical JSON, SQLite store, effect bus, projections) — and adds the primitives that use a trace: branching execution, workspace capture/restore, reversible settlement, and supervised task handles.

Module scope. The kernel module owns the durable trace: the record model, the store, the effect bus, and the execution/history projections. This module owns everything above it. Trace types (trace.Record, trace.SQLiteTraceStore, trace.WorkspaceState, …) come from github.com/buchenberg/shepherd-kernel-go (package trace, the kernel's module root).

Packages

Package	Contents
shepherd (module root)	the runtime hub — Scope, ScopeManager, checkpoints, workspace operations, the merge gate, settlement (Seal, Settle, RetainedOutput), recovery (RecoverScopes)
sandbox	the Sandbox interface and the backend registry
sandbox/git	the git backend (in-place, worktree); self-registers as "git"
substrate	the Substrate contract, the registry (unknown substrate refs fail the materialization), the echo/KV/workspace substrates
materialize	the materialize orchestrator (owns the append; substrates never do)
task	the task runtime — StartTask/StartTaskSync, Run, TaskControl, the task Registry, ChildHandle
supervisor	the rules engine — Supervisor, SupervisionRule, the built-in rules (DestructiveToolRule, HighErrorRateRule, StuckDetectionRule, DestructiveToolGuard)

Containment backends

The execution boundary is enforced by containment backends: each lowers a set of writable roots to an OS-specific profile, probes it fail-closed, and launches a command confined by it. Each lives in its own nested module, so the root runtime keeps its "kernel + stdlib" dependency footprint.

Module	Platform	Mechanism	Enforcement tier
sandbox/landlock	Linux	an unprivileged kernel LSM (Landlock): self-restrict, then exec	native-syscall-deny
sandbox/winacl	Windows	a write-restricted token, per-root capability SIDs, a Low integrity level, and a mandatory label	native-access-check-deny

Both implement the same containment.Backend contract, so a host picks a backend per platform and passes it to the sandbox constructor — the root module does not change:

sb := git.NewLocalGitSandbox(repoPath, git.WithExec(backend, confinement.PlacementPaddock))

A backend that cannot enforce the requested network mode refuses it rather than running with the axis silently ignored. Each backend's own package documentation states what it does not confine.

Dependency

The runtime requires the kernel for the trace store, the record vocabulary, and the shared public testkit package:

require github.com/buchenberg/shepherd-kernel-go v0.17.0

The kernel is the only external dependency family it pulls in; the runtime carries no other third-party dependency. The module carries no replace directive, so it resolves from the module proxy like any other dependency.

What You Can Do With It

Fork and branch execution

Create isolated execution branches for speculative work. Fork records the branch point in the trace; merge or discard the branch later:

mgr := shepherd.NewScopeManager(store)

// A root scope carries the workspace it operates on.
parent, _ := mgr.Create("agent:main", git.NewLocalGitSandbox(repoPath))
child, _ := mgr.Fork(parent.ID(), "agent:experimental", nil)

// Child runs in isolation...
// If it works:
parent.Merge(child)
// If it fails:
parent.Discard(child)  // child's records remain, but semantically abandoned

Roll the workspace back

Checkpoints capture workspace state plus an opaque caller snapshot (typically conversation history). A restore rewinds both:

cp, _ := mgr.CreateCheckpoint(ctx, scope.ID(), conversationSnapshot)

// ...the sub-agent makes destructive changes...

snapshot, _ := mgr.RestoreCheckpoint(ctx, cp.ID)  // workspace reverted, snapshot returned
// Checkpoints are single-use: restore marks them used and releases the snapshot.

Reusable (non-consuming) workspace states are available when you need to apply the same state more than once — that is what fork-and-choose requires:

forkState, _ := scope.CaptureWorkspace(ctx)   // no lifecycle, stays valid
// ...variant A...
scope.ApplyWorkspace(ctx, forkState)          // rewind to the fork point
// ...variant B...

Checkpoint snapshots persist in the trace database (bounded by a per-store blob limit), so they survive the process.

Recover after a restart

Scopes, checkpoints, and sealed outputs are rebuilt from the durable trace after a crash or restart — the trace is the registry. Sandboxes are never invented: the host's resolver decides which worktrees and containers to re-adopt and which to orphan-clean:

mgr, err := shepherd.RecoverScopes(store, shepherd.RecoverOptions{
    // backend and path come from the fork record the trace durably holds;
    // both are empty for roots, and shared children are never resolved —
    // they inherit the parent's re-adopted sandbox.
    Resolver: func(scopeID, backend, path string) (sandbox.Sandbox, error) {
        return mySandboxFor(scopeID, backend, path) // nil = recover this scope causally only
    },
})
// The tree, terminal states, fork baselines, checkpoints, and retained
// outputs are back. A checkpoint whose recorded revision no longer matches
// the re-adopted workspace is CheckpointInvalid — never silently restorable.
cp := mgr.LatestCheckpoint(scopeID)

Recovered children still propose merges against their recovered fork baselines, and unconsumed outputs settle exactly as they would have before the restart. Long-lived hosts can call mgr.PruneTerminal() to unregister merged/discarded scopes when their settled state is no longer needed in-process.

Isolate a branch in its own worktree

The git backend has two modes. In-place operates on your repository directly. Worktree allocates a detached git worktree per scope, so a speculative branch cannot touch the parent tree and Discard becomes a physical rollback:

sb := git.NewWorktreeSandbox(repoPath, worktreePath)
sb.Create(ctx, sandbox.SandboxSpec{})
defer sb.Destroy(ctx)

// Scope-scoped ownership: the sandbox is destroyed by Discard/Merge/Halt.
child, _ := mgr.ForkIsolated(parent.ID(), "agent:experimental", nil, sb)

Forking is Create then Apply(forkState): the worktree lands on the fork HEAD, then the fork-point state is replayed so the variant sees the parent's exact (uncommitted) tree. Captured state survives teardown because worktrees share the repository's object store.

Supervise sub-agents

Define rules that watch for dangerous patterns and intervene automatically:

sup := supervisor.NewSupervisor(mgr, bus)
sup.AddRule(supervisor.DestructiveToolRule())          // advisory warning on destructive calls
sup.AddRule(supervisor.DestructiveToolGuard())         // deny blockable destructive calls at check time
sup.AddRule(supervisor.HighErrorRateRule(0.5, 10))     // alert at 50% errors
sup.AddRule(supervisor.StuckDetectionRule(3))          // detect loops
sup.Start("orchestrator")

for iv := range sup.Interventions() {
    switch iv.Type {
    case supervisor.InterventionInject:
        sup.Inject(iv.ScopeID, "try a different approach")
    case supervisor.InterventionHalt:
        sup.Halt(iv.ScopeID)
    }
}

Review before merge / settle outputs

Nothing from a child reaches the parent without passing a review gate. CommitMerge proposes the child's diff (against the baseline captured at fork time), records it, runs your reviewer, and either merges or discards — a denial returns *SupervisorDeniedError and never touches the parent:

// Commit-time review: shared-sandbox child, rollback via checkpoint restore.
err := mgr.CommitMerge(ctx, child.ID(), shepherd.DraftsOnlyReviewer("drafts"))
var denied *shepherd.SupervisorDeniedError
if errors.As(err, &denied) {
    _, _ = mgr.RestoreCheckpoint(ctx, cp.ID) // roll the shared workspace back
    // retry with the denial reason injected as guidance
}

For isolated children (worktrees), physical propagation belongs to settlement: seal each output, compare the frozen Changes, then settle exactly one — best-of-n in a dozen lines:

var outs []*shepherd.RetainedOutput
for i := 0; i < n; i++ {
    child, _ := mgr.ForkIsolated(parent.ID(), fmt.Sprintf("try-%d", i), nil, worktreeFor(repo, i))
    // ... the attempt does its work in the worktree ...
    out, _ := mgr.Seal(ctx, child.ID())
    outs = append(outs, out) // out.Changes and out.Diff are frozen at seal time
}

if err := mgr.Settle(ctx, outs[winner].ID, shepherd.SettleSelect); err != nil { // parent unmoved: fast-forward
    // parent moved since the fork? three-way instead:
    //   disjoint paths  -> mgr.Settle(ctx, outs[winner].ID, shepherd.SettleApply)
    //   path overlap    -> ErrApplyConflict, output stays unconsumed
}
for _, out := range losers {
    _ = mgr.Settle(ctx, out.ID, shepherd.SettleDiscard)
}

Every settlement is consume-once (ErrOutputConsumed on a second settle); selected requires the parent to be exactly at the fork baseline, applied requires path-disjoint parent changes, released keeps the sealed state addressable without integrating it, and discarded tears the child down keeping the sealed state as a record. Rule engines join the same gate via Supervisor.ReviewMerge, which presents proposals to the rules as merge:proposed events.

Materialize recorded intents

Declarations are commitments: records that say what an agent intended to do. Materialization is the other half of that rhythm — applying recorded declarations through a typed substrate and appending the receipts as captures, so the trace describes both sides of the transition:

registry := substrate.NewSubstrateRegistry()
registry.Register(substrate.NewWorkspaceSubstrate(sandbox))  // or a KV, echo, custom substrate

receipt, err := materialize.Materialize(ctx, store,
    appendCtx.ToOperationContext(trace.OpMaterialize),
    materialize.MaterializationRequest{
        AppendIntentID:            "run:42:materialize",
        TargetTraceOwnerID:        "agent:main",
        TargetRecordIDs:           declarationIDs, // workspace.file.write.v1 records
        TargetThroughOwnerOrdinal: materialize.MaxOwnerOrdinal,
    },
    registry,
)
// receipt.Outcome, receipt.ProducedRecordIDs — applied captures, caused by
// their declarations, witness-stamped with the substrate that applied them.

The transition is idempotent once an intent has completed: a same-intent retry replays the stored receipt from the completed-intent ledger without redispatching the substrate, and a same-intent request with different content is a conflict, not a silent second write. Dispatch itself is at-least-once: in the window between the substrate call and the ledger write — including a crash there — the substrate runs again. Deterministic substrates (the reference Echo and KV substrates) converge to the identical receipt in that window; a non-deterministic one fails loudly as an intent conflict rather than double-writing. The substrate is selected by the declarations' own witnesses — the substrate_ref stamped when they were appended — never guessed from schemas.

WorkspaceSubstrate bridges the kernel's Sandbox interface, so recorded file-write and exec intents reach a real workspace (git or worktree) through the same declare→capture rhythm, gated on the backend's capabilities: the git backend honestly reports clean_failure instead of half-applying. Outcome honesty is the contract: clean_failure is claimed only when nothing was attempted — a failure after a sandbox call was made is reported as split_state even when no capture landed, because backends write files in chunks and the world may have changed without a trace. The reference KV substrate (substrate.NewKVSubstrate) keeps a world-side SQLite store separate from the trace, and the echo substrate is the test fixture.

Run tasks

Executions give runs identities: a create/complete/fail lifecycle folded from retained facts, parent/child relations, and a synchronous task facade over the store. Every id is derived exactly as the Python reference derives it — ExecutionIDFor and the full run sequence are pinned by Python-generated vectors — so a Go-written trace is readable by shepherd2 and vice versa.

reg := task.NewRegistry()
reg.Register("MyTask", func(c *task.TaskControl) (map[string]any, error) {
    c.Publish(ctx, "note", map[string]any{"msg": "working"})
    child, err := c.Spawn(ctx, "LeafTask", inputs, reg)
    if err != nil { return nil, err }
    exec, err := c.AwaitTerminal(ctx, child)
    return map[string]any{"child": exec.ExecutionID}, nil
})

run, err := task.StartTaskSync(ctx, store, reg, "MyTask", "run:fixed-id", inputs)
exec, err := run.Wait(ctx)      // terminal projection: succeeded / failed

// or async: StartTask returns a live handle; Wait blocks until terminal
run, err = task.StartTask(ctx, store, reg, "MyTask", "", inputs)

Spawns and adopts are recorded as parent-owned relation facts; abandoning a child writes an abandoned relation under the same relation id, so the effective-history fold drops it. The whole tree projects from one terminal frontier, and re-opening the store file re-projects it identically. The underlying projection (trace.ProjectEffectiveHistoryFromStore) lives in the kernel module.

Concepts

Scopes

A scope represents a branch of execution. Fork creates a child scope with its own trace owner; merge propagates causal links; discard abandons the branch. Lifecycle events are recorded in the trace.

A scope also carries an optional sandbox — its execution substrate. A scope created without one is pure-causal: it participates in the trace DAG but workspace operations return ErrNoSandbox. Fork inherits the parent's sandbox without owning it; ForkIsolated attaches a new sandbox that the child owns, so Discard/Merge/Halt destroy it.

Sandboxes

A Sandbox is the substrate a scope forks, snapshots, restores, diffs, and executes in. It is the seam that keeps the runtime independent of any particular isolation technology:

type Sandbox interface {
    Backend() string
    Capabilities() SandboxCapabilities
    Create(ctx context.Context, spec SandboxSpec) error
    Destroy(ctx context.Context) error
    Capture(ctx context.Context) (WorkspaceState, error)
    Apply(ctx context.Context, ws WorkspaceState) error
    Diff(ctx context.Context, ws WorkspaceState, maxLines int) (string, []string, error)
    Exec(ctx context.Context, req ExecRequest) (ExecResult, error)
    ReadFile(ctx context.Context, path string) ([]byte, error)
    WriteFile(ctx context.Context, path string, data []byte, perm fs.FileMode) error
}

Backends need not implement everything. Capabilities() reports what is available, and unimplemented operations return ErrUnsupported rather than pretending to succeed. Capture and Diff are non-mutating: the caller's git index is left as it was found.

Two backend modes ship, packaged on purpose:

Backend	Mode	Lifecycle	Isolated	Containment	Package
git	in-place (git.NewLocalGitSandbox)	No — validates only	No	uncontained	sandbox/git
git	worktree (git.NewWorktreeSandbox)	Yes — real worktrees	Yes	buffered	sandbox/git

A backend is a materializer, not necessarily an isolation boundary; the git backends have zero external dependencies and work on any platform.

Any type implementing the Sandbox interface is swappable in — the runtime never inspects a backend beyond the opaque WorkspaceState.Backend string. For config-driven selection, backends register factories and the host resolves a configured name:

// Config-driven construction: the git backend (sandbox/git) self-registers; a
// host-supplied backend registers itself the same way when its package is
// imported.
sb, err := sandbox.OpenSandbox("git", git.GitSandboxConfig{RepoPath: repoPath})

_, err = sandbox.OpenSandbox("podman", nil)    // -> ErrUnknownSandboxBackend, listing what IS registered
backends := sandbox.SandboxBackends()          // sorted names, for config diagnostics

// Direct constructors remain available:
inPlace := git.NewLocalGitSandbox(repoPath) // materializes on your repository. Destroy is a no-op.
worktree := git.NewWorktreeSandbox(repoPath, wt) // a detached git worktree this scope owns

err = sb.Create(ctx, sandbox.SandboxSpec{})    // OpenSandbox does NOT provision; Create is the lifecycle step
defer sb.Destroy(ctx)

ws, err := sb.Capture(ctx)
err = sb.Apply(ctx, ws)
diff, files, err := sb.Diff(ctx, ws, maxLines)

caps := sb.Capabilities()   // capability negotiation: Lifecycle/Exec/FileIO/Diff/Isolated/Containment

API

Scope Manager

mgr := shepherd.NewScopeManager(store)

scope, err := mgr.Create("owner-id", sb)                            // sb may be nil (pure-causal)
child, err := mgr.Fork("parent-scope-id", "child-owner-id", nil)    // inherits parent's sandbox
child, err := mgr.ForkIsolated("parent-scope-id", "child-owner-id", nil, sb)
err := parent.Merge(child)                                          // takes *Scope, not an ID
err := parent.Discard(child)
err := mgr.DestroyScopeSandbox(ctx, "scope-id")                     // release provisioned resources
scope, ok := mgr.Get("scope-id")
active := mgr.ActiveScopes()

cp, err := mgr.CreateCheckpoint(ctx, "scope-id", snapshot)          // single-use
snapshot, err := mgr.RestoreCheckpoint(ctx, cp.ID)
latest := mgr.LatestCheckpoint("scope-id")
mgr.PruneCheckpoints(ctx, "scope-id")                 // also deletes persisted blobs

// Merge gate (plan 04): propose, review, commit-or-discard
proposal, err := mgr.ProposeMerge(ctx, "child-scope-id", 2000)      // query; classifies create/modify/delete
err = mgr.CommitMerge(ctx, "child-scope-id", shepherd.DraftsOnlyReviewer("drafts"))
// deny -> *SupervisorDeniedError, child discarded; isolated children are refused (use Seal + Settle)

// Retained outputs: seal, then settle exactly once
out, err := mgr.Seal(ctx, "scope-id")                              // scope stays active
got, ok := mgr.Output(out.ID)
outs := mgr.OutputsForScope("scope-id")
err = mgr.Settle(ctx, out.ID, shepherd.SettleSelect)               // or SettleApply / SettleRelease / SettleDiscard

// Restart durability: rebuild the registry from the trace
mgr, err = shepherd.RecoverScopes(store, shepherd.RecoverOptions{ /* Resolver: re-adopt sandboxes */ })
pruned := mgr.PruneTerminal()                                      // unregister merged/discarded scopes

Scope workspace and checkpoint operations

scope := shepherd.NewScope(store, "owner-id").WithSandbox(sb, false)  // owns=false: do not Destroy

ws, err := scope.CaptureWorkspace(ctx)          // reusable state, no lifecycle
err = scope.ApplyWorkspace(ctx, ws)
diff, files, err := scope.DiffWorkspace(ctx, ws, 2000)

cp, err := scope.CreateCheckpoint(ctx, snapshot)
snapshot, err := scope.RestoreCheckpoint(ctx, cp)

Supervisor

A rule-based engine that watches the effect bus and emits interventions when rules match. Built-in rules guard against destructive operations, high error rates, and stuck detection.

sup := supervisor.NewSupervisor(mgr, bus)
sup.AddRule(supervisor.DestructiveToolRule())
sup.AddRule(supervisor.HighErrorRateRule(0.5, 10))
sup.AddRule(supervisor.StuckDetectionRule(3))
sup.Start("orchestrator")
defer sup.Close()

for iv := range sup.Interventions() { ... }

err := sup.Inject(scopeID, "guidance text")
err := sup.Halt(scopeID)

Materialization

registry := substrate.NewSubstrateRegistry()
err := registry.Register(substrate)               // Substrate interface: SubstrateRef,
                                                  // DeclarationSchemas, CaptureSchemas,
                                                  // Containment, Materialize
substrate, err := registry.Get(substrateRef)      // ErrUnknownSubstrate when missing

receipt, err := materialize.Materialize(ctx, store, opCtx, req, registry)
// opCtx.Operation must be trace.OpMaterialize
// req: AppendIntentID, TargetTraceOwnerID, TargetRecordIDs,
//      TargetThroughOwnerOrdinal (materialize.MaxOwnerOrdinal), CaptureTraceOwnerID
// receipt: Outcome (success | clean_failure | split_state), SubstrateRef,
//          TargetRecordIDs, ProducedRecordIDs, WorldSideAnchors

echo := substrate.NewEchoSubstrate("my.echo.v1", "example.write.v1")
kv, err := substrate.NewKVSubstrate("world.sqlite")   // kv.sqlite.local.v1
ws := substrate.NewWorkspaceSubstrate(sandbox)       // workspace.sandbox.v1

result, err := substrate.Materialize(ctx, records)   // substrates never append;
                                                     // dispatch owns the append

Trace schemas

The checkpoint and workspace schemas embed a backend-neutral revision and state_digest rather than git-specific SHAs, so a trace reads the same whether the workspace was a git tree, a worktree, or a container snapshot:

Schema	Mode	Payload
shepherd.checkpoint.created.v2	declaration	checkpoint_id, backend, revision, state_digest, has_snapshot
shepherd.checkpoint.restored.v2	capture	checkpoint_id, backend, revision, state_digest
shepherd.workspace.captured.v1	declaration	backend, revision, state_digest
shepherd.workspace.applied.v1	capture	backend, revision, state_digest

These are kernel extensions, not part of shepherd.kernel.abi.v0, so they do not affect the golden vectors. Trace writes are advisory: a failed append never fails the workspace operation, because the rollback guarantee does not depend on the audit record.

Constants

shepherd.SchemaScopeForked          // "shepherd.scope.forked.v1"
shepherd.SchemaScopeMerged          // "shepherd.scope.merged.v1"
shepherd.SchemaScopeDiscarded       // "shepherd.scope.discarded.v1"
shepherd.SchemaSupervisorInject     // "shepherd.supervisor.inject.v1"
shepherd.SchemaSupervisorHalt       // "shepherd.supervisor.halt.v1"
shepherd.SchemaCheckpointCreated    // "shepherd.checkpoint.created.v2"
shepherd.SchemaCheckpointRestored   // "shepherd.checkpoint.restored.v2"
shepherd.SchemaWorkspaceCaptured    // "shepherd.workspace.captured.v1"
shepherd.SchemaWorkspaceApplied     // "shepherd.workspace.applied.v1"

shepherd.CheckpointValid            // "valid"
shepherd.CheckpointUsed             // "used"
shepherd.CheckpointInvalid          // "invalid"

sandbox.ErrNoSandbox               // scope has no sandbox
sandbox.ErrUnsupported             // backend does not implement the capability

The trace.* constants (modes, trusted contexts, the ABI/canonical versions) live in the kernel module.

Build & test

go build ./...
go test ./...
go vet ./...
gofmt -l .            # must be empty

History

Extracted unchanged from github.com/buchenberg/shepherd-kernel-go (fresh history; the original history remains in that repository's log).

Documentation

Overview

Package shepherd is the runtime layer of the shepherd kernel: the scope tree an agent operates in, and the orchestration that composes the kernel's components around it.

A Scope is an owner's causal unit of work — a branch of activity with an append-only trace of its records, an optional sandboxed workspace, and a lifecycle (active, merged, discarded, halted). ScopeManager owns the tree: forking (inheriting or isolating the workspace), merging back through a reviewable proposal, discarding, checkpointing, and settling sealed outputs. RecoverScopes rebuilds the tree from the trace after a restart — the trace is the registry.

On top of the scope tree sit the merge gate (MergeProposal, MergeReviewer) and settlement (RetainedOutput, Scope.Seal, ScopeManager.Settle).

The kernel's components publish as their own packages, imported directly:

• github.com/buchenberg/shepherd-kernel-go/trace — the record model, canonical JSON, the SQLite trace store, and the effect bus
• github.com/buchenberg/shepherd-runtime-go/sandbox — the Sandbox interface and the backend registry
• github.com/buchenberg/shepherd-runtime-go/sandbox/git — the git backend (in-place and detached-worktree modes); self-registers as "git"
• github.com/buchenberg/shepherd-runtime-go/substrate — the Substrate contract and the echo/KV/workspace substrates
• github.com/buchenberg/shepherd-runtime-go/materialize — the materialize orchestrator
• github.com/buchenberg/shepherd-runtime-go/task — the task runtime: StartTaskSync/StartTask, Run, TaskControl, ChildHandle with spawn/ adopt/abandon relations
• github.com/buchenberg/shepherd-runtime-go/supervisor — the rules engine that observes the effect bus and can inject, fork, discard, halt, or deny

Compatibility: the record digests, canonical JSON, and the vector fixtures are frozen ABI surface shared with the Python reference; the runtime layer follows the module's semver. Pre-1.0, minor releases may break — see the CHANGELOG for the v0.11.0 import-path migration table.

Index

• Constants
• Variables
• type Checkpoint
• type CheckpointState
• type MergeProposal
• type MergeReviewer
  • func DestructivePathReviewer(patterns []string) MergeReviewer
  • func DraftsOnlyReviewer(prefix string) MergeReviewer
• type OutputState
• type ProposedChange
• type RecoverOptions
• type RecoveryStats
• type RetainedOutput
  • func (o *RetainedOutput) Action() SettlementAction
  • func (o *RetainedOutput) Baseline() trace.WorkspaceState
  • func (o *RetainedOutput) Changes() []ProposedChange
  • func (o *RetainedOutput) Diff() string
  • func (o *RetainedOutput) ID() string
  • func (o *RetainedOutput) ScopeID() string
  • func (o *RetainedOutput) SealedAt() time.Time
  • func (o *RetainedOutput) Settle(ctx context.Context, action SettlementAction) error
  • func (o *RetainedOutput) SettledAt() time.Time
  • func (o *RetainedOutput) State() OutputState
  • func (o *RetainedOutput) Workspace() trace.WorkspaceState
• type Scope
  • func NewScope(store *trace.SQLiteTraceStore, ownerID string) *Scope
  • func (s *Scope) AppendFact(ctx context.Context, schemaRef, kindLabel string, payload map[string]any) (trace.AppendReceipt, error)
  • func (s *Scope) ApplyWorkspace(ctx context.Context, ws trace.WorkspaceState) error
  • func (s *Scope) Baseline() (trace.WorkspaceState, bool)
  • func (s *Scope) CaptureWorkspace(ctx context.Context) (trace.WorkspaceState, error)
  • func (s *Scope) Children() []*Scope
  • func (s *Scope) CreateCheckpoint(ctx context.Context, snapshot []byte) (*Checkpoint, error)
  • func (s *Scope) DiffWorkspace(ctx context.Context, ws trace.WorkspaceState, maxLines int) (string, []string, error)
  • func (s *Scope) Discard(child *Scope) error
  • func (s *Scope) Fork(childOwnerID string, snapshot any) (*Scope, error)
  • func (s *Scope) ForkPoint() string
  • func (s *Scope) Halt() error
  • func (s *Scope) ID() string
  • func (s *Scope) Inject(guidance string) error
  • func (s *Scope) Merge(child *Scope) error
  • func (s *Scope) OwnerID() string
  • func (s *Scope) Parent() *Scope
  • func (s *Scope) RestoreCheckpoint(ctx context.Context, cp *Checkpoint) ([]byte, error)
  • func (s *Scope) Sandbox() sandbox.Sandbox
  • func (s *Scope) Seal(ctx context.Context) (*RetainedOutput, error)
  • func (s *Scope) Snapshot() any
  • func (s *Scope) State() ScopeState
  • func (s *Scope) WithSandbox(sb sandbox.Sandbox, owns bool) *Scope
• type ScopeManager
  • func NewScopeManager(store *trace.SQLiteTraceStore) *ScopeManager
  • func RecoverScopes(store *trace.SQLiteTraceStore, opts RecoverOptions) (*ScopeManager, error)
  • func (m *ScopeManager) ActiveScopes() []*Scope
  • func (m *ScopeManager) AllScopes() []*Scope
  • func (m *ScopeManager) CommitMerge(ctx context.Context, childID string, review MergeReviewer) error
  • func (m *ScopeManager) Create(ownerID string, sb sandbox.Sandbox) (*Scope, error)
  • func (m *ScopeManager) CreateCheckpoint(ctx context.Context, scopeID string, snapshot []byte) (*Checkpoint, error)
  • func (m *ScopeManager) DestroyScopeSandbox(ctx context.Context, scopeID string) error
  • func (m *ScopeManager) Discard(childID string) error
  • func (m *ScopeManager) Fork(parentID, childOwnerID string, snapshot any) (*Scope, error)
  • func (m *ScopeManager) ForkIsolated(parentID, childOwnerID string, snapshot any, sb sandbox.Sandbox) (*Scope, error)
  • func (m *ScopeManager) Get(id string) (*Scope, bool)
  • func (m *ScopeManager) LatestCheckpoint(scopeID string) *Checkpoint
  • func (m *ScopeManager) Merge(childID string) error
  • func (m *ScopeManager) Output(id string) (*RetainedOutput, bool)
  • func (m *ScopeManager) OutputsForScope(scopeID string) []*RetainedOutput
  • func (m *ScopeManager) ProposeMerge(ctx context.Context, childID string, maxDiffLines int) (*MergeProposal, error)
  • func (m *ScopeManager) PruneCheckpoints(ctx context.Context, scopeID string)
  • func (m *ScopeManager) PruneTerminal() int
  • func (m *ScopeManager) RestoreCheckpoint(ctx context.Context, checkpointID string) ([]byte, error)
  • func (m *ScopeManager) Seal(ctx context.Context, scopeID string) (*RetainedOutput, error)
  • func (m *ScopeManager) Settle(ctx context.Context, outputID string, action SettlementAction) error
• type ScopeState
• type SettlementAction
• type SupervisorDeniedError
  • func (e *SupervisorDeniedError) Error() string

Constants

const (
	SchemaCheckpointCreated  = "shepherd.checkpoint.created.v2"
	SchemaCheckpointRestored = "shepherd.checkpoint.restored.v2"
)

Schema refs for checkpoint trace recording. Version 2 carries backend-neutral fields (the v1 payload embedded git-specific stash_sha and head_sha keys).

const (
	// SchemaMergeProposed is recorded on the child's owner path when a merge
	// is proposed for review, before the reviewer runs.
	SchemaMergeProposed = "shepherd.merge.proposed.v1"
	// SchemaSupervisorDecision records the review outcome on the parent's
	// owner path, citing the proposed record as its causal parent.
	SchemaSupervisorDecision = "shepherd.supervisor.decision.v1"
)

Merge gate schema refs for trace recording (plan 04 §1).

const (
	ChangeCreate = "create"
	ChangeModify = "modify"
	ChangeDelete = "delete"
)

Proposed change kinds.

const (
	SchemaScopeForked    = "shepherd.scope.forked.v1"
	SchemaScopeMerged    = "shepherd.scope.merged.v1"
	SchemaScopeDiscarded = "shepherd.scope.discarded.v1"
	// SchemaSupervisorInject is recorded by Scope.Inject — the supervisor
	// vocabulary the scope itself writes; the supervisor component reads it.
	SchemaSupervisorInject = "shepherd.supervisor.inject.v1"
	// SchemaSupervisorHalt is recorded by Scope.Halt and scanned by recovery
	// as a terminal state.
	SchemaSupervisorHalt = "shepherd.supervisor.halt.v1"
)

Scope schema refs for trace recording.

const (
	// SchemaRunOutputSealed is recorded on the sealing scope's owner path
	// when an output is sealed. Declaration: the sealing is stated intent
	// that the settlement records then observe.
	SchemaRunOutputSealed = "shepherd.run_output.sealed.v1"
	// SchemaRunOutputSettled is recorded on the same owner path when the
	// output is settled, citing the sealed record.
	SchemaRunOutputSettled = "shepherd.run_output.settled.v1"
)

Settlement schema refs for trace recording (plan 04 §2). These are Go-kernel extensions like the scope/checkpoint schemas — documented as such, not pretending to be shepherd2.run_output.v1.

const (
	SchemaWorkspaceCaptured = "shepherd.workspace.captured.v1"
	SchemaWorkspaceApplied  = "shepherd.workspace.applied.v1"
)

Workspace schema refs for trace recording. These replace the git-specific shepherd.tree.* schemas: the payload is backend-neutral and carries a state digest rather than stash/head SHAs.

const DefaultProposalDiffLines = 1000

DefaultProposalDiffLines bounds the diff a CommitMerge proposal carries when the caller has not chosen a bound. It is deliberately generous: the reviewer sees the change, and the trace only stores the digest.

Variables

var ErrApplyConflict = errors.New("shepherd: apply conflict")

ErrApplyConflict reports a SettleApply whose changes overlap paths the parent changed since the fork baseline. The output is NOT consumed by a conflict — apply failure never consumes.

var ErrOutputConsumed = errors.New("shepherd: retained output already settled")

ErrOutputConsumed reports a second settlement of an already-settled output. Settlement is consume-once: every action is terminal.

var ErrSettleEffectLanded = errors.New("shepherd: settle effect landed")

ErrSettleEffectLanded reports a settle whose physical effect landed on the parent's workspace but whose causal merge failed to complete. The workspace change happens before the merge, so a blind retry would re-apply an effect that already happened; callers detecting this sentinel should treat the work as standing on the parent and end the output's lifecycle rather than retry. The failure may be at any merge stage — validation, the trace append, or the post-record sandbox teardown — and the work is landed in all of them. The lifecycle ending differs by stage: SettleRelease always works and keeps the landed work; SettleDiscard additionally destroys the child's sandbox, which requires the child still active — true when the merge failed before its record (validation, append), false after a teardown failure (the child is already merged; discard refuses it). The settle's error message names the ending its child's state allows. The output stays unconsumed — the kernel's own partial-settlement contract. Wrapped in the settle error alongside the merge failure.

Types

type Checkpoint

type Checkpoint struct {

	// ID is a unique checkpoint identifier, derived from a monotonic sequence
	// rather than wall-clock time so concurrent checkpoints cannot collide.
	ID string
	// Seq is the process-wide monotonic creation order. It is the authoritative
	// ordering key (CreatedAt is informational and can tie).
	Seq uint64
	// ScopeID is the scope this checkpoint belongs to.
	ScopeID string
	// Workspace is the captured backend-neutral workspace state.
	Workspace trace.WorkspaceState
	// Snapshot is the opaque caller-provided state (typically conversation
	// history). Released after a successful restore.
	Snapshot []byte
	// CreatedAt is when the checkpoint was taken.
	CreatedAt time.Time
	// State tracks usability.
	State CheckpointState
	// contains filtered or unexported fields
}

Checkpoint is a scope-owned, single-use capture of workspace state plus an opaque caller-provided snapshot.

The workspace state is backend-neutral: a git stash/HEAD pair, a container snapshot key, or anything else the scope's Sandbox understands. The snapshot is opaque to the kernel — typically JSON-marshaled conversation history from the agent loop — and the caller knows its concrete type.

Checkpoints are single-use: RestoreCheckpoint marks them as used and releases the snapshot. Restore is guarded by mu so concurrent restores of the same checkpoint cannot both claim the valid state.

type CheckpointState

type CheckpointState string

CheckpointState tracks whether a checkpoint is usable.

const (
	// CheckpointValid means the checkpoint can be restored.
	CheckpointValid CheckpointState = "valid"
	// CheckpointUsed means the checkpoint has already been restored.
	CheckpointUsed CheckpointState = "used"
	// CheckpointInvalid means a restore failed and the checkpoint is unusable.
	CheckpointInvalid CheckpointState = "invalid"
)

type MergeProposal

type MergeProposal struct {
	// ChildScopeID identifies the scope whose work is being merged.
	ChildScopeID string
	// BaselineDigest is the canonical digest of the parent's workspace state
	// captured at fork time — the state the diff is against.
	BaselineDigest string
	// Changes lists every file the merge would introduce, in diff order.
	Changes []ProposedChange
	// Diff is the full unified diff output (truncated to the proposal's
	// line limit).
	Diff string
}

MergeProposal is what a reviewer sees: the child scope, the baseline the changes are against, and the changes themselves.

type MergeReviewer

type MergeReviewer func(proposal MergeProposal) error

MergeReviewer decides whether a proposed merge may proceed. Approve by returning nil; deny by returning an error whose text becomes the denial reason recorded in the trace and carried on SupervisorDeniedError.

func DestructivePathReviewer

func DestructivePathReviewer(patterns []string) MergeReviewer

DestructivePathReviewer denies any change touching a path that matches one of the given patterns. A pattern is matched against the workspace-relative path with path.Match semantics ("*.pem", "secrets/*.key"); a pattern ending in "/" matches everything beneath that directory ("secrets/"). An empty pattern list denies nothing — an explicit list is an explicit contract, and a nil list should not silently become deny-everything.

func DraftsOnlyReviewer

func DraftsOnlyReviewer(prefix string) MergeReviewer

DraftsOnlyReviewer denies any change outside prefix — the Go analogue of the reference drafts_only_supervisor. prefix is a directory: "out" matches "out" and "out/anything", but not "outline.txt". An empty prefix approves everything (useful as a chain element, not as a gate).

type OutputState

type OutputState string

OutputState is the consume-once lifecycle state of a retained output.

const (
	OutputUnconsumed OutputState = "unconsumed"
	OutputSelected   OutputState = "selected"
	OutputApplied    OutputState = "applied"
	OutputReleased   OutputState = "released"
	OutputDiscarded  OutputState = "discarded"
	// OutputInvalid marks an output whose sealed state was discovered
	// unusable. No core path sets it today; it exists so a caller (or a
	// future recovery pass, plan 05 §4) can mark a state that must never be
	// settled without pretending it was consumed.
	OutputInvalid OutputState = "invalid"
)

type ProposedChange

type ProposedChange struct {
	// Path is the workspace-relative path of the changed file.
	Path string
	// Kind is "create", "modify", or "delete".
	Kind string
	// Diff is the unified diff hunk for a modify or delete; the added
	// content (already bounded by the proposal's line limit) for a create.
	Diff string
}

ProposedChange is one workspace change a merge would introduce.

type RecoverOptions

type RecoverOptions struct {
	// Sandbox re-adoption. Recovery calls it for every recovered root scope
	// and every isolated child, with the backend and durable address the
	// trace recorded (both empty for roots, which have no record; the path
	// empty for isolated children whose backend does not implement the
	// Addressed capability). Return (nil, nil) to recover the
	// scope without a sandbox — deliberate non-adoption, not an error: the
	// host uses it to orphan-clean worktrees it does not want back. An
	// error fails the recovery: half-adopted sandboxes are worse than none.
	//
	// Shared-sandbox children are never resolved: they inherit the parent's
	// re-adopted sandbox, exactly as they inherited it at fork time.
	//
	// The path is a normalized absolute address recorded at fork time
	// (plan 06, U2); a recorded non-absolute path fails the recovery
	// before the resolver sees it.
	Resolver func(scopeID, backend, path string) (sandbox.Sandbox, error)

	// Stats, when non-nil, receives the recovery's scan measurements:
	// records scanned per lifecycle schema and the total scan duration
	// (the scan phase — fork records, terminal states, checkpoint blobs,
	// sealed outputs — not the sandbox re-adoption). The recovery scan
	// is O(history) with no bound; hosts that run recovery at startup
	// hand this to their log so the cost is visible long before it
	// matters (plan 07, H5). The counts are written as their scans
	// complete — a recovery that fails partway reports what it had
	// counted, so an all-zero Stats after an error means the failure
	// preceded the first scan. ScanDuration is written only when the
	// scan phase completes.
	Stats *RecoveryStats
}

RecoverOptions configures scope recovery.

type RecoveryStats

type RecoveryStats struct {
	ForkRecords     int
	TerminalRecords int
	CheckpointBlobs int
	SealedOutputs   int
	// ScanDuration is the wall time of the whole scan phase.
	ScanDuration time.Duration
}

RecoveryStats measures one recovery's scan phase. Every counter reports the raw records the scans read — the O(history) cost — not the derived state: terminal records are counted per record even when several fold into one owner's state, and corrupt checkpoint rows are counted even though they recover nothing.

type RetainedOutput

type RetainedOutput struct {
	// contains filtered or unexported fields
}

RetainedOutput is a sealed child result: the child's captured workspace frozen at seal time, settle-able exactly once. It outlives the scope that sealed it — sealing does not merge — so a supervisor can compare sealed outputs and choose, the fork-and-choose rhythm.

All fields are unexported behind lock-taking getters: Settle mutates the lifecycle fields under mu, and exported fields would let any caller race that write.

func (*RetainedOutput) Action

func (o *RetainedOutput) Action() SettlementAction

Action returns the settlement that consumed the output; the zero value until settled.

func (*RetainedOutput) Baseline

func (o *RetainedOutput) Baseline() trace.WorkspaceState

Baseline returns the parent's fork-time state the sealed changes are against.

func (*RetainedOutput) Changes

func (o *RetainedOutput) Changes() []ProposedChange

Changes returns the sealed output's proposed changes, frozen at seal time.

func (*RetainedOutput) Diff

func (o *RetainedOutput) Diff() string

Diff returns the sealed output's full diff against the baseline.

func (*RetainedOutput) ID

func (o *RetainedOutput) ID() string

ID returns the output's unique identifier ("out:<owner>:<seq>").

func (*RetainedOutput) ScopeID

func (o *RetainedOutput) ScopeID() string

ScopeID returns the scope that sealed this output.

func (*RetainedOutput) SealedAt

func (o *RetainedOutput) SealedAt() time.Time

SealedAt returns when the output was sealed.

func (*RetainedOutput) Settle

func (o *RetainedOutput) Settle(ctx context.Context, action SettlementAction) error

Settle consumes the output with the given action. It requires the output to be registered with a ScopeManager (ScopeManager.Seal registers; Scope.Seal alone does not) — use the manager's Settle directly otherwise.

func (*RetainedOutput) SettledAt

func (o *RetainedOutput) SettledAt() time.Time

SettledAt returns when the output was settled; the zero time until then.

func (*RetainedOutput) State

func (o *RetainedOutput) State() OutputState

State returns the consume-once lifecycle state.

func (*RetainedOutput) Workspace

func (o *RetainedOutput) Workspace() trace.WorkspaceState

Workspace returns the sealed sandbox state.

type Scope

type Scope struct {
	// contains filtered or unexported fields
}

Scope represents a branch of agent execution. It wraps a trace owner and supports fork/merge/discard over the causal graph.

Unlike the Python Shepherd, this scope does NOT carry filesystem state or context bindings — yaah sub-agents are independent LLM loops, not shared-fs containers. "Fork" means "branch the causal graph and start a new execution from this point."

The scope records lifecycle events (forked, merged, discarded) in the trace store so the supervisor can inspect the full branching history. If the store has a bus attached (via WithBus), lifecycle events are also published to the bus automatically — no separate bus field needed.

Lock ordering: Merge and Discard always lock parent before child. Never lock child then parent — that deadlocks. If a future API needs to lock from the child side, it must acquire parent first or use a try-lock pattern.

Snapshots

A scope can carry a snapshot — an opaque value captured at fork time that represents the execution state (typically conversation history). Snapshots enable:

• Retry from checkpoint (fork from snapshot, inject guidance, retry suffix)
• Parallel strategy exploration (N forks from same snapshot, different prompts)
• Guard-and-retry (halt dangerous branch, fork safe branch from snapshot)
• Counterfactual replay (fork from mid-point, try different continuation)

The snapshot is `any` so callers can store whatever they need ([]byte, []types.Message, etc.) without the scope package importing their types.

func NewScope

func NewScope(store *trace.SQLiteTraceStore, ownerID string) *Scope

NewScope creates a root scope (no parent) for the given trace owner. If the store has a bus attached (via WithBus), lifecycle events will be published automatically.

func (*Scope) AppendFact

func (s *Scope) AppendFact(ctx context.Context, schemaRef, kindLabel string, payload map[string]any) (trace.AppendReceipt, error)

AppendFact appends a capture record on this scope's owner path and returns the append receipt. It is the runtime's general "record something on this scope" seam — the run wrap uses it to persist its execution record (authority evidence and outcome).

schemaRef is free-form (the kernel validates only that it is non-empty), so a caller can introduce a runtime-level record without a kernel ABI change.

func (*Scope) ApplyWorkspace

func (s *Scope) ApplyWorkspace(ctx context.Context, ws trace.WorkspaceState) error

ApplyWorkspace resets the scope's workspace to a previously captured state.

The state is not consumed — the same WorkspaceState can be applied repeatedly by backends whose workspace state is reusable (the git backend). A backend that provisions fresh resources on Apply may make it single-use; see Sandbox.Apply.

Records a "workspace.applied" capture in the scope's trace (advisory).

func (*Scope) Baseline

func (s *Scope) Baseline() (trace.WorkspaceState, bool)

Baseline returns the workspace state captured from the parent's sandbox at fork time, and whether one exists. It is the diff base for merge review and the fast-forward reference for settlement's select verb. Pure-causal forks (parent without a sandbox) report false; children without a baseline cannot propose merges or seal outputs.

func (*Scope) CaptureWorkspace

func (s *Scope) CaptureWorkspace(ctx context.Context) (trace.WorkspaceState, error)

CaptureWorkspace records the scope's current workspace state as a reusable WorkspaceState.

Unlike CreateCheckpoint, this has no lifecycle: it consumes nothing, records no checkpoint, and the returned state stays valid indefinitely. This is what fork-and-choose needs — snapshot the fork point, run variant A, re-apply the fork point, run variant B, then apply the winner.

Records a "workspace.captured" declaration in the scope's trace (advisory).

func (*Scope) Children

func (s *Scope) Children() []*Scope

Children returns a copy of the child scopes.

func (*Scope) CreateCheckpoint

func (s *Scope) CreateCheckpoint(ctx context.Context, snapshot []byte) (*Checkpoint, error)

CreateCheckpoint captures the scope's workspace state plus the caller's snapshot.

The workspace capture is delegated to the scope's Sandbox, so this works unchanged for an in-place git tree, a detached worktree, or a future container backend. A scope created without a sandbox returns sandbox.ErrNoSandbox.

The snapshot is persisted durably into the trace database's checkpoint_snapshots table (bounded by the store's checkpoint blob limit; an oversized snapshot fails the checkpoint rather than silently ballooning the file), which is what makes a checkpoint restorable after a process restart via RecoverScopes. The "checkpoint.created" trace record remains advisory.

func (*Scope) DiffWorkspace

func (s *Scope) DiffWorkspace(ctx context.Context, ws trace.WorkspaceState, maxLines int) (string, []string, error)

DiffWorkspace returns the unified diff and changed file paths between ws and the scope's current workspace. maxLines <= 0 means unbounded.

This is a query: it records no trace event and consumes nothing.

func (*Scope) Discard

func (s *Scope) Discard(child *Scope) error

Discard abandons the child scope's effects. Records a "scope.discarded" capture in the parent's trace. The child's records remain in the store (append-only) but are semantically abandoned.

If the child owns its sandbox, the sandbox is destroyed, which is what makes discard a physical rollback for an isolating backend.

func (*Scope) Fork

func (s *Scope) Fork(childOwnerID string, snapshot any) (*Scope, error)

Fork creates a child scope branched from the current scope's state. The child gets a new trace owner ID and inherits the parent's causal context — its first record cites the parent's head as a causal parent.

The snapshot parameter captures the execution state at fork time (typically the conversation history). It's stored opaquely — the scope doesn't inspect it. Pass nil if no snapshot is needed.

When the parent has a sandbox, its workspace state is captured as the fork baseline (see Baseline). A "scope.forked" declaration is recorded in the parent's trace.

func (*Scope) ForkPoint

func (s *Scope) ForkPoint() string

ForkPoint returns the record ID at which this scope was forked, or empty string for root scopes.

func (*Scope) Halt

func (s *Scope) Halt() error

Halt forces this scope into the discarded state and records a supervisor halt event in the trace. Unlike Discard, Halt operates on the scope itself (not a child) and is used by the supervisor to force-stop a sub-agent.

Halt is idempotent — calling it on an already-discarded scope returns nil.

If the scope owns its sandbox, the sandbox is destroyed so a force-stopped sub-agent does not leak its worktree.

func (*Scope) ID

func (s *Scope) ID() string

ID returns the scope's unique identifier.

func (*Scope) Inject

func (s *Scope) Inject(guidance string) error

Inject records a supervisor inject action in this scope's trace. The guidance text is recorded as a declaration that the sub-agent can see in its next turn.

func (*Scope) Merge

func (s *Scope) Merge(child *Scope) error

Merge propagates the child's effects into this (parent) scope. Records a "scope.merged" capture in the parent's trace, citing the child's fork point as a causal parent. Marks the child as merged.

Merge is causal only: it never rewrites the workspace. Physical propagation is an explicit ApplyWorkspace by the caller, so merge semantics stay stable across backends. If the child owns its sandbox, it is destroyed after the lifecycle event is recorded.

func (*Scope) OwnerID

func (s *Scope) OwnerID() string

OwnerID returns the trace owner ID this scope writes to.

func (*Scope) Parent

func (s *Scope) Parent() *Scope

Parent returns the parent scope, or nil for root scopes.

func (*Scope) RestoreCheckpoint

func (s *Scope) RestoreCheckpoint(ctx context.Context, cp *Checkpoint) ([]byte, error)

RestoreCheckpoint reverts the workspace to the checkpointed state and returns the stored snapshot for the caller to deserialize and restore into its agent loop's context manager.

The workspace restore is delegated to the scope's Sandbox. The checkpoint's snapshot field is released after a successful restore, and the checkpoint is marked used — calling Restore twice returns an error.

The single-use transition is race-safe: cp.mu is held for the entire validate → apply → mark-used sequence, so concurrent restore attempts cannot both claim the valid state.

Records a "checkpoint.restored" capture in the scope's trace (advisory).

func (*Scope) Sandbox

func (s *Scope) Sandbox() sandbox.Sandbox

Sandbox returns the scope's execution substrate, or nil for a pure-causal scope. Workspace and checkpoint operations on a nil sandbox return sandbox.ErrNoSandbox.

func (*Scope) Seal

func (s *Scope) Seal(ctx context.Context) (*RetainedOutput, error)

Seal captures the scope's workspace as a retained output and freezes its changes against the fork baseline. The scope stays active — sealing does not merge, and the output outlives the scope by design.

The scope must have a sandbox and a fork baseline (a parent with a sandbox at fork time); a scope without a baseline has nothing to diff against and sealing it would freeze changes with no base to settle them onto.

Records shepherd.run_output.sealed.v1 on the scope's owner path. The append is hard, not advisory: the sealed record is the durable half of the output (plan 05 §4 rebuilds metadata from it), so a trace that cannot carry it must not hand out an output whose settlement would be unrecordable.

func (*Scope) Snapshot

func (s *Scope) Snapshot() any

Snapshot returns the execution state captured at fork time. For root scopes this is nil. For forked scopes, it's the value passed to Fork.

The caller knows the concrete type — typically []types.Message for yaah conversation history, or []byte for serialized state.

func (*Scope) State

func (s *Scope) State() ScopeState

State returns the current lifecycle state.

func (*Scope) WithSandbox

func (s *Scope) WithSandbox(sb sandbox.Sandbox, owns bool) *Scope

WithSandbox attaches an execution substrate to the scope and reports whether the scope owns its lifecycle.

When owns is true, Merge, Discard, and Halt call sandbox.Sandbox.Destroy. When false the sandbox outlives the scope, which is correct for an in-place materializer over the user's repository (Destroy is a no-op there) and for a sandbox inherited from a parent scope.

Returns the scope so construction can be chained.

type ScopeManager

type ScopeManager struct {
	// contains filtered or unexported fields
}

ScopeManager tracks active scopes and their lifecycle. It provides a centralized registry for creating, looking up, forking, merging, discarding scopes, and managing checkpoints.

The manager is safe for concurrent use.

func NewScopeManager

func NewScopeManager(store *trace.SQLiteTraceStore) *ScopeManager

NewScopeManager creates a scope manager backed by the given store. If the store has a bus attached (via WithBus), scope lifecycle events will be published automatically.

func RecoverScopes

func RecoverScopes(store *trace.SQLiteTraceStore, opts RecoverOptions) (*ScopeManager, error)

RecoverScopes rebuilds a ScopeManager from the durable trace: the scope tree and terminal states from lifecycle records, the checkpoint registry from the persisted blobs (with staleness detection against the re-adopted sandboxes), and retained outputs from sealed/settled records.

What is and is not recoverable:

• Every scope that appears in a fork record — as parent or child — is recovered, with its lifecycle state (merged/discarded children and halted scopes included: terminal states are data too).
• A root scope that never forked left no record and is not recovered; the host re-creates it. RecoverScopes returns the recovered roots so the host can decide whether they are new or forgotten.
• Checkpoints recover from the checkpoint_snapshots table. A recovered checkpoint whose recorded workspace revision no longer matches a fresh capture of the re-adopted sandbox is marked CheckpointInvalid — never silently restorable. Revision, not digest: the revision is the timeline position, and that is what staleness means — a checkpoint recorded at a HEAD the workspace has left is stale even if its content would still digest equal. Sandboxes that cannot be captured at recovery (no sandbox resolved, capture error) leave their checkpoints Valid — restore re-validates against the live workspace anyway.
• Retained outputs recover settle-able: the sealed payload carries the full sealed and baseline states and the change list. A recovered output's Diff is empty (the diff body was never recorded) and its SealedAt/SettledAt are zero times — the records carry no timestamps.

Sandboxes are never invented: the resolver decides what is re-adopted, and a recovery without a resolver rebuilds the causal tree only.

func (*ScopeManager) ActiveScopes

func (m *ScopeManager) ActiveScopes() []*Scope

ActiveScopes returns all scopes in the Active state.

func (*ScopeManager) AllScopes

func (m *ScopeManager) AllScopes() []*Scope

AllScopes returns all registered scopes regardless of state.

func (*ScopeManager) CommitMerge

func (m *ScopeManager) CommitMerge(ctx context.Context, childID string, review MergeReviewer) error

CommitMerge runs the check-at-commit gate: propose, record, review, then merge or discard.

The sequence is:

1. ProposeMerge builds the proposal from the child's current workspace diffed against the fork baseline.
2. A shepherd.merge.proposed.v1 capture is recorded on the child's owner path (change paths, kinds, count, and the diff digest — never the diff body).
3. review(proposal) decides.
4. Approval records shepherd.supervisor.decision.v1 {approved: true} on the parent's path and merges the child (the plain, ungated Merge). Denial records the decision with the reason, discards the child — never merges — and returns *SupervisorDeniedError.

Merge is causal-only: it never rewrites the parent's workspace. CommitMerge is therefore for children that share the parent's sandbox — their work is already in the parent's tree, and a denial's rollback is the caller's checkpoint restore. A child with an isolated sandbox (a worktree) is refused with an error pointing at Seal + SettleSelect: merging it would causally record an approval and then destroy the worktree, losing the work the reviewer just approved. Settlement owns physical propagation for isolated children.

A nil reviewer is rejected: skipping review is what plain Merge is for, and an accidental nil must not become an accidental approve.

func (*ScopeManager) Create

func (m *ScopeManager) Create(ownerID string, sb sandbox.Sandbox) (*Scope, error)

Create registers a new root scope for the given trace owner and attaches sb as its execution substrate. Pass a nil sandbox for a pure-causal scope, whose workspace and checkpoint operations will return sandbox.ErrNoSandbox.

Returns an error if a scope with this ID already exists.

func (*ScopeManager) CreateCheckpoint

func (m *ScopeManager) CreateCheckpoint(ctx context.Context, scopeID string, snapshot []byte) (*Checkpoint, error)

CreateCheckpoint creates a checkpoint for a scope and registers it. The checkpoint captures the scope's workspace state via its sandbox, plus the opaque caller-provided snapshot (typically serialized conversation history).

The manager lock is NOT held during the workspace capture — resolving the scope takes a read lock, the (potentially slow) capture runs outside it, and only the final map registration takes the write lock.

func (*ScopeManager) DestroyScopeSandbox

func (m *ScopeManager) DestroyScopeSandbox(ctx context.Context, scopeID string) error

DestroyScopeSandbox tears down a scope's owned sandbox without changing the scope's lifecycle state. Callers use this on the abort path to release worktrees and other provisioned resources when a session ends without a merge or discard.

It is a no-op for scopes with no sandbox or one they do not own, so an in-place materializer over the user's repository is never touched.

func (*ScopeManager) Discard

func (m *ScopeManager) Discard(childID string) error

Discard abandons a child scope's effects.

func (*ScopeManager) Fork

func (m *ScopeManager) Fork(parentID, childOwnerID string, snapshot any) (*Scope, error)

Fork creates a child scope branched from the parent. The child is automatically registered with the manager. The snapshot parameter captures execution state at fork time (pass nil if not needed).

The fork baseline (the parent's workspace state) is captured BEFORE the manager's write lock is taken: a capture is a backend subprocess bounded only by its own timeout (git: up to 30s), and the registry must not block every other operation on it. The parent is re-validated inside forkWithBaseline, so a parent discarded during the capture still fails the fork.

Returns an error if the parent doesn't exist or a scope with the child owner ID already exists.

func (*ScopeManager) ForkIsolated

func (m *ScopeManager) ForkIsolated(parentID, childOwnerID string, snapshot any, sb sandbox.Sandbox) (*Scope, error)

ForkIsolated creates a child scope with its own execution substrate instead of inheriting the parent's. The child owns sb, so Discard, Merge, and Halt destroy it — this is what makes a discard a physical rollback for an isolating backend such as a git worktree. The isolation is recorded in the fork payload, which is what recovery uses to decide that a re-adopted sandbox belongs to this child alone.

Returns an error if the parent doesn't exist or a scope with the child owner ID already exists.

func (*ScopeManager) Get

func (m *ScopeManager) Get(id string) (*Scope, bool)

Get returns a scope by its ID, or false if not found.

func (*ScopeManager) LatestCheckpoint

func (m *ScopeManager) LatestCheckpoint(scopeID string) *Checkpoint

LatestCheckpoint returns the most recent valid checkpoint for a scope, or nil if none exists. Only checkpoints still in the Valid state are considered — a used or invalid checkpoint is not a candidate for restore.

Ordering is by the monotonic Seq (creation order), not CreatedAt, which can tie for concurrent checkpoints and would leave selection ambiguous.

func (*ScopeManager) Merge

func (m *ScopeManager) Merge(childID string) error

Merge propagates a child scope's effects into its parent.

func (*ScopeManager) Output

func (m *ScopeManager) Output(id string) (*RetainedOutput, bool)

Output returns a registered retained output by ID, or false if not found.

func (*ScopeManager) OutputsForScope

func (m *ScopeManager) OutputsForScope(scopeID string) []*RetainedOutput

OutputsForScope returns the retained outputs sealed by a scope, oldest first (by seal sequence).

func (*ScopeManager) ProposeMerge

func (m *ScopeManager) ProposeMerge(ctx context.Context, childID string, maxDiffLines int) (*MergeProposal, error)

ProposeMerge diffs the child's current workspace against the baseline captured at fork time and returns the proposal a reviewer would see, without running any review or changing any scope state. maxDiffLines <= 0 means unbounded.

The child must be active, have a sandbox with the Diff capability, and have been forked from a parent with a sandbox (the fork baseline). ProposeMerge is a query: it records no trace event and consumes nothing.

func (*ScopeManager) PruneCheckpoints

func (m *ScopeManager) PruneCheckpoints(ctx context.Context, scopeID string)

PruneCheckpoints removes all checkpoints belonging to a scope and deletes their persisted blobs. Call this when a scope is discarded or merged so consumed checkpoint metadata (and any still-retained snapshot) is not held for the process lifetime — or in the database — after the scope is gone. RestoreCheckpoint already releases the snapshot on use; this handles checkpoints that were never restored.

func (*ScopeManager) PruneTerminal

func (m *ScopeManager) PruneTerminal() int

PruneTerminal removes merged and discarded scopes from the registry (plan 05 §4). Terminal scopes stay lookupable until pruned — their state is settled data a supervisor may still read — but a long-lived host that accumulates them grows the map without bound.

Pruning a scope does not touch its checkpoints or retained outputs: those have their own registries and lifecycles (PruneCheckpoints, Settle). The trace records survive either way. Returns the number of scopes pruned.

func (*ScopeManager) RestoreCheckpoint

func (m *ScopeManager) RestoreCheckpoint(ctx context.Context, checkpointID string) ([]byte, error)

RestoreCheckpoint restores a registered checkpoint and returns the stored conversation snapshot.

func (*ScopeManager) Seal

func (m *ScopeManager) Seal(ctx context.Context, scopeID string) (*RetainedOutput, error)

Seal seals a scope's workspace as a retained output and registers it, which is what makes the output settle-able: Settle and RetainedOutput.Settle resolve the output and its scopes through the manager. The scope stays active — sealing does not merge.

func (*ScopeManager) Settle

func (m *ScopeManager) Settle(ctx context.Context, outputID string, action SettlementAction) error

Settle consumes a registered output exactly once.

The verbs (plan 04 §2, mirroring the reference settlement machine):

• selected: the parent's workspace must be bit-identical to the fork baseline (fast-forward-only, strictly). The sealed state is applied onto the parent and the child is causally merged. A parent that moved since the fork — even one whose content is identical but whose timeline advanced — gets an actionable error pointing at apply.
• applied: three-way. The parent's changes since the baseline must be path-disjoint from the output's changes; the sealed delta is then merged onto the parent's current workspace (sandbox.DeltaApplier). A path overlap returns ErrApplyConflict and consumes nothing; so does any merge failure — apply failure never consumes.

Both physical verbs record the causal merge AFTER the workspace change: if the recording fails, the work has already landed and the error wraps ErrSettleEffectLanded — the caller knows the effect happened and must not retry it, and the output stays unconsumed for exactly that distinction.

• released: mark consumed, keep the sealed state addressable. No workspace change; the scope's lifecycle is the caller's to finish.
• discarded: the child scope is discarded (destroying a sandbox it owns); the sealed state is retained as a record.

Select and apply end with a causal merge of the child into the parent, which also destroys a child-owned sandbox — safe because the sealed state outlives it (git worktrees share the object store; a snapshot-backed backend pins captured states on a lease).

Records shepherd.run_output.settled.v1 on the sealing scope's owner path. The append is advisory, checkpoint-style: by the time it runs the settlement's physical effects have happened and cannot be rolled back.

Select and apply are not transactional across the two systems they touch: the workspace effect lands first, and the causal merge (trace append plus scope teardown) runs after. If that recording fails, the parent already holds the output's changes and the error says so; the output stays unconsumed, so the caller can end its lifecycle with SettleRelease or SettleDiscard rather than retry a physical effect that already happened.

type ScopeState

type ScopeState string

ScopeState tracks the lifecycle of a scope.

const (
	ScopeActive    ScopeState = "active"
	ScopeMerged    ScopeState = "merged"
	ScopeDiscarded ScopeState = "discarded"
)

type SettlementAction

type SettlementAction string

SettlementAction is how a sealed output is consumed exactly once.

const (
	SettleSelect  SettlementAction = "selected"
	SettleApply   SettlementAction = "applied"
	SettleRelease SettlementAction = "released"
	SettleDiscard SettlementAction = "discarded"
)

type SupervisorDeniedError

type SupervisorDeniedError struct {
	// Reason is the reviewer's error text.
	Reason string
	// Changes are the proposed changes that were denied.
	Changes []ProposedChange
}

SupervisorDeniedError is returned by CommitMerge when the reviewer denied the merge. The child scope has been discarded; the parent's workspace was never touched.

func (*SupervisorDeniedError) Error

func (e *SupervisorDeniedError) Error() string