Documentation
¶
Overview ¶
Package engine is the orchestration loop for mallcop's self-extension code-authoring engine. It composes the runtime credential/billing SEAM (session), the worktree jail (sandbox), and the opencode adapter (opencode) into one serialized, destructive-safe build:
anti-thrash → session.Authorize → open jail → author (opencode) → commit → gate (mallcop validate-proposal) → measure cost → session.Record → GREEN: emit a human-review ARTIFACT / RED: poison the fingerprint → teardown (session.Close, remove worktree) — ALWAYS.
Two rails, one safety surface ¶
The session seam hides exactly ONE axis of variation: where inference is billed. A commercial billing session (provided by the commercial layer) runs the spend-cap + run-key lifecycle (authorize → mint → usage-delta → revoke); a session.BYOISession points at an OSS user's OWN endpoint+key with NO cap and NO minted key. The seam is the ONLY difference between the rails — EVERY safety rail below is byte-identical on both, and a BYOISession holds no Gate/Minter/billing handle so a BYOI build makes zero billing calls by construction.
Safety invariants (identical on both rails) ¶
- A known-reject fingerprint is SKIPPED before the session is even consulted — zero Authorize, zero mint, zero inference, on EITHER rail.
- On the metered rail the spend gate is consulted BEFORE a run key exists; a refusal spends nothing (no network call to inference is possible without a key). session.Close is deferred immediately after Authorize and fires regardless of success, gate verdict, or a panic — on the metered rail this revokes the per-build run key (the load-bearing teardown step).
- The engine NEVER pushes to a remote or opens a PR, at ANY autonomy setting — that requires operator credentials the worktree's scrubbed env never carries (see sandbox's package doc). A GREEN gate always produces a reviewable proposal artifact (diff + GateResult + provenance) in a human-review directory. ADDITIONALLY, when Engine.Autonomy is "fully" (the operator-owned autonomy dial — see package autonomy), a GREEN proposal is ALSO merge-automated: a local branch ref in the TARGET repo (never origin/main, never a push) is force-updated to the authored HEAD, so the operator's own tooling can act on it immediately. At "non"/"semi" this step never runs — a human always merges authored code by hand. Contribute-back to the shared OSS pool has no code-lane equivalent at all: it is a router (DATA lane) concept, and there it is NEVER auto-merged regardless of dial — that hard line is untouched by anything in this package. ADDITIONALLY (BOTH ruling, part B), a GREEN gate with GateResult.NovelGap==true — the customer-tree gate's declared family has ZERO labeled must_fire rows in the reference corpus, so the corpus cannot independently grade it — ALSO withholds merge automation at "fully", the same dial-independent forced-human-review treatment as OSS contribute-back. The proposal is still emitted as a reviewable artifact; it is simply never merged.
- The authoring prompt is built from TRUSTED structural signals only.
Process boundary ¶
The gate runs as a separate trusted binary (`mallcop validate-proposal`); its versioned JSON GateResult is the seam (see gate.go). This engine execs that binary rather than importing the gate package.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Authorer ¶
type Authorer interface {
Invoke(ctx context.Context, wt *sandbox.Worktree, apiKey, task string) (opencode.Result, error)
// BuildTaskPrompt builds the authoring instruction. customerShaped is the
// SAME trusted signal the gate itself uses (hasCmdMallcop on the worktree
// jail's own TargetRepo — see Run's customerShaped computation and
// runValidateProposal) — never anything derived from untrusted proposal
// content. true routes the prompt (and, in Run, the post-authoring
// registry step) onto the customer-tree SIDECAR shape instead of the
// in-tree own-package shape.
BuildTaskPrompt(gap opencode.TrustedGap, customerShaped bool) string
}
Authorer is the code-authoring surface the engine needs. *opencode.Adapter satisfies it; tests inject a fake stub (including one that panics, to prove deferred teardown still fires). The Adapter's inference endpoint is bound at CLI wiring (metered: the provider URL; BYOI: the user's URL); only the run's key flows through Invoke — the minted run key OR the BYOI user key, from session.Credentials.
type Engine ¶
type Engine struct {
// Session is the runtime seam that supplies inference credentials and, on the
// metered rail, the spend-cap + run-key lifecycle. A commercial billing
// session reproduces the metered billing preamble (authorize → mint →
// usage-delta → revoke); a session.BYOISession points at the user's OWN
// endpoint+key with NO cap and NO minted key. Required. It is the ONLY place
// billing differs between the two rails — every safety rail below is identical.
Session session.Session
// Jail opens the target-repo worktree write jail. Required.
Jail jailOpener
// Adapter drives headless opencode. Its endpoint is bound at wiring; the run's
// key flows through Invoke via session.Credentials. Required.
Adapter Authorer
// Fingerprints is the anti-thrash reject set. Required.
Fingerprints *RejectSet
// ValidateBin is the mallcop binary that runs `validate-proposal`. When set
// it is trusted verbatim (operator/test-injected configuration). Empty →
// "mallcop" is resolved from PATH and then VERSION-PROBED (see
// resolveValidateBin/probeGoMallcopBinary) before the engine ever trusts
// it as the gate — a bare, unverified PATH lookup can silently resolve to
// an unrelated or stale "mallcop" (e.g. the deprecated python-legacy shim
// some machines still have on PATH), which then fails opaquely mid-run
// instead of never being invoked. The gate itself builds per-tree binaries.
ValidateBin string
// ExamRepo is the path to a REFERENCE mallcop tree (has
// its own cmd/mallcop + pinned exam corpus) the gate uses to grade a
// CUSTOMER-SHAPED TargetRepo — one with no cmd/mallcop of its own (the
// THIN-EMBED shape: go.mod pins mallcop, detectors/<name>/ carries the
// authored detector). It is passed to the gate as `--exam-repo` ONLY when
// the worktree jail's TargetRepo lacks cmd/mallcop (see gate.go's
// hasCmdMallcop); when TargetRepo already has its own cmd/mallcop (the
// existing "author into the mallcop repo itself" lane), ExamRepo is
// ignored and the gate runs its unchanged in-tree lane. Empty is the
// fail-safe default — a misconfigured customer-tree build then surfaces
// the gate's own loud, actionable error (it names --exam-repo) rather than
// silently building nothing. ExamRepo is caller-owned configuration, NEVER
// derived from the untrusted target repo's own contents (the trust
// boundary core/selfgate's package doc establishes on the mallcop side).
ExamRepo string
// ArtifactDir is the human-review lane directory GREEN proposals land in.
// Required.
ArtifactDir string
// Class is the stable spend-cap class string recorded in provenance for
// self-ext authoring. Empty → "selfext-author". The spend attribution class
// itself lives on the commercial billing session.
Class string
// AuthoringLane is the lane the run key is scoped to and opencode authors
// under (e.g. "heal"). Required.
AuthoringLane string
// Sovereignty is recorded in provenance (a caller-supplied account tier label).
Sovereignty string
// BudgetUSD is the per-build spend estimate handed to session.Authorize.
// On the metered rail it is the spend-cap estimate and pool ceiling; BYOI
// ignores it. Required (> 0).
BudgetUSD float64
// MaxAuthoringAttempts bounds how many times Run re-drives opencode as a FRESH
// session when an exit-0 attempt authored NOTHING — the narrate-then-die shape
// (rd 4a1): the model narrates an intention as a text-only turn and opencode
// ends the session with exit 0 having issued zero write tool calls. Each retry
// appends opencode.NoOpAuthoringEscalation to the (byte-identical) base prompt.
// This is DISTINCT from the adapter's MaxAttempts (which retries a TRANSIENT
// NON-ZERO-exit fast-fail WITHIN one Invoke): a no-op exit-0 run is a success
// to the adapter, so only the engine can catch it. ≤0 →
// defaultMaxAuthoringAttempts (3). The metered run key is authorized ONCE with
// BudgetUSD and every attempt spends against that same capped key, so the cap
// already bounds cumulative spend across ALL attempts — no per-attempt billing.
MaxAuthoringAttempts int
// Autonomy is the operator-owned dial. ONLY "fully"
// merge-automates a GREEN proposal (see AutoApplyBranchPrefix); "non" and
// "semi" both always leave it as an artifact for a human to review and
// merge by hand. Zero value normalizes to autonomy.NonAutonomy (fail-safe:
// an unconfigured Engine never auto-applies) via autonomy.Dial.Normalized.
Autonomy autonomy.Dial
// AutoApplyBranchPrefix is the local branch prefix "fully" autonomy merge
// automation force-updates in the TARGET repo: <prefix>/<detector-id>.
// Empty -> "selfext/applied". Never origin/main, never a push.
AutoApplyBranchPrefix string
// Logger receives non-secret lifecycle events. Nil → discard.
Logger *slog.Logger
// Now is the clock, for tests. Nil → time.Now.
Now func() time.Time
}
Engine orchestrates one authoring build per Run. All fields are set once; Run is serialized by the caller (one build at a time).
func (*Engine) Run ¶
Run executes one authoring build for gap. It returns an Outcome describing the terminal state; the error return is reserved for engine-infrastructure failures (bad config, unwritable artifact dir) — expected terminal states (skipped/refused/rejected/failed) are Outcomes with a nil error.
Named returns let the deferred panic-guard convert a panic in opencode or the gate into Outcome{Failed} while the deferred teardown (revoke, worktree remove) still fires.
type GateFinding ¶
type GateFinding struct {
Path string `json:"path"`
Rule string `json:"rule"`
Detail string `json:"detail"`
}
GateFinding mirrors selfgate.GuardFinding.
type GateResult ¶
type GateResult struct {
SchemaVersion int `json:"schema_version"`
Tier string `json:"tier"`
Passed bool `json:"passed"`
BaseSHA string `json:"base_sha"`
HeadSHA string `json:"head_sha"`
Stages []GateStage `json:"stages"`
CoveragePlus int `json:"coverage_plus"`
NewFirings []string `json:"new_firings"`
// NovelGap mirrors selfgate.GateResult.NovelGap (BOTH
// ruling, part B): CUSTOMER-TREE MODE ONLY — true when at least one family
// the customer detector's own scenarios declare has ZERO labeled must_fire
// rows anywhere in the reference corpus. The held-out-corpus new-firing
// control (part A) has no independent ground truth for such a family — it
// can prove the detector does not fire WRONGLY elsewhere, but cannot prove
// its claimed detection is itself sound. A GREEN gate with NovelGap==true
// is NOT a weaker verdict (every stage still passed) but BOTH the router
// (DATA lane) and the engine (CODE lane, below) treat it as a
// dial-independent forced-human-review signal — mirroring the existing
// dial-independent hard line already applied to OSS contribute-back (see
// package autonomy's doc). Omitted/false for the in-tree lane and for any
// run that never reaches customer-tree mode.
NovelGap bool `json:"novel_gap,omitempty"`
// NovelGapFamilies mirrors selfgate.GateResult.NovelGapFamilies —
// informational detail only, never consulted to compute NovelGap.
NovelGapFamilies []string `json:"novel_gap_families,omitempty"`
}
GateResult mirrors mallcop core/selfgate.GateResult over the PROCESS BOUNDARY. This engine does not import the gate package: the gate is a separate trusted binary (`mallcop validate-proposal`), and its versioned JSON is the seam (exactly as selfgate itself decodes exam-detect's JSON locally rather than importing core/eval). SchemaVersion lets us reject a report shape we do not understand.
func RunValidateProposal ¶
func RunValidateProposal(ctx context.Context, bin, workdir, baseSHA, examRepo string) (GateResult, int, error)
RunValidateProposal is the exported entrypoint the operator K8 propose→gate→route pipeline uses to gate an already-applied add-only overlay change: it execs the SAME trusted `mallcop validate-proposal` the engine's Run uses internally, over base..HEAD in workdir, and returns the parsed GateResult. The router then decides where the gate-GREEN proposal goes.
examRepo is the path to a REFERENCE mallcop tree (has its own cmd/mallcop + pinned exam corpus) — the engine's own pinned checkout, caller-supplied, NEVER derived from workdir's own contents. It is only EVER USED when workdir itself has no cmd/mallcop (a customer-shaped THIN-EMBED target repo): see hasCmdMallcop below. Pass "" to preserve the prior behavior byte-for-byte (the flag is simply omitted from argv) — the existing mallcop-repo-as-TargetRepo lane is completely untouched.
CUSTOMER-TREE MODE (orchestrator ruling, resolving the 72d/97b collision): on the mallcop side, `--exam-repo` is not just stage 3's reference-tree path — core/selfgate.ValidateProposal now ALSO derives its stage-1 guard's customerTreeMode strictly as `opts.ExamRepo != ""` (see selfgate/validate.go's Options.ExamRepo doc). There is deliberately NO separate `--customer-tree` flag on either side of this process boundary: customer-tree mode and --exam-repo are THE SAME SIGNAL, so they cannot travel apart by construction (a caller cannot pass one without the other, unlike two independent flags that could drift out of sync). That signal still originates ONLY here — the trusted engine's own hasCmdMallcop gate on workdir, never anything the untrusted proposal content could set — so the "never inferred from the target tree" invariant holds end to end.
type GateStage ¶
type GateStage struct {
Name string `json:"name"`
Passed bool `json:"passed"`
Evidence string `json:"evidence"`
Findings []GateFinding `json:"findings"`
}
GateStage mirrors selfgate.StageResult.
type Outcome ¶
type Outcome struct {
Skipped bool // anti-thrash: known-reject fingerprint; spent nothing
Refused bool // spend gate denied; spent nothing
Proposed bool // GREEN gate: reviewable artifact written
Rejected bool // RED gate: findings; fingerprint poisoned
Failed bool // operational/authoring failure (transient; no fingerprint)
Reason string // human-readable cause (denial reason, error, ...)
Fingerprint string // the gap fingerprint
ArtifactPath string // GREEN: path to the proposal artifact directory
CostUSD float64 // measured spend for this run
Gate *GateResult // parsed gate verdict when the gate ran
// Applied is true ONLY when Engine.Autonomy is "fully" AND the gate was
// GREEN: the authored change was ALSO merge-automated (see AppliedBranch).
// At "non"/"semi" this is always false — Proposed alone means "artifact
// only, a human must merge it by hand".
Applied bool
// AppliedBranch is the local target-repo branch merge automation
// force-updated to the authored HEAD, set only when Applied is true.
AppliedBranch string
}
Outcome is the terminal state of one Run. Exactly one of Skipped / Refused / Proposed / Rejected / Failed is true. Skipped and Refused spend nothing.
type Provenance ¶
type Provenance struct {
Fingerprint string `json:"fingerprint"`
DetectorID string `json:"detector_id"`
EventType string `json:"event_type"`
Lane string `json:"lane"`
Model string `json:"model"`
// Endpoint is the inference base URL this build was billed to (the provider
// URL on the metered rail, the user's URL on BYOI). It is recorded from
// session.Credentials for an auditable "billed-to" record. It is the base URL
// only — NEVER the key.
Endpoint string `json:"endpoint"`
Sovereignty string `json:"sovereignty"`
Class string `json:"class"`
CostUSD float64 `json:"cost_usd"`
BaseSHA string `json:"base_sha"`
HeadSHA string `json:"head_sha"`
GatePassed bool `json:"gate_passed"`
Timestamp time.Time `json:"timestamp"`
// Applied/AppliedBranch mirror Outcome.Applied/AppliedBranch — recorded in
// the artifact's provenance.json for an auditable "was this auto-merged"
// trail, distinct from GatePassed (GREEN but NOT applied happens at
// "non"/"semi").
Applied bool `json:"applied"`
AppliedBranch string `json:"applied_branch,omitempty"`
}
Provenance is the auditable record emitted alongside every artifact.
type RejectSet ¶
type RejectSet struct {
// contains filtered or unexported fields
}
RejectSet is the persisted set of gap fingerprints the gate has already rejected. Consulting it FIRST means the loop never re-spends inference re-deriving a known dead end. Safe for concurrent use within one process; the JSON file is the cross-run record.
func LoadRejectSet ¶
LoadRejectSet loads (or initializes) the reject set from dir. An empty dir resolves via $MALLCOP_SPEND_DIR then ~/.cache/mallcop. A missing file is an empty set, not an error.
type SpendController ¶
type SpendController = session.SpendController
SpendController is the spend-cap surface the engine needs. It is a type ALIAS to session.SpendController so the commercial billing session, the engine, and the proposer all share ONE definition (a reviewer confirms the spend-cap surface in a single place). The commercial layer's spend-cap implementation satisfies it; tests inject a spy.
Source Files