researchctl

README

researchctl

researchctl manages research projects as typed claim/evidence/decision graphs. A project can be authored as YAML/JSON or as a trusted JavaScript grammar file, validated structurally, materialized into deterministic files, checked with completion gates, and rendered into Markdown reports.

What it models

A research project is a graph of stable IDs:

• goals describe outcomes.
• questions connect goals to hypotheses and sources.
• hypotheses capture claims that can be supported, rejected, or left open.
• workPackages track implementation or analysis work.
• experiments define runs, expected artifacts, metrics, and success criteria.
• sources and evidence connect outside material and artifacts back to claims.
• decisions record conclusions, evidence, confidence, and reversal conditions.
• reports select graph entities and render them through report blocks.
• reviewRules define completion gates for check-done.
• views describe future board/status renderings.

All IDs are globally unique and all references are validated.

Project YAML format

Minimal YAML:

schemaVersion: 2
kind: ResearchProject
id: PRJ-EXAMPLE-SPRINT
name: Example research sprint
hypotheses:
  - id: H-001
    claim: Simulation gives enough signal to choose the first backend.
    status: open
    priority: P1
    confidence: unknown
experiments:
  - id: EXP-001
    title: Run the first simulation
    status: planned
    priority: P1
    hypotheses: [H-001]
    successCriteria:
      - The output identifies one next decision.

Validate it with:

researchctl validate project.yaml --output json

JavaScript grammar format

JavaScript project files are trusted local programs executed by go-go-goja. They can build reusable project templates with require("researchctl"):

const { project } = require("researchctl");

module.exports = project("PRJ-EXAMPLE-JS", "Example JS project")
  .goal("Choose a backend", g => g.id("GOAL-001").status("active").priority("P1"))
  .hypothesis("Simulation gives enough signal", h => h.id("H-001").status("open").priority("P1").confidence("unknown"))
  .experiment("Run simulation", e => e.id("EXP-001").status("planned").priority("P1").tests("H-001"));

The loader exposes only the researchctl native module to project files. Treat .js and .cjs files as code: do not execute untrusted project files. TypeScript input is intentionally rejected in this phase; transpile it first if needed.

CLI commands

researchctl validate <project>              # structural validation
researchctl export <project> --format yaml  # normalize YAML/JSON/JS to YAML or JSON
researchctl status --project <project>      # count graph entities
researchctl lab init --project <project>    # initialize project-local immutable ledger
researchctl experiment import-run <export> --project <project> --experiment <id> --dry-run
researchctl apply <project> --dry-run       # preview generated files
researchctl apply <project> --out ./out     # write generated files safely
researchctl check-done <id> --project <p>    # run completion rules for one entity
researchctl render <report-id> --project <p>  # render a Markdown report
researchctl experiment run <experiment.js> --project <project> --max-attempts 2 --timeout 30s
researchctl experiment execute-spec <specification.json> --project <project> --experiment-id <id> --runner-command <worker>
researchctl experiment validate-plan <plan.js> --output json
researchctl experiment explain-plan <plan.js> --output json
researchctl experiment run-plan <plan.js> --project <project> --runner-command <worker> --runner-name <name> --runner-version <version>
researchctl experiment plan-status <plan.js> --project <project> --output json
researchctl experiment rerun <run-id> --reason <code> --project <project> --runner-command <worker> --runner-name <name> --runner-version <version>
researchctl lab runs list --project <project> --output json
researchctl lab runs show <run-id> --project <project> --output json

Safe filesystem writer

researchctl apply builds a deterministic filesystem plan before writing. The plan classifies every file as:

• create — the file does not exist.
• update-generated — the existing file has the researchctl generated marker and can be replaced.
• skip-existing — the existing content already matches.
• conflict — the existing file is hand-authored and will not be overwritten.
• update-forced — --force intentionally overwrites a non-generated file.

Generated files include Code generated by researchctl; DO NOT EDIT.. By default, hand-authored files are protected and conflicts fail before partial writes occur.

Completion rules and reports

Built-in review rules cover common “done” gates:

• done-experiment requires a done experiment with linked hypotheses and success criteria.
• accepted-decision requires an accepted decision with a conclusion and evidence.
• resolved-hypothesis requires a supported or rejected hypothesis with evidence and non-unknown confidence.

Reports render Markdown from report blocks such as summary, hypotheses, experiments, evidence, decisions, source-cards, codesign-metrics, and risks.

CPU/GPU codesign experiments

researchctl experiment run executes one exported codesign specification through the project-local laboratory. Initialize the laboratory first, then run the reference example:

researchctl lab init --project examples/projects/minimal.js
researchctl experiment run examples/lab/codesign-reference.js \
  --project examples/projects/minimal.js

The JavaScript file is pure authoring input: it exports one codesign.runSpec(...) builder and does not call .run() or writeArtifacts(). The laboratory assigns the specification, run, attempt, event-sequence, and recorded-time identities; writes verified run-spec and event-stream artifacts beneath .researchctl/artifacts; records scalar or structured JSON metrics without flattening; and closes immutable attempt/run summaries. --max-attempts enables retry and --timeout bounds each in-process attempt.

Codesign JavaScript is descriptor-only. Direct codesign.run(), script-owned result artifacts, and comparison helpers were removed; use Researchctl laboratory commands, experiment plans, and checked-in analysis specifications. Project loading remains side-effect-free and exposes only require("researchctl").

Generic experiment plans

researchctl experiment run-plan executes multi-case, multi-replicate JavaScript plans through the authoritative laboratory. Researchctl owns deterministic declared, blocked, or seeded-randomized ordering; bounded concurrency; retry-versus-replicate separation; identity-based resume; plan status; and explicit child reruns. Plan loading is pure and domain neutral: each case contains a canonical lab.SpecificationRecord, and no domain-specific type enters the plan package.

The runnable examples/lab/experiment-plan.js fixture creates two cases with three replicates each. Use validate-plan and explain-plan before execution, then plan-status to inspect desired versus existing runs without opening SQLite directly. Run researchctl help researchctl-experiment-plans for the complete workflow.

Scraper Workflow V3 runner

Researchctl can execute a compiled Scraper Workflow V3 plan as one laboratory attempt through the existing process boundary. The integration preserves exact plan and task-package identities, verified input/output custody, lineage between both durable run IDs, internal Workflow retries, failed external-operation evidence, cancellation, timeout classification, and experiment-plan resume.

The two-case, two-replicate fixture lives in examples/lab/scraper-workflow-plan.js. Execute it with scraper-workflow-runner, --runner-name scraper-workflow-runner, and --runner-version v1. Run researchctl help researchctl-scraper-workflow-runner for complete staging, execution, evidence, and recovery instructions.

External domain workers

researchctl experiment execute-spec executes any canonical generic laboratory specification through a worker speaking researchctl-runner-stdio/v1. Researchctl validates only the generic identity, artifact, observation, attempt, and run contracts; domain-specific authoring, schemas, capabilities, and help belong to the domain repository.

xgoja, jsverbs, and plugins

pkg/xgoja/providers/researchctl exposes the researchctl and codesign JavaScript modules to xgoja/v2 generated binaries. examples/xgoja/researchctl-jsverbs shows a generated workbench binary that selects both modules, scans examples/jsverbs, and provides run, repl, and verbs commands.

examples/jsverbs contains go-go-goja/jsverbs examples that build, validate, and summarize project graphs with require("researchctl") and run explicit codesign workbench examples with require("codesign"). examples/plugins contains in-process plugin examples for custom review rules and report blocks. The executable project-local plugin runtime is intentionally deferred; the core registry contracts are present so later runtime adapters can register templates, review rules, report blocks, source importers, evidence validators, and view renderers.

Development

Use GOWORK=off in this workspace because the parent go.work can lag the module Go version.

GOWORK=off go test ./...
make test
make logcopter-check

Workbench dev server with devctl

This repo includes a devctl plugin that supervises both workbench services:

• backend — researchctl serve on http://127.0.0.1:8080
• frontend — Vite workbench on http://127.0.0.1:5173

Common commands:

devctl plugins list          # verify plugin handshake
devctl validate              # check tools, deps, ports, and local go-go-goja use
devctl plan                  # inspect the backend/frontend launch plan
devctl up --force            # run/start the full workbench system
devctl status                # see supervised service PIDs and log paths
devctl logs --service backend --stderr --follow
devctl restart backend       # restart only the backend
devctl stop-service frontend # stop one service
devctl start frontend        # start a stopped/crashed service
devctl down                  # stop all supervised services and remove state

The plugin creates an isolated var/devctl/gowork/go.work when sibling ../go-go-goja is present, avoiding the stale parent workspace while still using local replapi changes. Runtime state/logs are ignored under .devctl/ and var/devctl/.