mdriver

README

m-driver-sdk — the shared engine-driver SDK for the m toolchain

m-driver-sdk (Go package mdriver) is the single coupling point between m-cli and the engine drivers. It encodes the vendor-neutral engine-driver contract v1.0 as Go types and the verb-level Transport interface every m-<engine> driver implements. m-cli speaks only the contract; all proprietary detail lives behind the Transport in each driver (m-ydb, m-iris).

It was extracted at the Phase-0 reconciliation checkpoint by freezing the two driver Transport sketches against one interface — see phase-0-reconciliation.md.

Orchestration

This repo is the owner and orchestrator of the m-engine-drivers implementation. Cross-repo status and the protocols that keep m-ydb + m-iris parallel and consistent live here:

• docs/implementation-tracker.md — the single source of truth for per-repo, per-milestone status. Reconcile it on every cross-repo change.
• docs/archive/continue-implementation.md — archived kickoff prompt; paste into a fresh session to resume work.

What's in it

Symbol	Purpose
Transport	the frozen verb-level seam: Health · Load · Exec · ReadGlobal · SetGlobal
ExecRequest/ExecResult	field-dispatched exec (Script > EntryRef > Command); EngineError carries §7 faults
LoadRequest/LoadResult, GlobalRef/GlobalNode, Health	the request/result types
EngineError	the §7 structured engine fault (mnemonic/routine/line/text)
Caps/Axes/Features, ContractVersion, Transport* consts	the capability document (§4); Axes.Wired() iterates advertised axes in contract order
FakeTransport	the function-field fake for driver unit tests (no engine)

Design rules

• Verb-level, not run(argv) (risk B1). A low-level argv seam cannot model both YottaDB's session-pipe (stdin→stdout) and IRIS's Atelier-SQL remote (HTTP PUT + SQL; results via a result global). Each transport implements its own strategy; the rest of a driver is transport-agnostic.
• Vendor-neutral only. No YottaDB/IRIS specifics here.
• No clikit dependency. clikit (the toolchain envelope/styling layer) is vendored into every toolchain repo, including non-driver ones; a clikit→SDK edge would couple the whole toolchain to the driver SDK. EngineError is therefore defined here for transport results, and clikit keeps its own copy for the JSON envelope — drivers convert at the boundary.
• Versioning. Adding verbs/fields is a back-compatible minor bump; changing or removing one is a major bump (contract §8). ContractVersion is the handshake m-cli negotiates on.

Consuming it (local development)

Until the module is published, drivers depend on it via a local replace:

require github.com/vista-forge/m-driver-sdk v0.0.0
replace github.com/vista-forge/m-driver-sdk => ../m-driver-sdk

Drop the replace and pin a tagged version once it is published.

Documentation

Overview

Package mdriver is the shared engine-driver SDK for the m toolchain: the vendor-neutral contract types (driver-contract.md v1.0) and the verb-level Transport seam that every m-<engine> driver implements and that m-cli depends on. It is the single coupling point between m-cli, m-ydb, and m-iris — m-cli speaks only the contract; all proprietary detail lives behind the Transport in each driver.

Extracted at the Phase-0 reconciliation checkpoint by freezing the two driver Transport sketches (m-ydb's session-pipe and m-iris's Atelier-SQL) against one interface. Adding verbs/fields is a back-compatible minor bump; changing or removing one is a major bump (driver-contract.md §8).

The package holds ONLY vendor-neutral types — no YottaDB or IRIS specifics. clikit (the toolchain envelope/styling layer) is intentionally NOT imported: clikit is vendored into every toolchain repo, including non-driver ones, so a clikit→SDK dependency would couple the whole toolchain to the driver SDK. EngineError is therefore defined here for Transport results; clikit keeps its own copy for the JSON envelope and drivers convert at the boundary.

Index

• Constants
• func ExecRunner(ctx context.Context, name string, args []string) (stdout, stderr []byte, exit int, err error)
• func Locate(engine string, d LocateDeps) (string, error)
• type Axes
  • func (a Axes) Wired() []AxisVerbs
• type AxisVerbs
• type Caps
• type Check
• type Client
  • func NewClient(bin, engine, transport string, connArgs []string, run CmdRunner) *Client
  • func (c *Client) Caps(ctx context.Context) (Caps, error)
  • func (c *Client) ExecEval(ctx context.Context, command string) (ExecResult, error)
  • func (c *Client) ExecRun(ctx context.Context, entryref string, args []string) (ExecResult, error)
  • func (c *Client) ExecScript(ctx context.Context, script string) (ExecResult, error)
  • func (c *Client) Load(ctx context.Context, paths []string) (LoadResult, error)
  • func (c *Client) LoadNoCompile(ctx context.Context, paths []string) (LoadResult, error)
  • func (c *Client) Status(ctx context.Context) (Status, error)
• type CmdRunner
• type CoverResult
• type DoctorResult
• type EngineError
• type ExecRequest
• type ExecResult
• type FakeCall
• type FakeTransport
  • func (f *FakeTransport) Exec(ctx context.Context, req ExecRequest) (ExecResult, error)
  • func (f *FakeTransport) Health(ctx context.Context) (Health, error)
  • func (f *FakeTransport) Load(ctx context.Context, req LoadRequest) (LoadResult, error)
  • func (f *FakeTransport) ReadGlobal(ctx context.Context, req GlobalRef) (GlobalNode, error)
  • func (f *FakeTransport) SetGlobal(ctx context.Context, ref, value string) error
• type Features
• type GlobalNode
• type GlobalRef
• type Health
• type LoadRequest
• type LoadResult
• type LocateDeps
  • func DefaultLocateDeps() LocateDeps
• type StateResult
• type Status
• type Transport

Constants

const (
	TransportLocal  = "local"
	TransportDocker = "docker"
	TransportRemote = "remote"
)

Transport selectors (driver-contract.md §3). A driver advertises which it supports via caps.transports; YottaDB supports local+docker, IRIS adds remote.

const ContractVersion = "1.0"

ContractVersion is the driver-contract major.minor this SDK encodes. A driver advertises it in caps.contract; m-cli refuses a driver whose major it does not understand, with an upgrade hint (driver-contract.md §8).

Functions

func ExecRunner

func ExecRunner(ctx context.Context, name string, args []string) (stdout, stderr []byte, exit int, err error)

ExecRunner is the production CmdRunner: it runs the driver binary, capturing stdout and stderr separately. A non-zero exit is a result; only a launch failure is an error.

func Locate

func Locate(engine string, d LocateDeps) (string, error)

Locate finds the m-<engine> driver binary, in the contract's resolution order (driver-contract.md §4): $M_<ENGINE>_BIN → next to the running executable → the sibling source checkout's dist/ (…/m-<engine>/dist/m-<engine>) → $PATH.

Types

type Axes

type Axes struct {
	Lifecycle []string `json:"lifecycle,omitempty"`
	Sync      []string `json:"sync,omitempty"`
	Exec      []string `json:"exec,omitempty"`
	Data      []string `json:"data,omitempty"`
	Cover     []string `json:"cover,omitempty"`
	Admin     []string `json:"admin,omitempty"`
	Meta      []string `json:"meta,omitempty"`
}

Axes lists the advertised verbs per contract axis (driver-contract.md §5). It is a struct (not a map) so the JSON field order is the contract's logical order; each axis is omitempty so a driver advertising only its wired axes leaves the rest nil → absent (the honest-incremental model both drivers use).

func (Axes) Wired

func (a Axes) Wired() []AxisVerbs

Wired returns the non-empty axes in the contract's logical order, so callers (caps text rendering, conformance) can iterate advertised axes without re-listing field names. Empty (unwired) axes are skipped.

type AxisVerbs

type AxisVerbs struct {
	Name  string
	Verbs []string
}

AxisVerbs pairs an axis name with its advertised verbs (for iteration).

type Caps

type Caps struct {
	Engine     string   `json:"engine"`
	Contract   string   `json:"contract"`
	Transports []string `json:"transports"`
	Axes       Axes     `json:"axes"`
	Features   Features `json:"features"`
}

Caps is the capability document emitted by `meta caps` (driver-contract.md §4). m-cli probes it before optional verbs and adapts to exactly what is advertised; calling an unadvertised verb yields exit 7. Caps MUST be honest — advertise only axes/verbs actually wired in the build — and conformance enforces it.

type Check

type Check struct {
	Name   string `json:"name"`
	OK     bool   `json:"ok"`
	Detail string `json:"detail,omitempty"`
	Fix    string `json:"fix,omitempty"`
}

Check is one doctor preflight result (driver-contract.md §5.7, plan §3): a typed, named diagnostic with an optional human detail and a fix hint.

type Client

type Client struct {
	Bin       string   // path to the m-<engine> binary
	Engine    string   // "ydb" | "iris" (for error messages)
	Transport string   // local | docker | remote
	ConnArgs  []string // extra connection flags (e.g. --container, --base-url); usually empty (driver reads M_<ENGINE>_* env)
	// contains filtered or unexported fields
}

Client is the SDK's reference engine client: the single, shared way any tool reaches a live engine over the driver contract. It invokes an m-<engine> driver binary over the contract's subprocess + JSON-envelope seam (driver-contract.md §2) — `m-<engine> <axis> <verb> [args] --transport <t> [conn] --output json` — and parses the one JSON envelope it writes. A caller speaks only the neutral contract (§1, §11: "zero changes to add an engine"); all vendor detail lives behind the binary, which the drivers reach themselves (m-ydb via local/docker/SSH, m-iris via Atelier REST), so this client never knows the wire, only the contract.

It is the seam's transport monopoly: m-cli and every `v` tool import this Client rather than hand-rolling transport or vendoring a copy (~/vista-forge/CLAUDE.md, waterline rule 3). One Client drives one m-<engine> binary over one transport+connection.

func NewClient

func NewClient(bin, engine, transport string, connArgs []string, run CmdRunner) *Client

NewClient builds a driver client. A nil run uses the real subprocess runner.

func (*Client) Caps

func (c *Client) Caps(ctx context.Context) (Caps, error)

Caps fetches the driver's capability document.

func (*Client) ExecEval

func (c *Client) ExecEval(ctx context.Context, command string) (ExecResult, error)

ExecEval evaluates a single M command. A §7 engine fault is returned in ExecResult.EngineError (data), not as a Go error — only a transport/launch failure is a Go error.

func (*Client) ExecRun

func (c *Client) ExecRun(ctx context.Context, entryref string, args []string) (ExecResult, error)

ExecRun runs an entryref (args become $ZCMDLINE / the formallist).

func (*Client) ExecScript

func (c *Client) ExecScript(ctx context.Context, script string) (ExecResult, error)

ExecScript runs a multi-line M/ObjectScript script (ExecRequest.Script) and returns its raw captured device output — the transport feeds it to the engine's direct mode (YDB `-direct`) or a raw `iris session` pipe, appending the terminating halt. Unlike ExecEval/ExecRun there is no $ETRAP/TRY-CATCH wrapper: a script is a self-contained compound operation (e.g. the coverage TRACE / LineByLine passes) whose caller parses the raw dump itself, so faults surface only as absent/garbled output, never as a structured engineError. The script is base64-passed on argv so newlines and quotes cross the subprocess seam unmangled without widening the CmdRunner signature with a stdin channel.

func (*Client) Load

func (c *Client) Load(ctx context.Context, paths []string) (LoadResult, error)

Load stages + compiles routine source (exec load).

func (*Client) LoadNoCompile

func (c *Client) LoadNoCompile(ctx context.Context, paths []string) (LoadResult, error)

LoadNoCompile stages routine source WITHOUT the eager ZLINK/OBJ.Load compile (exec load --no-compile). This is the lazy-staging model a test/coverage run needs: drop every candidate routine on the engine's source path and let it auto-compile only what a suite actually references — so an optional module that fails to compile on this engine (e.g. a YDB-only deviceparam in STDNET) does not fail the whole stage when no running suite references it.

func (*Client) Status

func (c *Client) Status(ctx context.Context) (Status, error)

Status runs `lifecycle status` — the reachability + identity probe (running, healthy, version). This is the engine-neutral way to prove a VistA is live and learn its version banner (the portable replacement for `W $ZV`, which only captures device output on YottaDB).

type CmdRunner

type CmdRunner func(ctx context.Context, name string, args []string) (stdout, stderr []byte, exit int, err error)

CmdRunner runs the driver binary and returns its stdout, stderr, and process exit. Only a launch failure (binary missing, etc.) is a Go error; a non-zero engine/driver exit is a result. Injected so the client is testable without a real driver binary.

type CoverResult

type CoverResult struct {
	LCOV         string  `json:"lcov"`
	CoveredLines int     `json:"coveredLines"`
	TotalLines   int     `json:"totalLines"`
	LinePercent  float64 `json:"linePercent"`
}

CoverResult is the `cover trace` payload (driver-contract.md §5.5). The driver runs the entryref under the engine's line tracer, reconciles the raw per-line hit data against the executable-line set, and emits an LCOV tracefile plus the rolled-up line totals. Both engines converge on executable-line granularity:

• m-ydb: view "TRACE":1:"^ycov" … zwrite ^ycov (label-relative offsets, reconciled to absolute lines via the parse tree);
• m-iris: %Monitor.System.LineByLine → MLINE:<routine>:<line>:<count> (already absolute); remote rides the runner class.

The neutral shape is intentionally line-coverage only (LCOV DA/LF/LH) — neither engine natively emits function or branch coverage. m-cli aggregates LCOV across drivers and applies --min-percent; the counts here let it gate without re-parsing.

All fields always render: a zero is meaningful (0% covered is a fact, not absence), the same convention as Status.LatencyMs and the Features flags.

type DoctorResult

type DoctorResult struct {
	Transport string  `json:"transport"`
	OK        bool    `json:"ok"`
	Checks    []Check `json:"checks"`
}

DoctorResult is the `meta doctor` payload: the resolved transport, an overall ok (all checks green), and the typed checks. Exit code is carried by the envelope (0 all-green / 6 unreachable / 5 a check failed), not duplicated here.

type EngineError

type EngineError struct {
	Routine  string `json:"routine,omitempty"`
	Line     int    `json:"line,omitempty"`
	Mnemonic string `json:"mnemonic,omitempty"`
	Text     string `json:"text,omitempty"`
}

EngineError is the structured engine fault (driver-contract.md §7). On any compile/runtime fault, exec/cover verbs set ok=false AND populate this so a RED suite shows the real cause (e.g. a <NOROUTINE> at a line) instead of passed:0, failed:0. Mnemonic carries the engine code: %YDB-E-…/%GTM-E… for YottaDB (from $ZSTATUS), or the IRIS <…> code (from $ZERROR).

A driver builds this from the transport result and maps it onto the clikit envelope's own EngineError field when rendering a failure.

type ExecRequest

type ExecRequest struct {
	Script   string
	EntryRef string
	Args     []string
	Command  string
	Stdin    string // optional principal-device input
	Prefix   string // ephemeral-run prefix (zzt<runid>)
}

ExecRequest runs one of three shapes, selected by which field is set rather than an explicit mode enum (the union of both drivers' needs). Precedence when more than one is set: Script, then EntryRef, then Command.

• Script → a multi-line M/ObjectScript script (YDB `-direct`; the transport appends the terminating halt). Engines without a direct mode ignore it.
• EntryRef→ run an entryref (`yottadb -run <ref>` / IRIS `do <ref>`); Args become $ZCMDLINE / the entry's formallist.
• Command → evaluate a single M command (YDB `%XCMD` / IRIS `xecute`).

type ExecResult

type ExecResult struct {
	Stdout      string       `json:"stdout"`
	Status      int          `json:"status"`
	EngineError *EngineError `json:"engineError,omitempty"`
}

ExecResult is the unified outcome. Stdout is the captured device output (session transports) or the runner's result-global text (IRIS remote). EngineError, when non-nil, is the §7 structured fault — set instead of a Go error so the caller can render a RED-with-cause envelope.

type FakeCall

type FakeCall struct {
	Verb string
	Req  any
}

FakeCall is one recorded interaction. Req holds the request struct (or, for SetGlobal, a [2]string{ref, value}).

type FakeTransport

type FakeTransport struct {
	HealthFn     func(ctx context.Context) (Health, error)
	LoadFn       func(ctx context.Context, req LoadRequest) (LoadResult, error)
	ExecFn       func(ctx context.Context, req ExecRequest) (ExecResult, error)
	ReadGlobalFn func(ctx context.Context, req GlobalRef) (GlobalNode, error)
	SetGlobalFn  func(ctx context.Context, ref, value string) error

	Calls []FakeCall
}

FakeTransport is the injected Transport for driver unit tests (no engine). It records every call and returns canned results, so a command's behavior — envelope shape, engineError mapping, verb sequencing — is asserted without a real engine. Real transports appear only in the gated integration tier (plan §1, "No hidden engine in unit tests").

Set the *Fn fields to script behavior; an unset verb returns a zero result. Calls records an ordered trace for sequencing/argument assertions.

func (*FakeTransport) Exec

func (f *FakeTransport) Exec(ctx context.Context, req ExecRequest) (ExecResult, error)

func (*FakeTransport) Health

func (f *FakeTransport) Health(ctx context.Context) (Health, error)

func (*FakeTransport) Load

func (f *FakeTransport) Load(ctx context.Context, req LoadRequest) (LoadResult, error)

func (*FakeTransport) ReadGlobal

func (f *FakeTransport) ReadGlobal(ctx context.Context, req GlobalRef) (GlobalNode, error)

func (*FakeTransport) SetGlobal

func (f *FakeTransport) SetGlobal(ctx context.Context, ref, value string) error

type Features

type Features struct {
	Remote          bool `json:"remote"`
	Prune           bool `json:"prune"`
	EphemeralPrefix bool `json:"ephemeralPrefix"`
	Snapshot        bool `json:"snapshot"`
}

Features advertises optional capabilities m-cli negotiates for graceful degradation (driver-contract.md §4, §10). All fields are always rendered (no omitempty): a false flag is meaningful information, not absence.

type GlobalNode

type GlobalNode struct {
	Ref   string       `json:"ref"`
	Value string       `json:"value,omitempty"`
	Nodes []GlobalNode `json:"nodes,omitempty"`
}

GlobalNode is a global value, with children for a subtree read.

type GlobalRef

type GlobalRef struct {
	Ref   string
	Order string // "forward" | "reverse"
	Depth int    // 0 = this node only
}

GlobalRef addresses a global for a read. Ref is the full global reference (leading ^ optional), e.g. `^ycov("RTN",12)`. Order/Depth shape a subtree query (data.query); the zero value (empty Order, Depth 0) is a single-node get.

type Health

type Health struct {
	Running   bool   `json:"running"`
	Healthy   bool   `json:"healthy"`
	Version   string `json:"version,omitempty"`
	LatencyMs int64  `json:"latencyMs"`
}

Health is the probe result (driver-contract.md §3 health probes).

type LoadRequest

type LoadRequest struct {
	Paths  []string
	Dir    string
	Prefix string
}

LoadRequest stages source for exec (contract input `<paths…> | <dir>`). Paths are explicit files; Dir is an alternative directory of source. Prefix (e.g. zzt<runid>) namespaces an ephemeral run so teardown is scoped to that prefix.

type LoadResult

type LoadResult struct {
	Loaded      []string     `json:"loaded"`
	EngineError *EngineError `json:"engineError,omitempty"`
}

LoadResult reports what was staged + compiled; EngineError carries a compile fault (§7) when staging compiled routines fails.

type LocateDeps

type LocateDeps struct {
	Getenv   func(string) string          // environment lookup
	LookPath func(string) (string, error) // PATH resolution
	ExeDir   func() (string, error)       // directory of the running executable
	IsFile   func(string) bool            // path exists and is an executable file
}

LocateDeps are the injectable lookups used to find a driver binary (so the resolution order is unit-testable without a real filesystem/PATH).

func DefaultLocateDeps

func DefaultLocateDeps() LocateDeps

DefaultLocateDeps binds Locate to the real environment.

type StateResult

type StateResult struct {
	State    string `json:"state"`
	Endpoint string `json:"endpoint,omitempty"`
}

StateResult is the lifecycle up/down/restart payload: the resulting state (e.g. "started"/"stopped"/"attached") and, where meaningful, the endpoint.

type Status

type Status struct {
	Transport  string   `json:"transport"`
	Running    bool     `json:"running"`
	Healthy    bool     `json:"healthy"`
	Version    string   `json:"version,omitempty"`
	Namespaces []string `json:"namespaces,omitempty"`
	LatencyMs  int64    `json:"latencyMs"`
	Endpoint   string   `json:"endpoint,omitempty"`
}

Status is the `lifecycle status` payload (driver-contract.md §5.1) — the liveness/readiness snapshot plus engine identity. Namespaces/Version/Endpoint are omitempty so a driver that cannot cheaply report one simply omits it.

type Transport

type Transport interface {
	// Health is the readiness/liveness probe behind `lifecycle status --probe`
	// and `wait` (driver-contract.md §3, plan §3). YDB: `%XCMD 'write 1'` → 1;
	// IRIS remote: GET /api/atelier/v1/ → 200 + version.
	Health(ctx context.Context) (Health, error)

	// Load stages routine source and compiles it (exec.load: stage + compile).
	// YDB: copy .m onto $ydb_routines (compile is implicit on first run);
	// IRIS: $SYSTEM.OBJ.Load(path,"ck") / Atelier PUT + action/compile.
	Load(ctx context.Context, req LoadRequest) (LoadResult, error)

	// Exec runs an entryref, evaluates a command, or runs a direct-mode script
	// (exec.run / exec.eval). On a compile/runtime fault it returns the fault in
	// ExecResult.EngineError, NOT as a Go error — the fault is data (§7); a Go
	// error means the transport itself failed (could not reach/launch).
	Exec(ctx context.Context, req ExecRequest) (ExecResult, error)

	// ReadGlobal reads a global node, or a subtree per Depth/Order — data.get /
	// data.query, and the result-global read that backs exec/cover orchestration.
	ReadGlobal(ctx context.Context, req GlobalRef) (GlobalNode, error)

	// SetGlobal sets a single global node (data.set), used to seed fixtures. It
	// is a first-class verb (not Exec of a "set" command) so the IRIS remote
	// substrate can route it through a parameterized, role-gated runner method
	// rather than splicing values into ObjectScript.
	SetGlobal(ctx context.Context, ref, value string) error
}

Transport is the frozen verb-level seam between a driver's vendor logic and its engine. It is deliberately NOT a low-level run(argv): the two driver shapes it must fit cannot share an argv seam (risk B1) —

• m-ydb local/docker: pipe M into a `yottadb` session (stdin → stdout), compile implicit;
• m-iris local/docker: pipe ObjectScript into `iris session -U NS`, compile via $SYSTEM.OBJ.Load; m-iris remote: Atelier PUT + action/compile + a SQL action/query into a role-gated runner class — NO raw "run" endpoint, no stdout, results returned through a result global the transport reads.

A verb-level interface lets each transport implement its own strategy while the rest of the driver stays transport-agnostic. Frozen at the Phase-0 checkpoint against both shapes.