cli

package
v0.10.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 25, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package cli handles detection, invocation, and version extraction for LLM CLI tools.

Index

Constants

View Source 
const DefaultProbeTimeout = 10 * time.Second

DefaultProbeTimeout is the per-LLM deadline for the pre-flight probe. 10s is long enough that a healthy CLI's startup + minimal round-trip fits comfortably (claude-code is the slowest in our roster at ~3-5s on a warm cache) and short enough that a hung CLI doesn't make the readiness block itself feel like the 4-minute gemini hang we're trying to eliminate.

Public constant so doctor and future callers can use the same value as the runner — if we tune it later, every consumer updates at once instead of drifting.

View Source 
const DefaultTimeoutSec = 600

DefaultTimeoutSec is the per-agent timeout in seconds used when the resolved LLMConfig has no explicit value. Single source of truth shared across the runner's applyConfig fallback, the orchestrator's RunParallel fallback, the merge-step fallback, and the roster line that shows "timeout: Ns" on the pre-run roster. Centralising prevents drift between what the user sees ("timeout: Ns") and what actually fires.

10 minutes accommodates worst-case agents (Anthropic Sonnet on a thinking model) on worst-case diffs. Users wanting shorter timeouts override per-agent via `llms.<agent>.timeout_seconds:` in `.local-review.yml`.

Variables

View Source 
var CanonicalAPIKeyEnv = map[string]string{
	"claude": "ANTHROPIC_API_KEY",
	"gemini": "GEMINI_API_KEY",
	"codex":  "OPENAI_API_KEY",
}

CanonicalAPIKeyEnv is the env var each CLI itself reads to find its API key. Doctor uses these as the default check when the user hasn't configured a custom APIKeyEnv; invokers use them as the injection target when the user *has* (so a key in $MY_GEMINI_KEY still ends up as $GEMINI_API_KEY for the gemini subprocess).

Functions

func ClassifyExit added in v0.6.3 

func ClassifyExit(ctx context.Context, err error, combinedOutput []byte, agent string) string

ClassifyExit produces a short, user-facing summary of why a CLI subprocess failed, suitable for the per-LLM line: 

[1/3] claude ✗ <ClassifyExit output>

Every classification that we recognise ends with an *actionable* next step — "try a smaller diff", "raise timeout_sec", etc. — so a user staring at a failure has a path forward without diving into stderr or strace. Pre-fix the failure line was the bare wrapped error ("claude review failed: signal: killed (output: )"), which reliably triggered "delete the tool" frustration.

agent is the LLM name ("claude", "gemini", "codex") — embedded into hints that reference per-agent config (e.g. `llms.<agent>.timeout_seconds`). The YAML key is `timeout_seconds` (see internal/config/config.go LLMConfig struct tag). A previous version of this hint said `timeout_sec` — wrong, and would silently leave the user's config untouched if they pasted it.

func ContextWindow added in v0.6.5 

func ContextWindow(agent string) int

ContextWindow returns the conservative per-LLM context window we preflight against, in tokens. Values are the smallest current- stable model in each vendor's lineup so we never falsely pass a diff that would 4xx on a smaller model.

Why a static table instead of probing the CLI: each CLI exposes model context differently (or not at all), and the values change monthly. A small table here, conservative by design, fails-safe if a vendor *shrinks* a context window (rare but happens during rate-limit shifts) at the cost of over-skipping when a vendor *grows* one. Users can override per-agent via `llms.<agent>.context_window:` once that config field lands.

Returns 0 for unknown agents — caller should treat that as "don't preflight, let the agent decide" so a future agent type doesn't get silently filtered.

func EstimatePromptPayload added in v0.6.5 

func EstimatePromptPayload(systemPrompt, diff string) int

EstimatePromptPayload returns an upper-bound token estimate for the exact bytes the invokers ship to the CLI: the wrapped system prompt + separator + diff. Using this — rather than estimating systemPrompt + diff in isolation — prevents preflight from passing an agent that will then 4xx because the actual sent payload was bigger than what we measured.

func EstimateTokens added in v0.6.5 

func EstimateTokens(s string) int

EstimateTokens approximates the token count of s using a fixed byte-to-token ratio. Returns a true upper bound (math.Ceil over the byte/3.5 division) so preflight biases toward skipping rather than feeding an oversized prompt to a model that will 4xx.

We don't pull in a real tokeniser (tiktoken, sentencepiece): they require SDK dependencies we deliberately avoid (vendor lock-in, binary bloat) and the preflight only needs an upper bound to decide "fit or skip". Off-by-20% in the conservative direction is fine.

func FormatSkipReason added in v0.6.5 

func FormatSkipReason(s SkippedAgent) string

FormatSkipReason returns the user-facing one-liner explaining why an agent was preflight-skipped, with the same actionable-hint shape as ClassifyExit's failure messages: numbers + concrete fix.

"prompt+diff" rather than "diff" because the estimate covers the full payload (system prompt + glue + diff) — saying only "diff is X tokens" misled users into thinking the raw diff alone exceeded the context window. Both the prompt+diff total AND the response-reservation appear because the gate is "payload + margin > window" — saying only "payload exceeds window" is factually wrong near the limit.

func PreflightFilter added in v0.6.5 

func PreflightFilter(active []LLM, systemPrompt, diff string) (kept []LLM, skipped []SkippedAgent, promptDiffTokens int)

PreflightFilter takes the active agent list and the system-prompt + diff that will be sent to each agent, returns the subset that fits in each agent's context window plus a list of skipped agents with their numbers.

The filter is purely based on length — we don't know the user's model precisely (CLIs may pick their own default) so we use the conservative-floor table from ContextWindow. An agent whose name isn't in the table is passed through unchanged ("don't preflight what we don't know").

Returns the filtered active list, the skipped-agent details, and the prompt+diff token estimate (useful for the caller's summary line on the all-skipped path).

func SkipSummary added in v0.6.5 

func SkipSummary(skipped []SkippedAgent) string

SkipSummary returns a multi-line summary block listing every skipped agent and what the user should do. Empty string when none were skipped.

func SplitReady added in v0.10.1 

func SplitReady(results []ProbeResult) (ready, notReady []string)

SplitReady partitions a ProbeResult slice into "ready" and "not ready" subsets, preserving order within each subset. Used by the runner to filter the active LLM list to the ones that passed pre-flight, while keeping the not-ready ones around for rendering the readiness block.

Operates on names so the caller can correlate with their own LLM slice without a second lookup. Returned slices are nil when empty (Go idiom) — callers should `len(...)`-check rather than `!= nil`-check.

Types

type ClaudeInvoker 

type ClaudeInvoker struct {
	// contains filtered or unexported fields
}

ClaudeInvoker runs the Anthropic Claude CLI. Uses stdin pipe similar to Gemini.

func (*ClaudeInvoker) Review 

func (c *ClaudeInvoker) Review(ctx context.Context, systemPrompt, diff string) (string, TokenUsage, error)

func (*ClaudeInvoker) RunPrompt 

func (c *ClaudeInvoker) RunPrompt(ctx context.Context, prompt string) (string, TokenUsage, error)

type CodexInvoker 

type CodexInvoker struct {
	// contains filtered or unexported fields
}

CodexInvoker runs the OpenAI Codex CLI.

Bare `codex` (no subcommand) opens an interactive TUI — that's what the pre-v0.5.1 invoker was doing, which is why every codex review failed with `exit status 1`. We use `codex exec` (non-interactive), pipe the prompt over stdin, and have codex write only the final assistant message to a temp file via --output-last-message. That sidesteps both the interactive-TUI failure AND the noisy "session id / tokens used" preamble that codex exec normally prints to stdout.

We deliberately don't use `codex review` (the dedicated review subcommand) because it re-extracts the diff itself from the local git tree, conflicting with the orchestrator's "extract once, fan out to all LLMs with the same diff string" contract.

func (*CodexInvoker) Review 

func (c *CodexInvoker) Review(ctx context.Context, systemPrompt, diff string) (string, TokenUsage, error)

func (*CodexInvoker) RunPrompt 

func (c *CodexInvoker) RunPrompt(ctx context.Context, prompt string) (string, TokenUsage, error)

type GeminiInvoker 

type GeminiInvoker struct {
	// contains filtered or unexported fields
}

GeminiInvoker runs the Google Gemini CLI. Uses: git diff | gemini -p "Review these changes for bugs and security issues"

func (*GeminiInvoker) Review 

func (g *GeminiInvoker) Review(ctx context.Context, systemPrompt, diff string) (string, TokenUsage, error)

func (*GeminiInvoker) RunPrompt 

func (g *GeminiInvoker) RunPrompt(ctx context.Context, prompt string) (string, TokenUsage, error)

type Invoker 

type Invoker interface {
	Review(ctx context.Context, systemPrompt, diff string) (string, TokenUsage, error)
	// RunPrompt sends a raw prompt to the LLM without wrapping it
	// in a code-review context. Used by the merger; tokens returned
	// for cost-attribution symmetry with Review.
	RunPrompt(ctx context.Context, prompt string) (string, TokenUsage, error)
}

Invoker runs an LLM CLI with a diff and returns the review output plus token-usage metadata.

Review takes a `systemPrompt` (the language-specific prompt pack content the runner has already loaded via lang.Dominant + prompts.Get) so each agent reviews against the same review-rules, severity tiering, and hard rules the single-LLM path uses. Empty systemPrompt means "fall back to the agent's built-in generic prompt" — useful for tests and as a defensive default.

Returns the review text, a TokenUsage from the CLI's structured output (zero values when the CLI version or invocation didn't surface token counts), and any error.

func NewInvoker 

func NewInvoker(llm LLM) Invoker

NewInvoker creates an invoker for the given LLM. The Model and APIKey fields on LLM are threaded into each invoker so per-agent --claude-model / --gemini-model / --codex-model flag overrides actually reach the CLI command line, and so a key sourced from a user-named env var (cfg.LLMs[name].APIKeyEnv) still reaches the subprocess under the canonical name the CLI itself expects. An empty Model leaves the CLI on its default; an empty APIKey means "rely on the CLI's own auth flow / OAuth session."

Returns nil if the LLM name is unknown.

type LLM 

type LLM struct {
	Name    string // "claude", "gemini", "codex"
	Path    string // full path to the binary (e.g., "/usr/local/bin/claude")
	Version string // version string (e.g., "2.1.0"), or "unknown" if version probe failed
	// Available is true only when both the binary is in PATH AND the
	// `--version` probe returned a parseable string. A binary that
	// resolves but whose version probe fails (broken symlink, corrupted
	// install, missing runtime) is reported Available=false so callers
	// don't try to invoke an unusable tool.
	Available bool
	// Model is the agent-specific model id (e.g., "claude-opus-4-7",
	// "gemini-2.0-flash", "gpt-5"). Threaded through from cfg.LLMs[*]
	// .Model so per-agent model overrides on the runner actually reach
	// the invoker — pre-fix the field was set in config and printed in
	// the roster but the invoker only got Path, so users got false
	// confirmation that the requested model ran.
	Model string
	// APIKey is the resolved API-key value sourced from
	// cfg.LLMs[name].APIKeyEnv (or the legacy hard-coded env var when
	// APIKeyEnv is empty). The invoker injects it into the subprocess
	// env under the *canonical* variable name each CLI expects
	// (ANTHROPIC_API_KEY / GEMINI_API_KEY / OPENAI_API_KEY) so users
	// can stash the key under any env name they like and the agent
	// still authenticates.
	APIKey     string
	TimeoutSec int // timeout in seconds for this LLM (from config)
}

LLM represents a detected CLI tool with its metadata.

func Detect 

func Detect(name string) LLM

Detect checks if a specific LLM CLI is installed and returns its metadata. Available is true only if the binary is found AND its version probe succeeded; see the LLM.Available field doc for the precise contract.

func DetectAll 

func DetectAll() []LLM

DetectAll checks for all supported LLM CLIs and returns their status. Returns a slice of LLM structs, one per supported CLI (even if not found). Runs detections concurrently to avoid sequential timeouts.

Equivalent to DetectAllWithOverrides(nil) — kept for callers that don't need cli_path overrides (e.g. tests).

func DetectAllWithOverrides added in v0.6.0 

func DetectAllWithOverrides(overrides map[string]string) []LLM

DetectAllWithOverrides is DetectAll plus per-LLM cli_path overrides from config (cfg.LLMs[name].CLIPath). An override may be an absolute path (`/opt/corporate/bin/claude`) or a bare binary name; exec.LookPath handles both — anything with a slash bypasses $PATH per Go's docs.

Pre-fix the orchestrator detected only the hardcoded `claude` / `gemini` / `codex` binary names, so the cli_path field shipped in every example config was inert. Users with corporate installs at non-standard paths got "✗ not installed" with no path-override escape hatch.

type ProbeResult added in v0.10.1 

type ProbeResult struct {
	LLM      string
	Status   ProbeStatus
	Err      error         // populated iff Status != ProbeReady
	Duration time.Duration // wall-clock of the probe call
}

ProbeResult is the per-LLM outcome of a pre-flight readiness probe. Symmetric with multi.ReviewResult in that we always emit one of these per LLM (even on failure) so callers can render a complete readiness block without back-tracking.

func Probe added in v0.10.1 

func Probe(ctx context.Context, inv Invoker, name string) ProbeResult

Probe invokes inv with a tiny "reply OK" prompt and returns the outcome. The caller owns the context's deadline: this function does no inner timeout management so there's a single source of truth for "how long do we wait before declaring an LLM unresponsive" (the per-LLM ctx deadline set by ProbeAll, or by a test passing context.WithTimeout directly).

name is plumbed through to ProbeResult.LLM so the caller can build the readiness block without re-deriving identity from the invoker (Invoker is an interface and doesn't expose name).

func ProbeAll added in v0.10.1 

func ProbeAll(ctx context.Context, llms []LLM, perLLMTimeout time.Duration) []ProbeResult

ProbeAll runs Probe on each LLM in parallel and returns the results in roster order. Each LLM gets its own context with perLLMTimeout — slow LLMs don't hold up faster ones, and a hung CLI on one agent doesn't block the entire readiness block (which was the v0.10.0-RC observed failure: gemini's exhausted-capacity error surfacing after a ~4 min hang while claude and codex sat idle waiting for it).

Returns one ProbeResult per input LLM, indexed by roster position (not completion order, unlike orchestrator.RunParallel). Roster order matches the readiness block's left-to-right layout in the runner; reordering by completion would make the block non-deterministic across runs, hurting readability when users compare two runs side by side.

invokerFactory is an unexported test seam (see ProbeAllWithInvokerFactory). Production callers use ProbeAll which wires in cli.NewInvoker.

func ProbeAllWithInvokerFactory added in v0.10.1 

func ProbeAllWithInvokerFactory(ctx context.Context, llms []LLM, perLLMTimeout time.Duration, factory func(LLM) Invoker) []ProbeResult

ProbeAllWithInvokerFactory is the test-seam variant of ProbeAll. Production code calls ProbeAll; this is exported only so the runner's tests can substitute fake invokers (the same pattern internal/multi.Orchestrator uses for its unexported invokerFactory field). Kept as a public function rather than a struct field so callers don't have to construct a Prober type for a single override — the seam is the function pointer itself.

func (ProbeResult) IsReady added in v0.10.1 

func (r ProbeResult) IsReady() bool

IsReady is a named predicate so call sites don't litter the runner with `r.Status == cli.ProbeReady` comparisons.

type ProbeStatus added in v0.10.1 

type ProbeStatus int

ProbeStatus is the outcome of a pre-flight readiness probe.

Distinct from a per-LLM review error because the user-facing rendering differs: a probe failure prints a one-character marker (✗) in the readiness block and gets a single-line error summary, not the multi-line stderr-tail treatment a real-review failure gets. Conflating the two would surface noise in the readiness block (which is meant to read at a glance) and hide auth-vs- capacity-vs-network distinctions a user needs to act on. 

const (
	// ProbeReady: the CLI returned a non-empty response within the
	// per-LLM timeout window. The LLM is fit to participate in the
	// real review run.
	ProbeReady ProbeStatus = iota

	// ProbeError: the invoker returned an error other than a context
	// deadline — typically auth-not-ready, capacity exhausted on the
	// model, or network/connectivity. The error message carries the
	// vendor's own string (ClassifyExit'd) so the user sees what to
	// fix without re-running the real review.
	ProbeError

	// ProbeTimeout: the probe context's deadline expired before the
	// CLI replied. Distinct from ProbeError because it usually means
	// "vendor SLOW today" rather than "permanently broken" — the
	// real-review timeout is much longer and might still complete.
	// Treated the same as ProbeError for run-or-skip purposes; the
	// distinct status exists so the readiness block can render a
	// clearer "(timeout after Ns)" hint instead of a generic
	// error string.
	ProbeTimeout
)

func (ProbeStatus) String added in v0.10.1 

func (s ProbeStatus) String() string

String returns a one-word identifier suitable for rendering / logging. The readiness block uses ✓/✗ glyphs at the call site; this is for log lines and tests.

type SkippedAgent added in v0.6.5 

type SkippedAgent struct {
	Name             string
	PromptDiffTokens int
	ContextWindow    int
}

SkippedAgent describes one agent that was preflight-skipped, with enough detail for the user to understand why and what to do about it.

PromptDiffTokens is the upper-bound estimate of the *combined* payload sent to the CLI: the system prompt (after wrapping by buildReviewPrompt) plus the diff, with the "# Diff" header glue included. Named "PromptDiffTokens" rather than "EstimatedTokens" so the user-facing message can't accidentally claim "diff is X tokens" when X is actually prompt+diff.

type TokenUsage added in v0.6.6 

type TokenUsage struct {
	// InputTokens is the size of the prompt sent to the model
	// (system prompt + diff + any wrapper text). Anthropic calls
	// the components `input_tokens` + `cache_read_input_tokens` +
	// `cache_creation_input_tokens`; we sum all three since they
	// all represent prompt content the user wrote. Google calls
	// it `promptTokenCount`; OpenAI calls it `prompt_tokens`.
	// All three vendor names normalise to this single field.
	InputTokens int

	// OutputTokens is the size of the response generated by the
	// model. Anthropic: `output_tokens`. Google: `candidatesTokenCount`.
	// OpenAI: `completion_tokens`.
	OutputTokens int

	// TotalOnly indicates the source CLI reported a single combined
	// total without an input/output split. Codex's pre-v0.128
	// stdout metadata is the canonical example ("tokens used: N").
	// When set, the total is in InputTokens (we fold it there
	// instead of inventing a 50/50 split) and display callers
	// should render as "Nk total" rather than "Nk in / 0 out" —
	// the latter would mislead users into thinking the model
	// produced no output. Tools that aggregate across calls should
	// still use Total() for the running sum.
	TotalOnly bool
}

TokenUsage captures the size of a single LLM call's prompt and response, in tokens. Populated by each invoker from the vendor CLI's structured output (claude/gemini JSON mode) or stdout metadata (codex).

**These are prompt/response *size* numbers, not billing numbers.** For Anthropic specifically, InputTokens includes prompt-cache reads and creation tokens — Anthropic discounts cache reads to ~10% of list price, so summing InputTokens across calls does not give dollars-spent (and nor does Total()). The numbers exist so users can answer "how big was the prompt I sent?" / "how big was the reply?" — the kind of question relevant when scoping a review or estimating context-window pressure. For exact spend, check the vendor's own billing dashboard. See parseClaudeJSON in parsers.go for the v0.7.1 rationale (pre-fix InputTokens excluded cache reads to be cost-shaped, and the displayed "in" collapsed to single digits on cached prompts which read as a broken parser).

Zero values mean "we couldn't determine usage from this call" — the CLI version may not support a structured-output flag, or its output shape didn't match what we expected. Callers should treat `usage.IsZero()` as "unknown" rather than "no tokens used."

func (TokenUsage) IsZero added in v0.6.6 

func (u TokenUsage) IsZero() bool

IsZero reports whether neither token count was populated. The caller uses this to decide between "show '12.3k in / 4.5k out'" and "omit the token suffix entirely" on the per-LLM completion line — printing "0 in / 0 out" would mislead users into thinking the call was free when actually we just didn't know.

func (TokenUsage) Total added in v0.6.6 

func (u TokenUsage) Total() int

Total is the sum of input + output, useful for closing-line summaries that show overall token volume across all agents. For TotalOnly usage, InputTokens already holds the full total so the math still works — OutputTokens is 0 by definition there.

Like the underlying fields, this is a *size* aggregate, not a dollar/spend aggregate. For Anthropic specifically, the sum over-states billed amount because cache-read tokens are discounted ~10×. See the TokenUsage struct doc above for why we picked size over spend.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.