multi

package
v0.15.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 29, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package multi handles multi-LLM orchestration, storage, and merging.

Index

Constants

View Source 
const MaxReviewBytesForMerge = 8 * 1024

MaxReviewBytesForMerge caps how much of any single per-LLM review gets concatenated into the merger prompt. The merger LLM has a finite context window; a verbose-enough reviewer (or one that hallucinates a long preamble) can blow it out by itself, killing the whole pipeline at the very last step.

8 KB ≈ 2k tokens at typical English density — enough room for a detailed review while leaving plenty of context budget for the other reviewers and the merger's own template / instructions. Saved per-LLM review files on disk are NOT truncated; this limit applies only to what the merger sees.

Variables

View Source 
var ErrNoReviewsToMerge = fmt.Errorf("no reviews to merge")

ErrNoReviewsToMerge signals that Merge was called with an empty review set. The runner short-circuits before reaching this in normal flow (see classifyRunMode + the no-mergeable-output path), but the guard here is defense-in-depth — pre-fix, ReviewCount==0 fell into the multi-review template branch and rendered "0 separate code review reports from: " with an empty reviewer list, producing an incoherent prompt to the merger LLM.

Functions

func CountSuccessful 

func CountSuccessful(results []ReviewResult) int

CountSuccessful returns the number of successful reviews (Error == nil); for framing decisions prefer CountWithOutput.

func CountWithOutput added in v0.6.1 

func CountWithOutput(results []ReviewResult) int

CountWithOutput returns the number of reviews with non-blank Output (matches BuildMergeInput's filter).

func HasMergeableOutput added in v0.6.1 

func HasMergeableOutput(r ReviewResult) bool

HasMergeableOutput reports whether r has non-whitespace Output (single source of truth across CountWithOutput, BuildMergeInput, selectMergeLLM).

Types

type GateDecision added in v0.10.1 

type GateDecision struct {
	Total      int // len(results)
	Successful int // Error == nil (regardless of Output)
	WithOutput int // HasMergeableOutput == true (regardless of Error)
}

GateDecision is a single-pass summary of a result set, capturing both the "Error == nil" view (Successful) and the "has mergeable output" view (WithOutput). These two views CAN diverge and historically did, producing two distinct bugs:

  1. SaveReview-failed-with-output: Error != nil but Output != "". Counted in WithOutput, not Successful — the merger can still consolidate the in-memory output even though persistence failed.

  2. CLI-exited-zero-with-empty-output: Error == nil but Output == "". Counted in Successful, not WithOutput — the merger has nothing to consume even though the per-LLM call "succeeded."

Pre-fix the runner threaded both counts through six call sites; this type is the single source so the metrics can't drift again (audit/tech-debt.md "## cmd/local-review [part 2/3]" major finding on runner.go:156, surfaced by `local-review audit --topic tech-debt`).

func DecideGate added in v0.10.1 

func DecideGate(results []ReviewResult) GateDecision

DecideGate computes a GateDecision from a result set in a single pass. The intent: derive both counts at the call site that needs them once, then thread the GateDecision through everything downstream — no second traversal, no possibility of the two metrics observing different snapshots of `results` (results is a slice, so the underlying array won't change, but the historical concern was call-site drift, not data-race drift).

func (GateDecision) ClassifyZero added in v0.10.1 

func (g GateDecision) ClassifyZero() ZeroMergeableReason

ClassifyZero categorises why a result set produced nothing mergeable. Only meaningful when HasMergeable() is false; the runner uses it to pick the right error message ("all N failed" vs "all N returned empty output" vs the mixed-mode case).

When HasMergeable() is true the result is meaningless — guard at the call site with `if !g.HasMergeable() { ... ClassifyZero() ... }`.

Implementation note: when WithOutput == 0, every Successful entry is by definition "succeeded but produced no output" (no entry can be in WithOutput, so the intersection of Successful and WithOutput is empty). That's why the AllEmpty case compares Successful to Total — it's exact in the only state ClassifyZero is consulted in.

func (GateDecision) Failed added in v0.10.1 

func (g GateDecision) Failed() int

Failed returns the count of reviews where Error != nil (regardless of Output). Symmetric with Successful = Total - Failed.

func (GateDecision) HasMergeable added in v0.10.1 

func (g GateDecision) HasMergeable() bool

HasMergeable reports whether at least one review has mergeable output. Use this instead of `WithOutput > 0` at gate sites — the named predicate documents intent ("is there anything for the merger to consume?") instead of leaving the reader to infer it from the count.

type MergeInput 

type MergeInput struct {
	ReviewCount        int
	LLMNames           string
	ConsensusThreshold int
	Reviews            []ReviewContent
}

MergeInput holds data for the merge template.

func BuildMergeInput 

func BuildMergeInput(results []ReviewResult, consensusThreshold int) MergeInput

BuildMergeInput creates MergeInput from review results. Includes all reviews that have output, even if saving to disk failed. Per-review output is truncated to MaxReviewBytesForMerge with a trailing notice so the merger can still pick up an explicit signal that some content was clipped.

ConsensusThreshold is clamped to the actual reviewer count so the merge prompt never asks for "3+ reviewers agree" when only 2 ran — the LLM was apologising for the impossible-by-design ask in its own summary line ("0 (only 2 reviewers, but 3 issues have 2/2 consensus)"), which read as a broken template.

type MergeMeta 

type MergeMeta struct {
	LLM                  string `json:"llm"`
	Status               string `json:"status"`
	FinalFindingsCount   int    `json:"final_findings_count,omitempty"`
	DeduplicationRemoved int    `json:"deduplication_removed,omitempty"`
	DurationMs           int64  `json:"duration_ms,omitempty"`
	Error                string `json:"error,omitempty"`
	// InputTokens / OutputTokens for the merge step's own LLM call,
	// same shape and semantics as ReviewMeta — prompt/response
	// *size*, not billed amount. Aggregating across ReviewMeta +
	// MergeMeta gives approximate prompt volume but inflates on
	// Anthropic cached re-runs (see ReviewMeta doc above for the
	// caveat). For dollar spend, the vendor billing dashboard is
	// authoritative.
	InputTokens     int  `json:"input_tokens,omitempty"`
	OutputTokens    int  `json:"output_tokens,omitempty"`
	TotalOnlyTokens bool `json:"total_only_tokens,omitempty"`
}

MergeMeta holds details about the merge operation.

type Merger 

type Merger struct {
	// contains filtered or unexported fields
}

Merger consolidates review findings from multiple LLMs.

func NewMerger 

func NewMerger(llm cli.LLM) (*Merger, error)

NewMerger creates a new Merger that uses the given LLM for merging.

func (*Merger) Merge 

func (m *Merger) Merge(ctx context.Context, input MergeInput) (string, cli.TokenUsage, error)

Merge consolidates multiple reviews into one using an LLM.

Returns the merged markdown report, the merge step's own token usage (for cost-attribution alongside per-LLM review usage), and any error. Tokens may be zero when the merge LLM's CLI doesn't surface usage data.

Returns ErrNoReviewsToMerge when input has no reviewable content. The runner is expected to filter zero-review cases upstream (single-LLM fallback path or "all agents failed" branch); this is just a backstop.

type Metadata 

type Metadata struct {
	Commit    string       `json:"commit"`
	Branch    string       `json:"branch"`
	Timestamp time.Time    `json:"timestamp"`
	Reviews   []ReviewMeta `json:"reviews"`
	Merge     MergeMeta    `json:"merge"`
}

Metadata tracks details about a multi-LLM review run.

func Load 

func Load(path string) (*Metadata, error)

Load reads metadata from a JSON file.

func (*Metadata) Save 

func (m *Metadata) Save(path string) error

Save writes the metadata to a JSON file.

type Orchestrator 

type Orchestrator struct {
	// contains filtered or unexported fields
}

Orchestrator coordinates parallel reviews from multiple LLMs.

invokerFactory is an in-package test seam. Production code uses cli.NewInvoker; orchestrator_test.go swaps in fake invokers with controlled durations so the streaming contract (channel emits in completion order, closes after all done) can be pinned without shelling out to real CLIs. Kept unexported to avoid leaking the hook into the package's public API.

func NewOrchestrator 

func NewOrchestrator(llms []cli.LLM, storage *ReviewStorage) *Orchestrator

NewOrchestrator creates a new Orchestrator with the given LLMs and storage.

func (*Orchestrator) RunParallel 

func (o *Orchestrator) RunParallel(ctx context.Context, systemPrompt, diff, commit, branch string) (<-chan ReviewResult, error)

RunParallel executes reviews concurrently for all configured LLMs and returns a channel that emits each ReviewResult as its agent finishes. The channel is closed after every agent has reported, so callers can range over it. Buffered to len(llms) so a slow consumer can't deadlock the workers.

Streaming (added in v0.6.7) replaced the prior "block on wg.Wait, return slice" shape because users with one slow agent (gemini-3.x preview at 5+ min) saw a blank terminal for the whole run with zero feedback. Now each per-LLM line prints as the agent completes; the runner accumulates into a slice for the merge step. Emission order = completion order, not roster order — the CLI dropped the N/M prefix in favor of bare agent names so the new ordering doesn't read as misleading numbering.

systemPrompt is the language-specific prompt pack content the caller has already loaded (lang.Dominant + prompts.Get). Passed to every invoker so all agents review the diff against identical review-rules and severity tiering. Empty string is allowed — each invoker has a generic fallback.

The error return is reserved for synchronous setup failures (none today; always nil). Per-agent failures travel inside ReviewResult .Error so the channel still reports them and the runner can surface a per-LLM failure line in completion order.

type ReviewContent 

type ReviewContent struct {
	LLM     string
	Content string
}

ReviewContent holds a single review's content.

type ReviewMeta 

type ReviewMeta struct {
	LLM           string `json:"llm"`
	Version       string `json:"version"`
	Mode          string `json:"mode"`   // "cli" or "api"
	Status        string `json:"status"` // "success" or "failed"
	DurationMs    int64  `json:"duration_ms"`
	FindingsCount int    `json:"findings_count,omitempty"`
	OutputFile    string `json:"output_file,omitempty"`
	Error         string `json:"error,omitempty"`
	// InputTokens / OutputTokens are prompt and response *size* in
	// tokens — not billing numbers. Sourced from each CLI's
	// structured output (claude / gemini JSON, codex stdout
	// metadata) when available. For Anthropic, InputTokens
	// includes cache-read and cache-creation tokens (Anthropic
	// discounts cache reads ~10× at the billing layer, but for
	// "how big was the prompt" we want the full count).
	//
	// **Aggregation caveat:** summing across ReviewMeta entries
	// gives an *approximate prompt-byte volume*, not a billable
	// total. For Anthropic specifically, cache reads inflate the
	// sum on repeat runs of the same prompt — the cached portion
	// is counted in every ReviewMeta even though Anthropic only
	// charges full price the first time. For dollar spend, use
	// the vendor's billing dashboard. Both 0 means usage was
	// indeterminate. omitempty keeps backward-compat for readers
	// that don't know about these fields.
	InputTokens  int `json:"input_tokens,omitempty"`
	OutputTokens int `json:"output_tokens,omitempty"`
	// TotalOnlyTokens=true means input/output split is unknown and
	// InputTokens holds the combined total (codex pre-v0.128 stdout
	// shape). Aggregation tools should still sum InputTokens +
	// OutputTokens — the total is in InputTokens, OutputTokens is 0
	// — but display-layer tools should show "Nk total" rather than
	// "Nk in / 0 out".
	TotalOnlyTokens bool `json:"total_only_tokens,omitempty"`
}

ReviewMeta holds details about a single LLM's review.

type ReviewResult 

type ReviewResult struct {
	LLM      string
	Version  string
	Mode     string
	Output   string
	Error    error
	Duration time.Duration
	FilePath string
	Tokens   cli.TokenUsage
}

ReviewResult holds the result of a single LLM review.

Tokens is populated from the CLI's structured output (claude / gemini JSON, codex stdout metadata) when available. Zero values mean "we couldn't determine usage" — the CLI version may be too old to support a JSON flag, or the output shape didn't match what we expected. Display callers should check Tokens.IsZero() rather than printing "0 in / 0 out" which would mislead users.

func GetSuccessful 

func GetSuccessful(results []ReviewResult) []ReviewResult

GetSuccessful returns only successful review results.

type ReviewStorage 

type ReviewStorage struct {
	// contains filtered or unexported fields
}

ReviewStorage handles saving review outputs to disk.

func NewStorage 

func NewStorage(basePath string) *ReviewStorage

NewStorage creates a new ReviewStorage with the given base path.

func (*ReviewStorage) SaveMerged 

func (s *ReviewStorage) SaveMerged(branch, commit, content string) (string, error)

SaveMerged writes the merged review to disk. Returns the path to the saved file.

func (*ReviewStorage) SaveMetadata 

func (s *ReviewStorage) SaveMetadata(branch, commit string, meta *Metadata) (string, error)

SaveMetadata writes the metadata JSON to disk. Returns the path to the saved file.

func (*ReviewStorage) SaveReview 

func (s *ReviewStorage) SaveReview(branch, commit, llmName, llmVersion, content string) (string, error)

SaveReview writes an LLM's review output to disk. Returns the path to the saved file.

type ZeroMergeableReason added in v0.10.1 

type ZeroMergeableReason int

ZeroMergeableReason classifies *why* a result set has no mergeable output. Only meaningful when GateDecision.WithOutput == 0. 

const (
	// ZeroMergeableAllFailed: every review's Error is non-nil. The
	// LLMs crashed or the CLI invocation errored.
	ZeroMergeableAllFailed ZeroMergeableReason = iota

	// ZeroMergeableAllEmpty: every review's Error is nil, but no
	// review produced non-blank Output. The CLI exited zero with
	// empty stdout — historically a real bug (CodeRabbit caught a
	// case where this slipped past the gate with "all succeeded"
	// framing). Distinct from AllFailed because "succeeded with
	// empty output" is an LLM-emitter pathology, not a crash.
	ZeroMergeableAllEmpty

	// ZeroMergeableMixed: a mix of failures (Error != nil) and
	// successes-with-empty-output. Pre-consolidation the runner
	// would print "all returned empty output" here, misleading
	// users into debugging the wrong problem.
	ZeroMergeableMixed
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.