report

Documentation

Overview

Package report turns the telemetry-only ledger into per-task quality stats (no model calls). It is the observational complement to package eval's labeled gold-set metrics — it reports raw production signals, not AURC/AUDC.

Index

• func SortedTasks(m map[string]TaskStat) []string
• func Summarize(entries []ledger.Entry, marginThreshold float64) map[string]TaskStat
• type TaskStat

Functions

func SortedTasks

func SortedTasks(m map[string]TaskStat) []string

SortedTasks returns stable task keys.

func Summarize

func Summarize(entries []ledger.Entry, marginThreshold float64) map[string]TaskStat

Summarize aggregates ledger entries per task. marginThreshold flags the "barely confident" accept band (the Phase-5 two-pass target). Cache hits are skipped (they carry no fresh model signal).

Types

type TaskStat

type TaskStat struct {
	Task               string  `json:"task"`
	N                  int     `json:"n"`
	Deferred           int     `json:"deferred"`
	EscalationResolved int     `json:"escalation_resolved"` // a larger tier produced an accepted answer
	ReasoningReclaimed int     `json:"reasoning_reclaimed"` // completed via the terminal reasoning tier
	LabeledGrounded    int     `json:"labeled_grounded"`
	Grounded           int     `json:"grounded"`
	LowMarginAccepts   int     `json:"low_margin_accepts"` // accepted but margin in (0, threshold)
	DeferRate          float64 `json:"defer_rate"`
	GroundedRate       float64 `json:"grounded_rate"`
	TokensOut          int     `json:"tokens_out"`
}