metrics

Documentation

Overview

Package metrics provides a centralized observability pipeline for M31A sessions. It tracks tool execution metrics, LLM interaction metrics, and workflow phase metrics with thread-safe collection and JSON persistence.

Index

• func HashPrompt(prompt string) string
• func Save(sessionsDir, sessionID string, m *SessionMetrics) error
• type Collector
  • func NewCollector(sessionID, sessionsDir string, enabled bool) *Collector
  • func (c *Collector) Enabled() bool
  • func (c *Collector) Flush() error
  • func (c *Collector) RecordBisectOutcome(phase m31types.WorkflowPhase, success bool)
  • func (c *Collector) RecordBisectTrigger(phase m31types.WorkflowPhase)
  • func (c *Collector) RecordEditStrategy(strategy string)
  • func (c *Collector) RecordHealDuration(phase m31types.WorkflowPhase, durationMs int64)
  • func (c *Collector) RecordHealLoop(phase m31types.WorkflowPhase)
  • func (c *Collector) RecordHealOutcome(phase m31types.WorkflowPhase, success bool)
  • func (c *Collector) RecordHealTrigger(phase m31types.WorkflowPhase)
  • func (c *Collector) RecordLLMInteraction(phase m31types.WorkflowPhase, usage *m31types.Usage, cost float64)
  • func (c *Collector) RecordLLMInteractionWithPrompt(phase m31types.WorkflowPhase, usage *m31types.Usage, cost float64, ...)
  • func (c *Collector) RecordPhaseDuration(phase m31types.WorkflowPhase, durationMs int64, success bool)
  • func (c *Collector) RecordPhaseTransition(phase m31types.WorkflowPhase)
  • func (c *Collector) RecordToolCall(name string, success bool, durationMs int64)
  • func (c *Collector) Snapshot() *SessionMetrics
  • func (c *Collector) Stop()
• type EditStrategyMetric
• type LLMMetric
• type PhaseMetric
• type SessionMetrics
  • func Load(sessionsDir, sessionID string) (*SessionMetrics, error)
• type ToolMetric

Functions

func HashPrompt

func HashPrompt(prompt string) string

HashPrompt computes a short SHA-256 hash of a prompt string. Returns the first 16 hex characters for compactness.

func Save

func Save(sessionsDir, sessionID string, m *SessionMetrics) error

Save persists the given SessionMetrics to the session directory.

Types

type Collector

type Collector struct {
	// contains filtered or unexported fields
}

Collector provides thread-safe collection and persistence of session metrics. It is designed for a single-session lifetime: create once at session start, record events during the session, and flush/stop at session end.

func NewCollector

func NewCollector(sessionID, sessionsDir string, enabled bool) *Collector

NewCollector creates a new Collector scoped to the given session. If enabled is false, all recording methods become no-ops.

func (*Collector) Enabled

func (c *Collector) Enabled() bool

Enabled reports whether metrics collection is active.

func (*Collector) Flush

func (c *Collector) Flush() error

Flush persists the current metrics state to the session directory as JSON. Creates the directory and file if they do not exist.

func (*Collector) RecordBisectOutcome

func (c *Collector) RecordBisectOutcome(phase m31types.WorkflowPhase, success bool)

RecordBisectOutcome records whether a bisect heal attempt succeeded or failed.

func (*Collector) RecordBisectTrigger

func (c *Collector) RecordBisectTrigger(phase m31types.WorkflowPhase)

RecordBisectTrigger records a bisect trigger event.

func (*Collector) RecordEditStrategy

func (c *Collector) RecordEditStrategy(strategy string)

RecordEditStrategy records which edit replacement strategy was used.

func (*Collector) RecordHealDuration

func (c *Collector) RecordHealDuration(phase m31types.WorkflowPhase, durationMs int64)

RecordHealDuration adds heal duration to the phase metric.

func (*Collector) RecordHealLoop

func (c *Collector) RecordHealLoop(phase m31types.WorkflowPhase)

RecordHealLoop records a detected heal loop (same error repeating).

func (*Collector) RecordHealOutcome

func (c *Collector) RecordHealOutcome(phase m31types.WorkflowPhase, success bool)

RecordHealOutcome records whether a self-heal attempt succeeded or failed.

func (*Collector) RecordHealTrigger

func (c *Collector) RecordHealTrigger(phase m31types.WorkflowPhase)

RecordHealTrigger records a self-heal trigger event.

func (*Collector) RecordLLMInteraction

func (c *Collector) RecordLLMInteraction(phase m31types.WorkflowPhase, usage *m31types.Usage, cost float64)

RecordLLMInteraction records an LLM call's token usage and cost.

func (*Collector) RecordLLMInteractionWithPrompt

func (c *Collector) RecordLLMInteractionWithPrompt(phase m31types.WorkflowPhase, usage *m31types.Usage, cost float64, promptHash string, truncated bool)

RecordLLMInteractionWithPrompt records an LLM call with prompt hash and truncation info.

func (*Collector) RecordPhaseDuration

func (c *Collector) RecordPhaseDuration(phase m31types.WorkflowPhase, durationMs int64, success bool)

RecordPhaseDuration records the final duration of a completed phase.

func (*Collector) RecordPhaseTransition

func (c *Collector) RecordPhaseTransition(phase m31types.WorkflowPhase)

RecordPhaseTransition records a phase transition event.

func (*Collector) RecordToolCall

func (c *Collector) RecordToolCall(name string, success bool, durationMs int64)

RecordToolCall records a completed tool execution.

func (*Collector) Snapshot

func (c *Collector) Snapshot() *SessionMetrics

Snapshot returns a deep copy of the current metrics state. The returned copy is safe for concurrent reads without holding the lock.

func (*Collector) Stop

func (c *Collector) Stop()

Stop persists metrics and logs a summary. Intended for session cleanup.

type EditStrategyMetric

type EditStrategyMetric struct {
	Strategy string `json:"strategy"`
	Count    int64  `json:"count"`
}

EditStrategyMetric tracks which edit replacement strategy was used.

type LLMMetric

type LLMMetric struct {
	Phase            m31types.WorkflowPhase `json:"phase"`
	PromptTokens     int64                  `json:"prompt_tokens"`
	CompletionTokens int64                  `json:"completion_tokens"`
	TotalTokens      int64                  `json:"total_tokens"`
	Cost             float64                `json:"cost"`
	InteractionCount int64                  `json:"interaction_count"`
	PromptHash       string                 `json:"prompt_hash,omitempty"`
	TruncatedInput   bool                   `json:"truncated_input,omitempty"`
}

LLMMetric captures token usage and cost for an LLM interaction in a phase.

type PhaseMetric

type PhaseMetric struct {
	Phase              m31types.WorkflowPhase `json:"phase"`
	DurationMs         int64                  `json:"duration_ms"`
	TransitionCount    int64                  `json:"transition_count"`
	HealTriggerCount   int64                  `json:"heal_trigger_count"`
	HealSuccessCount   int64                  `json:"heal_success_count"`
	HealFailCount      int64                  `json:"heal_fail_count"`
	HealDurationMs     int64                  `json:"heal_duration_ms,omitempty"`
	HealLoopCount      int64                  `json:"heal_loop_count,omitempty"`
	BisectTriggerCount int64                  `json:"bisect_trigger_count"`
	BisectSuccessCount int64                  `json:"bisect_success_count"`
	BisectFailCount    int64                  `json:"bisect_fail_count"`
	Success            bool                   `json:"success"`
}

PhaseMetric captures workflow phase execution metrics.

type SessionMetrics

type SessionMetrics struct {
	SessionID      string               `json:"session_id"`
	StartedAt      time.Time            `json:"started_at"`
	UpdatedAt      time.Time            `json:"updated_at"`
	Tools          []ToolMetric         `json:"tools"`
	EditStrategies []EditStrategyMetric `json:"edit_strategies,omitempty"`
	LLMs           []LLMMetric          `json:"llms"`
	Phases         []PhaseMetric        `json:"phases"`
}

SessionMetrics is the top-level metrics container persisted per session.

func Load

func Load(sessionsDir, sessionID string) (*SessionMetrics, error)

Load reads a SessionMetrics from the session directory. Returns nil and a nil error if the file does not exist.

type ToolMetric

type ToolMetric struct {
	Name         string  `json:"name"`
	CallCount    int64   `json:"call_count"`
	SuccessCount int64   `json:"success_count"`
	FailCount    int64   `json:"fail_count"`
	TotalDurMs   int64   `json:"total_duration_ms"`
	AvgDurMs     float64 `json:"avg_duration_ms"`
}

ToolMetric captures the execution metrics for a single tool type.