Documentation
¶
Overview ¶
Package catrace implements finite-state Markov models for autonomous-agent networks.
Core concept ¶
The central operation is the trace of a Markov kernel: given a Markov chain on a large state space N, the trace onto a subset A ⊆ N is the induced chain you would observe if you watched only the states in A and integrated out all hidden excursions through the complement. Concretely, if P is partitioned as
P = [ a b ]
[ d c ]
with rows and columns of a indexed by A, then the trace kernel is
P_A = a + b (I - c)⁻¹ d
The term b (I-c)⁻¹ d captures the contribution of all paths that leave A, wander through the hidden states, and eventually return. This is not the same as deleting the hidden rows and columns.
A key property: the stationary distribution of the trace kernel equals the stationary distribution of the parent kernel restricted and renormalized to A.
Agent model ¶
An Agent is defined by three rectangular row-stochastic maps that form a closed loop:
D : X → G (decision: experience states → action states) A : G → W (effect: action states → world states) P : W → X (perception: world states → experience states)
Composing these in different orders gives square kernels on each space:
QualiaKernel Q = D·A·P on X (experience dynamics) StrategyKernel S = A·P·D on G (action dynamics) WorldKernel W = P·D·A on W (world dynamics)
The three kernels are cyclic permutations of the same product and share eigenvalues, providing three perspectives on the same closed-loop system.
Kernel operations ¶
All analysis methods are defined on Kernel, a square row-stochastic matrix with named states:
- Kernel.Trace — compute the trace onto a subset of states
- Kernel.Stationary — stationary distribution via power iteration (ergodic chains only)
- Kernel.EntropyRate — entropy rate in a specified log base
- Kernel.MeanFirstPassage — expected steps from state i to state j
- Kernel.CommuteTime — mean first-passage time i→j plus j→i
- Kernel.Classes — communicating class decomposition (recurrent/transient, period)
- Kernel.Sample — forward sample one step given a starting state
- Kernel.LeftAction — evolve a distribution one step: π' = π·P
Sampling and estimation ¶
- EstimateKernelFromSequence — empirical transition counts with pseudocount smoothing
- SampleTraceFromSequence — filter a trajectory to observed states
- WindowedTraceEstimates — sliding-window kernel estimates from a trajectory
Index ¶
- func CanonicalSubset(subset []int) []int
- func RestrictDistribution(pi []float64, subset []int, tol float64) ([]float64, error)
- func SampleTraceFromSequence(seq []int, subset map[int]bool) []int
- type Agent
- type ClassDecomposition
- type Kernel
- func (k *Kernel) Classes(tol float64) (*ClassDecomposition, error)
- func (k *Kernel) Clone() *Kernel
- func (k *Kernel) CommuteTime(i, j int) (float64, error)
- func (k *Kernel) EntropyRate(base float64) (float64, error)
- func (k *Kernel) IsTraceOf(parent *Kernel, subset []int, tol float64) (bool, error)
- func (k *Kernel) LeftAction(dist []float64) ([]float64, error)
- func (k *Kernel) MeanFirstPassage(i, j int) (float64, error)
- func (k *Kernel) Multiply(other *Kernel) (*Kernel, error)
- func (k *Kernel) NormalizeRows(tol float64) error
- func (k *Kernel) NumStates() int
- func (k *Kernel) PersonalizedPageRank(restart []float64, alpha, tol float64, maxIter int) ([]float64, error)
- func (k *Kernel) Sample(rowIdx int, rng *rand.Rand) (int, error)
- func (k *Kernel) Stationary(tol float64, maxIter int) ([]float64, error)
- func (k *Kernel) StationaryFrom(start []float64, tol float64, maxIter int) ([]float64, error)
- func (k *Kernel) TeleportingKernel(restart []float64, alpha float64) (*Kernel, error)
- func (k *Kernel) ToHTML(opts *VisualiseOptions) ([]byte, error)
- func (k *Kernel) Trace(subset []int, tol float64) (*Kernel, error)
- func (k *Kernel) Validate(tol float64) error
- type KernelEstimate
- type RectKernel
- type VisualiseOptions
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CanonicalSubset ¶
CanonicalSubset returns sorted unique subset indices.
func RestrictDistribution ¶
RestrictDistribution normalizes the restriction of a distribution to subset.
Types ¶
type Agent ¶
type Agent struct {
D *mat.Dense // decision kernel X → G
A *mat.Dense // effect kernel G → W
P *mat.Dense // perception kernel W → X
XNames []string // experience state labels
GNames []string // action state labels
WNames []string // world state labels
}
Agent represents the three stochastic maps of the finite-state agent model.
The maps form a closed loop over three spaces:
D : X → G (decision: experience → action) A : G → W (effect: action → world) P : W → X (perception: world → experience)
func (*Agent) QualiaKernel ¶
QualiaKernel computes Q = D·A·P, a square kernel on the experience space X.
Example ¶
package main
import (
"fmt"
"github.com/stephen-mcelhose/catrace"
"gonum.org/v1/gonum/mat"
)
func main() {
D := mat.NewDense(2, 2, []float64{
0.9, 0.1,
0.2, 0.8,
})
A := mat.NewDense(2, 2, []float64{
0.7, 0.3,
0.4, 0.6,
})
P := mat.NewDense(2, 2, []float64{
0.8, 0.2,
0.1, 0.9,
})
agent := &catrace.Agent{D: D, A: A, P: P}
Q, _ := agent.QualiaKernel()
fmt.Printf("%.3f %.3f\n", Q.P.At(0, 0), Q.P.At(0, 1))
fmt.Printf("%.3f %.3f\n", Q.P.At(1, 0), Q.P.At(1, 1))
}
Output: 0.569 0.431 0.422 0.578
func (*Agent) StrategyKernel ¶
StrategyKernel computes S = A·P·D, a square kernel on the action space G.
func (*Agent) Validate ¶
Validate checks that D, A, P are non-nil, dimensionally consistent, and row-stochastic.
func (*Agent) WorldKernel ¶
WorldKernel computes W = P·D·A, a square kernel on the world space W.
type ClassDecomposition ¶
type ClassDecomposition struct {
SCCs [][]int // all strongly connected components, each sorted
Recurrent [][]int // closed (recurrent) components
Transient []int // transient states, sorted
Periods map[int]int // period of each recurrent class keyed by index into SCCs
}
ClassDecomposition summarizes the communicating class structure of a Markov kernel.
SCCs holds all strongly connected components identified by Kosaraju's algorithm. Recurrent holds the closed SCCs; once entered, these classes cannot be escaped. Transient holds all states that eventually leave their SCC with probability 1. Periods maps each recurrent SCC index (into SCCs) to its period.
type Kernel ¶
Kernel represents a row-stochastic square Markov kernel. StateNames labels both the row and column indices.
func NewKernel ¶
NewKernel constructs a validated square row-stochastic kernel. names may be nil, in which case default state labels are assigned.
func NewRandomWalkKernel ¶
NewRandomWalkKernel constructs a Kernel from a weighted adjacency matrix.
adj[i][j] is the weight of the edge from node i to node j. Zero means no edge. For undirected graphs the matrix should be symmetric.
The transition probability is:
P(i, j) = adj[i][j] / sum_k adj[i][k]
In plain terms: from node i, move to each neighbor with probability proportional to the edge weight. This is the standard random walk on a graph — the graph and the Markov kernel are two views of the same object.
For undirected graphs the stationary distribution has a closed form:
π(i) = degree(i) / sum_k degree(k)
meaning high-degree nodes are visited most often. This can be verified by calling Stationary on the returned kernel.
Returns an error if any row of adj is all-zero (an isolated node with no outgoing edges, which would make the row of P undefined).
Example ¶
package main
import (
"fmt"
"github.com/stephen-mcelhose/catrace"
"gonum.org/v1/gonum/mat"
)
func main() {
// Symmetric adjacency matrix (1 = edge present, 0 = no edge).
adj := mat.NewDense(4, 4, []float64{
// A B C D
0, 1, 1, 1, // A connects to B, C, D
1, 0, 1, 0, // B connects to A, C
1, 1, 0, 0, // C connects to A, B
1, 0, 0, 0, // D connects to A only
})
k, err := catrace.NewRandomWalkKernel(adj, []string{"A", "B", "C", "D"})
if err != nil {
panic(err)
}
// Each row of the kernel is the adjacency row divided by the node's degree.
fmt.Printf("P(A→B) = %.3f\n", k.P.At(0, 1)) // degree(A)=3, so 1/3
fmt.Printf("P(D→A) = %.3f\n", k.P.At(3, 0)) // degree(D)=1, so 1/1
// Stationary distribution: π(i) = degree(i) / total degree.
pi, _ := k.Stationary(1e-12, 5000)
fmt.Printf("π(A) = %.3f\n", pi[0])
fmt.Printf("π(B) = %.3f\n", pi[1])
fmt.Printf("π(C) = %.3f\n", pi[2])
fmt.Printf("π(D) = %.3f\n", pi[3])
}
Output: P(A→B) = 0.333 P(D→A) = 1.000 π(A) = 0.375 π(B) = 0.250 π(C) = 0.250 π(D) = 0.125
func NewTeleportingKernelFromAdj ¶
func NewTeleportingKernelFromAdj(adj *mat.Dense, restart []float64, alpha float64, names []string) (*Kernel, error)
NewTeleportingKernelFromAdj constructs a teleporting Markov kernel directly from a raw weighted adjacency matrix, combining row-normalisation and teleportation in a single step:
T[i][j] = α·v[j] + (1−α)·(adj[i][j] / rowsum[i]) if rowsum[i] > 0 T[i][j] = v[j] if rowsum[i] = 0
Sink nodes (rows that sum to zero — pages with no outgoing links) collapse entirely to the restart distribution v. No artificial uniform edges are inserted; the teleportation term carries them instead. This is semantically cleaner than pre-filling sink rows with 1/n before calling NewRandomWalkKernel.
The returned kernel is the same object as TeleportingKernel would produce on a pre-normalised input, but avoids the two-step NewRandomWalkKernel → TeleportingKernel pipeline that errors on sink nodes.
alpha must be in [0, 1]. restart must be a valid probability vector (non- negative, sums to 1 within 1e-9). adj must be square with non-negative entries.
func (*Kernel) Classes ¶
func (k *Kernel) Classes(tol float64) (*ClassDecomposition, error)
Classes decomposes k into communicating classes using Kosaraju's algorithm. An edge i→j is included when k.P[i,j] exceeds tol. Returns the full SCC decomposition, recurrent and transient states, and the period of each recurrent class.
func (*Kernel) CommuteTime ¶
CommuteTime returns m(i,j)+m(j,i).
func (*Kernel) EntropyRate ¶
EntropyRate returns the entropy rate of the chain in the specified log base. For base 2 the unit is bits per step.
func (*Kernel) IsTraceOf ¶
IsTraceOf checks whether k matches the trace of parent on subset within tol.
func (*Kernel) LeftAction ¶
LeftAction evolves dist one step by computing π·P, returning the next distribution. dist must have length equal to NumStates.
func (*Kernel) MeanFirstPassage ¶
MeanFirstPassage returns the expected number of steps to hit target j starting from i.
func (*Kernel) Multiply ¶
Multiply returns the matrix product k·other as a new Kernel. Both operands must be square and of the same dimension.
func (*Kernel) NormalizeRows ¶
NormalizeRows rescales each row to sum to 1, clamping near-zero negatives to 0. Returns an error if any row contains a significantly negative entry or a near-zero sum.
func (*Kernel) PersonalizedPageRank ¶
func (k *Kernel) PersonalizedPageRank(restart []float64, alpha, tol float64, maxIter int) ([]float64, error)
PersonalizedPageRank computes the Personalized PageRank vector for the given restart distribution and teleportation weight alpha ∈ [0, 1].
Each iteration step blends the propagated distribution back toward restart:
x ← α·restart + (1−α)·(x·P)
The fixed point satisfies x* = α·restart + (1−α)·x*·P, which is the stationary distribution of the teleporting chain α·restart·𝟙ᵀ + (1−α)·P.
For alpha ∈ (0, 1] convergence is guaranteed regardless of chain structure, because the teleporting chain is strongly connected. For alpha = 0 this reduces to plain power iteration from restart (equivalent to StationaryFrom(restart, tol, maxIter)).
Example ¶
ExampleKernel_PersonalizedPageRank shows how seeding on a leaf node concentrates PPR mass near that node compared to the global stationary.
The same 4-node graph (A–B–C–D) is used. D is a structural dead-end (degree 1). Seeding entirely on D with alpha=0.15 biases the PPR vector toward D and its immediate neighbour A, compared with the global π which assigns D the least mass.
package main
import (
"fmt"
"github.com/stephen-mcelhose/catrace"
"gonum.org/v1/gonum/mat"
)
func main() {
adj := mat.NewDense(4, 4, []float64{
0, 1, 1, 1, // A connects to B, C, D
1, 0, 1, 0, // B connects to A, C
1, 1, 0, 0, // C connects to A, B
1, 0, 0, 0, // D connects to A only
})
k, _ := catrace.NewRandomWalkKernel(adj, []string{"A", "B", "C", "D"})
// Seed entirely on D (the leaf). alpha=0.15 is the standard PageRank damping.
ppr, _ := k.PersonalizedPageRank([]float64{0, 0, 0, 1}, 0.15, 1e-12, 5000)
fmt.Printf("ppr(A) = %.3f\n", ppr[0])
fmt.Printf("ppr(B) = %.3f\n", ppr[1])
fmt.Printf("ppr(C) = %.3f\n", ppr[2])
fmt.Printf("ppr(D) = %.3f\n", ppr[3])
}
Output:
func (*Kernel) Sample ¶
Sample draws one transition from row rowIdx using inverse CDF sampling. If rng is nil, a new source seeded from the current time is used.
func (*Kernel) Stationary ¶
Stationary computes a stationary distribution pi such that pi P = pi. It uses power iteration on an initial uniform distribution.
This method requires the chain to be ergodic (irreducible and aperiodic). If the chain has multiple recurrent classes, power iteration from a uniform start will not converge to a unique stationary distribution and the method will return an error after maxIter iterations. Use Classes to inspect the chain structure before calling Stationary on reducible chains.
func (*Kernel) StationaryFrom ¶
StationaryFrom runs power iteration initialised from start rather than the uniform distribution. start must be a valid probability distribution over k.NumStates() states: all entries ≥ 0 (entries in (−tol, 0) are clamped to zero) and the sum within tol of 1.
Use Stationary for the standard uniform-start case. Use StationaryFrom when the initial distribution carries semantic meaning, or to verify that convergence is distribution-independent (ergodicity check).
Note: for non-ergodic chains convergence is not guaranteed. Use Classes to inspect chain structure before calling StationaryFrom on reducible chains.
Example ¶
ExampleNewRandomWalkKernel demonstrates how a graph becomes a Markov kernel.
Consider this small network:
D – A – B
| /
C
A is the hub — it connects to B, C, and D. D is a dead end — it connects only to A.
The random walk assigns each node a transition probability proportional to its edge weights. From A (degree 3), you go to each neighbor with prob 1/3. From D (degree 1), you always go to A.
The stationary distribution equals the normalized degree of each node:
degree(A)=3, degree(B)=2, degree(C)=2, degree(D)=1 → total=8 π = [3/8, 2/8, 2/8, 1/8] = [0.375, 0.250, 0.250, 0.125]
A is visited most often. D is visited least — it is a structural dead end. ExampleKernel_StationaryFrom shows that a skewed starting distribution converges to the same stationary distribution as the uniform start.
The same 4-node graph as ExampleNewRandomWalkKernel (A–B–C–D) is used. Starting from [0.7, 0.1, 0.1, 0.1] (mass concentrated on A) converges to the degree-proportional stationary distribution π = [3/8, 2/8, 2/8, 1/8].
package main
import (
"fmt"
"github.com/stephen-mcelhose/catrace"
"gonum.org/v1/gonum/mat"
)
func main() {
adj := mat.NewDense(4, 4, []float64{
0, 1, 1, 1, // A connects to B, C, D
1, 0, 1, 0, // B connects to A, C
1, 1, 0, 0, // C connects to A, B
1, 0, 0, 0, // D connects to A only
})
k, _ := catrace.NewRandomWalkKernel(adj, []string{"A", "B", "C", "D"})
// Skewed start: most mass on A.
pi, _ := k.StationaryFrom([]float64{0.7, 0.1, 0.1, 0.1}, 1e-12, 5000)
fmt.Printf("π(A) = %.3f\n", pi[0])
fmt.Printf("π(B) = %.3f\n", pi[1])
fmt.Printf("π(C) = %.3f\n", pi[2])
fmt.Printf("π(D) = %.3f\n", pi[3])
}
Output: π(A) = 0.375 π(B) = 0.250 π(C) = 0.250 π(D) = 0.125
func (*Kernel) TeleportingKernel ¶
TeleportingKernel constructs the teleporting Markov chain
T = α·restart·𝟙ᵀ + (1−α)·P
whose stationary distribution equals the PersonalizedPageRank vector for the same restart and alpha. The teleporting chain is strongly connected for any alpha ∈ (0, 1] and any stochastic restart, so its stationary distribution exists and is unique.
For alpha = 1 every row of T equals restart (the chain ignores P entirely). For alpha = 0 T equals P unchanged.
TeleportingKernel is useful for visualisation: call ToHTML on the returned kernel and nodes will be sized by PPR score. Use MinEdge in VisualiseOptions to suppress low-weight teleportation arcs.
func (*Kernel) ToHTML ¶
func (k *Kernel) ToHTML(opts *VisualiseOptions) ([]byte, error)
ToHTML generates a self-contained HTML file that renders the kernel as an interactive force-directed graph, similar to Obsidian's graph view.
Node radius is proportional to the stationary distribution π. Edge stroke width and opacity are proportional to the transition probability. Directed edges are drawn as curved arcs with arrowheads so that A→B and B→A are visually distinct. Self-loops are rendered as small arcs above the node. Nodes are coloured by communicating class; transient nodes are grey.
The returned bytes are a self-contained UTF-8 HTML document. Write them to a .html file and open in any modern browser. D3.js v7 is loaded from CDN; an internet connection is required on first open (after that, browser cache handles it).
Example:
html, err := kernel.ToHTML(&catrace.VisualiseOptions{Title: "Q kernel"})
if err != nil { log.Fatal(err) }
os.WriteFile("graph.html", html, 0644)
func (*Kernel) Trace ¶
Trace computes the induced trace kernel on a subset A of states. If P is partitioned as
P = [ a b ]
[ d c ]
with rows/columns of a indexed by A, then the trace is
P_A = a + b (I - c)^(-1) d,
provided the excursion operator is well-defined.
Example ¶
package main
import (
"fmt"
"github.com/stephen-mcelhose/catrace"
"gonum.org/v1/gonum/mat"
)
func main() {
k, _ := catrace.NewKernel(mat.NewDense(3, 3, []float64{
0.6, 0.3, 0.1,
0.2, 0.6, 0.2,
0.4, 0.1, 0.5,
}), nil)
tr, _ := k.Trace([]int{0, 1}, 1e-12)
fmt.Printf("%.3f %.3f\n", tr.P.At(0, 0), tr.P.At(0, 1))
fmt.Printf("%.3f %.3f\n", tr.P.At(1, 0), tr.P.At(1, 1))
}
Output: 0.680 0.320 0.360 0.640
type KernelEstimate ¶
type KernelEstimate struct {
Counts *mat.Dense // observed transition counts
Kernel *Kernel // estimated kernel; nil if any row was unobserved
RowsObserved []bool // true for each row that appeared at least once as a source state
}
KernelEstimate holds the result of estimating a transition kernel from a state sequence. Kernel is nil when one or more rows had no observed outgoing transitions.
func EstimateKernelFromSequence ¶
func EstimateKernelFromSequence(seq []int, nStates int, pseudocount float64) (*KernelEstimate, error)
EstimateKernelFromSequence builds a transition frequency estimate from seq. pseudocount adds a Laplace smoothing term to each cell before normalizing. Kernel is nil if any state had no outgoing transitions in seq.
func WindowedTraceEstimates ¶
func WindowedTraceEstimates(seq []int, subset map[int]bool, windowSize, step int, pseudocount float64) ([]*KernelEstimate, error)
WindowedTraceEstimates partitions seq into overlapping windows of windowSize steps, each offset by step, and returns a kernel estimate per window restricted to subset. Useful for detecting drift in transition probabilities over time.
type RectKernel ¶
RectKernel represents a row-stochastic rectangular kernel. RowNames and ColNames label the row and column spaces respectively.
func NewRectKernel ¶
func NewRectKernel(p *mat.Dense, rowNames, colNames []string) (*RectKernel, error)
NewRectKernel constructs a validated rectangular row-stochastic kernel. rowNames and colNames may be nil, in which case default labels are assigned.
func (*RectKernel) Validate ¶
func (k *RectKernel) Validate(tol float64) error
Validate checks that every row is non-negative and sums to 1 within tol.
type VisualiseOptions ¶
type VisualiseOptions struct {
// Title is shown in the browser tab and as the graph heading.
// Defaults to the kernel's first state name or "Markov Kernel".
Title string
// MinEdge omits directed edges whose transition probability is below this
// threshold. Reduces visual clutter on dense kernels. Default: 0.01.
MinEdge float64
// Width and Height of the SVG canvas in pixels. Defaults: 960 × 680.
Width int
Height int
// StationaryTol and StationaryMaxIter are passed to Stationary.
// If stationary computation fails (e.g. reducible chain), all nodes
// are rendered at equal size.
StationaryTol float64
StationaryMaxIter int
// NodeMass, if non-nil, is used directly as the node-size distribution
// instead of computing the stationary distribution of this kernel.
// Must have length equal to the number of states. Use this to visualise
// a base link graph (no teleportation edges) with nodes sized by a
// pre-computed PPR vector from a separate teleporting kernel.
NodeMass []float64
}
VisualiseOptions controls the HTML graph output.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
prompt_chaining
command
|
|
|
self_healing_nodes
command
|
|
|
simple_agent
command
|
|
|
trace_analysis
command
|
|
|
validator_repair
command
|