thread

Documentation

Overview

Package thread assembles conversation threads that fit a model's context window, and detects when an agent has stopped making progress.

A thread is the list of messages sent in a single request. BuildThread trims that list to a token budget; the cycle heuristics in cycle.go and the streaming guard in guard.go decide whether the conversation has degenerated into a loop.

The behaviour here is pinned by testdata/corpus.json, a case corpus seeded from a production implementation. See corpus_test.go.

Index

• Constants
• func DescribeThreadCycle(messages []Message, options CycleOptions) string
• func HasRepeatedActivityTail(messages []Message, _ CycleOptions) bool
• func HasRepeatedResultRun(messages []Message, options CycleOptions) bool
• func HasRepeatedSuffix(messages []Message, options CycleOptions) bool
• func HasRepeatedTextRun(text string, options TextRunOptions) bool
• func IsThreadCyclic(messages []Message, options CycleOptions) bool
• func Opt[T any](v T) *T
• type BuildOptions
• type BuildResult
  • func BuildThread(options BuildOptions) (BuildResult, error)
• type CycleOptions
• type EstimateFunc
• type Guard
  • func NewGuard(options GuardOptions) *Guard
  • func (g *Guard) Push(text string) bool
  • func (g *Guard) Reason() *GuardReason
• type GuardOptions
• type GuardReason
• type InclusiveFunc
• type Message
  • func (m Message) Clone() Message
  • func (m Message) HasMeta() bool
  • func (m Message) Meta() (map[string]any, bool)
  • func (m Message) Text() string
  • func (m Message) Type() string
• type TextRunOptions
• type Usage

Constants

const DefaultMinResultRepetitions = 3

DefaultMinResultRepetitions is how many consecutive identical tool results HasRepeatedResultRun requires before reporting a loop.

Functions

func DescribeThreadCycle

func DescribeThreadCycle(messages []Message, options CycleOptions) string

DescribeThreadCycle names the first heuristic to fire, or returns the empty string when the thread is progressing.

Attribution matters more than it looks: without it a stuck run is only ever "stopped for looping", and the four heuristics fail in very different ways.

func HasRepeatedActivityTail

func HasRepeatedActivityTail(messages []Message, _ CycleOptions) bool

HasRepeatedActivityTail reports whether the thread ends in a run of tool calls that keeps re-treading the same small set of signatures.

It compresses the trailing contiguous activity messages to (type, name, input, output) and asks whether that set has collapsed. Unlike HasRepeatedSuffix it tolerates the text around the calls varying, which catches a model that narrates differently each time while doing the same thing.

A request whose repeats produce genuinely different outputs is spared: polling an endpoint until it changes is progress, not a loop.

func HasRepeatedResultRun

func HasRepeatedResultRun(messages []Message, options CycleOptions) bool

HasRepeatedResultRun reports whether the last few tool results are identical - the model issuing the same call and getting the same answer, learning nothing.

It walks only the tool-result stream, skipping every message in between. That is the gap it fills: HasRepeatedSuffix needs the surrounding messages to repeat byte-for-byte, and HasRepeatedActivityTail needs a contiguous run of activities, so a single interleaved reasoning message defeats both - and a reasoning model emits one between every tool call.

Three details are load-bearing:

• Arguments are part of the signature. A model issuing different calls that happen to return the same trivial result (several distinct shell commands each producing empty output) is making progress, not looping.
• Synthetic "_"-prefixed activities are skipped, so a notice injected to nudge the model out of a loop cannot break the run and mask the loop.
• Results still streaming are skipped: not a settled outcome to compare.

func HasRepeatedSuffix

func HasRepeatedSuffix(messages []Message, options CycleOptions) bool

HasRepeatedSuffix reports whether the thread ends in a repeating block of messages - the plainest form of a loop, where the model and its context cycle through the same exchange verbatim.

Messages are reduced to a fingerprint of (type, text, meta), and patterns are tried shortest-first because tight loops are both more common and more urgent than long ones.

It requires the repeats to be byte-identical and adjacent, which is its blind spot: one interleaved message - a reasoning turn between tool calls - breaks the run. HasRepeatedResultRun covers that case.

func HasRepeatedTextRun

func HasRepeatedTextRun(text string, options TextRunOptions) bool

HasRepeatedTextRun reports a runaway repetition inside a single block of text - a model stuck cycling the same handful of sentences.

Unlike the thread-level heuristics this works within one message, so it catches a turn that loops in its own reasoning without ever emitting a tool call - which the thread-level checks never see.

func IsThreadCyclic

func IsThreadCyclic(messages []Message, options CycleOptions) bool

IsThreadCyclic reports whether any heuristic considers the thread stuck.

func Opt

func Opt[T any](v T) *T

Opt returns a pointer to v, for populating the optional fields of CycleOptions, TextRunOptions and GuardOptions.

These are pointers rather than plain ints because zero is a meaningful value that clamps to the minimum, and a plain int cannot tell "unset" from "explicitly zero" - a caller passing MinRepetitions: 0 means "as low as possible", not "use the default of 2".

Types

type BuildOptions

type BuildOptions struct {
	// Messages is the full conversation, oldest first.
	Messages []Message

	// Estimate reports the token cost of a message. Only consulted for messages
	// that do not already carry a non-negative usage.
	Estimate EstimateFunc

	// MaxTokens is the budget the assembled thread must fit within.
	MaxTokens float64

	// MinMessages is the number of newest messages kept even when they exceed
	// MaxTokens. Without a floor, one oversized message starves the whole turn.
	MinMessages int

	// Inclusive, when set, keeps the message that straddles the budget boundary
	// rather than dropping it whole.
	//
	// InclusiveAll keeps it untrimmed; Inclusive trims it to the remaining
	// budget. Both stop the walk at that message.
	Inclusive InclusiveFunc

	// InclusiveAll keeps the straddling message without trimming it. Ignored
	// when Inclusive is set.
	InclusiveAll bool
}

BuildOptions configures BuildThread.

type BuildResult

type BuildResult struct {
	Messages []Message `json:"messages"`
	Usage    Usage     `json:"usage"`
}

BuildResult is an assembled thread and its total cost.

func BuildThread

func BuildThread(options BuildOptions) (BuildResult, error)

BuildThread assembles the largest suffix of the conversation that fits within MaxTokens.

It walks from the newest message backwards, which is what makes the fit exact: the budget is spent on the most recent context first, and the walk stops the moment the next message would not fit. The result is returned oldest-first.

The function is deliberately indifferent to message type and ordering - it only counts tokens. Keeping paired messages together (a tool-call request and its response, which a provider rejects if split) is the caller's job, and must be done before trimming rather than after.

type CycleOptions

type CycleOptions struct {
	// MinRepetitions is how many consecutive repeats of a pattern constitute a
	// cycle. At the default of 2, [A B A B] is cyclic; at 3 it is not. Clamped
	// to at least 1.
	MinRepetitions *int

	// MinPatternLength is the shortest pattern considered, in messages. The
	// default of 2 catches the common ask/answer/ask/answer loop. Clamped to at
	// least 1.
	MinPatternLength *int

	// MaxPatternLength bounds how long a repeating sequence may be. Unset means
	// len(messages)/MinRepetitions, the longest that could repeat enough times.
	MaxPatternLength *int

	// MinResultRepetitions overrides DefaultMinResultRepetitions. Clamped to at
	// least 2.
	MinResultRepetitions *int
}

CycleOptions tunes cycle detection. The zero value is the production default.

type EstimateFunc

type EstimateFunc func(Message) (Usage, error)

EstimateFunc reports the token usage of a message.

type Guard

type Guard struct {
	// contains filtered or unexported fields
}

Guard is an incremental runaway-repetition detector.

Where HasRepeatedTextRun re-scans a whole block, Guard maintains a rolling window of normalised words and a running count of every phrase in it. Each pushed chunk costs O(1) amortised, so it can run on every streamed token and latch within a few repeats - long before the heavier backstop would react.

func NewGuard

func NewGuard(options GuardOptions) *Guard

NewGuard creates a repetition guard.

func (*Guard) Push

func (g *Guard) Push(text string) bool

Push feeds streamed text into the guard and reports whether a runaway has been detected. Once tripped it stays tripped.

func (*Guard) Reason

func (g *Guard) Reason() *GuardReason

Reason returns why the guard tripped, or nil while it has not.

type GuardOptions

type GuardOptions struct {
	// Ngram is how many consecutive words make a tracked phrase. Longer phrases
	// recur by chance less often. Clamped to at least 2.
	Ngram *int

	// Window is how many words of history the guard keeps. Clamped to at least
	// Ngram.
	Window *int

	// MaxRepeats is how many times a phrase must recur within the window before
	// the guard latches. Clamped to at least 2.
	MaxRepeats *int

	// MaxUniqueRatio is the lexical-diversity ceiling. A stuck loop churns the
	// same few words; a progressing list keeps introducing new ones and must not
	// be cut off. Clamped to [0, 1].
	MaxUniqueRatio *float64

	// MinChars is the minimum output length before the guard may trip. Short
	// repetitive output is harmless - it ends on its own - and was the bulk of
	// the false positives this exists to prevent.
	MinChars *int
}

GuardOptions tunes the incremental repetition guard. The zero value is the production default.

type GuardReason

type GuardReason struct {
	// Phrase is the normalised form that recurred - stable for grouping.
	Phrase string `json:"phrase"`

	// Count is how many times it recurred within the window.
	Count int `json:"count"`

	// Text is the phrase as the model actually emitted it, suitable for showing
	// back to a user or to the model.
	Text string `json:"text"`

	// UniqueRatio and HapaxRatio are the window's diversity and novelty at the
	// trip - the signals that separate a stuck loop (both low) from a wrongly
	// flagged progressing list (both higher). Reported so a stop can be triaged
	// from telemetry rather than re-derived by hand.
	UniqueRatio float64 `json:"uniqueRatio"`
	HapaxRatio  float64 `json:"hapaxRatio"`
}

GuardReason explains a trip.

type InclusiveFunc

type InclusiveFunc func(message Message, trimTo float64) (trimmed Message, ok bool, err error)

InclusiveFunc trims a message to fit trimTo tokens.

Returning ok=false drops the message instead, ending the thread there. The returned message is expected to carry an updated usage; BuildThread clamps it to trimTo regardless, so a trimmer that overshoots cannot blow the budget.

type Message

type Message map[string]any

Message is one entry in a conversation thread.

It is map-backed rather than a struct because messages carry provider- and caller-specific fields that must survive a round trip untouched: BuildThread hands the whole message to a token estimator and returns it with a usage annotation, and dropping an unrecognised field on the way through would silently corrupt the caller's data.

func (Message) Clone

func (m Message) Clone() Message

Clone returns a shallow copy, so annotating a message does not mutate the caller's.

func (Message) HasMeta

func (m Message) HasMeta() bool

HasMeta reports whether the message carries a meta field at all, including an explicit null. The distinction matters: a fingerprint treats "no meta" and "meta present but empty" as different messages.

func (Message) Meta

func (m Message) Meta() (map[string]any, bool)

Meta returns the message metadata, or nil when absent.

func (Message) Text

func (m Message) Text() string

Text returns the message text, or the empty string when absent or not a string.

func (Message) Type

func (m Message) Type() string

Type returns the message type ("user", "bot", "activity", "reasoning", ...), or the empty string when absent or not a string.

type TextRunOptions

type TextRunOptions struct {
	// MinUnits is the number of trailing sentence-like units required before a
	// runaway is even considered. Short repetitive snippets end on their own.
	// Clamped to at least 2.
	MinUnits *int

	// Window bounds how many trailing units are inspected, so a degenerate tail
	// is still caught after a long healthy prefix. Clamped to at least MinUnits.
	Window *int

	// MaxUniqueRatio is the unique-to-inspected ratio at or below which the text
	// counts as a runaway. Healthy prose almost never repeats whole normalised
	// sentences that densely.
	MaxUniqueRatio *float64
}

TextRunOptions tunes HasRepeatedTextRun. The zero value is the production default.

type Usage

type Usage struct {
	Tokens float64 `json:"tokens"`
}

Usage is the token cost of a message or a thread.