compact

Documentation

Index

• Constants
• func EstimateTokens(messages []llm.Message) int
• func SafeSplitPoint(messages []llm.Message, desired int) int
• func TriggerTokensForContextWindow(contextWindowTokens int) int
• type Compactor
• type NoCompaction
  • func (NoCompaction) Compact(_ context.Context, messages []llm.Message) ([]llm.Message, error)
• type Summarizer
  • func NewSummarizer(p llm.Provider, model string) Summarizer
  • func (s Summarizer) Compact(ctx context.Context, messages []llm.Message) ([]llm.Message, error)

Constants

const (
	DefaultContextWindowTokens = 200_000
	DefaultKeepRecent          = 20
)

Compaction defaults. The context window default is intentionally conservative for modern coding models; users on larger-context models can override it in config without requiring a model catalog.

Functions

func EstimateTokens

func EstimateTokens(messages []llm.Message) int

EstimateTokens approximates the token count of a transcript at ~4 characters per token. It is deliberately rough: it only needs to be in the right order of magnitude to decide when to compact.

func SafeSplitPoint

func SafeSplitPoint(messages []llm.Message, desired int) int

SafeSplitPoint walks backward from desired until it finds a fresh user turn. Splitting there avoids keeping a tool_result without its preceding tool_use.

func TriggerTokensForContextWindow

func TriggerTokensForContextWindow(contextWindowTokens int) int

TriggerTokensForContextWindow returns the estimated transcript size at which compaction should run for a model context window.

Types

type Compactor

type Compactor interface {
	Compact(ctx context.Context, messages []llm.Message) ([]llm.Message, error)
}

type NoCompaction

type NoCompaction struct{}

func (NoCompaction) Compact

func (NoCompaction) Compact(_ context.Context, messages []llm.Message) ([]llm.Message, error)

type Summarizer

type Summarizer struct {
	Provider llm.Provider
	Model    string
	// TriggerTokens is the estimated transcript size at which compaction runs
	// (default TriggerTokensForContextWindow(DefaultContextWindowTokens)).
	TriggerTokens int
	// KeepRecent is the number of trailing messages preserved verbatim
	// (default DefaultKeepRecent).
	KeepRecent int
}

Summarizer compacts long transcripts by asking the provider to summarize the oldest turns, replacing them with a single user message that carries the summary. The most recent messages are kept verbatim, cut at a safe split point so no tool_result loses its tool_use.

func NewSummarizer

func NewSummarizer(p llm.Provider, model string) Summarizer

NewSummarizer builds a Summarizer with default thresholds.

func (Summarizer) Compact

func (s Summarizer) Compact(ctx context.Context, messages []llm.Message) ([]llm.Message, error)