conversation

package
v0.180.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 24, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Index

Constants

View Source 
const CancelledToolResponseContent = "[Tool execution cancelled by user before completion]"

CancelledToolResponseContent is the sentinel body used for synthetic tool responses inserted by EnsureToolCallsClosed. The wording is surfaced to the model in the next turn and to the user via the conversation view, so it must be self-explanatory and stable.

Variables

This section is empty.

Functions

func BuildAgentMessagesFromEntries added in v0.178.1 

func BuildAgentMessagesFromEntries(entries []convdomain.ConversationEntry) []sdk.Message

BuildAgentMessagesFromEntries converts conversation entries into the flat slice of SDK messages sent to the model, dropping the three UI-only classes: plan entries, user-typed `!command` entries, and pending-approval placeholders. Plan and bash entries carry no reasoning_content, which thinking-mode providers reject with HTTP 400; a placeholder between an assistant tool_calls message and its tool response breaks provider adjacency.

func NewConversationOptimizer 

func NewConversationOptimizer(config OptimizerConfig) convdomain.ConversationOptimizer

NewConversationOptimizer creates a new conversation optimizer with configuration

func NewPricingService 

func NewPricingService(cfg *config.PricingConfig) convdomain.PricingService

NewPricingService creates a new pricing service instance.

Types

type ConversationOptimizer 

type ConversationOptimizer struct {
	// contains filtered or unexported fields
}

ConversationOptimizer provides methods to optimize conversation history for token efficiency

func (*ConversationOptimizer) GenerateLLMSummary 

func (co *ConversationOptimizer) GenerateLLMSummary(messages []sdk.Message, model string) (string, error)

GenerateLLMSummary creates a concise summary of conversation messages using an LLM. It uses the SDK client to generate an intelligent summary focused on key tasks, decisions, critical context, and next steps. The summary is limited to 2-3 sentences.

func (*ConversationOptimizer) OptimizeMessages 

func (co *ConversationOptimizer) OptimizeMessages(messages []sdk.Message, model string, force bool) []sdk.Message

OptimizeMessages reduces token usage by intelligently managing conversation history with LLM summarization. When force is false it is a no-op unless the model has a configured context window and the conversation has crossed auto_at percent of it; when force is true it always compacts (manual /compact and session rollover both rely on this).

type ConversationTitleGenerator 

type ConversationTitleGenerator struct {
	// contains filtered or unexported fields
}

ConversationTitleGenerator generates titles for conversations using AI

func NewConversationTitleGenerator 

func NewConversationTitleGenerator(client sdk.Client, storage storage.ConversationStorage, config *config.Config) *ConversationTitleGenerator

NewConversationTitleGenerator creates a new conversation title generator

func (*ConversationTitleGenerator) GenerateTitleForConversation 

func (g *ConversationTitleGenerator) GenerateTitleForConversation(ctx context.Context, conversationID string) error

GenerateTitleForConversation generates a title for a specific conversation

func (*ConversationTitleGenerator) InvalidateTitle 

func (g *ConversationTitleGenerator) InvalidateTitle(ctx context.Context, conversationID string) error

InvalidateTitle marks a conversation title as needing regeneration

func (*ConversationTitleGenerator) ProcessPendingTitles 

func (g *ConversationTitleGenerator) ProcessPendingTitles(ctx context.Context) error

ProcessPendingTitles processes a batch of conversations that need title generation

type EventBridge 

type EventBridge struct {
	// contains filtered or unexported fields
}

EventBridge multicasts chat events to multiple subscribers without modifying the existing event flow to the terminal UI.

func NewEventBridge 

func NewEventBridge() *EventBridge

NewEventBridge creates a new event bridge with a circular buffer

func (*EventBridge) Close 

func (eb *EventBridge) Close()

Close closes all subscriber channels and clears the subscribers list

func (*EventBridge) Publish 

func (eb *EventBridge) Publish(event agentdomain.ChatEvent)

Publish broadcasts an event to every subscriber. Delivery is non-blocking so one slow subscriber can never stall the bus for the others.

func (*EventBridge) Subscribe 

func (eb *EventBridge) Subscribe() chan agentdomain.ChatEvent

Subscribe registers a subscriber and replays the recent-event ring buffer so backfill-less subscribers catch up on connect.

func (*EventBridge) SubscribeFuture 

func (eb *EventBridge) SubscribeFuture() chan agentdomain.ChatEvent

SubscribeFuture is Subscribe without the ring-buffer replay, for subscribers that backfill history another way (the extension bridge's conversation snapshot) where a replay would double-render the last turn.

func (*EventBridge) Tap 

func (eb *EventBridge) Tap(input <-chan agentdomain.ChatEvent) <-chan agentdomain.ChatEvent

Tap intercepts an event stream and multicasts it to all subscribers Returns a new channel that mirrors the input channel for the terminal UI

func (*EventBridge) Unsubscribe 

func (eb *EventBridge) Unsubscribe(ch chan agentdomain.ChatEvent)

Unsubscribe removes a subscriber and closes its channel

type HTTPModelService 

type HTTPModelService struct {
	// contains filtered or unexported fields
}

HTTPModelService implements ModelService using SDK client

func NewHTTPModelService 

func NewHTTPModelService(client sdk.Client) *HTTPModelService

NewHTTPModelService creates a new HTTP-based model service with pre-configured client

func (*HTTPModelService) GetCurrentModel 

func (s *HTTPModelService) GetCurrentModel() string

func (*HTTPModelService) IsModelAvailable 

func (s *HTTPModelService) IsModelAvailable(modelID string) bool

func (*HTTPModelService) ListModels 

func (s *HTTPModelService) ListModels(ctx context.Context) ([]string, error)

func (*HTTPModelService) SelectModel 

func (s *HTTPModelService) SelectModel(modelID string) error

func (*HTTPModelService) ValidateModel 

func (s *HTTPModelService) ValidateModel(modelID string) error

type InMemoryConversationRepository 

type InMemoryConversationRepository struct {
	// contains filtered or unexported fields
}

InMemoryConversationRepository implements ConversationRepository using in-memory storage

func NewInMemoryConversationRepository 

func NewInMemoryConversationRepository(formatterService ToolFormatter, pricingService convdomain.PricingService) *InMemoryConversationRepository

NewInMemoryConversationRepository creates a new in-memory conversation repository

func (*InMemoryConversationRepository) AddCachedTokens 

func (r *InMemoryConversationRepository) AddCachedTokens(tokens int)

AddCachedTokens accumulates provider-reported cached prompt tokens (usage.prompt_tokens_details.cached_tokens) into the session totals.

func (*InMemoryConversationRepository) AddMessage 

func (r *InMemoryConversationRepository) AddMessage(msg convdomain.ConversationEntry) error

func (*InMemoryConversationRepository) AddPendingToolCall 

func (r *InMemoryConversationRepository) AddPendingToolCall(toolCall sdk.ChatCompletionMessageToolCall, responseChan chan agentdomain.ApprovalAction) error

AddPendingToolCall adds a pending tool call entry that requires approval

func (*InMemoryConversationRepository) AddTokenUsage 

func (r *InMemoryConversationRepository) AddTokenUsage(model string, inputTokens, outputTokens, totalTokens, cachedTokens, cacheWriteTokens int) error

AddTokenUsage adds token usage from a single API call to session totals with model tracking. The model parameter is required for cost tracking. Use empty string for unknown models.

func (*InMemoryConversationRepository) Clear 

func (r *InMemoryConversationRepository) Clear() error

func (*InMemoryConversationRepository) ClearExceptFirstUserMessage 

func (r *InMemoryConversationRepository) ClearExceptFirstUserMessage() error

func (*InMemoryConversationRepository) DeleteMessagesAfterIndex 

func (r *InMemoryConversationRepository) DeleteMessagesAfterIndex(index int) error

DeleteMessagesAfterIndex deletes all messages after the specified index (keeps messages from 0 to index inclusive, deletes index+1 onward)

func (*InMemoryConversationRepository) Export 

func (r *InMemoryConversationRepository) Export(format convdomain.ExportFormat) ([]byte, error)

func (*InMemoryConversationRepository) FormatToolResultExpanded 

func (r *InMemoryConversationRepository) FormatToolResultExpanded(result *agentdomain.ToolExecutionResult, terminalWidth int) string

FormatToolResultExpanded formats expanded tool execution results

func (*InMemoryConversationRepository) FormatToolResultForLLM 

func (r *InMemoryConversationRepository) FormatToolResultForLLM(result *agentdomain.ToolExecutionResult) string

FormatToolResultForLLM formats tool execution results for LLM consumption

func (*InMemoryConversationRepository) FormatToolResultForUI 

func (r *InMemoryConversationRepository) FormatToolResultForUI(result *agentdomain.ToolExecutionResult, terminalWidth int) string

FormatToolResultForUI formats tool execution results for UI display

func (*InMemoryConversationRepository) GetCurrentConversationID 

func (r *InMemoryConversationRepository) GetCurrentConversationID() string

GetCurrentConversationID returns the current conversation ID (empty for in-memory)

func (*InMemoryConversationRepository) GetCurrentConversationTitle 

func (r *InMemoryConversationRepository) GetCurrentConversationTitle() string

GetCurrentConversationTitle returns the current conversation title

func (*InMemoryConversationRepository) GetMessageCount 

func (r *InMemoryConversationRepository) GetMessageCount() int

func (*InMemoryConversationRepository) GetMessages 

func (r *InMemoryConversationRepository) GetMessages() []convdomain.ConversationEntry

func (*InMemoryConversationRepository) GetSessionCostStats 

func (r *InMemoryConversationRepository) GetSessionCostStats() convdomain.SessionCostStats

GetSessionCostStats returns the accumulated cost statistics for the session

func (*InMemoryConversationRepository) GetSessionTokens 

func (r *InMemoryConversationRepository) GetSessionTokens() convdomain.SessionTokenStats

GetSessionTokens returns the accumulated token statistics for the session

func (*InMemoryConversationRepository) LoadConversation 

func (r *InMemoryConversationRepository) LoadConversation(ctx context.Context, conversationID string) error

LoadConversation always fails for in-memory storage: there is nothing persisted to load from.

func (*InMemoryConversationRepository) MarkLastMessageAsPlan 

func (r *InMemoryConversationRepository) MarkLastMessageAsPlan()

MarkLastMessageAsPlan marks the last assistant message as a plan with pending approval

func (*InMemoryConversationRepository) MarkMessageAsPlanByIndex 

func (r *InMemoryConversationRepository) MarkMessageAsPlanByIndex(index int)

MarkMessageAsPlanByIndex marks a specific message by index as a plan with pending approval

func (*InMemoryConversationRepository) RemovePendingToolCallByID 

func (r *InMemoryConversationRepository) RemovePendingToolCallByID(toolCallID string)

RemovePendingToolCallByID removes a specific pending tool call by its ID

func (*InMemoryConversationRepository) SetSessionStats 

func (r *InMemoryConversationRepository) SetSessionStats(tokenStats convdomain.SessionTokenStats, costStats convdomain.SessionCostStats)

SetSessionStats sets the session token and cost statistics (used when loading conversations)

func (*InMemoryConversationRepository) StartNewConversation 

func (r *InMemoryConversationRepository) StartNewConversation(title string) error

StartNewConversation clears the current conversation (in-memory doesn't persist)

func (*InMemoryConversationRepository) UpdateLastMessage 

func (r *InMemoryConversationRepository) UpdateLastMessage(content string) error

func (*InMemoryConversationRepository) UpdateLastMessageToolCalls 

func (r *InMemoryConversationRepository) UpdateLastMessageToolCalls(toolCalls *[]sdk.ChatCompletionMessageToolCall) error

func (*InMemoryConversationRepository) UpdatePlanStatus 

func (r *InMemoryConversationRepository) UpdatePlanStatus(action agentdomain.PlanApprovalAction)

UpdatePlanStatus updates the status of the most recent pending plan

func (*InMemoryConversationRepository) UpdateToolApprovalStatus 

func (r *InMemoryConversationRepository) UpdateToolApprovalStatus(action agentdomain.ApprovalAction)

UpdateToolApprovalStatus updates the approval status of the most recent pending tool

type MessageQueueService 

type MessageQueueService struct {
	// contains filtered or unexported fields
}

MessageQueueService manages a centralized queue for messages waiting to be processed

func NewMessageQueueService 

func NewMessageQueueService() *MessageQueueService

NewMessageQueueService creates a new message queue service

func (*MessageQueueService) Clear 

func (mq *MessageQueueService) Clear()

Clear removes all messages from the queue

func (*MessageQueueService) Dequeue 

func (mq *MessageQueueService) Dequeue() *convdomain.QueuedMessage

Dequeue removes and returns the next message from the queue Returns nil if the queue is empty

func (*MessageQueueService) Enqueue 

func (mq *MessageQueueService) Enqueue(message sdk.Message, requestID string)

Enqueue adds a message to the queue

func (*MessageQueueService) GetAll 

func (mq *MessageQueueService) GetAll() []convdomain.QueuedMessage

GetAll returns all messages in the queue without removing them

func (*MessageQueueService) IsEmpty 

func (mq *MessageQueueService) IsEmpty() bool

IsEmpty returns true if the queue has no messages

func (*MessageQueueService) Peek 

func (mq *MessageQueueService) Peek() *convdomain.QueuedMessage

Peek returns the next message without removing it Returns nil if the queue is empty

func (*MessageQueueService) Size 

func (mq *MessageQueueService) Size() int

Size returns the number of messages in the queue

type OptimizerConfig 

type OptimizerConfig struct {
	Enabled           bool
	AutoAt            int
	BufferSize        int
	KeepFirstMessages int
	Client            sdk.Client
	Config            *config.Config
	Tokenizer         *TokenizerService
	// Repo is optional. When provided, OptimizeMessages reads
	// LastInputTokens from the repo's session stats and uses it as the
	// trigger value (the gateway-reported count includes system prompt and
	// tool definitions, matching what `/context` displays). When nil, the
	// gate falls back to the entries-only estimate.
	Repo convdomain.ConversationRepository
}

OptimizerConfig represents configuration for the conversation optimizer

type PersistentConversationRepository 

type PersistentConversationRepository struct {
	*InMemoryConversationRepository
	// contains filtered or unexported fields
}

PersistentConversationRepository wraps the InMemoryConversationRepository and adds persistence capabilities using a storage backend

func NewPersistentConversationRepository 

func NewPersistentConversationRepository(formatterService ToolFormatter, pricingService convdomain.PricingService, storageBackend storage.ConversationStorage) *PersistentConversationRepository

NewPersistentConversationRepository creates a new persistent conversation repository

func (*PersistentConversationRepository) AddMessage 

func (r *PersistentConversationRepository) AddMessage(msg convdomain.ConversationEntry) error

Override AddMessage to trigger auto-save

func (*PersistentConversationRepository) AddTokenUsage 

func (r *PersistentConversationRepository) AddTokenUsage(model string, inputTokens, outputTokens, totalTokens, cachedTokens, cacheWriteTokens int) error

AddTokenUsage wraps the in-memory implementation with persistence and auto-save

func (*PersistentConversationRepository) Clear 

func (r *PersistentConversationRepository) Clear() error

Override Clear to handle conversation state

func (*PersistentConversationRepository) Close 

func (r *PersistentConversationRepository) Close() error

Close closes the storage connection

func (*PersistentConversationRepository) DeleteMessagesAfterIndex 

func (r *PersistentConversationRepository) DeleteMessagesAfterIndex(index int) error

DeleteMessagesAfterIndex wraps the in-memory implementation with auto-save

func (*PersistentConversationRepository) DeleteSavedConversation 

func (r *PersistentConversationRepository) DeleteSavedConversation(ctx context.Context, conversationID string) error

DeleteSavedConversation deletes a saved conversation

func (*PersistentConversationRepository) GetCurrentConversationID 

func (r *PersistentConversationRepository) GetCurrentConversationID() string

GetCurrentConversationID returns the current conversation ID

func (*PersistentConversationRepository) GetCurrentConversationMetadata 

func (r *PersistentConversationRepository) GetCurrentConversationMetadata() convdomain.ConversationMetadata

GetCurrentConversationMetadata returns the current conversation metadata

func (*PersistentConversationRepository) GetCurrentConversationTitle 

func (r *PersistentConversationRepository) GetCurrentConversationTitle() string

GetCurrentConversationTitle returns the current conversation title

func (*PersistentConversationRepository) ListSavedConversations 

func (r *PersistentConversationRepository) ListSavedConversations(ctx context.Context, limit, offset int) ([]convdomain.ConversationSummary, error)

ListSavedConversations returns a list of saved conversations

func (*PersistentConversationRepository) LoadConversation 

func (r *PersistentConversationRepository) LoadConversation(ctx context.Context, conversationID string) error

LoadConversation loads a conversation from persistent storage

func (*PersistentConversationRepository) SaveConversation 

func (r *PersistentConversationRepository) SaveConversation(ctx context.Context) error

SaveConversation saves the current conversation to persistent storage

func (*PersistentConversationRepository) SetA2ATaskTracker 

func (r *PersistentConversationRepository) SetA2ATaskTracker(taskTracker scheddomain.A2AClearer)

SetA2ATaskTracker sets the task tracker for context ID persistence

func (*PersistentConversationRepository) SetAutoSave 

func (r *PersistentConversationRepository) SetAutoSave(enabled bool)

SetAutoSave enables or disables automatic saving after each operation

func (*PersistentConversationRepository) SetConversationID 

func (r *PersistentConversationRepository) SetConversationID(id string)

SetConversationID pre-sets the conversation ID so that subsequent AddMessage calls use this ID instead of generating a random one. This is used when resuming a session by ID that doesn't exist yet in storage.

func (*PersistentConversationRepository) SetConversationTags 

func (r *PersistentConversationRepository) SetConversationTags(tags []string)

SetConversationTags sets tags for the current conversation

func (*PersistentConversationRepository) SetConversationTitle 

func (r *PersistentConversationRepository) SetConversationTitle(title string)

SetConversationTitle sets the title for the current conversation

func (*PersistentConversationRepository) SetTitleGenerator 

func (r *PersistentConversationRepository) SetTitleGenerator(titleGenerator *ConversationTitleGenerator)

SetTitleGenerator sets the title generator for automatic title invalidation

func (*PersistentConversationRepository) StartNewConversation 

func (r *PersistentConversationRepository) StartNewConversation(title string) error

StartNewConversation saves the current conversation (if any), then begins a new conversation with a unique ID

type PricingServiceImpl 

type PricingServiceImpl struct {
	// contains filtered or unexported fields
}

PricingServiceImpl implements the PricingService interface.

func (*PricingServiceImpl) CalculateCost 

func (p *PricingServiceImpl) CalculateCost(model string, inputTokens, outputTokens, cachedTokens, cacheWriteTokens int) (inputCost, outputCost, totalCost float64)

CalculateCost computes the total cost for a given number of input, output and cached-prompt tokens. cachedTokens and cacheWriteTokens are the cache-read and cache-creation subsets of inputTokens; they are billed at the gateway's cache-read/cache-write rates when known, otherwise at the full input rate. Returns inputCost, outputCost, and totalCost in USD (or configured currency).

func (*PricingServiceImpl) FormatModelPricing 

func (p *PricingServiceImpl) FormatModelPricing(model string) string

FormatModelPricing returns a formatted string describing the model's pricing. Returns empty string if pricing is disabled or the model has no pricing entry (callers should not assume "no entry" means "free"). Returns "free" only when an explicit pricing entry sets both prices to 0.0. Returns "$X.XX/$Y.YY per MTok" for paid models.

func (*PricingServiceImpl) GetInputPrice 

func (p *PricingServiceImpl) GetInputPrice(model string) float64

GetInputPrice retrieves the input price per million tokens for a specific model. Returns 0.0 for unknown models (e.g., Ollama, custom models).

func (*PricingServiceImpl) GetOutputPrice 

func (p *PricingServiceImpl) GetOutputPrice(model string) float64

GetOutputPrice retrieves the output price per million tokens for a specific model. Returns 0.0 for unknown models (e.g., Ollama, custom models).

func (*PricingServiceImpl) IsEnabled 

func (p *PricingServiceImpl) IsEnabled() bool

IsEnabled returns whether pricing is enabled in the configuration.

func (*PricingServiceImpl) RequiresPro 

func (p *PricingServiceImpl) RequiresPro(model string) bool

RequiresPro reports whether the model is gated behind a paid Pro subscription. Returns false when pricing is disabled or the model has no entry.

type SessionRolloverManager 

type SessionRolloverManager struct {
	// contains filtered or unexported fields
}

SessionRolloverManager decides when to roll over a long-running conversation into a new file (matching the chat-mode `/compact` behavior) and exposes the machinery to perform that rollover. It also resolves "group key" inputs from the channel manager (e.g. `channel-telegram-XYZ`) to the current session UUID via the configured SessionGroupStorage backend, so callers like the channel manager can keep using a stable, deterministic identifier without worrying about which physical session it points at right now.

func NewSessionRolloverManager 

func NewSessionRolloverManager(
	cfg *config.Config,
	optimizer convdomain.ConversationOptimizer,
	repo *PersistentConversationRepository,
	tokenizer *TokenizerService,
	groupStore storage.SessionGroupStorage,
) *SessionRolloverManager

NewSessionRolloverManager constructs a manager. The optimizer is required for PerformRollover to work; if it's nil, ShouldRollover always returns false and PerformRollover returns an error. This mirrors how the chat-mode /compact shortcut behaves when the optimizer is disabled. groupStore is required for non-UUID session-id resolution; if it is nil, group-keyed lookups will fall back to passing the raw id through.

func (*SessionRolloverManager) MaybeRollover 

func (m *SessionRolloverManager) MaybeRollover(ctx context.Context, model, groupKey string) (string, bool)

MaybeRollover combines the ShouldRollover gate with PerformRollover and the uniform warn-on-failure fallback that both `chat` and `infer headless` need. Returns the new session id and true if a rollover fired; "" and false otherwise (whether because the gate was closed or because PerformRollover errored - callers do not need to distinguish).

func (*SessionRolloverManager) PerformRollover 

func (m *SessionRolloverManager) PerformRollover(ctx context.Context, model, groupKey string) (string, error)

PerformRollover runs the optimizer with force=true to produce a summary, calls StartNewConversation on the repo to begin a fresh conversation file, re-adds the summarized messages, and (if groupKey != "") updates the configured SessionGroupStorage to point the group at the new session ID.

This mirrors performCompactAsync in chat_shortcut_handler.go:510-615 - same optimizer call, same StartNewConversation call, same AddMessage loop.

Returns the new session UUID on success.

func (*SessionRolloverManager) ResolveSessionID 

func (m *SessionRolloverManager) ResolveSessionID(rawID string) (string, string, error)

ResolveSessionID maps a raw --session-id value to the conversation ID that should actually be loaded.

  • If rawID parses as a UUID, it is treated as a literal session ID and returned unchanged with an empty groupKey.
  • Otherwise it is treated as a group key. The configured SessionGroupStorage is consulted: if the group exists, its current_session_id is returned; if it does not, the group is registered with rawID as its initial current_session_id (this is the migration path for existing channel JSONL files named after the deterministic group key).

Returns (sessionID, groupKey, error). On any error reading/writing the store, the function logs a warning and falls back to passing rawID through unchanged so the agent can still run.

func (*SessionRolloverManager) ShouldRollover 

func (m *SessionRolloverManager) ShouldRollover(model string) bool

ShouldRollover checks the currently loaded conversation in the repo against both rollover triggers (idle and token threshold) and returns true if either fires. Returns false on a fresh/empty conversation, when the optimizer is disabled, or when compact.enabled=false.

type SyntheticToolResponse 

type SyntheticToolResponse struct {
	Message    sdk.Message
	ToolCallID string
	ToolName   string
}

SyntheticToolResponse pairs a synthesized Tool-role message with the metadata of the assistant tool_call it closes. Callers persist Message to the conversation repository and use ToolCallID / ToolName when emitting UI events.

func EnsureToolCallsClosed 

func EnsureToolCallsClosed(conv []sdk.Message) ([]sdk.Message, []SyntheticToolResponse)

EnsureToolCallsClosed enforces the OpenAI-style invariant that every assistant message with tool_calls is followed by a Tool-role message for each tool_call_id. For every unmatched tool_call_id it inserts a synthetic Tool-role response with CancelledToolResponseContent at the tail of the existing tool responses for that assistant turn, preserving the original tool_calls order.

The returned conversation is a copy; the input slice is not mutated. The returned synthetics list is in insertion order and is non-nil only when at least one synthetic was added.

Idempotent: running on an already-repaired conversation returns the input shape with an empty synthetics slice.

type TokenizerConfig 

type TokenizerConfig struct {
	// CharsPerToken is the average characters per token (default: 4.0)
	CharsPerToken float64

	// MessageOverhead is tokens per message for formatting (default: 4)
	MessageOverhead int

	// ToolCallOverhead is extra tokens per tool call (default: 10)
	ToolCallOverhead int
}

TokenizerConfig holds configuration for the tokenizer service

func DefaultTokenizerConfig 

func DefaultTokenizerConfig() TokenizerConfig

DefaultTokenizerConfig returns the default tokenizer configuration

type TokenizerService 

type TokenizerService struct {
	// contains filtered or unexported fields
}

TokenizerService provides token counting functionality for LLM messages. This is a polyfill for providers (like Ollama Cloud) that don't return token usage metrics in their API responses.

func NewTokenizerService 

func NewTokenizerService(config TokenizerConfig) *TokenizerService

NewTokenizerService creates a new tokenizer service with the given configuration

func (*TokenizerService) AdjustedEstimate 

func (t *TokenizerService) AdjustedEstimate(text string) int

AdjustedEstimate provides a more accurate estimate for code vs prose

func (*TokenizerService) CalculateUsagePolyfill 

func (t *TokenizerService) CalculateUsagePolyfill(
	inputMessages []sdk.Message,
	outputContent string,
	outputToolCalls []sdk.ChatCompletionMessageToolCall,
	tools []sdk.ChatCompletionTool,
) *sdk.CompletionUsage

CalculateUsagePolyfill creates a CompletionUsage estimate for providers that don't return usage metrics. This is the main entry point for the polyfill.

func (*TokenizerService) EffectiveContextTokens 

func (t *TokenizerService) EffectiveContextTokens(lastInputTokens int, messages []sdk.Message) int

EffectiveContextTokens returns the larger of lastInputTokens and a fresh estimate, so a single-turn tool-output spike isn't masked by a stale count.

func (*TokenizerService) EstimateMessageTokens 

func (t *TokenizerService) EstimateMessageTokens(msg sdk.Message) int

EstimateMessageTokens estimates the total tokens for a single message

func (*TokenizerService) EstimateMessagesTokens 

func (t *TokenizerService) EstimateMessagesTokens(messages []sdk.Message) int

EstimateMessagesTokens estimates the total tokens for a slice of messages. This is useful for estimating the prompt/input token count.

func (*TokenizerService) EstimateResponseTokens 

func (t *TokenizerService) EstimateResponseTokens(response string) int

EstimateResponseTokens estimates the tokens in an LLM response string

func (*TokenizerService) EstimateTokenCount 

func (t *TokenizerService) EstimateTokenCount(text string) int

EstimateTokenCount estimates the number of tokens in a text string. This uses a character-based heuristic that provides a reasonable approximation for most English text.

func (*TokenizerService) EstimateToolDefinitionsTokens 

func (t *TokenizerService) EstimateToolDefinitionsTokens(tools []sdk.ChatCompletionTool) int

EstimateToolDefinitionsTokens estimates tokens for tool definitions

func (*TokenizerService) GetToolStats 

func (t *TokenizerService) GetToolStats(toolService agentdomain.ToolService, agentMode agentdomain.AgentMode) (tokens int, count int)

GetToolStats returns token count and tool count for a given agent mode

func (*TokenizerService) IsLikelyCodeContent 

func (t *TokenizerService) IsLikelyCodeContent(text string) bool

IsLikelyCodeContent checks if the text appears to be code Code typically has a higher tokens-per-character ratio

func (*TokenizerService) ShouldUsePolyfill 

func (t *TokenizerService) ShouldUsePolyfill(usage *sdk.CompletionUsage) bool

ShouldUsePolyfill determines if token estimation should be used based on whether the provider returned valid usage metrics

type ToolFormatter 

type ToolFormatter interface {
	FormatToolCall(toolName string, args map[string]any) string
	FormatToolResultForLLM(result *agentdomain.ToolExecutionResult) string
	FormatToolResultExpanded(result *agentdomain.ToolExecutionResult, terminalWidth int) string
	FormatToolResultForUI(result *agentdomain.ToolExecutionResult, terminalWidth int) string
}

ToolFormatter is the slice of the tool-formatter service the conversation repositories use to render tool calls and results.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.