tracker

Documentation

Overview

Package tracker records LLM token usage and request duration as OpenTelemetry metrics following the GenAI semantic conventions (https://github.com/open-telemetry/semantic-conventions-genai):

• gen_ai.client.token.usage histogram {token}, split by gen_ai.token.type
• gen_ai.client.operation.duration histogram s, errors via error.type

Request counts and error counts are intentionally NOT separate counters: the duration histogram's count is the request count, and error.type on it classifies failures — that is the standard shape.

Index

• Variables
• type TokenTracker
  • func NewTokenTracker(meter metric.Meter) (*TokenTracker, error)
  • func (tt *TokenTracker) RecordUsage(ctx context.Context, opts UsageOptions)
• type UsageOptions

Variables

var (
	AttrScenario     = attribute.Key("tingly.scenario")
	AttrProviderUUID = attribute.Key("tingly.provider.uuid")
	AttrRuleUUID     = attribute.Key("tingly.rule.uuid")
	AttrStreaming    = attribute.Key("tingly.streaming")
	AttrUserTier     = attribute.Key("tingly.user.tier")
)

Gateway-specific metric attributes. These have no gen_ai equivalent; they live in the tingly.* namespace instead of squatting on a standard one. This is the single home for these keys — pkg/otel aliases them for spans (tracker cannot import pkg/otel, which imports tracker).

Types

type TokenTracker

type TokenTracker struct {
	// contains filtered or unexported fields
}

TokenTracker records token usage and operation duration using the OpenTelemetry GenAI client metrics.

func NewTokenTracker

func NewTokenTracker(meter metric.Meter) (*TokenTracker, error)

NewTokenTracker creates a new TokenTracker with the provided meter. The genaiconv constructors supply the spec-exact instrument names, units and descriptions.

func (*TokenTracker) RecordUsage

func (tt *TokenTracker) RecordUsage(ctx context.Context, opts UsageOptions)

RecordUsage records token usage and duration for one request.

type UsageOptions

type UsageOptions struct {
	// Operation is the gen_ai.operation.name ("chat", "embeddings", ...).
	// Defaults to "chat" when empty. Callers MUST pass a bounded set of
	// values — every distinct operation mints permanent timeseries.
	Operation string

	// Provider is the name of the LLM provider (e.g., "openai", "anthropic")
	Provider string

	// ProviderUUID is the unique identifier of the provider
	ProviderUUID string

	// Model is the actual model used (not the requested model)
	Model string

	// RequestModel is the original model name requested by the user
	RequestModel string

	// RuleUUID is the load balancer rule UUID
	RuleUUID string

	// Scenario is the API scenario (e.g., "openai", "anthropic", "claude_code")
	Scenario string

	// InputTokens is the number of input/prompt tokens consumed (excluding cache)
	InputTokens int

	// OutputTokens is the number of output/completion tokens consumed
	OutputTokens int

	// CacheReadTokens is the number of cache-READ tokens consumed
	CacheReadTokens int

	// CacheWriteTokens is the number of tokens written to the prompt cache.
	// Billed separately (Anthropic cache_creation, OpenAI cache_write_tokens
	// since gpt-5.6) and already included in InputTokens, so summing the two
	// would double count.
	CacheWriteTokens int

	// SystemTokens represents tokens consumed by system-level operations
	SystemTokens int

	// Streamed indicates whether this was a streaming request
	Streamed bool

	// Status is the request status - "success", "error", or "canceled"
	Status string

	// ErrorCode is the error code if status is not "success"
	ErrorCode string

	// LatencyMs is the request processing time in milliseconds
	LatencyMs int

	// UserTier is a low-cardinality class for enterprise observability.
	UserTier string
}

UsageOptions contains the options for recording token usage.