usage

package
v0.40.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 19, 2026 License: MIT Imports: 6 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Budget 

type Budget struct {
	MaxCostUSD      float64
	MaxInputTokens  int
	MaxOutputTokens int
	MaxTotalTokens  int
}

Budget defines spending and token ceilings. Zero value means "no limit".

func (Budget) Exceeded 

func (b Budget) Exceeded(agg Record) bool

Exceeded returns true when agg violates any non-zero limit.

type Cost 

type Cost struct {
	Total      float64 `json:"total"`
	Input      float64 `json:"input,omitempty"`
	Output     float64 `json:"output,omitempty"`
	Reasoning  float64 `json:"reasoning,omitempty"` // zero when KindReasoning not present
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`

	// Source describes how the cost was determined.
	//   "calculated" — via CalcCost from KnownPricing or the built-in catalog
	//   "reported"   — API-provided total (OpenRouter)
	//   "estimated"  — pre-request estimate from CountTokens
	//   ""           — no pricing available (Ollama local, unknown model)
	Source string `json:"source,omitempty"`
}

Cost holds monetary amounts in USD. All fields are derived UNLESS Source == "reported" (provider sent the value directly).

func CalcCost 

func CalcCost(items TokenItems, p Pricing) Cost

func (Cost) IsZero 

func (c Cost) IsZero() bool

type CostCalculator 

type CostCalculator interface {
	// Calculate returns (Cost, true) when pricing is known,
	// (Cost{}, false) when the provider+model has no entry.
	Calculate(provider, model string, tokens TokenItems) (Cost, bool)
}

CostCalculator computes a Cost for a given provider, model, and token items.

func Compose 

func Compose(calculators ...CostCalculator) CostCalculator

func Default 

func Default() CostCalculator

type CostCalculatorFunc 

type CostCalculatorFunc func(provider, model string, tokens TokenItems) (Cost, bool)

CostCalculatorFunc is a function that implements CostCalculator.

func (CostCalculatorFunc) Calculate 

func (f CostCalculatorFunc) Calculate(p, m string, t TokenItems) (Cost, bool)

type Dims 

type Dims struct {
	Provider  string `json:"provider,omitempty"`
	Model     string `json:"model,omitempty"`
	RequestID string `json:"request_id,omitempty"`
	TurnID    string `json:"turn_id,omitempty"`    // caller-assigned turn identifier
	SessionID string `json:"session_id,omitempty"` // caller-assigned session identifier

	// Labels are arbitrary string key-value annotations on the Record.
	// Used to distinguish sub-breakdowns within a request, e.g. in estimates:
	//   {"category": "system"}       — system prompt tokens
	//   {"category": "conversation"} — conversation history tokens
	//   {"category": "tools"}        — tool definition tokens
	// Provider-reported records carry no labels.
	Labels map[string]string `json:"labels,omitempty"`
}

Dims carries attribution context for a Record.

type Drift 

type Drift struct {
	Dims Dims // from the actual Record (provider, model, requestID, ...)

	EstimatedInput int // TotalInput() of the unlabeled estimate (Input+CacheRead+CacheWrite)
	ActualInput    int // TotalInput() of the actual record (non-cache + cache-read + cache-write)

	// InputDelta = ActualInput - EstimatedInput.
	// Positive = underestimate (provider used more tokens than predicted).
	// Negative = overestimate (provider used fewer tokens than predicted).
	InputDelta int

	// InputPct = InputDelta / EstimatedInput * 100.
	// math.NaN() when EstimatedInput == 0.
	InputPct float64

	Estimate Record // the matched estimate (IsEstimate == true, Labels == nil)
	Actual   Record // the matched provider-reported actual
}

Drift holds the delta between the unlabeled pre-request estimate and the provider-reported actual for a single request.

func ComputeDrift 

func ComputeDrift(estimate, actual *Record) *Drift

ComputeDrift creates a Drift from an estimate and actual record. Returns nil if estimate or actual is nil, or if estimate is not an estimate.

type DriftStats 

type DriftStats struct {
	N       int     // number of matched pairs
	MinPct  float64 // best-case (most negative = largest overestimate)
	MaxPct  float64 // worst-case (most positive = largest underestimate)
	MeanPct float64
	P50Pct  float64 // median
	P95Pct  float64 // 95th percentile — useful for worst-case budget planning
}

DriftStats aggregates drift across multiple matched request pairs.

type FilterFunc 

type FilterFunc func(Record) bool

FilterFunc is a predicate for filtering records.

func ByLabel 

func ByLabel(key, value string) FilterFunc

ByLabel filters records that have the given label key=value.

func ByModel 

func ByModel(model string) FilterFunc

ByModel filters records matching the given model ID.

func ByProvider 

func ByProvider(name string) FilterFunc

ByProvider filters records matching the given provider name.

func BySessionID 

func BySessionID(id string) FilterFunc

BySessionID filters records matching the given session ID.

func ByTurnID 

func ByTurnID(id string) FilterFunc

ByTurnID filters records matching the given turn ID.

func EstimatesOnly 

func EstimatesOnly() FilterFunc

EstimatesOnly filters to estimate records only.

func ExcludeEstimates 

func ExcludeEstimates() FilterFunc

ExcludeEstimates filters to non-estimate records only.

func Since 

func Since(t time.Time) FilterFunc

Since filters records recorded after the given time.

type Pricing 

type Pricing struct {
	Input       float64 `json:"input"`
	Output      float64 `json:"output"`
	Reasoning   float64 `json:"reasoning,omitempty"`
	CachedInput float64 `json:"cached_input,omitempty"`
	CacheWrite  float64 `json:"cache_write,omitempty"`
}

type Record 

type Record struct {
	Tokens     TokenItems `json:"tokens"`
	Cost       Cost       `json:"cost"`
	Dims       Dims       `json:"dims"`
	IsEstimate bool       `json:"is_estimate,omitempty"`
	RecordedAt time.Time  `json:"recorded_at"`

	// Source describes how the token count was obtained.
	//   "api"       — exact count from the provider's token-counting endpoint
	//   "heuristic" — local BPE approximation (e.g. cl100k_base)
	//   ""          — actual usage reported by the provider (post-request)
	Source string `json:"source,omitempty"`

	// Encoder identifies the tokenizer or counting method used for heuristic estimates.
	// Examples: "cl100k_base", "o200k_base", "cl100k_base+anthropic_overhead".
	// Empty for API-sourced counts and post-request actual usage records.
	Encoder string `json:"encoder,omitempty"`

	// Extras holds provider-specific metadata captured at request time.
	// Mirrors the StreamStartedEvent.Extra convention (same map[string]any type).
	// Keys and value types are defined per provider:
	//
	//   Anthropic / Claude-OAuth: "rate_limits" -> *llm.RateLimits
	//     Contains 5h/7d window utilisation, overage status, fallback percentage,
	//     and representative claim. Populated from HTTP response headers.
	//
	// nil for estimate records and for providers that return no extras.
	Extras map[string]any `json:"extras,omitempty"`
}

Record is a single, fully-attributed usage record.

type TokenItem 

type TokenItem struct {
	Kind  TokenKind `json:"kind"`
	Count int       `json:"count"`
}

TokenItem is one independently-priced token entry.

type TokenItems 

type TokenItems []TokenItem

TokenItems is the ordered list of token items for a Record. Each kind appears at most once per record.

func (TokenItems) Count 

func (t TokenItems) Count(kind TokenKind) int

Count returns the count for the item with the given kind. Returns 0 if no item with that kind exists.

func (TokenItems) NonZero 

func (t TokenItems) NonZero() TokenItems

NonZero returns a new slice with zero-count items removed.

func (TokenItems) Total 

func (t TokenItems) Total() int

Total returns TotalInput + TotalOutput.

func (TokenItems) TotalInput 

func (t TokenItems) TotalInput() int

TotalInput returns Input + CacheRead + CacheWrite.

func (TokenItems) TotalOutput 

func (t TokenItems) TotalOutput() int

TotalOutput returns Output + Reasoning.

type TokenKind 

type TokenKind string

TokenKind identifies one independently-priced token category. 

const (
	// KindInput are regular (non-cache) input tokens.
	// For providers with cache support this is:
	//   total_input_tokens - cache_read_tokens - cache_write_tokens
	// Priced at the model's standard input rate.
	KindInput TokenKind = "input"

	// KindOutput are generated/completion tokens, EXCLUDING reasoning tokens.
	// When the upstream API reports reasoning tokens separately, providers MUST
	// subtract them here so that KindOutput + KindReasoning == total completions
	// with no overlap. When the API cannot distinguish, KindOutput holds the
	// full completion count and KindReasoning is omitted.
	// Priced at the model's standard output rate.
	KindOutput TokenKind = "output"

	// KindReasoning are thinking/reasoning tokens reported by the provider.
	// Only emitted when the upstream API reports them as a distinct value.
	// They are NOT included in KindOutput (no double-counting).
	// Priced at the model's reasoning rate; falls back to output rate when
	// Pricing.Reasoning == 0 (true for all current providers).
	KindReasoning TokenKind = "reasoning"

	// KindCacheRead are input tokens served from an existing prompt cache entry.
	// Anthropic: cache_read_input_tokens.
	// OpenAI:    prompt_tokens_details.cached_tokens.
	// Priced at a reduced cache-read rate; NOT included in KindInput.
	KindCacheRead TokenKind = "cache_read"

	// KindCacheWrite are input tokens written to a new prompt cache entry.
	// Anthropic / Bedrock only (cache_creation_input_tokens).
	// Priced at a cache-write rate; NOT included in KindInput.
	KindCacheWrite TokenKind = "cache_write"
)

type Tracker 

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker accumulates usage records, enriches costs, enforces budgets, and computes drift between pre-request estimates and actual usage.

func NewTracker 

func NewTracker(opts ...TrackerOption) *Tracker

NewTracker creates a new Tracker with the provided options.

func (*Tracker) Aggregate 

func (t *Tracker) Aggregate() Record

Aggregate returns a Record whose Tokens and Cost fields are the sum of all non-estimate records. Dims is zero-valued (the aggregate has no single owner).

func (*Tracker) Drift 

func (t *Tracker) Drift(requestID string) (*Drift, bool)

Drift computes the drift for a given requestID. Matches the first unlabeled estimate against the first actual record. Returns (nil, false) if no complete estimate+actual pair exists.

func (*Tracker) DriftStats 

func (t *Tracker) DriftStats() DriftStats

DriftStats returns aggregate statistics across all matched pairs. Returns zero-value DriftStats when no pairs exist.

func (*Tracker) Drifts 

func (t *Tracker) Drifts() []Drift

Drifts returns drift for all requests with a complete pair, ordered by Actual.RecordedAt.

func (*Tracker) Filter 

func (t *Tracker) Filter(fs ...FilterFunc) []Record

Filter returns all records matching all provided filter functions.

func (*Tracker) Record 

func (t *Tracker) Record(r Record)

Record appends r to the history. If r.Cost.IsZero() and a CostCalculator is configured, the tracker attempts to fill cost before storing. Records with Source == "reported" are never recalculated.

func (*Tracker) Records 

func (t *Tracker) Records() []Record

Records returns a copy of all stored records.

func (*Tracker) Reset 

func (t *Tracker) Reset()

Reset clears all stored records.

func (*Tracker) WithinBudget 

func (t *Tracker) WithinBudget() bool

WithinBudget returns true when the aggregate does not exceed the configured budget.

type TrackerOption 

type TrackerOption func(*Tracker)

TrackerOption configures a Tracker.

func WithBudget 

func WithBudget(b Budget) TrackerOption

WithBudget sets the spending/token ceiling for the Tracker.

func WithCostCalculator 

func WithCostCalculator(c CostCalculator) TrackerOption

WithCostCalculator sets the calculator used to enrich cost-less records.

func WithSessionID 

func WithSessionID(id string) TrackerOption

WithSessionID sets the session ID stamped on records that have no session ID.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.