otellix

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 30, 2026 License: MIT Imports: 13 Imported by: 0

README

Otellix

Production-grade LLM observability for Go backends — built for cost-constrained markets.

Go Reference Go Report Card

Why Otellix

  • Every Go LLM observability tool is Python-first. If you're building LLM features in Go, you're on your own for tracing, cost tracking, and budget control. Otellix is Go-native, built on OpenTelemetry, and designed for production from day one.
  • LLM costs are invisible until they're catastrophic. Without per-user and per-feature cost attribution, a single runaway prompt can blow your monthly budget. Otellix attaches accurate USD costs to every OTel span, broken down by user, feature, and project.
  • Budget guardrails don't exist in other SDKs. Otellix is the only Go library that lets you set per-user and per-project daily spend ceilings with automatic enforcement — block, downgrade, or notify when limits are hit. Built by someone who watched LLM API costs eat into product budgets at BudgIT (civic tech) and Class54 (edtech, 400K users) in Nigeria.

Quick Start

go get github.com/oluwajubelo1/otellix

package main

import (
    "context"
    "log"

    "github.com/oluwajubelo1/otellix"
    "github.com/oluwajubelo1/otellix/providers"
    "github.com/oluwajubelo1/otellix/providers/anthropic"
)

func main() {
    shutdown := otellix.SetupDev()
    defer shutdown()

    provider := anthropic.New()

    result, err := otellix.Trace(context.Background(), provider,
        providers.CallParams{
            Model:    "claude-sonnet-4-6",
            MaxTokens: 256,
            Messages: []providers.Message{
                {Role: "user", Content: "What is OpenTelemetry?"},
            },
        },
        otellix.WithFeatureID("demo"),
        otellix.WithUserID("usr_001"),
    )
    if err != nil {
        log.Fatal(err)
    }
    _ = result
}

Output:

[otellix] anthropic/claude-sonnet-4-6 | feature: demo | user: usr_001
          tokens: 245 in + 891 out | cost: $0.014070 | latency: 1.2s
          prompt_fingerprint: a3f8b1c2 | budget: $0.014/$1.000 (1%)

Features

Feature Description Providers
Automatic tracing OTel spans with provider, model, tokens, cost, latency All
Cost attribution Accurate USD cost per call, per user, per feature, per project All
Budget guardrails Per-user/project daily spend ceilings with enforcement All
Prompt fingerprinting SHA256 fingerprint for prompt drift detection All
Prometheus metrics Counters and histograms for all LLM telemetry All
HTTP middleware Auto-extract user/project from JWT and headers Gin, Echo
Prompt Decorators Dynamic prompt optimization based on real-time budget All
Dev mode Human-readable stdout printer for local development All

Configuration

otellix.Trace(ctx, provider, params,
    otellix.WithProvider("anthropic"),       // LLM provider name
    otellix.WithModel("claude-sonnet-4-6"),  // Model identifier
    otellix.WithFeatureID("chat"),           // Product feature (cost attribution)
    otellix.WithUserID("usr_123"),           // User (per-user tracking + budget)
    otellix.WithProjectID("proj_456"),       // Tenant/project (multi-tenant billing)
    otellix.WithSpanName("custom.span"),     // Override default span name
    otellix.WithAttributes(map[string]string{"env": "prod"}),
    otellix.WithPromptFingerprint("system prompt + user message"),
    otellix.WithFallbackModel("claude-haiku-4-5"),
    otellix.WithBudgetConfig(&otellix.BudgetConfig{...}),
)

Budget Guardrails

budgetCfg := &otellix.BudgetConfig{
    PerUserDailyLimit:    0.50,  // $0.50/day per user
    PerProjectDailyLimit: 10.00, // $10.00/day per project
    FallbackAction:       otellix.FallbackBlock,
    ResetInterval:        24 * time.Hour,
}

result, err := otellix.Trace(ctx, provider, params,
    otellix.WithUserID("usr_123"),
    otellix.WithBudgetConfig(budgetCfg),
)
if err != nil {
    var budgetErr *otellix.BudgetExceededError
    if errors.As(err, &budgetErr) {
        log.Printf("Budget exceeded: spend=$%.2f limit=$%.2f resets=%s",
            budgetErr.CurrentSpend, budgetErr.Limit, budgetErr.ResetAt)
    }
}

Fallback modes:

Mode Behaviour
FallbackBlock Return BudgetExceededError immediately — LLM is never called
FallbackNotify Call the LLM but emit a budget.warning event on the span
FallbackDowngrade Swap to a cheaper model (Config.FallbackModel) and proceed

Dynamic Prompt Optimization

Prompt Decorators allow your application to react to telemetry data in real-time. For example, you can inject the remaining budget directly into a system prompt to force the LLM to be more concise.

decorator := func(ctx context.Context, status otellix.BudgetStatus, params *providers.CallParams) {
    if status.Remaining < 0.05 {
        params.SystemPrompt += "\n[BUDGET: You have less than $0.05 left. Be concise.]"
        params.MaxTokens = 100 // Hard-cap output tokens to save cost
    }
}

otellix.Trace(ctx, provider, params,
    otellix.WithBudgetConfig(budgetCfg),
    otellix.WithPromptDecorator(decorator),
)

Metrics Emitted

Metric Type Labels Description
otellix.llm.tokens.input Counter provider, model, feature_id, user_id, project_id Total input tokens
otellix.llm.tokens.output Counter Same Total output tokens
otellix.llm.cost.usd Counter (float) Same Total cost in USD
otellix.llm.latency.ms Histogram Same Call latency in milliseconds
otellix.llm.errors.total Counter provider, model, error_type Total errors

OTel Span Attributes

Attribute Type Description
llm.provider string Provider name (anthropic, openai, gemini, ollama)
llm.model string Model identifier
llm.feature_id string Product feature that triggered the call
llm.user_id string User who triggered the call
llm.project_id string Tenant/project identifier
llm.input_tokens int64 Input token count
llm.output_tokens int64 Output token count
llm.cached_tokens int64 Cached token count
llm.cost_usd float64 Calculated cost in USD
llm.latency_ms float64 Latency in milliseconds
llm.prompt_fingerprint string SHA256 fingerprint (first 8 hex chars)
llm.error bool Whether the call errored
llm.budget_blocked bool Whether the call was blocked by budget

Grafana Dashboard

A pre-built Grafana dashboard is included. Quick start with Docker Compose:

cd examples/docker-compose
docker-compose up

Open http://localhost:3000 — panels include: Total LLM Cost Today, Cost per Feature, Token Usage by Model, Budget Utilisation, Error Rate, P95 Latency.

Supported Providers

Provider Models Notes
Anthropic claude-opus-4-6, claude-sonnet-4-6, claude-haiku-4-5 Prompt caching tracked
OpenAI gpt-4o, gpt-4o-mini Cached tokens tracked
Google Gemini gemini-2.5-pro, gemini-2.5-flash Via google.golang.org/genai
Ollama Any local model $0 cost, HTTP API

Custom models can be registered:

otellix.RegisterModel("anthropic", "claude-4-opus", otellix.PricingEntry{
    InputPricePerMToken:  20.0,
    OutputPricePerMToken: 100.0,
    CachePricePerMToken:  2.0,
})

Contributing

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Run tests: go test ./... -race
  4. Run vet: go vet ./...
  5. Commit (git commit -m 'feat: add amazing feature')
  6. Push (git push origin feature/amazing-feature)
  7. Open a Pull Request

Licence

MIT — see LICENSE.

Documentation

Overview

Package otellix provides OpenTelemetry-native LLM observability for Go.

Package otellix provides OpenTelemetry-native LLM observability for Go.

Otellix wraps LLM API calls with automatic tracing, token counting, cost attribution, and budget enforcement. Built for production Go backends with cost-constrained environments as a first-class concern.

Quick start: 

result, err := otellix.Trace(ctx, provider, params,
    otellix.WithModel("claude-sonnet-4-6"),
    otellix.WithFeatureID("chat-completion"),
    otellix.WithUserID("usr_123"),
    otellix.WithProjectID("proj_456"),
)

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CalculateCost 

func CalculateCost(provider, model string, result providers.CallResult) float64

CalculateCost computes the USD cost for an LLM call based on the provider, model, and token counts in the result.

Returns 0 for unknown provider/model combinations (does not panic).

func EstimateCost 

func EstimateCost(provider, model string, estimatedOutputTokens int64) float64

EstimateCost estimates the cost assuming the given number of output tokens. Used for pre-call budget checks where actual token counts are unknown.

func EstimateTokens 

func EstimateTokens(text string) int64

EstimateTokens provides a rough token count estimate from text length. This uses the common heuristic of ~4 characters per token for English text. It's intended for pre-call budget estimation, not exact accounting.

func EstimateTokensFromMessages 

func EstimateTokensFromMessages(messages []string) int64

EstimateTokensFromMessages estimates total input tokens across multiple messages.

func RegisterDevPrinter 

func RegisterDevPrinter()

RegisterDevPrinter enables the human-readable stdout printer without setting up an OTel trace exporter. Useful when you already have your own TracerProvider configured.

func RegisterModel 

func RegisterModel(provider, model string, entry PricingEntry)

RegisterModel adds or updates pricing for a provider/model combination. This allows callers to add custom models or override built-in pricing.

Example: 

otellix.RegisterModel("anthropic", "claude-4-opus", otellix.PricingEntry{
    InputPricePerMToken:  20.0,
    OutputPricePerMToken: 100.0,
})

func SetupDev 

func SetupDev() func()

SetupDev initialises Otellix for local development with a stdout span exporter and the human-readable dev printer. Call this in main() for local development.

Usage: 

shutdown := otellix.SetupDev()
defer shutdown()

func Trace 

func Trace(ctx context.Context, provider providers.Provider, params providers.CallParams, opts ...Option) (providers.CallResult, error)

Trace executes an LLM call through the given provider, wrapping it with full OpenTelemetry tracing, cost attribution, and optional budget enforcement.

Usage: 

result, err := otellix.Trace(ctx, provider, params,
    otellix.WithModel("claude-sonnet-4-6"),
    otellix.WithFeatureID("chat"),
    otellix.WithUserID("usr_123"),
)

Types

type BudgetConfig 

type BudgetConfig struct {
	// PerUserDailyLimit is the maximum USD spend per user in the rolling window.
	PerUserDailyLimit float64

	// PerProjectDailyLimit is the maximum USD spend per project in the rolling window.
	PerProjectDailyLimit float64

	// FallbackAction determines what to do when the budget is exceeded.
	FallbackAction FallbackAction

	// FallbackModel is the cheaper model to use when FallbackDowngrade is triggered.
	FallbackModel string

	// ResetInterval is the rolling window duration. Default: 24 hours.
	ResetInterval time.Duration

	// Store is the backing store for budget data. Default: in-memory.
	Store BudgetStore
}

BudgetConfig holds budget enforcement settings.

type BudgetEnforcer 

type BudgetEnforcer struct {
	// contains filtered or unexported fields
}

BudgetEnforcer checks and records budget usage.

func NewBudgetEnforcer 

func NewBudgetEnforcer(config *BudgetConfig) *BudgetEnforcer

NewBudgetEnforcer creates a new enforcer. Uses InMemoryBudgetStore if config.Store is nil.

func (*BudgetEnforcer) BudgetError 

func (e *BudgetEnforcer) BudgetError(userID, projectID string) *BudgetExceededError

BudgetError creates a BudgetExceededError with current state.

func (*BudgetEnforcer) Check 

func (e *BudgetEnforcer) Check(ctx context.Context, userID, projectID string, estimatedCost float64) (bool, FallbackAction)

Check determines whether a call is allowed given current budget usage. Returns (allowed, fallbackAction).

func (*BudgetEnforcer) Record 

func (e *BudgetEnforcer) Record(ctx context.Context, userID, projectID string, actualCost float64)

Record adds actual cost to the budget store after a successful call.

func (*BudgetEnforcer) Remaining 

func (e *BudgetEnforcer) Remaining(userID, projectID string) float64

Remaining returns the remaining budget for a user/project (whichever is lower).

func (*BudgetEnforcer) Status added in v1.1.0 

func (e *BudgetEnforcer) Status(ctx context.Context, userID, projectID string) BudgetStatus

Status returns the current detailed budget status for a user/project.

type BudgetExceededError 

type BudgetExceededError struct {
	UserID       string
	ProjectID    string
	CurrentSpend float64
	Limit        float64
	ResetAt      time.Time
}

BudgetExceededError is returned when a call is blocked by budget enforcement.

func (*BudgetExceededError) Error 

func (e *BudgetExceededError) Error() string

type BudgetStatus added in v1.1.0 

type BudgetStatus struct {
	// Remaining is the USD amount left in the budget.
	Remaining float64
	// Usage is the USD amount spent so far in the current interval.
	Usage float64
	// IsExceeded is true if the budget limit has been reached.
	IsExceeded bool
	// Mode is the active fallback action (Block, Notify, Downgrade).
	Mode FallbackAction
}

BudgetStatus represents the current state of a user's budget.

type BudgetStore 

type BudgetStore interface {
	// GetSpend returns the current spend for a key within the rolling window.
	GetSpend(ctx context.Context, key string) float64

	// AddSpend records additional spend for a key.
	AddSpend(ctx context.Context, key string, amount float64)

	// GetResetTime returns when the current budget window resets for a key.
	GetResetTime(ctx context.Context, key string) time.Time
}

BudgetStore is the interface for pluggable budget storage backends.

type Config 

type Config struct {
	// Provider identifies the LLM provider (e.g. "anthropic", "openai", "gemini", "ollama").
	Provider string

	// Model is the specific model identifier (e.g. "claude-sonnet-4-6", "gpt-4o").
	Model string

	// FeatureID identifies which product feature triggered this call (for cost attribution).
	FeatureID string

	// UserID identifies who triggered the call (for per-user cost attribution and budget enforcement).
	UserID string

	// ProjectID identifies which tenant or project this call belongs to (for multi-tenant billing).
	ProjectID string

	// SpanName overrides the default OTel span name ("llm.call").
	SpanName string

	// Attributes holds arbitrary key-value metadata attached to the OTel span.
	Attributes map[string]string

	// EnablePromptFingerprint enables SHA256 fingerprinting of the prompt for drift detection.
	EnablePromptFingerprint bool

	// PromptText is the raw prompt text used for fingerprinting (system prompt + first user message).
	PromptText string

	// FallbackModel is the cheaper model to switch to when FallbackDowngrade is triggered.
	FallbackModel string

	// BudgetConfig holds budget enforcement settings. Nil means no budget enforcement.
	BudgetConfig *BudgetConfig

	// PromptDecorator is a function that can modify prompt params based on budget context.
	PromptDecorator PromptDecorator
}

Config holds the configuration for a single traced LLM call.

func NewConfig 

func NewConfig(opts ...Option) *Config

NewConfig creates a Config from functional options.

type FallbackAction 

type FallbackAction int

FallbackAction determines what happens when a budget ceiling is exceeded. 

const (
	// FallbackBlock returns a BudgetExceededError immediately — the LLM is never called.
	FallbackBlock FallbackAction = iota

	// FallbackNotify calls the LLM but emits a budget.exceeded event on the span.
	FallbackNotify

	// FallbackDowngrade swaps to a cheaper model (Config.FallbackModel) and proceeds.
	FallbackDowngrade
)

type InMemoryBudgetStore 

type InMemoryBudgetStore struct {
	// contains filtered or unexported fields
}

InMemoryBudgetStore is a thread-safe, in-memory budget store using rolling time-bucketed counters. Suitable for single-process deployments.

func NewInMemoryBudgetStore 

func NewInMemoryBudgetStore(interval time.Duration) *InMemoryBudgetStore

NewInMemoryBudgetStore creates a new in-memory store with the given window.

func (*InMemoryBudgetStore) AddSpend 

func (s *InMemoryBudgetStore) AddSpend(ctx context.Context, key string, amount float64)

func (*InMemoryBudgetStore) GetResetTime 

func (s *InMemoryBudgetStore) GetResetTime(ctx context.Context, key string) time.Time

func (*InMemoryBudgetStore) GetSpend 

func (s *InMemoryBudgetStore) GetSpend(ctx context.Context, key string) float64

type Option 

type Option func(*Config)

Option is a functional option for configuring a traced LLM call.

func WithAttributes 

func WithAttributes(attrs map[string]string) Option

WithAttributes sets arbitrary key-value metadata on the span.

func WithBudgetConfig 

func WithBudgetConfig(bc *BudgetConfig) Option

WithBudgetConfig sets budget enforcement configuration.

func WithFallbackModel 

func WithFallbackModel(model string) Option

WithFallbackModel sets the cheaper model for budget downgrade fallback.

func WithFeatureID 

func WithFeatureID(featureID string) Option

WithFeatureID sets the product feature identifier for cost attribution.

func WithModel 

func WithModel(model string) Option

WithModel sets the LLM model identifier.

func WithProjectID 

func WithProjectID(projectID string) Option

WithProjectID sets the project/tenant identifier.

func WithPromptDecorator added in v1.1.0 

func WithPromptDecorator(pd PromptDecorator) Option

WithPromptDecorator sets a function to dynamically decorate prompts based on budget.

func WithPromptFingerprint 

func WithPromptFingerprint(promptText string) Option

WithPromptFingerprint enables prompt fingerprinting with the given prompt text.

func WithProvider 

func WithProvider(provider string) Option

WithProvider sets the LLM provider name.

func WithSpanName 

func WithSpanName(name string) Option

WithSpanName overrides the default span name.

func WithUserID 

func WithUserID(userID string) Option

WithUserID sets the user identifier for per-user cost tracking.

type PricingEntry 

type PricingEntry struct {
	InputPricePerMToken  float64 // USD per 1 million input tokens
	OutputPricePerMToken float64 // USD per 1 million output tokens
	CachePricePerMToken  float64 // USD per 1 million cached tokens (0 if N/A)
}

PricingEntry holds the per-million-token pricing for a specific model.

func GetPricing 

func GetPricing(provider, model string) (PricingEntry, bool)

GetPricing returns the pricing entry for a specific provider/model combination. Returns (entry, false) if not found.

type PromptDecorator added in v1.1.0 

type PromptDecorator func(ctx context.Context, status BudgetStatus, params *providers.CallParams)

PromptDecorator is a function that can modify LLM call parameters based on the current budget status before the provider is called.

Source Files

View all Source files

Directories

Path Synopsis
examples
basic command
examples/basic/main.go — Minimal working example of Otellix.
 examples/basic/main.go — Minimal working example of Otellix.
budget-guardrail command
examples/budget-guardrail/main.go — Budget enforcement demonstration.
 examples/budget-guardrail/main.go — Budget enforcement demonstration.
docker-compose/app command
Docker Compose demo app — makes mock LLM calls to demonstrate Otellix metrics.
 Docker Compose demo app — makes mock LLM calls to demonstrate Otellix metrics.
gin-middleware command
examples/gin-middleware/main.go — Gin integration with automatic attribution.
 examples/gin-middleware/main.go — Gin integration with automatic attribution.
prompt-optimization command
Package exporters provides OTel exporters for Otellix telemetry data.
 Package exporters provides OTel exporters for Otellix telemetry data.
Package middleware provides HTTP middleware for automatic LLM call attribution.
 Package middleware provides HTTP middleware for automatic LLM call attribution.
Package providers defines the interface that all LLM provider wrappers implement.
 Package providers defines the interface that all LLM provider wrappers implement.
anthropic
Package anthropic provides an Otellix provider wrapper for the Anthropic API.
 Package anthropic provides an Otellix provider wrapper for the Anthropic API.
gemini
Package gemini provides an Otellix provider wrapper for Google's Gemini API.
 Package gemini provides an Otellix provider wrapper for Google's Gemini API.
ollama
Package ollama provides an Otellix provider wrapper for Ollama (local LLM inference).
 Package ollama provides an Otellix provider wrapper for Ollama (local LLM inference).
openai
Package openai provides an Otellix provider wrapper for the OpenAI API.
 Package openai provides an Otellix provider wrapper for the OpenAI API.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.