pricing

Documentation

Overview

Package pricing provides model pricing data from LiteLLM's maintained database.

Index

• Constants
• type Config
• type ModelPrice
• type Source
  • func NewSource(cfg Config) *Source
  • func (s *Source) Count() int
  • func (s *Source) GetPrice(provider, model string) *ModelPrice
  • func (s *Source) Load(ctx context.Context) error
  • func (s *Source) RefreshIfStale(ctx context.Context)

Constants

const (
	// LiteLLMPricingURL is the raw GitHub URL for LiteLLM's pricing JSON.
	LiteLLMPricingURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"

	// DefaultTTL is how long to cache pricing data before refreshing.
	DefaultTTL = 24 * time.Hour

	// DefaultTimeout for HTTP requests.
	DefaultTimeout = 30 * time.Second
)

Types

type Config

type Config struct {
	CacheDir string        // Directory for caching pricing data
	TTL      time.Duration // How long before refreshing (0 = use default)
	Logger   *slog.Logger
}

Config configures the pricing source.

type ModelPrice

type ModelPrice struct {
	Provider              string   // Normalized provider name
	Model                 string   // Model identifier
	InputCostPer1k        float64  // Cost per 1000 input tokens
	OutputCostPer1k       float64  // Cost per 1000 output tokens
	CacheCreationPer1k    *float64 // Cost per 1000 cache creation tokens (nil if not supported)
	CacheReadPer1k        *float64 // Cost per 1000 cache read tokens (nil if not supported)
	MaxInputTokens        int      // Maximum input context
	MaxOutputTokens       int      // Maximum output tokens
	SupportsPromptCaching bool
	SupportsVision        bool
	SupportsFunctionCall  bool
}

ModelPrice contains pricing information for a model.

type Source

type Source struct {
	// contains filtered or unexported fields
}

Source provides model pricing lookups.

func NewSource

func NewSource(cfg Config) *Source

NewSource creates a new pricing source.

func (*Source) Count

func (s *Source) Count() int

Count returns the number of loaded pricing entries.

func (*Source) GetPrice

func (s *Source) GetPrice(provider, model string) *ModelPrice

GetPrice looks up pricing for a model. It tries exact match first, then prefix match for versioned models.

func (*Source) Load

func (s *Source) Load(ctx context.Context) error

Load fetches pricing data from LiteLLM (or cache) and loads it into memory.

func (*Source) RefreshIfStale

func (s *Source) RefreshIfStale(ctx context.Context)

RefreshIfStale checks if pricing data needs refresh and fetches if needed.