modelsdev

Documentation

Overview

Package modelsdev resolves per-model context-window limits from the models.dev open catalog (https://models.dev/api.json).

Provider-hosted /models endpoints routinely under-report real limits (e.g. GitHub Copilot advertises a 200K prompt cap for claude-fable-5 while the deployment accepts 1M-context requests), so models.dev is treated as the authoritative source for compaction thresholds. The catalog is fetched lazily, cached in memory and on disk with a TTL, and served stale on fetch errors so offline runs keep working.

Index

• Constants
• type Catalog
  • func ParseCatalog(body []byte) (Catalog, error)
• type ModelLimits
• type Option
  • func WithCachePath(path string) Option
  • func WithHTTPClient(c *http.Client) Option
  • func WithTTL(ttl time.Duration) Option
  • func WithURL(url string) Option
• type Resolver
  • func NewResolver(opts ...Option) *Resolver
  • func (r *Resolver) CompactionResolverFunc(defaultProvider string) func(ctx context.Context, model string) (int, int, bool)
  • func (r *Resolver) CompactionThresholds(ctx context.Context, provider, model string) (triggerTokens, targetTokens int, ok bool)
  • func (r *Resolver) Lookup(ctx context.Context, provider, model string) (ModelLimits, bool)

Constants

const (
	// CatalogURL is the models.dev catalog endpoint.
	CatalogURL = "https://models.dev/api.json"
)

Types

type Catalog

type Catalog map[string]map[string]ModelLimits

Catalog is a parsed models.dev catalog: provider ID -> model ID -> limits.

func ParseCatalog

func ParseCatalog(body []byte) (Catalog, error)

ParseCatalog parses the models.dev api.json payload into a Catalog.

type ModelLimits

type ModelLimits struct {
	ContextTokens int `json:"context"`
	OutputTokens  int `json:"output"`
}

ModelLimits are the context/output token limits for one model.

type Option

type Option func(*Resolver)

Option configures a Resolver.

func WithCachePath

func WithCachePath(path string) Option

WithCachePath sets the on-disk cache location (default: os.UserCacheDir()/gratefulagents/modelsdev.json).

func WithHTTPClient

func WithHTTPClient(c *http.Client) Option

WithHTTPClient overrides the HTTP client.

func WithTTL

func WithTTL(ttl time.Duration) Option

WithTTL overrides the cache freshness window.

func WithURL

func WithURL(url string) Option

WithURL overrides the catalog URL (tests).

type Resolver

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver lazily fetches and caches the models.dev catalog and answers per-(provider, model) limit lookups. Safe for concurrent use.

func NewResolver

func NewResolver(opts ...Option) *Resolver

NewResolver builds a catalog resolver.

func (*Resolver) CompactionResolverFunc

func (r *Resolver) CompactionResolverFunc(defaultProvider string) func(ctx context.Context, model string) (int, int, bool)

CompactionResolverFunc adapts the resolver to the SDK's CompactionModelResolver shape. The model's route prefix (before "/") selects the provider; defaultProvider applies to unprefixed models.

func (*Resolver) CompactionThresholds

func (r *Resolver) CompactionThresholds(ctx context.Context, provider, model string) (triggerTokens, targetTokens int, ok bool)

CompactionThresholds derives compaction trigger/target tokens from the model's context window: trigger at 90% (10% headroom for the next turn), target at 50%.

func (*Resolver) Lookup

func (r *Resolver) Lookup(ctx context.Context, provider, model string) (ModelLimits, bool)

Lookup returns the limits for a model under a provider. The provider is an SDK provider or route prefix name ("copilot", "openai-oauth", ...); the model may carry a "prefix/" which is stripped for matching. Model IDs are matched case-insensitively, exact first and then with date/version suffix tolerance (catalog "claude-fable-5" matches request "claude-fable-5-20260601").