modelcache

Documentation

Overview

Package modelcache provides model-specific cache types and interfaces. It defines the data structures for caching LLM provider model lists and the Cache interface that local and Redis backends implement.

Index

• Constants
• type Cache
  • func NewRedisModelCache(cfg RedisModelCacheConfig) (Cache, error)
• type CachedModel
• type CachedProvider
• type LocalCache
  • func NewLocalCache(filePath string) *LocalCache
  • func (c *LocalCache) Close() error
  • func (c *LocalCache) Get(ctx context.Context) (*ModelCache, error)
  • func (c *LocalCache) Set(ctx context.Context, cache *ModelCache) error
• type ModelCache
• type RedisModelCacheConfig

Constants

const (
	// DefaultRedisKey is the Redis key used to store the model registry cache.
	DefaultRedisKey = "gomodel:models"
)

Types

type Cache

type Cache interface {
	// Get retrieves the model cache data.
	// Returns nil, nil if no cache exists yet.
	Get(ctx context.Context) (*ModelCache, error)

	// Set stores the model cache data.
	Set(ctx context.Context, cache *ModelCache) error

	// Close releases any resources held by the cache.
	Close() error
}

Cache defines the interface for model cache storage. Implementations must be safe for concurrent use.

func NewRedisModelCache

func NewRedisModelCache(cfg RedisModelCacheConfig) (Cache, error)

NewRedisModelCache creates a Cache backed by a Redis store.

type CachedModel

type CachedModel struct {
	ID      string `json:"id"`
	Created int64  `json:"created"`
}

CachedModel represents a single cached model entry within a provider group.

type CachedProvider

type CachedProvider struct {
	ProviderType string        `json:"provider_type"`
	OwnedBy      string        `json:"owned_by"`
	Models       []CachedModel `json:"models"`
}

CachedProvider holds shared fields for all models from a single provider.

type LocalCache

type LocalCache struct {
	// contains filtered or unexported fields
}

LocalCache implements Cache using local file storage. This is suitable for single-instance deployments.

func NewLocalCache

func NewLocalCache(filePath string) *LocalCache

NewLocalCache creates a new local file-based cache. The filePath specifies where the cache file will be stored.

func (*LocalCache) Close

func (c *LocalCache) Close() error

Close is a no-op for local cache.

func (*LocalCache) Get

func (c *LocalCache) Get(ctx context.Context) (*ModelCache, error)

Get retrieves the model cache from the local file.

func (*LocalCache) Set

func (c *LocalCache) Set(ctx context.Context, cache *ModelCache) error

Set stores the model cache to the local file.

type ModelCache

type ModelCache struct {
	UpdatedAt time.Time                 `json:"updated_at"`
	Providers map[string]CachedProvider `json:"providers"`
	// ModelListData holds the raw JSON model registry bytes for cache persistence,
	// allowing the registry to restore its full model list without re-fetching.
	ModelListData json.RawMessage `json:"model_list_data,omitempty"`
}

ModelCache represents the cached model data structure. Models are grouped by provider to avoid repeating shared fields (provider_type, owned_by) on every model entry.

type RedisModelCacheConfig

type RedisModelCacheConfig struct {
	// URL is the Redis connection URL (e.g. "redis://localhost:6379").
	// Required; NewRedisModelCache returns an error if the URL is invalid or
	// the server is unreachable.
	URL string

	// Key is the Redis key under which the serialised ModelCache is stored.
	// Defaults to DefaultRedisKey ("gomodel:models") when empty.
	Key string

	// TTL is how long a cached entry lives in Redis before expiring.
	// Defaults to cache.DefaultRedisTTL (24 h) when zero.
	TTL time.Duration
}

RedisModelCacheConfig is the configuration passed to NewRedisModelCache when creating a Redis-backed model registry cache. All fields are optional and fall back to sensible defaults when zero.