session

Documentation

Overview

Package session provides in-memory session persistence with timeout-based expiry, three-strike escalation, and per-session cost tracking.

Index

• func CacheTTLForProvider(provider string) int64
• func DeriveSessionID(firstUserMessage string) string
• func GetSessionID(header http.Header, headerName string) string
• func HashRequestContent(content string, toolNames []string) string
• func ProviderFromModel(modelID string) string
• func ShouldCacheSticky(messages []MessageInfo) bool
• type Config
  • func DefaultConfig() Config
• type Entry
  • func (e *Entry) String() string
• type MessageInfo
• type SessionInfo
• type Stats
• type Store
  • func NewStore(cfg Config) *Store
  • func (s *Store) AddSessionCost(sessionID string, micros int64)
  • func (s *Store) ClearAll()
  • func (s *Store) ClearSession(sessionID string)
  • func (s *Store) Close()
  • func (s *Store) EscalateSession(sessionID string)
  • func (s *Store) GetSession(sessionID string) *Entry
  • func (s *Store) GetSessionCostUSD(sessionID string) float64
  • func (s *Store) GetStats() Stats
  • func (s *Store) IsCacheSticky(sessionID string) bool
  • func (s *Store) RecordRequestHash(sessionID, hash string) int
  • func (s *Store) SetCacheSticky(sessionID, provider string)
  • func (s *Store) SetSession(sessionID, model string, tier router.Tier, userExplicit bool)
  • func (s *Store) TouchSession(sessionID string)

Functions

func CacheTTLForProvider

func CacheTTLForProvider(provider string) int64

CacheTTLForProvider returns the prefix cache TTL for a given provider name.

func DeriveSessionID

func DeriveSessionID(firstUserMessage string) string

DeriveSessionID derives a session ID from the first user message via SHA-256.

func GetSessionID

func GetSessionID(header http.Header, headerName string) string

GetSessionID extracts a session ID from HTTP headers.

func HashRequestContent

func HashRequestContent(content string, toolNames []string) string

HashRequestContent produces a short hash from request content. It normalizes whitespace, truncates to 500 chars, and includes tool names.

func ProviderFromModel

func ProviderFromModel(modelID string) string

ProviderFromModel extracts the provider prefix from a model ID. e.g. "anthropic/claude-sonnet-4.6" → "anthropic", "gpt-4o" → "default"

func ShouldCacheSticky

func ShouldCacheSticky(messages []MessageInfo) bool

ShouldCacheSticky checks whether a request's context is large enough to warrant pinning the model for prefix cache optimization. It estimates token count from the raw message content (1 token ≈ 4 chars for English, 2 chars for CJK/code — we use 3 as a conservative average).

Types

type Config

type Config struct {
	Enabled    bool
	TimeoutMs  int64
	HeaderName string
}

Config controls session store behavior.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns a Config with sensible defaults.

type Entry

type Entry struct {
	Model             string
	Tier              router.Tier
	CreatedAt         int64 // unix millis
	LastUsedAt        int64 // unix millis
	RequestCount      int
	RecentHashes      []string // sliding window of recent request hashes
	Strikes           int
	Escalated         bool
	UserExplicit      bool   // true when user explicitly selected a model (not a routing profile)
	CacheSticky       bool   // true when session is pinned for prefix cache optimization
	CacheStickyAt     int64  // when cache sticky was activated (unix millis)
	Provider          string // provider name for cache TTL lookup (e.g. "anthropic", "vllm")
	SessionCostMicros int64  // cost in micro-USD (1 USD = 1_000_000 micros)
}

Entry holds the state of a single session.

func (*Entry) String

func (e *Entry) String() string

String returns a one-line summary of a session entry.

type MessageInfo

type MessageInfo struct {
	Role    string
	Content string
}

MessageInfo is a minimal message representation for cache-sticky evaluation.

type SessionInfo

type SessionInfo struct {
	ID    string `json:"id"`
	Model string `json:"model"`
	Age   int64  `json:"age"` // seconds
}

SessionInfo is a summary of a single session.

type Stats

type Stats struct {
	Count    int           `json:"count"`
	Sessions []SessionInfo `json:"sessions"`
}

Stats is a snapshot of session store state for debugging.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a concurrent-safe in-memory session store with automatic cleanup.

func NewStore

func NewStore(cfg Config) *Store

NewStore creates a session store. If config.Enabled is true, a background goroutine periodically evicts expired sessions.

func (*Store) AddSessionCost

func (s *Store) AddSessionCost(sessionID string, micros int64)

AddSessionCost adds cost in micro-USD to the session.

func (*Store) ClearAll

func (s *Store) ClearAll()

ClearAll removes every session.

func (*Store) ClearSession

func (s *Store) ClearSession(sessionID string)

ClearSession removes a specific session.

func (*Store) Close

func (s *Store) Close()

Close stops the background cleanup goroutine.

func (*Store) EscalateSession

func (s *Store) EscalateSession(sessionID string)

EscalateSession bumps the session tier one level up. If the session has UserExplicit set, escalation is skipped entirely - the user's explicit model choice wins unconditionally.

func (*Store) GetSession

func (s *Store) GetSession(sessionID string) *Entry

GetSession returns the session entry for the given ID, or nil if expired.

func (*Store) GetSessionCostUSD

func (s *Store) GetSessionCostUSD(sessionID string) float64

GetSessionCostUSD returns the session cost in USD.

func (*Store) GetStats

func (s *Store) GetStats() Stats

GetStats returns a snapshot of all active sessions for debugging.

func (*Store) IsCacheSticky

func (s *Store) IsCacheSticky(sessionID string) bool

IsCacheSticky returns true if the session has an active cache-sticky pin (i.e., the pin hasn't expired past the provider's prefix cache TTL).

func (*Store) RecordRequestHash

func (s *Store) RecordRequestHash(sessionID, hash string) int

RecordRequestHash tracks a request content hash in the session sliding window. If 2+ consecutive identical hashes, increment strikes. Returns strike count.

func (*Store) SetCacheSticky

func (s *Store) SetCacheSticky(sessionID, provider string)

SetCacheSticky marks a session as cache-sticky for a specific provider. The model is pinned until the provider's prefix cache TTL expires.

func (*Store) SetSession

func (s *Store) SetSession(sessionID, model string, tier router.Tier, userExplicit bool)

SetSession pins a model to a session. Creates or updates. If userExplicit is true, the user explicitly chose this model (not a routing profile), and session pinning will skip tier escalation and profile-based routing for the lifetime of the session.

func (*Store) TouchSession

func (s *Store) TouchSession(sessionID string)

TouchSession extends a session timeout and increments its request count.