selectacct

Documentation

Index

• Constants
• Variables
• func ModelKey(model string) string
• func ScoreKey(provider account.Provider, accountID string) string
• type LimitWindow
• type Scheduler
  • func NewScheduler(scores []Score) Scheduler
  • func (s Scheduler) Exhausted(provider account.Provider, accountID string) bool
  • func (s Scheduler) ForModel(model string) Scheduler
  • func (s Scheduler) HasModelPool(model string) bool
  • func (s Scheduler) Pick(candidates []account.Account) (account.Account, error)
  • func (s Scheduler) PickBest(candidates []account.Account) (account.Account, error)
  • func (s Scheduler) ScoreFor(provider account.Provider, accountID string) Score
  • func (s Scheduler) UsableForNewSession(provider account.Provider, accountID string) bool
  • func (s Scheduler) UsableForStickySession(provider account.Provider, accountID string) bool
  • func (s Scheduler) WithLiveDebits(debits map[string]int) Scheduler
  • func (s Scheduler) WithScore(score Score) Scheduler
  • func (s Scheduler) WithSessionCounts(counts map[string]int) Scheduler
• type SchedulerRef
  • func NewSchedulerRef(scheduler Scheduler) *SchedulerRef
  • func (r *SchedulerRef) AdvanceAccountGeneration(generation uint64)
  • func (r *SchedulerRef) BeginRefreshIfStale(ttl time.Duration) bool
  • func (r *SchedulerRef) BeginRefreshIfStaleForAccountGeneration(ttl time.Duration, generation uint64) bool
  • func (r *SchedulerRef) ExhaustedUntilFor(provider account.Provider, accountID, poolKey string) (time.Time, bool)
  • func (r *SchedulerRef) FinishRefresh(scheduler Scheduler, update bool)
  • func (r *SchedulerRef) FinishRefreshForAccountGeneration(scheduler Scheduler, update bool, generation uint64) bool
  • func (r *SchedulerRef) Get() Scheduler
  • func (r *SchedulerRef) LiveDebits() map[string]int
  • func (r *SchedulerRef) MarkExhausted(provider account.Provider, accountID, poolKey string)
  • func (r *SchedulerRef) MarkExhaustedUntil(provider account.Provider, accountID, poolKey string, until time.Time)
  • func (r *SchedulerRef) MarkModelIncompatible(provider account.Provider, accountID, model string)
  • func (r *SchedulerRef) MarkModelIncompatibleUntil(provider account.Provider, accountID, model string, until time.Time)
  • func (r *SchedulerRef) ModelIncompatibleUntilFor(provider account.Provider, accountID, model string) (time.Time, bool)
  • func (r *SchedulerRef) NoteRouted(provider account.Provider, accountID string)
  • func (r *SchedulerRef) Set(scheduler Scheduler)
  • func (r *SchedulerRef) SetForAccountGeneration(scheduler Scheduler, generation uint64) bool
  • func (r *SchedulerRef) SetUpdatedAt(updatedAt time.Time)
  • func (r *SchedulerRef) Stale(ttl time.Duration) bool
  • func (r *SchedulerRef) Touch()
• type Score
  • func ScoreFromLimitWindows(accountID string, sessions int, windows []LimitWindow) Score

Constants

const (
	VelocityProjectionMinutes = 20.0
	LiveDebitPerRequest       = 0.020
)

VelocityProjectionMinutes is how far ahead a refresh-delta velocity signal projects headroom (kept for the experiment's comparison policies), and LiveDebitPerRequest is the assumed cost of one routed request as a fraction of an account's short window: Pick debits the snapshot headroom by it for every request the proxy itself routed to the account since the last usage refresh, so concurrent fleets spread across accounts instead of herding onto the snapshot's single best account until it cooks. Deliberately an over-estimate (real mean is ~0.3%): the experiment shows 2% minimizes user-visible failovers (-31% vs no debit), with degradation on both sides (0.6%: -23%, 5%: -27%, 10%: -22%). Chosen by TestVelocityPolicyExperiment (a deterministic 24h 3-user/19-thread fleet-burst workload over 24 seeds); rerun with -v when tuning.

const DefaultExhaustedTTL = 10 * time.Minute

DefaultExhaustedTTL bounds an exhaustion mark when the upstream response gave no reset time. Short on purpose: re-marking a still-cooked account costs one failed attempt, while over-holding a recovered account starves routing of real quota.

const MinNewSessionHeadroom = 0.40

const MinStickyRetentionHeadroom = 0.05

MinStickyRetentionHeadroom is the headroom an account must still have for an idle session to stay on it. It sits far below MinNewSessionHeadroom on purpose. Placing a NEW session on an account needs room for a whole conversation, but a session that is already running built the upstream prompt cache on that account, and the cache is per account: moving the session re-bills its entire conversation prefix as uncached input. Holding the session until its account is nearly empty costs less than moving it.

Variables

var ErrNoAccounts = errors.New("no accounts available")

Functions

func ModelKey

func ModelKey(model string) string

ModelKey reduces a request model (or a window feature name) to a comparison key by lowercasing and dropping non-alphanumeric characters, so a request model "gpt-5.3-codex-spark" and the upstream quota name "GPT-5.3-Codex-Spark" resolve to the same key. Matching is strict equality on this key: a regular model key ("gpt53codex") is a prefix of the Spark key ("gpt53codexspark"), so any looser (substring) match would misroute regular traffic into the Spark pool.

func ScoreKey

func ScoreKey(provider account.Provider, accountID string) string

ScoreKey is the scheduler's identity for an account. A Codex account's ID is its bare email and a Claude account's ID is its profile name, which can also be an email, so the same string routinely identifies one account per provider (e.g. lawrence@cmux.com exists as both a Codex account and a Claude profile). Keying scores by the bare ID alone lets one provider's score silently overwrite the other's; scoping the key by provider keeps them distinct.

Types

type LimitWindow

type LimitWindow struct {
	Name               string
	UsedPercent        float64
	LimitWindowSeconds int64
	ResetAfterSeconds  int64
	// Feature is the upstream limit_name of the model-specific quota pool this
	// window belongs to (e.g. "GPT-5.3-Codex-Spark"); empty for account-wide
	// windows.
	Feature string
}

type Scheduler

type Scheduler struct {
	// contains filtered or unexported fields
}

func NewScheduler

func NewScheduler(scores []Score) Scheduler

func (Scheduler) Exhausted

func (s Scheduler) Exhausted(provider account.Provider, accountID string) bool

func (Scheduler) ForModel

func (s Scheduler) ForModel(model string) Scheduler

ForModel narrows scoring to a model's dedicated quota pool when one exists. If the model maps to no known pool (the common case: regular models), the base scheduler is returned unchanged so account-wide quota is used. When a pool exists but a given account lacks it, that account scores zero so it is not picked for a model it cannot serve.

func (Scheduler) HasModelPool

func (s Scheduler) HasModelPool(model string) bool

HasModelPool reports whether the model maps to a dedicated quota pool present on at least one scored account.

func (Scheduler) Pick

func (s Scheduler) Pick(candidates []account.Account) (account.Account, error)

Pick chooses an account for placing work (a new session, a failover retry). Codex pools spread across the usable accounts (see spreadPool); everything else takes the sort's leading candidate.

func (Scheduler) PickBest

func (s Scheduler) PickBest(candidates []account.Account) (account.Account, error)

PickBest returns the deterministic argmax of the selection order, with no placement spreading. Callers that maintain ONE choice over time (sr auto-switch re-picks the single active CLI account on an interval) use this so an unchanged pool keeps an unchanged answer instead of rotating through equally-usable accounts.

func (Scheduler) ScoreFor

func (s Scheduler) ScoreFor(provider account.Provider, accountID string) Score

ScoreFor returns the stored score for an account, or an optimistic default (treated as fully healthy) when none is known. Callers use it to seed a refresh so accounts whose fresh usage could not be fetched preserve their last known score instead of being clobbered with low-confidence data.

func (Scheduler) UsableForNewSession

func (s Scheduler) UsableForNewSession(provider account.Provider, accountID string) bool

func (Scheduler) UsableForStickySession

func (s Scheduler) UsableForStickySession(provider account.Provider, accountID string) bool

UsableForStickySession reports whether an account still has enough headroom to keep serving a session already assigned to it. Callers placing a fresh session use UsableForNewSession instead. Retention reads the measured score, not the live-debited one. The debit floors headroom at 0.01, so on any busy account it sits below every retention threshold and would evict the session the debit was only meant to steer new picks away from.

func (Scheduler) WithLiveDebits

func (s Scheduler) WithLiveDebits(debits map[string]int) Scheduler

WithLiveDebits attaches per-account routed-request counts accumulated since the last usage refresh. score() debits LiveDebitPerRequest of headroom per routed request, so between refreshes the scheduler sees its OWN traffic draining the snapshot instead of herding every pick onto the same account. Keys are ScoreKey(provider, accountID).

func (Scheduler) WithScore

func (s Scheduler) WithScore(score Score) Scheduler

func (Scheduler) WithSessionCounts

func (s Scheduler) WithSessionCounts(counts map[string]int) Scheduler

type SchedulerRef

type SchedulerRef struct {
	// contains filtered or unexported fields
}

func NewSchedulerRef

func NewSchedulerRef(scheduler Scheduler) *SchedulerRef

func (*SchedulerRef) AdvanceAccountGeneration

func (r *SchedulerRef) AdvanceAccountGeneration(generation uint64)

AdvanceAccountGeneration invalidates refresh work computed from an older account snapshot. Callers advance immediately after publishing a new AccountRef snapshot, before any potentially slow usage scoring begins.

func (*SchedulerRef) BeginRefreshIfStale

func (r *SchedulerRef) BeginRefreshIfStale(ttl time.Duration) bool

func (*SchedulerRef) BeginRefreshIfStaleForAccountGeneration

func (r *SchedulerRef) BeginRefreshIfStaleForAccountGeneration(ttl time.Duration, generation uint64) bool

func (*SchedulerRef) ExhaustedUntilFor

func (r *SchedulerRef) ExhaustedUntilFor(provider account.Provider, accountID, poolKey string) (time.Time, bool)

ExhaustedUntilFor reports the expiry recorded for an account's exhaustion mark, if any. Used by tests and diagnostics to verify TTL selection.

func (*SchedulerRef) FinishRefresh

func (r *SchedulerRef) FinishRefresh(scheduler Scheduler, update bool)

func (*SchedulerRef) FinishRefreshForAccountGeneration

func (r *SchedulerRef) FinishRefreshForAccountGeneration(scheduler Scheduler, update bool, generation uint64) bool

func (*SchedulerRef) Get

func (r *SchedulerRef) Get() Scheduler

func (*SchedulerRef) LiveDebits

func (r *SchedulerRef) LiveDebits() map[string]int

LiveDebits returns the per-account routed-request counts since the last successful refresh, for Scheduler.WithLiveDebits.

func (*SchedulerRef) MarkExhausted

func (r *SchedulerRef) MarkExhausted(provider account.Provider, accountID, poolKey string)

func (*SchedulerRef) MarkExhaustedUntil

func (r *SchedulerRef) MarkExhaustedUntil(provider account.Provider, accountID, poolKey string, until time.Time)

MarkExhaustedUntil records an exhaustion overlay until the given time, after which routing reads the base snapshot again. An empty poolKey marks the whole account; a non-empty poolKey marks only that model pool. Callers pass the upstream's own reset time (anthropic-ratelimit-unified-reset / Retry-After) when available.

func (*SchedulerRef) MarkModelIncompatible

func (r *SchedulerRef) MarkModelIncompatible(provider account.Provider, accountID, model string)

func (*SchedulerRef) MarkModelIncompatibleUntil

func (r *SchedulerRef) MarkModelIncompatibleUntil(provider account.Provider, accountID, model string, until time.Time)

MarkModelIncompatibleUntil excludes one account from one model until the supplied expiry. Unlike quota exhaustion, usage-score refreshes cannot clear this mark because they do not carry entitlement evidence.

func (*SchedulerRef) ModelIncompatibleUntilFor

func (r *SchedulerRef) ModelIncompatibleUntilFor(provider account.Provider, accountID, model string) (time.Time, bool)

func (*SchedulerRef) NoteRouted

func (r *SchedulerRef) NoteRouted(provider account.Provider, accountID string)

NoteRouted records that one request was routed to the account, debiting its live score until the next successful usage refresh.

func (*SchedulerRef) Set

func (r *SchedulerRef) Set(scheduler Scheduler)

func (*SchedulerRef) SetForAccountGeneration

func (r *SchedulerRef) SetForAccountGeneration(scheduler Scheduler, generation uint64) bool

SetForAccountGeneration publishes a scheduler only when it was computed from the current account snapshot. The comparison and write share one lock, so a concurrent account reload cannot slip between them.

func (*SchedulerRef) SetUpdatedAt

func (r *SchedulerRef) SetUpdatedAt(updatedAt time.Time)

func (*SchedulerRef) Stale

func (r *SchedulerRef) Stale(ttl time.Duration) bool

func (*SchedulerRef) Touch

func (r *SchedulerRef) Touch()

type Score

type Score struct {
	AccountID              string
	Provider               account.Provider
	Headroom               float64
	ShortHeadroom          float64
	ShortResetAfterSeconds int64
	ExpiryPressure         float64
	Sessions               int
	ModelScores            map[string]Score
	// Fresh marks a score computed from a successful, current usage fetch, as
	// opposed to a seed carried forward from the previous scheduler (fetch
	// failed/stale) or a request-time exhaustion mark. Expiry reconciliation
	// uses it to tell "fresh evidence re-confirmed exhausted" apart from "old
	// zero score dragged along".
	Fresh bool
}

func ScoreFromLimitWindows

func ScoreFromLimitWindows(accountID string, sessions int, windows []LimitWindow) Score

ScoreFromLimitWindows computes an account's base score from its account-wide windows, plus a per-feature score for every model-specific quota pool present in the windows. Model-specific windows are excluded from the base score so a request for a regular model is not penalized by an unrelated pool's usage, and a request for a metered model (e.g. Spark) is scored only against its own pool. This is fully general: any additional rate limit the upstream reports becomes its own pool, keyed by the limit name, with no per-model special cases.