core

package
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 23, 2026 License: MIT Imports: 34 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EstimateTokens added in v1.1.0 

func EstimateTokens(s string) int64

EstimateTokens exposes the rough char/4 estimator for count_tokens.

func Run 

func Run()

Run is the entry point called from the root main().

func SaverSnapshot added in v1.2.0 

func SaverSnapshot() map[string]interface{}

SaverSnapshot is the admin/dashboard view of the saver state.

func ValidPromptMode added in v1.2.0 

func ValidPromptMode(m string) bool

ValidPromptMode reports whether a mode id is supported.

Types

type APIKey 

type APIKey struct {
	Key        string `json:"key"`
	Name       string `json:"name"`
	CreatedAt  int64  `json:"created_at"`
	Enabled    bool   `json:"enabled"`
	Requests   int64  `json:"requests"`
	Errors     int64  `json:"errors"`
	TokensIn   int64  `json:"tokens_in"`
	TokensOut  int64  `json:"tokens_out"`
	LastUsedAt int64  `json:"last_used_at"`
	// Client identifies the OAuth device-flow connection that minted this
	// key ("cline", "kilo", "hermes", …). Empty = created by hand.
	Client string `json:"client,omitempty"`
	// UserCode is the device-flow code the user approved for this key.
	UserCode string `json:"user_code,omitempty"`
	// MaxRequests caps total requests for this key (0 = unlimited).
	MaxRequests int64 `json:"max_requests,omitempty"`
	// AllowedModels restricts which models this key may request.
	// Entries are exact model ids or provider wildcards like "qwen/*".
	// Empty = all models allowed.
	AllowedModels []string `json:"allowed_models,omitempty"`
}

type Config 

type Config struct {
	Host string
	Port int

	// AdminPassword guards the dashboard + /admin/api.
	AdminPassword string
	// RouterKey seeds the first client API key (ROUTER_KEY). When empty a
	// random sk- key is generated and stored.
	RouterKey string
	// InternalToken is the shared bridge secret (AUTH_TOKEN from .env). The
	// embedded bridges were configured from the very same env at package
	// init, so core and bridges always agree.
	InternalToken string

	DataDir  string
	LogLevel string

	// RetryPerProvider: attempts against one provider before failing over.
	RetryPerProvider int
	// CooldownSeconds: after a provider exhausts its retries it is benched
	// for this long so later requests prefer healthier candidates.
	CooldownSeconds int64
	// RequestTimeoutSec: hard upstream timeout for NON-stream requests
	// (streams are unbounded — reasoning models think for minutes).
	RequestTimeoutSec int64
}

type CustomProvider 

type CustomProvider struct {
	Name      string   `json:"name"`     // unique slug, used as prefix "name/model"
	BaseURL   string   `json:"base_url"` // OpenAI-compatible root (…/v1)
	APIKey    string   `json:"api_key"`  // upstream key (Bearer)
	Models    []string `json:"models"`   // advertised models (or fetched live)
	Enabled   bool     `json:"enabled"`
	CreatedAt int64    `json:"created_at"`
	// ModelMap maps public model names → the upstream model id actually
	// sent to this provider (aliasing, yolorouter-style).
	ModelMap map[string]string `json:"model_map,omitempty"`
}

type DeviceGrant added in v1.4.0 

type DeviceGrant struct {
	DeviceCode string `json:"device_code"`
	UserCode   string `json:"user_code"`
	Client     string `json:"client"` // client id (cline, kilo, hermes, …)
	Status     string `json:"status"` // pending | approved | denied
	Key        string `json:"key,omitempty"`
	CreatedAt  int64  `json:"created_at"`
	ExpiresAt  int64  `json:"expires_at"`
}

DeviceGrant is one OAuth Device Authorization Grant (RFC 8628): an agent asks for a code, the user approves it in the dashboard/consent page, and polling returns a freshly minted virtual API key.

type HourBucket added in v1.1.0 

type HourBucket struct {
	Hour      int64 `json:"hour"` // unix hour
	Requests  int64 `json:"requests"`
	Errors    int64 `json:"errors"`
	TokensIn  int64 `json:"tokens_in"`
	TokensOut int64 `json:"tokens_out"`
}

type LogEntry 

type LogEntry struct {
	Time      int64  `json:"time"`
	Key       string `json:"key"`
	Model     string `json:"model"`
	Provider  string `json:"provider"`
	Status    int    `json:"status"`
	Stream    bool   `json:"stream"`
	LatencyMS int64  `json:"latency_ms"`
	TokensIn  int64  `json:"tokens_in,omitempty"`
	TokensOut int64  `json:"tokens_out,omitempty"`
	Saved     int64  `json:"saved,omitempty"` // bytes shaved by the Token Saver
	Err       string `json:"error,omitempty"`
}

func RecentLogs 

func RecentLogs() []LogEntry

type ModelStat added in v1.1.0 

type ModelStat struct {
	Requests  int64 `json:"requests"`
	TokensIn  int64 `json:"tokens_in"`
	TokensOut int64 `json:"tokens_out"`
}

type Provider 

type Provider struct {
	ID      string       `json:"id"` // "glm" | "qwen" | "ds" | "gemini" | "custom:<name>"
	Kind    ProviderKind `json:"kind"`
	BaseURL string       `json:"base_url"` // internal listener or external root
	APIKey  string       `json:"-"`        // custom only
	Label   string       `json:"label"`
	Enabled bool         `json:"enabled"`
	Models  []string     `json:"models"`
	Healthy bool         `json:"healthy"`
	LastErr string       `json:"last_error,omitempty"`
	// Aliases: public model name → upstream model id for THIS provider.
	Aliases map[string]string `json:"aliases,omitempty"`
}

type ProviderKind 

type ProviderKind string
const (
	KindBridge ProviderKind = "bridge"
	KindCustom ProviderKind = "custom"
)

type ProviderStat added in v1.1.0 

type ProviderStat struct {
	Requests  int64 `json:"requests"`
	Errors    int64 `json:"errors"`
	TokensIn  int64 `json:"tokens_in"`
	TokensOut int64 `json:"tokens_out"`
}

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

func NewRegistry 

func NewRegistry(internalToken string, customs []CustomProvider) *Registry

func (*Registry) Catalog 

func (r *Registry) Catalog() ([]map[string]interface{}, bool)

Catalog returns the merged model list (sorted) with owning provider ids.

func (*Registry) Combos added in v1.2.0 

func (r *Registry) Combos() map[string][]string

Combos returns the named chains (dashboard view).

func (*Registry) Cooldown added in v1.1.0 

func (r *Registry) Cooldown(providerID string, seconds int64)

Cooldown marks a provider unhealthy for the next `seconds` so failover prefers other candidates.

func (*Registry) Cooling added in v1.1.0 

func (r *Registry) Cooling(providerID string) bool

Cooling reports whether a provider is currently on cooldown.

func (*Registry) DeleteCombo added in v1.2.0 

func (r *Registry) DeleteCombo(name string) bool

DeleteCombo removes a named chain; returns false when it did not exist.

func (*Registry) OwnerOf added in v1.1.0 

func (r *Registry) OwnerOf(model string) string

OwnerOf returns the provider id that owns a model id in the catalog (used by per-key allowlist checks).

func (*Registry) Providers 

func (r *Registry) Providers() []Provider

func (*Registry) RefreshModels 

func (r *Registry) RefreshModels(ctx context.Context)

RefreshModels re-fetches /v1/models from every enabled provider.

func (*Registry) Resolve 

func (r *Registry) Resolve(model string) []*Provider

Resolve maps a client model id to an ordered candidate chain: 

"auto"            → AUTO chain (env AUTO_CHAIN or a sane default across
                    enabled providers, best-first)
"provider/model"  → exactly that provider (single candidate)
"model"           → catalog owner + (optionally) other providers serving it

Each candidate's Models[0] is the upstream model id to send (alias applied).

func (*Registry) SetAliases added in v1.1.0 

func (r *Registry) SetAliases(providerID string, m map[string]string) bool

SetAliases installs/updates the public→upstream alias map of a provider (bridge id or custom name) and rebuilds the catalog immediately.

func (*Registry) SetBridgeEnabled 

func (r *Registry) SetBridgeEnabled(id string, enabled bool)

SetBridgeEnabled marks an embedded bridge enabled/disabled (e.g. no tokens).

func (*Registry) SetBridgeURL 

func (r *Registry) SetBridgeURL(id, baseURL string)

SetBridgeURL assigns the internal listener address of an embedded bridge.

func (*Registry) SetCombo added in v1.2.0 

func (r *Registry) SetCombo(name string, chain []string)

SetCombo installs/updates a named failover chain and rebuilds the catalog.

func (*Registry) SetInternalToken added in v1.2.0 

func (r *Registry) SetInternalToken(tok string)

SetInternalToken records the shared bridge secret for models fetching.

type Router 

type Router struct {
	// contains filtered or unexported fields
}

Router bundles the live state of the whole service.

type Stats added in v1.1.0 

type Stats struct {
	TotalRequests int64                    `json:"total_requests"`
	TotalErrors   int64                    `json:"total_errors"`
	TokensIn      int64                    `json:"tokens_in"`
	TokensOut     int64                    `json:"tokens_out"`
	Providers     map[string]*ProviderStat `json:"providers"`
	Models        map[string]*ModelStat    `json:"models"`
	Hourly        []HourBucket             `json:"hourly"`
	StartedAt     int64                    `json:"started_at"`
}

type Store 

type Store struct {
	// contains filtered or unexported fields
}

func NewStore 

func NewStore(dir string) *Store

func (*Store) AddProvider 

func (s *Store) AddProvider(p CustomProvider) error

func (*Store) ConnectionKeys added in v1.4.0 

func (s *Store) ConnectionKeys() []APIKey

ConnectionKeys returns the keys minted through OAuth connections.

func (*Store) CreateDeviceGrant added in v1.4.0 

func (s *Store) CreateDeviceGrant(deviceCode, userCode, client string) DeviceGrant

CreateDeviceGrant registers a new pending grant for a client.

func (*Store) CreateKey 

func (s *Store) CreateKey(name string, maxReq int64, allowedModels []string) APIKey

func (*Store) DeleteCombo added in v1.2.0 

func (s *Store) DeleteCombo(name string) bool

DeleteCombo removes a named chain; false when it did not exist.

func (*Store) DeleteDeviceGrant added in v1.4.0 

func (s *Store) DeleteDeviceGrant(deviceCode string)

DeleteDeviceGrant removes a consumed grant record (after successful poll).

func (*Store) DeleteKey 

func (s *Store) DeleteKey(key string) bool

func (*Store) DeleteProvider 

func (s *Store) DeleteProvider(name string) bool

func (*Store) EnsureSeedKey 

func (s *Store) EnsureSeedKey(routerKey string) (string, bool)

EnsureSeedKey creates the first key if none exists: ROUTER_KEY when set, else a random sk- key. Returns the key material for the boot banner.

func (*Store) Flush added in v1.1.0 

func (s *Store) Flush(force bool)

Flush persists dirty state (stats) at most once per 5 seconds. Called by a background ticker and before shutdown.

func (*Store) ListCombos added in v1.2.0 

func (s *Store) ListCombos() map[string][]string

ListCombos returns the persisted named chains.

func (*Store) ListFreebuffTokens added in v1.4.0 

func (s *Store) ListFreebuffTokens() []string

ListFreebuffTokens returns the dashboard-connected Freebuff auth tokens.

func (*Store) ListKeys 

func (s *Store) ListKeys() []APIKey

func (*Store) ListProviders 

func (s *Store) ListProviders() []CustomProvider

func (*Store) LookupDeviceGrant added in v1.4.0 

func (s *Store) LookupDeviceGrant(deviceCode string) (DeviceGrant, bool)

LookupDeviceGrant fetches a grant by its long device code.

func (*Store) LookupKey added in v1.1.0 

func (s *Store) LookupKey(key string) (APIKey, bool)

LookupKey returns a copy of the key record (quota + allowlist checks).

func (*Store) LookupUserGrant added in v1.4.0 

func (s *Store) LookupUserGrant(userCode string) (DeviceGrant, bool)

LookupUserGrant fetches the most recent grant for a user code.

func (*Store) PendingGrants added in v1.4.0 

func (s *Store) PendingGrants() []DeviceGrant

PendingGrants returns live pending grants (for the dashboard panel).

func (*Store) RecordStat added in v1.1.0 

func (s *Store) RecordStat(provider, model string, in, out int64, failed bool)

RecordStat bumps global + per-provider + per-model + hourly counters.

func (*Store) RecordUsage 

func (s *Store) RecordUsage(key string, inTok, outTok int64, failed bool)

RecordUsage updates per-key counters (called by the forwarder once the upstream attempt settled).

func (*Store) ResolveDeviceGrant added in v1.4.0 

func (s *Store) ResolveDeviceGrant(userCode, decision string) (DeviceGrant, bool)

ResolveDeviceGrant approves/denies a pending grant and, on approve, mints the linked virtual key (named after the client + user code). Returns the updated grant.

func (*Store) SetCombo added in v1.2.0 

func (s *Store) SetCombo(name string, chain []string)

SetCombo persists a named chain.

func (*Store) SetFreebuffTokens added in v1.4.0 

func (s *Store) SetFreebuffTokens(tokens []string)

SetFreebuffTokens persists the token list (empty slice disconnects).

func (*Store) SnapshotStats added in v1.1.0 

func (s *Store) SnapshotStats() Stats

func (*Store) ToggleKey 

func (s *Store) ToggleKey(key string, enabled bool) bool

func (*Store) ToggleProvider 

func (s *Store) ToggleProvider(name string, enabled bool) bool

func (*Store) TopModels added in v1.1.0 

func (s *Store) TopModels(n int) []string

TopModels returns up to n most-requested model ids.

func (*Store) UpdateKey added in v1.1.0 

func (s *Store) UpdateKey(key string, maxReq *int64, allowed *[]string) bool

UpdateKey edits quota/allowlist of an existing key.

func (*Store) UpdateProvider added in v1.1.0 

func (s *Store) UpdateProvider(p CustomProvider) error

UpdateProvider replaces an existing provider record (keeps CreatedAt).

func (*Store) ValidateKey 

func (s *Store) ValidateKey(key string) bool

ValidateKey checks a client key and bumps its request counter.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.