Help us shape the future of Go! Take the 2026 Go Developer Survey

runtime

package
v0.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 17, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package runtime resolves the active provider, model, deployment routing and credentials into a ready-to-use core.Provider: Load returns a *Runtime whose ChatProvider builds the transport for the current selection.

runtime is engine-internal. Hosts such as Decoder must not import it; the host contract is limited to github.com/GrayCodeAI/relay/engine, llm, graph and tools (see docs/architecture/HOST-ENGINE-BOUNDARY.md), and engine calls this package on the host's behalf. The package stays importable for Relay's own packages and non-Decoder integrations, but its exported names are not covered by engine.ContractVersion and may change in any release.

Provider metadata (IDs, credential variables, protocols, regions) comes from the catalog registry in github.com/GrayCodeAI/relay/catalog/registry; do not restate its size here, it is checked against the docs by catalog/registry/docs_test.go.

Index

Constants

View Source 
const (
	GatewayXiaomiTokenPlan = "xiaomi_mimo_token_plan" // #nosec G101 -- constant name, not a secret value

)

Variables

This section is empty.

Functions

func ActiveGateway 

func ActiveGateway(ctx context.Context) string

ActiveGateway returns the selected setup gateway derived from active provider/model state.

func ActiveModel 

func ActiveModel(ctx context.Context) string

ActiveModel returns the selected model from provider.json.

func ActiveProvider 

func ActiveProvider(ctx context.Context) string

ActiveProvider returns the selected provider from provider.json.

func ActiveProviderID 

func ActiveProviderID(provider string) string

ActiveProviderID canonicalizes a host-facing provider/gateway id through the runtime provider-id rules used by Relay.

func AllModelIDs 

func AllModelIDs(ctx context.Context) ([]string, error)

AllModelIDs returns every canonical model ID in the catalog (sorted).

func ApplyCredentialsForProvider 

func ApplyCredentialsForProvider(ctx context.Context, providerID string) (*setup.ApplyCredentialsResult, error)

ApplyCredentialsForProvider refreshes runtime catalog/provider state for one gateway after saving credentials.

func ApplyGatewayEnv 

func ApplyGatewayEnv(ctx context.Context, providerID string)

ApplyGatewayEnv applies derived env settings from provider.json for gateways that need them.

func AvailableProviders 

func AvailableProviders() []string

AvailableProviders lists built-in provider IDs from the canonical catalog registry. Instance-local custom providers belong to their owning client or Engine and are deliberately not exposed as process-global providers.

func CachedModelCountForProvider 

func CachedModelCountForProvider(ctx context.Context, provider string) int

CachedModelCountForProvider returns the on-disk catalog model count for a gateway.

func CatalogProviderID 

func CatalogProviderID(provider string) string

CatalogProviderID canonicalizes a host-facing provider into the catalog-facing id. Setup gateways stay on their registry ids; provider aliases like gemini/grok map to their catalog owners google/xai.

func ChatProvider 

func ChatProvider(ctx context.Context) (core.Provider, error)

ChatProvider builds the configured chat provider without requiring callers to load runtime state first. Host applications should prefer this over reaching into lower-level setup/config packages.

func ClearActiveSelection 

func ClearActiveSelection(ctx context.Context) error

ClearActiveSelection removes active provider/model from provider.json.

func CommitCredential 

func CommitCredential(ctx context.Context, inference CredentialInference, secret string) error

CommitCredential runs format + provider validation + probe (no persistence).

func CredentialEnvKeys 

func CredentialEnvKeys(providerID string) []string

CredentialEnvKeys returns the credential env vars associated with a provider, including registry fallbacks and compatibility aliases.

func DefaultModelForProvider 

func DefaultModelForProvider(ctx context.Context, provider string) string

DefaultModelForProvider returns the preferred model for a provider using cache first, then live discovery when credentials are configured.

func DefaultModelProviderFilter 

func DefaultModelProviderFilter(ctx context.Context) string

DefaultModelProviderFilter returns the catalog provider id to use when listing models with no explicit provider (e.g. /config model picker after paste-key). Order: provider.json default → first configured deployment (stable sort by id).

func DefaultPaths 

func DefaultPaths() (catalogPath, providerPath string)

DefaultPaths reports standard relay paths on disk.

func DeploymentRoutingEnabled 

func DeploymentRoutingEnabled(ctx context.Context, override *bool) bool

DeploymentRoutingEnabled resolves runtime routing policy plus an optional host override.

func DeploymentStatus 

func DeploymentStatus(ctx context.Context, activeModel string) (setup.StatusReport, error)

DeploymentStatus returns deployment-routing diagnostics for host UIs.

func FormatApplyCredentialsSummary 

func FormatApplyCredentialsSummary(result *setup.ApplyCredentialsResult) string

FormatApplyCredentialsSummary summarizes provider apply results for host UIs.

func FormatDeploymentStatus 

func FormatDeploymentStatus(report setup.StatusReport) string

FormatDeploymentStatus renders a deployment-routing diagnostics report for host UIs.

func FormatPreflightReport 

func FormatPreflightReport(r PreflightReport) string

FormatPreflightReport returns human-readable preflight output.

func FormatSetupError 

func FormatSetupError(providerID string, err error) error

FormatSetupError maps provider setup failures to user-facing hints.

func GatewayDisplayName 

func GatewayDisplayName(providerID string) string

GatewayDisplayName returns the registry display label for a setup gateway.

func GatewayNeedsRegion 

func GatewayNeedsRegion(providerID string) bool

GatewayNeedsRegion reports whether a gateway still requires a region selection.

func GatewayRegionLabel 

func GatewayRegionLabel(providerID string) string

GatewayRegionLabel returns the saved normalized region for gateways that require one.

func HasConfiguredDeployment 

func HasConfiguredDeployment(ctx context.Context) bool

HasConfiguredDeployment reports whether the engine has at least one usable deployment.

func HasStoredCredential 

func HasStoredCredential(ctx context.Context, providerID string) bool

HasStoredCredential reports whether the OS secret store has a usable secret for a gateway.

func IsSetupGateway 

func IsSetupGateway(providerID string) bool

IsSetupGateway reports whether the provider id resolves to a registered setup gateway.

func ModelsForProvider 

func ModelsForProvider(ctx context.Context, provider string) ([]catalog.ModelCatalogEntry, error)

ModelsForProvider loads the catalog cache and returns models for provider. Prefer ListModels for host UIs (registry-aware live vs cache).

func NewReplicaFromState 

func NewReplicaFromState(cfg *config.ProviderConfig) (*controlplane.Replica, error)

NewReplicaFromState creates an independent chat data plane. Shared manifests contain deployment IDs and routes, never secrets. This instance resolves each ID from an explicit, local provider configuration captured at creation. To rotate credentials, create a new replica with new local state and switch the host's provider reference after it has accepted a manifest.

func NormalizeProviderID 

func NormalizeProviderID(provider string) string

NormalizeProviderID resolves catalog aliases and host-facing variants to the runtime provider identifier used by Relay adapters and setup gateways.

func PreferredProvider 

func PreferredProvider(ctx context.Context) string

PreferredProvider returns the runtime-owned provider choice when a host has not pinned one explicitly. Active selection wins first, then inferred model ownership, then configured providers ordered by runtime preference, and finally credential detection as a last resort.

func PrepareCredentialDiscovery 

func PrepareCredentialDiscovery(ctx context.Context)

PrepareCredentialDiscovery applies runtime-owned gateway env derivations before probe/discovery.

func PrimaryAPIKeyEnv 

func PrimaryAPIKeyEnv(deploymentID string) string

PrimaryAPIKeyEnv returns the main env var for a deployment ID from the catalog.

func ProbeCredential 

func ProbeCredential(ctx context.Context, envKey, secret string) error

ProbeCredential validates a key against the provider HTTP API.

func ProviderIDForDeployment 

func ProviderIDForDeployment(deploymentID string) string

ProviderIDForDeployment maps a deployment id to catalog provider id.

func ProviderSupportsLiveModels 

func ProviderSupportsLiveModels(providerID string) bool

ProviderSupportsLiveModels reports whether a provider has a registered live model-list fetcher. Host applications use this as presentation metadata and do not need to inspect the catalog registry directly.

func RefreshGatewayCatalog 

func RefreshGatewayCatalog(ctx context.Context, providerID string) (string, error)

RefreshGatewayCatalog fetches live models for one gateway and updates the cached catalog.

func ResolveCanonicalModel 

func ResolveCanonicalModel(ctx context.Context, model string) string

ResolveCanonicalModel maps aliases/native IDs to canonical catalog model IDs.

func RoutingPreview 

func RoutingPreview(ctx context.Context, model string) (string, error)

RoutingPreview returns the effective routing preview JSON for a model ID.

func SaveCredential 

func SaveCredential(ctx context.Context, inference CredentialInference, secret string) error

SaveCredential validates, stores in the OS secret store, then probes the provider API. The key is persisted before the probe so a network or auth failure does not discard user input.

func SetActiveModel 

func SetActiveModel(ctx context.Context, modelID string) error

SetActiveModel persists the user's model choice to provider.json.

func SetActiveProvider 

func SetActiveProvider(ctx context.Context, provider string) error

SetActiveProvider persists active_provider to provider.json.

func SetCredential 

func SetCredential(ctx context.Context, envKey, secret string) error

SetCredential stores a provider secret (env var name + value). Never log the secret argument.

func SetGatewayRegion 

func SetGatewayRegion(providerID, region string) error

SetGatewayRegion persists a normalized gateway region and updates derived env/base-url state.

func SetupGatewayCredentialEnv 

func SetupGatewayCredentialEnv(providerID string) string

SetupGatewayCredentialEnv returns the primary credential env var for a setup gateway.

func SetupGatewayID 

func SetupGatewayID(provider string) string

SetupGatewayID canonicalizes a host-facing setup-gateway id through runtime rules.

func SetupGateways 

func SetupGateways() []string

SetupGateways returns the registry-backed gateway ids users configure in setup UIs.

func SetupUIFromCatalog 

func SetupUIFromCatalog(ctx context.Context, providerFilter string) (*setup.SetupUI, error)

SetupUIFromCatalog builds provider/model picker rows from the current catalog cache.

func ShouldClearSelectionAfterCredentialRemove 

func ShouldClearSelectionAfterCredentialRemove(ctx context.Context, removedProvider string) bool

ShouldClearSelectionAfterCredentialRemove reports whether removing a gateway key invalidates the active selection.

func SupportsNativeCompaction 

func SupportsNativeCompaction(ctx context.Context, provider, model string) bool

SupportsNativeCompaction reports whether Relay can compact this selection with a configured provider credential.

func SupportsNativeCompactionWithStore 

func SupportsNativeCompactionWithStore(ctx context.Context, provider, model string, store credentials.Store) bool

SupportsNativeCompactionWithStore is the host-neutral form using an explicit credential store.

func SyncSelectionWithCredentials 

func SyncSelectionWithCredentials(ctx context.Context)

SyncSelectionWithCredentials clears stale persisted selection when the selected gateway no longer has usable credentials.

func ValidateKeyFormat 

func ValidateKeyFormat(secret string) error

ValidateKeyFormat rejects empty/placeholder keys before provider selection.

Types

type ApplyResult 

type ApplyResult struct {
	Catalog      *catalog.RefreshResult
	Provider     *config.ProviderConfig
	ProviderPath string
	RoutingJSON  string
	Setup        *setup.SetupUI
}

ApplyResult summarizes an Apply call.

func Apply 

func Apply(ctx context.Context, creds catalog.Credentials) (*ApplyResult, error)

Apply discovers the model catalog and writes ~/.relay/provider.json (routing only; secrets stay in env).

func Discover 

func Discover(ctx context.Context) (*ApplyResult, error)

Discover runs a full catalog refresh then reloads runtime state.

type ChatTransport 

type ChatTransport struct {
	Selection SelectionState
	Provider  core.Provider
}

ChatTransport is the runtime-owned chat transport plan that host apps adapt into their local session/client abstractions.

func ResolveChatTransport 

func ResolveChatTransport(ctx context.Context, opts ChatTransportOpts) (ChatTransport, error)

ResolveChatTransport resolves the effective selection and constructs the runtime-owned provider transport for both deployment-routed and direct provider execution.

func ResolveChatTransportFromSelection 

func ResolveChatTransportFromSelection(ctx context.Context, selection SelectionState) (ChatTransport, error)

ResolveChatTransportFromSelection constructs a transport from an already resolved selection state, avoiding redundant provider/model discovery.

type ChatTransportOpts 

type ChatTransportOpts struct {
	Selection SelectionOpts
}

ChatTransportOpts supplies host-side overrides while transport ownership moves into the runtime package.

type ConfigureProviderOpts 

type ConfigureProviderOpts struct {
	ProviderID string
	Secret     string
}

ConfigureProviderOpts is the complete engine input for provider setup.

type ConfigureProviderResult 

type ConfigureProviderResult struct {
	ProviderID   string
	DeploymentID string
	Summary      string
	Models       []ModelEntry
}

ConfigureProviderResult is the normalized provider setup result for hosts.

func ConfigureProvider 

func ConfigureProvider(ctx context.Context, opts ConfigureProviderOpts) (*ConfigureProviderResult, error)

ConfigureProvider prepares provider environment, saves and probes its credential, refreshes its catalog, and returns normalized model rows.

type CredentialInference 

type CredentialInference = config.CredentialInference

Credential types re-exported for host apps.

func InferenceForProvider 

func InferenceForProvider(providerID string) (CredentialInference, error)

InferenceForProvider returns save metadata for a gateway selected in setup UI.

func LocalCredentialInference 

func LocalCredentialInference(providerID string) (CredentialInference, error)

LocalCredentialInference returns setup metadata for no-key providers.

type CredentialProviderOption 

type CredentialProviderOption = config.CredentialProviderOption

Credential types re-exported for host apps.

func ListCredentialProviders 

func ListCredentialProviders() []CredentialProviderOption

ListCredentialProviders returns all registry providers for setup UIs.

type CredentialResolveResult 

type CredentialResolveResult = config.CredentialResolveResult

Credential types re-exported for host apps.

func ResolveCredential 

func ResolveCredential(ctx context.Context, secret string) CredentialResolveResult

ResolveCredential validates format and lists registered providers (gateway must be chosen first).

type CredentialTarget 

type CredentialTarget struct {
	ProviderID   string
	DeploymentID string
	EnvVar       string
	Set          bool
}

CredentialTarget is one API key slot in a host /config UI.

type DeploymentRow 

type DeploymentRow struct {
	ID         string
	Name       string
	ProviderID string
	Configured bool
	Status     string
	PrimaryEnv string
}

DeploymentRow is a deployment plus env credential status.

type GatewayStatus 

type GatewayStatus struct {
	ID                      string `json:"id"`
	DisplayName             string `json:"display_name"`
	HasStoredCredential     bool   `json:"has_stored_credential"`
	HasConfiguredDeployment bool   `json:"has_configured_deployment"`
	ModelCount              int    `json:"model_count"`
	Active                  bool   `json:"active"`
	RegionLabel             string `json:"region_label,omitempty"`
	RegionRequired          bool   `json:"region_required,omitempty"`
}

GatewayStatus is one setup-gateway row for host UIs.

func GatewayStatuses 

func GatewayStatuses(ctx context.Context, opts GatewayStatusOpts) []GatewayStatus

GatewayStatuses returns runtime-owned setup-gateway summaries for host /config UIs.

type GatewayStatusOpts 

type GatewayStatusOpts struct {
	ActiveProvider string
	ActiveModel    string
}

GatewayStatusOpts controls runtime-owned gateway summaries for host setup UIs.

type ListModelSource 

type ListModelSource string

ListModelSource selects cache vs live model listing. 

const (
	ListSourceAuto  ListModelSource = "auto"
	ListSourceCache ListModelSource = "cache"
	ListSourceLive  ListModelSource = "live"
)

type ListModelsOpts 

type ListModelsOpts struct {
	ProviderID string
	Source     ListModelSource
	Refresh    bool
}

ListModelsOpts configures unified model listing for host UIs.

type ModelEntry 

type ModelEntry struct {
	ID               string          `json:"id"`
	DisplayName      string          `json:"display_name"`
	Owner            string          `json:"owner,omitempty"`
	ProviderID       string          `json:"provider_id"`
	ContextWindow    int             `json:"context_window,omitempty"`
	MaxOutput        int             `json:"max_output,omitempty"`
	InputPricePer1M  float64         `json:"input_price_per_1m,omitempty"`
	OutputPricePer1M float64         `json:"output_price_per_1m,omitempty"`
	LiveMetadata     json.RawMessage `json:"live_metadata,omitempty"`
	Source           string          `json:"source"`
	Installed        bool            `json:"installed,omitempty"`
}

ModelEntry is one row for host model pickers.

func ListModels 

func ListModels(ctx context.Context, opts ListModelsOpts) ([]ModelEntry, error)

ListModels returns models for a provider using registry-driven source selection.

type NativeCompactionOpts 

type NativeCompactionOpts struct {
	Provider        string
	Model           string
	Messages        []core.RelayMessage
	ContextWindow   int
	ThresholdPct    int
	MaxOutputTokens int
}

NativeCompactionOpts describes a provider-native conversation compaction request.

type NativeCompactionResult 

type NativeCompactionResult struct {
	Summary string
}

NativeCompactionResult is provider-neutral output from native compaction.

func CompactNativeConversation 

func CompactNativeConversation(ctx context.Context, opts NativeCompactionOpts) (*NativeCompactionResult, error)

CompactNativeConversation invokes the provider-native compaction protocol.

func CompactNativeConversationWithStore 

func CompactNativeConversationWithStore(ctx context.Context, opts NativeCompactionOpts, store credentials.Store) (*NativeCompactionResult, error)

CompactNativeConversationWithStore invokes provider-native compaction using an explicit host credential store.

type PreflightCheck 

type PreflightCheck struct {
	Name   string          `json:"name"`
	Status PreflightStatus `json:"status"`
	Detail string          `json:"detail"`
}

PreflightCheck is one readiness row.

type PreflightReport 

type PreflightReport struct {
	Ready  bool             `json:"ready"`
	Checks []PreflightCheck `json:"checks"`
}

PreflightReport summarizes whether decoder can chat.

func Preflight 

func Preflight(ctx context.Context) PreflightReport

Preflight evaluates catalog, credentials, model selection, and optional live model access.

type PreflightStatus 

type PreflightStatus string

PreflightStatus is ok, warn, or fail. 

const (
	PreflightOK   PreflightStatus = "ok"
	PreflightWarn PreflightStatus = "warn"
	PreflightFail PreflightStatus = "fail"
)

type ProviderSetupOption 

type ProviderSetupOption struct {
	Action string `json:"action"`
	Label  string `json:"label"`
}

ProviderSetupOption is one hub row in host /config.

func ListProviderSetupOptions 

func ListProviderSetupOptions(ctx context.Context) []ProviderSetupOption

ListProviderSetupOptions returns hub rows for host /config UIs.

type Runtime 

type Runtime struct {
	Catalog      *catalog.CompiledCatalog
	Provider     *config.ProviderConfig
	ProviderPath string
}

Runtime is a loaded relay control plane: catalog cache + routing + env-backed credentials.

func Load 

func Load(ctx context.Context) (*Runtime, error)

Load reads the on-disk catalog and provider config without network refresh.

func (*Runtime) ChatProvider 

func (r *Runtime) ChatProvider(ctx context.Context) (core.Provider, error)

ChatProvider builds the LLM client (deployment router when configured).

func (*Runtime) CredentialTargets 

func (r *Runtime) CredentialTargets() []CredentialTarget

CredentialTargets lists provider-facing API key env vars for simple UIs.

func (*Runtime) DeploymentRows 

func (r *Runtime) DeploymentRows() ([]DeploymentRow, error)

DeploymentRows lists deployments with credential status from env (not from provider.json secrets).

func (*Runtime) ModelEntriesForProvider 

func (r *Runtime) ModelEntriesForProvider(provider string) []catalog.ModelCatalogEntry

ModelEntriesForProvider returns models for a provider from this runtime's catalog snapshot.

func (*Runtime) ModelIDs 

func (r *Runtime) ModelIDs() []string

ModelIDs returns sorted canonical model IDs from the catalog.

func (*Runtime) ProviderConfigJSON 

func (r *Runtime) ProviderConfigJSON() (string, error)

ProviderConfigJSON returns provider.json as indented JSON.

func (*Runtime) RoutingPreviewJSON 

func (r *Runtime) RoutingPreviewJSON(model string) (string, error)

RoutingPreviewJSON returns effective routing for a model ID.

type SelectionOpts 

type SelectionOpts struct {
	ProviderOverride string
	ModelOverride    string
	// DeploymentRoutingOverride lets a host force the routing mode while the
	// migration to pure engine-owned policy is in progress.
	DeploymentRoutingOverride *bool
}

SelectionOpts supplies optional host overrides for provider/model selection. Empty overrides mean "use engine-persisted/default selection".

type SelectionState 

type SelectionState struct {
	Provider                string `json:"provider"`
	Model                   string `json:"model"`
	HasConfiguredDeployment bool   `json:"has_configured_deployment"`
	DeploymentRouting       bool   `json:"deployment_routing"`
}

SelectionState is the engine-resolved provider/model selection for chat.

func EffectiveSelection 

func EffectiveSelection(ctx context.Context, opts SelectionOpts) SelectionState

EffectiveSelection resolves the provider/model that chat should use after applying persisted selection, optional host overrides, credential-aware fallback, and optional canonicalization.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.