core

package module
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 24, 2026 License: MIT Imports: 11 Imported by: 0

README

llmgw-core

Headless, embeddable LLM routing engine and resilient multi-provider proxy for Go.

Highlights

  • Headless & Zero Database: Pure Go http.Handler. Mounts anywhere (net/http, Chi, Gin, Fiber, Echo) without requiring SQLite, PostgreSQL, or any storage engine.
  • Bi-Directional Wire Translation: Full Anthropic Messages API (/v1/messages) support translated dynamically to OpenAI backends (and SSE streaming) via llm-translate.
  • Resilient Multi-Provider Routing: Route aliases, ordered failover chains, rate-limit 429 detection, and circuit breaking.
  • Pluggable Extension Hooks:
    • Authenticator: Custom token, session, or API key validation.
    • PolicyGate: RBAC, project-level, or model-level permission gates.
    • CredentialResolver: Dynamic or multi-tenant credential overrides (BYOC).
    • UsageHook: Telemetry, cost tracking, and audit logging.
  • Shared Provider Contracts: Typed connection/auth kinds, catalog and completion evidence, health classification, exact target publication policy, and a persistence-agnostic ProviderConnector interface.

Installation

go get github.com/xibodev/llmgw-core

Quick Start (Embedded in 15 Lines)

package main

import (
    "net/http"
    "github.com/xibodev/llmgw-core"
    "github.com/xibodev/llmgw-core/providers"
)

func main() {
    engine := core.NewEngine(core.Config{
        Routes: map[string]core.RouteConfig{
            "smart": {
                Targets: []core.Target{
                    {Provider: "groq", Model: "llama-3.3-70b-versatile"},
                    {Provider: "openai", Model: "gpt-4o"},
                },
            },
        },
    })

    // Register backends
    engine.RegisterProvider("openai", providers.NewOpenAIProvider("openai", "https://api.openai.com/v1", os.Getenv("OPENAI_API_KEY"), nil))
    engine.RegisterProvider("groq", providers.NewOpenAIProvider("groq", "https://api.groq.com/openai/v1", os.Getenv("GROQ_API_KEY"), nil))

    // Mount on standard HTTP server
    http.Handle("/v1/", engine)
    http.ListenAndServe(":8080", nil)
}

Provider registry

providers.DefaultRegistry() is the reviewed, validated manifest of curated provider integrations. Core owns each entry's wire and safety facts: runtime, protocol, auth methods, connection scope, risk classification, and anonymous automation. A product layers its own curation on top with an overlay, and the result is validated like a manifest:

overlay, err := providers.DecodeOverlay(overlayJSON) // strict: unknown fields fail
registry, err := providers.DefaultRegistry().WithOverlay(overlay, providers.ValidationOptions{
    RuntimeTypes: []string{"openai_compatible", "anthropic"}, // what this product can execute
})
entry, ok := registry.Lookup("gpt") // ids and aliases, case-insensitive

An overlay can:

  • Override presentation, priority, and curation fields of existing entries: label, description, categories, icon, domain, docs URL, priority, availability, common models, onboarding fields, additional aliases, and product-private extensions.
  • Add product-only entries under their own ids.
  • Remove entries the product does not offer.

An override cannot change wire or safety facts. A product that needs different wire behavior adds its own entry instead. Entries() orders by priority, then by manifest order, so an overlay without priorities keeps the manifest order.

Anonymous providers use one mechanism:

  • Registry.AnonymousProfiles lists the entries curated for anonymous automation.
  • AdmitAnonymousModel applies each provider's reviewed free-model rules to a raw catalog row. It fails closed: unknown providers are never admitted, and OpenCode Zen is admitted only from its verified metadata.
  • SelectVerificationModel picks the model to probe.
  • DiscoverAnonymousModels combines them.

Whether to enroll an anonymous provider remains the product's policy.

License

MIT

Documentation

Index

Constants

View Source 
const ModelCapabilitiesSchemaVersion = 1

Variables

This section is empty.

Functions

func NewProviderOperationError added in v0.3.0 

func NewProviderOperationError(op string, statusCode int, retryAfter string, err error) error

NewProviderOperationError creates a classifiable provider error. Callers should not put response bodies, credentials, or private endpoints in op.

Types

type AllowAllPolicy 

type AllowAllPolicy struct{}

AllowAllPolicy is a PolicyGate that permits all requests.

func (AllowAllPolicy) Allows 

func (AllowAllPolicy) Allows(ctx context.Context, principal *Principal, target Target) (bool, string)

type Authenticator 

type Authenticator interface {
	Authenticate(r *http.Request) (*Principal, error)
}

Authenticator authenticates an incoming HTTP request. If not configured, requests proceed as anonymous.

type AuthenticatorFunc 

type AuthenticatorFunc func(r *http.Request) (*Principal, error)

AuthenticatorFunc is a functional adapter for Authenticator.

func (AuthenticatorFunc) Authenticate 

func (f AuthenticatorFunc) Authenticate(r *http.Request) (*Principal, error)

type CatalogEvidence added in v0.3.0 

type CatalogEvidence struct {
	Status     CatalogEvidenceStatus `json:"status"`
	Models     []ModelInfo           `json:"models,omitempty"`
	ObservedAt time.Time             `json:"observed_at,omitempty"`
}

CatalogEvidence records what a provider advertised. Discovery alone does not prove that any model can successfully perform inference.

type CatalogEvidenceStatus added in v0.3.0 

type CatalogEvidenceStatus string
const (
	CatalogNotProbed  CatalogEvidenceStatus = "not_probed"
	CatalogDiscovered CatalogEvidenceStatus = "discovered"
	CatalogEmpty      CatalogEvidenceStatus = "empty"
	CatalogFailed     CatalogEvidenceStatus = "failed"
)

type CompletionProbeEvidence added in v0.3.0 

type CompletionProbeEvidence struct {
	Target      Target                `json:"target"`
	Status      CompletionProbeStatus `json:"status"`
	ObservedAt  time.Time             `json:"observed_at,omitempty"`
	Latency     time.Duration         `json:"latency,omitempty"`
	FailureCode string                `json:"failure_code,omitempty"`
}

CompletionProbeEvidence records inference evidence for one exact target.

func (CompletionProbeEvidence) InferenceVerified added in v0.3.0 

func (e CompletionProbeEvidence) InferenceVerified() bool

type CompletionProbeStatus added in v0.3.0 

type CompletionProbeStatus string
const (
	CompletionNotProbed CompletionProbeStatus = "not_probed"
	CompletionVerified  CompletionProbeStatus = "verified"
	CompletionFailed    CompletionProbeStatus = "failed"
)

type Config 

type Config struct {
	Providers    map[string]ProviderConfig `json:"providers"`
	Routes       map[string]RouteConfig    `json:"routes"`
	DefaultRoute string                    `json:"default_route,omitempty"`

	// Pluggable extension hooks
	Authenticator      Authenticator
	PolicyGate         PolicyGate
	CredentialResolver CredentialResolver
	UsageHook          UsageHook
}

Config configures the core LLM Gateway engine.

type Credential 

type Credential struct {
	APIKey             string            `json:"api_key,omitempty"`
	Token              string            `json:"token,omitempty"`
	ConnectionID       string            `json:"connection_id,omitempty"`
	CredentialRevision int64             `json:"credential_revision,omitempty"`
	Headers            map[string]string `json:"headers,omitempty"`
}

Credential contains authentication secrets for an upstream provider.

type CredentialResolver 

type CredentialResolver interface {
	Resolve(ctx context.Context, principal *Principal, providerID string) (*Credential, error)
}

CredentialResolver resolves upstream credentials for a principal and provider.

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine is the core LLM Gateway routing and proxy engine. It implements http.Handler and can be mounted directly on any router.

func NewEngine 

func NewEngine(cfg Config) *Engine

func (*Engine) RegisterProvider 

func (e *Engine) RegisterProvider(id string, provider any)

RegisterProvider registers an active provider backend.

func (*Engine) ServeHTTP 

func (e *Engine) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements http.Handler.

type FreshnessStatus added in v0.3.0 

type FreshnessStatus string
const (
	FreshnessUnknown FreshnessStatus = "unknown"
	FreshnessFresh   FreshnessStatus = "fresh"
	FreshnessExpired FreshnessStatus = "expired"
)

type GenerateImagesRequest added in v0.6.0 

type GenerateImagesRequest struct {
	Model  string `json:"model"`
	Prompt string `json:"prompt"`
	Count  int    `json:"count,omitempty"`
}

GenerateImagesRequest is the storage-neutral input for image synthesis. Count is a caller-side result bound; providers may return fewer images.

type GenerateImagesResult added in v0.6.0 

type GenerateImagesResult struct {
	Images []GeneratedImage `json:"images"`
	Usage  map[string]any   `json:"usage,omitempty"`
}

GenerateImagesResult contains generated bytes and provider-native usage.

type GeneratedImage added in v0.6.0 

type GeneratedImage struct {
	Data     []byte `json:"-"`
	MimeType string `json:"mime_type"`
}

GeneratedImage contains one decoded inline image returned by a provider.

type ImageGenerator added in v0.6.0 

type ImageGenerator interface {
	GenerateImages(context.Context, GenerateImagesRequest, *Credential) (GenerateImagesResult, error)
}

ImageGenerator is implemented by providers with a typed image-generation operation. It makes no assumptions about credential or image storage.

type ModelCapabilities added in v0.3.0 

type ModelCapabilities struct {
	SchemaVersion     int                        `json:"schema_version"`
	Operations        ModelOperationCapabilities `json:"operations"`
	Surfaces          ModelSurfaceCapabilities   `json:"surfaces"`
	Inputs            ModelInputCapabilities     `json:"inputs"`
	Tools             Support                    `json:"tools"`
	Reasoning         Support                    `json:"reasoning"`
	StructuredOutput  Support                    `json:"structured_output"`
	Streaming         Support                    `json:"streaming"`
	StatefulResponses Support                    `json:"stateful_responses"`
	Limits            ModelCapabilityLimits      `json:"limits"`
	Provenance        ModelCapabilityProvenance  `json:"provenance"`
	Freshness         ModelCapabilityFreshness   `json:"freshness"`
}

ModelCapabilities is the bounded version 1 shared model capability contract.

func (ModelCapabilities) OperationCompatibility added in v0.3.0 

func (c ModelCapabilities) OperationCompatibility(operation ModelOperation) Support

OperationCompatibility returns unknown for both unreported capabilities and operation values outside this schema version.

func (ModelCapabilities) SurfaceCompatibility added in v0.3.0 

func (c ModelCapabilities) SurfaceCompatibility(surface ModelSurface) Support

SurfaceCompatibility returns unknown for both unreported capabilities and surface values outside this schema version.

type ModelCapabilityConfidence added in v0.3.0 

type ModelCapabilityConfidence string
const (
	ModelCapabilityConfidenceLow    ModelCapabilityConfidence = "low"
	ModelCapabilityConfidenceMedium ModelCapabilityConfidence = "medium"
	ModelCapabilityConfidenceHigh   ModelCapabilityConfidence = "high"
)

type ModelCapabilityFreshness added in v0.3.0 

type ModelCapabilityFreshness struct {
	DiscoveredAt *time.Time `json:"discovered_at,omitempty"`
	VerifiedAt   *time.Time `json:"verified_at,omitempty"`
	ExpiresAt    *time.Time `json:"expires_at,omitempty"`
}

func (ModelCapabilityFreshness) IsFresh added in v0.3.0 

func (f ModelCapabilityFreshness) IsFresh(at time.Time) bool

func (ModelCapabilityFreshness) Status added in v0.3.0 

func (f ModelCapabilityFreshness) Status(at time.Time) FreshnessStatus

Status reports unknown when no expiry was supplied. Expiry is inclusive: a capability is expired at its expires_at instant.

type ModelCapabilityLimits added in v0.3.0 

type ModelCapabilityLimits struct {
	ContextTokens   *int64 `json:"context_tokens,omitempty"`
	MaxOutputTokens *int64 `json:"max_output_tokens,omitempty"`
}

ModelCapabilityLimits uses pointers so an unknown limit is distinct from a provider-reported numeric value.

type ModelCapabilityProvenance added in v0.3.0 

type ModelCapabilityProvenance struct {
	Source     ModelCapabilitySource     `json:"source"`
	Confidence ModelCapabilityConfidence `json:"confidence"`
}

type ModelCapabilitySource added in v0.3.0 

type ModelCapabilitySource string
const (
	ModelCapabilitySourceUpstreamReported ModelCapabilitySource = "upstream_reported"
	ModelCapabilitySourceRegistryStatic   ModelCapabilitySource = "registry_static"
	ModelCapabilitySourceModelsDev        ModelCapabilitySource = "models_dev"
	ModelCapabilitySourceInferred         ModelCapabilitySource = "inferred"
	ModelCapabilitySourceOperatorOverride ModelCapabilitySource = "operator_override"
)

type ModelInfo 

type ModelInfo struct {
	ID            string             `json:"id"`
	Object        string             `json:"object"`
	Created       int64              `json:"created"`
	OwnedBy       string             `json:"owned_by"`
	Description   string             `json:"description,omitempty"`
	Tags          []string           `json:"tags,omitempty"`
	APIEligible   *bool              `json:"api_eligible,omitempty"`
	APIVisibility string             `json:"api_visibility,omitempty"`
	SupportedAPIs []string           `json:"supported_apis,omitempty"`
	Capabilities  *ModelCapabilities `json:"capabilities,omitempty"`
}

ModelInfo describes an available upstream model or alias.

type ModelInputCapabilities added in v0.3.0 

type ModelInputCapabilities struct {
	Text  Support `json:"text"`
	Image Support `json:"image"`
}

type ModelOperation added in v0.3.0 

type ModelOperation string
const (
	ModelOperationChat       ModelOperation = "chat"
	ModelOperationEmbeddings ModelOperation = "embeddings"
	ModelOperationImage      ModelOperation = "image"
	ModelOperationAudioIn    ModelOperation = "audio_in"
	ModelOperationAudioOut   ModelOperation = "audio_out"
	ModelOperationVideo      ModelOperation = "video"
	ModelOperationTokenCount ModelOperation = "token_count"
)

type ModelOperationCapabilities added in v0.3.0 

type ModelOperationCapabilities struct {
	Chat       Support `json:"chat"`
	Embeddings Support `json:"embeddings"`
	Image      Support `json:"image"`
	AudioIn    Support `json:"audio_in"`
	AudioOut   Support `json:"audio_out"`
	Video      Support `json:"video"`
	TokenCount Support `json:"token_count"`
}

type ModelSurface added in v0.3.0 

type ModelSurface string
const (
	ModelSurfaceChatCompletions ModelSurface = "chat_completions"
	ModelSurfaceResponses       ModelSurface = "responses"
	ModelSurfaceMessages        ModelSurface = "messages"
)

type ModelSurfaceCapabilities added in v0.3.0 

type ModelSurfaceCapabilities struct {
	ChatCompletions Support `json:"chat_completions"`
	Responses       Support `json:"responses"`
	Messages        Support `json:"messages"`
}

type PolicyGate 

type PolicyGate interface {
	Allows(ctx context.Context, principal *Principal, target Target) (bool, string)
}

PolicyGate controls whether a principal is authorized to access a given provider/model target.

type Principal 

type Principal struct {
	ID        string         `json:"id"`
	Type      string         `json:"type"` // "user", "api_key", "anonymous", "service"
	Name      string         `json:"name,omitempty"`
	ProjectID string         `json:"project_id,omitempty"`
	Metadata  map[string]any `json:"metadata,omitempty"`
}

Principal represents the authenticated caller making a request.

type ProviderAdapter added in v0.3.0 

type ProviderAdapter struct {
	ValidateAuthentication ProviderAuthenticationValidator
	DiscoverModels         ProviderModelDiscoverer
	SelectProbeTargets     ProviderProbeSelector
	Complete               ProviderCompletionRuntime
}

ProviderAdapter contains the small provider-specific operations used by the shared connector. Complete is both the probe and reusable runtime path.

type ProviderAuthKind added in v0.3.0 

type ProviderAuthKind string

ProviderAuthKind describes how a provider connection authenticates. 

const (
	ProviderAuthAPIKey           ProviderAuthKind = "api_key"
	ProviderAuthOAuthDevice      ProviderAuthKind = "oauth_device"
	ProviderAuthOAuthBrowser     ProviderAuthKind = "oauth_browser"
	ProviderAuthServiceAccount   ProviderAuthKind = "service_account"
	ProviderAuthWorkloadIdentity ProviderAuthKind = "workload_identity"
	ProviderAuthOfficialClient   ProviderAuthKind = "official_client"
	ProviderAuthAnonymous        ProviderAuthKind = "anonymous"
)

type ProviderAuthenticationValidator added in v0.3.0 

type ProviderAuthenticationValidator func(context.Context, ProviderConnection) error

type ProviderCompletionRuntime added in v0.3.0 

type ProviderCompletionRuntime func(context.Context, ProviderConnection, Target, map[string]any) (map[string]any, error)

type ProviderConfig 

type ProviderConfig struct {
	Type          string            `json:"type"`                     // "openai_compatible", "anthropic", "googleai", "ollama", "github_copilot", "codex", "edgetts"
	BaseURL       string            `json:"base_url,omitempty"`       // Upstream base endpoint
	APIKey        string            `json:"api_key,omitempty"`        // Static API key or token
	Timeout       time.Duration     `json:"timeout,omitempty"`        // Request timeout
	Headers       map[string]string `json:"headers,omitempty"`        // Extra HTTP headers
	AllowedModels []string          `json:"allowed_models,omitempty"` // Whitelist of models
	Disabled      bool              `json:"disabled,omitempty"`       // Disable provider
}

ProviderConfig configures a single upstream provider backend.

type ProviderConnectRequest added in v0.3.0 

type ProviderConnectRequest struct {
	Connection        ProviderConnection      `json:"connection"`
	PublicationPolicy TargetPublicationPolicy `json:"publication_policy"`
	// AuthenticationValidated permits callers that established authentication
	// before Connect to omit an adapter authentication validator.
	AuthenticationValidated bool `json:"authentication_validated,omitempty"`
}

type ProviderConnectResult added in v0.3.0 

type ProviderConnectResult struct {
	Connection ProviderConnection        `json:"connection"`
	Catalog    CatalogEvidence           `json:"catalog"`
	Probes     []CompletionProbeEvidence `json:"probes,omitempty"`
	Health     ProviderHealthEvidence    `json:"health"`
	Targets    []Target                  `json:"targets,omitempty"`
}

ProviderConnectResult is the complete, persistence-agnostic outcome of provider onboarding.

type ProviderConnection added in v0.3.0 

type ProviderConnection struct {
	ID         string                 `json:"id,omitempty"`
	ProviderID string                 `json:"provider_id"`
	Kind       ProviderConnectionKind `json:"kind"`
	AuthKind   ProviderAuthKind       `json:"auth_kind"`
	OwnerID    string                 `json:"owner_id,omitempty"`
	Credential *Credential            `json:"-"`
}

ProviderConnection is the persistence-agnostic description of a configured provider connection. Credential is available to connectors but never JSON encoded.

func (ProviderConnection) Validate added in v0.3.0 

func (c ProviderConnection) Validate() error

Validate rejects contradictory connection descriptions. Codex is always a human-owned subscription connection established through device OAuth.

type ProviderConnectionKind added in v0.3.0 

type ProviderConnectionKind string

ProviderConnectionKind describes who owns and may use a provider connection. 

const (
	ProviderConnectionSystem               ProviderConnectionKind = "system"
	ProviderConnectionPersonal             ProviderConnectionKind = "personal"
	ProviderConnectionPersonalSubscription ProviderConnectionKind = "personal_subscription"
	ProviderConnectionAnonymous            ProviderConnectionKind = "anonymous"
)

type ProviderConnector added in v0.3.0 

type ProviderConnector interface {
	Connect(ctx context.Context, request ProviderConnectRequest) (ProviderConnectResult, error)
}

ProviderConnector orchestrates provider-specific authentication, discovery, probing, classification, and exact target publication.

type ProviderErrorClass added in v0.3.0 

type ProviderErrorClass string
const (
	ProviderErrorNone        ProviderErrorClass = "none"
	ProviderErrorAuth        ProviderErrorClass = "auth"
	ProviderErrorForbidden   ProviderErrorClass = "forbidden"
	ProviderErrorRateLimited ProviderErrorClass = "rate_limited"
	ProviderErrorTransport   ProviderErrorClass = "transport"
	ProviderErrorUpstream    ProviderErrorClass = "upstream"
)

type ProviderErrorClassification added in v0.4.0 

type ProviderErrorClassification struct {
	StatusCode       int
	Retryable        bool
	FailoverEligible bool
	CircuitFailure   bool
}

ProviderErrorClassification is the provider-independent routing metadata for an operation failure. StatusCode is zero when no HTTP response was received.

type ProviderErrorClassifier added in v0.4.0 

type ProviderErrorClassifier interface {
	ProviderErrorClassification() ProviderErrorClassification
}

ProviderErrorClassifier exposes routing metadata without requiring callers to depend on a provider package's concrete error type.

type ProviderFailure added in v0.3.0 

type ProviderFailure struct {
	StatusCode int       `json:"status_code,omitempty"`
	RetryAfter string    `json:"retry_after,omitempty"`
	ObservedAt time.Time `json:"observed_at,omitempty"`
	Err        error     `json:"-"`
}

ProviderFailure contains only values needed for classification. Err is never serialized because transport errors may include credentials or private hosts.

type ProviderHealthEvidence added in v0.3.0 

type ProviderHealthEvidence struct {
	Status     ProviderHealthStatus `json:"status"`
	ErrorClass ProviderErrorClass   `json:"error_class"`
	Permanent  bool                 `json:"permanent,omitempty"`
	Retryable  bool                 `json:"retryable,omitempty"`
	RetryAfter time.Duration        `json:"retry_after,omitempty"`
	ObservedAt time.Time            `json:"observed_at,omitempty"`
}

func ClassifyProviderFailure added in v0.3.0 

func ClassifyProviderFailure(f ProviderFailure) ProviderHealthEvidence

ClassifyProviderFailure maps HTTP and transport outcomes into stable health semantics.

type ProviderHealthStatus added in v0.3.0 

type ProviderHealthStatus string
const (
	ProviderHealthUnknown   ProviderHealthStatus = "unknown"
	ProviderHealthHealthy   ProviderHealthStatus = "healthy"
	ProviderHealthDegraded  ProviderHealthStatus = "degraded"
	ProviderHealthUnhealthy ProviderHealthStatus = "unhealthy"
)

type ProviderModelDiscoverer added in v0.3.0 

type ProviderModelDiscoverer func(context.Context, ProviderConnection) ([]ModelInfo, error)

type ProviderOperationError added in v0.3.0 

type ProviderOperationError struct {
	Failure ProviderFailure
	Op      string
}

ProviderOperationError preserves classification inputs without exposing an upstream response body or transport error through serialized evidence.

func (*ProviderOperationError) Error added in v0.3.0 

func (e *ProviderOperationError) Error() string

func (*ProviderOperationError) ProviderErrorClassification added in v0.4.0 

func (e *ProviderOperationError) ProviderErrorClassification() ProviderErrorClassification

func (*ProviderOperationError) Unwrap added in v0.3.0 

func (e *ProviderOperationError) Unwrap() error

type ProviderOrchestrator added in v0.3.0 

type ProviderOrchestrator struct {
	// contains filtered or unexported fields
}

ProviderOrchestrator resolves only explicitly registered adapters. This is particularly important for Codex: its personal subscription OAuth flow must never fall through to anonymous OpenAI-compatible behavior.

func NewProviderOrchestrator added in v0.3.0 

func NewProviderOrchestrator() *ProviderOrchestrator

func (*ProviderOrchestrator) Connect added in v0.3.0 

func (o *ProviderOrchestrator) Connect(ctx context.Context, request ProviderConnectRequest) (ProviderConnectResult, error)

func (*ProviderOrchestrator) Register added in v0.3.0 

func (o *ProviderOrchestrator) Register(providerID string, adapter ProviderAdapter) error

type ProviderProbeSelector added in v0.3.0 

type ProviderProbeSelector func(ProviderConnection, []ModelInfo) []Target

type Resolution added in v0.1.1 

type Resolution struct {
	Targets  []Target `json:"targets"`
	Category string   `json:"category,omitempty"`
}

Resolution records both the resolved targets and the canonical category name, when the request addressed a category rather than a direct model.

type RouteConfig 

type RouteConfig struct {
	Targets     []Target `json:"targets"`               // Ordered targets to try
	Fallback    bool     `json:"fallback,omitempty"`    // Fall back to subsequent targets on error
	Description string   `json:"description,omitempty"` // User-facing description
}

RouteConfig configures an alias (failover chain) across multiple provider targets.

type StaticCredentialResolver 

type StaticCredentialResolver struct {
	Credentials map[string]Credential
}

StaticCredentialResolver returns pre-configured credentials from a map.

func (StaticCredentialResolver) Resolve 

func (s StaticCredentialResolver) Resolve(ctx context.Context, principal *Principal, providerID string) (*Credential, error)

type StreamIter added in v0.4.0 

type StreamIter interface {
	Next() ([]byte, error)
	Close() error
}

StreamIter yields complete SSE frames from an active provider stream.

type Support added in v0.3.0 

type Support uint8

Support is an explicitly tri-state capability value. Its zero value is SupportUnknown and marshals as "unknown" rather than false. 

const (
	SupportUnknown Support = iota
	SupportSupported
	SupportUnsupported
)

func (Support) IsKnown added in v0.3.0 

func (s Support) IsKnown() bool

func (Support) IsSupported added in v0.3.0 

func (s Support) IsSupported() bool

func (Support) MarshalJSON added in v0.3.0 

func (s Support) MarshalJSON() ([]byte, error)

func (Support) String added in v0.3.0 

func (s Support) String() string

func (*Support) UnmarshalJSON added in v0.3.0 

func (s *Support) UnmarshalJSON(data []byte) error

type Target 

type Target struct {
	Provider string `json:"provider"`
	Model    string `json:"model"`
}

Target represents a resolved provider and model.

func PublishExactTargets added in v0.3.0 

func PublishExactTargets(providerID string, catalog CatalogEvidence, probes []CompletionProbeEvidence, policy TargetPublicationPolicy) ([]Target, error)

PublishExactTargets converts catalog evidence to exact provider/model targets. Under verification policy, only targets with successful inference evidence publish.

type TargetPublicationPolicy added in v0.3.0 

type TargetPublicationPolicy string
const (
	PublishDiscoveredTargets TargetPublicationPolicy = "catalog_discovered"
	PublishVerifiedTargets   TargetPublicationPolicy = "inference_verified"
)

type TransportConfidence added in v0.4.0 

type TransportConfidence string
const (
	TransportConfidenceLow    TransportConfidence = "low"
	TransportConfidenceMedium TransportConfidence = "medium"
	TransportConfidenceHigh   TransportConfidence = "high"
)

type TransportDisposition added in v0.4.0 

type TransportDisposition string
const (
	TransportNative  TransportDisposition = "native"
	TransportAdapted TransportDisposition = "adapted"
	TransportReject  TransportDisposition = "reject"
)

type TransportInterface added in v0.4.0 

type TransportInterface struct {
	Surface ModelSurface `json:"surface"`
	Native  Support      `json:"native"`
}

TransportInterface is an available provider interface. Native records whether using this interface for its named surface preserves native provider semantics; SupportUnsupported therefore means available through adaptation, not unavailable.

type TransportPlan added in v0.4.0 

type TransportPlan struct {
	Disposition TransportDisposition  `json:"disposition"`
	Surface     ModelSurface          `json:"surface"`
	Confidence  TransportConfidence   `json:"confidence"`
	Losses      []translate.Loss      `json:"losses,omitempty"`
	Reason      TransportRejectReason `json:"reason,omitempty"`
}

TransportPlan is a deterministic decision over the supplied evidence.

func PlanTransport added in v0.4.0 

func PlanTransport(request TransportPlanRequest) TransportPlan

PlanTransport selects a native interface when one is confirmed, preserves unknown evidence as eligible at low confidence, and otherwise considers an explicitly reported adaptation. It performs no I/O and applies no gateway or provider policy.

type TransportPlanRequest added in v0.4.0 

type TransportPlanRequest struct {
	Operation            ModelOperation       `json:"operation"`
	Surface              ModelSurface         `json:"surface"`
	EvaluatedAt          time.Time            `json:"evaluated_at"`
	ExactTarget          bool                 `json:"exact_target"`
	Capabilities         ModelCapabilities    `json:"capabilities"`
	Interfaces           []TransportInterface `json:"interfaces"`
	Requirement          TransportRequirement `json:"requirement"`
	TranslationEvaluated bool                 `json:"translation_evaluated"`
	Translation          translate.Report     `json:"translation"`
}

TransportPlanRequest contains only evidence needed to choose a transport. Translation describes the conversion that would be used by an adapted plan.

type TransportRejectReason added in v0.4.0 

type TransportRejectReason string
const (
	TransportRejectInvalidRequirement     TransportRejectReason = "invalid_requirement"
	TransportRejectOperationUnsupported   TransportRejectReason = "operation_unsupported"
	TransportRejectNoInterface            TransportRejectReason = "no_interface"
	TransportRejectNativeRequired         TransportRejectReason = "native_required"
	TransportRejectExactTargetRequired    TransportRejectReason = "exact_target_required"
	TransportRejectNativeUnconfirmed      TransportRejectReason = "native_unconfirmed"
	TransportRejectTranslationUnevaluated TransportRejectReason = "translation_not_evaluated"
	TransportRejectTranslationLoss        TransportRejectReason = "translation_loss"
)

type TransportRequirement added in v0.4.0 

type TransportRequirement string

TransportRequirement constrains whether a request may use adaptation. 

const (
	TransportRequirementAny         TransportRequirement = "any"
	TransportRequirementNative      TransportRequirement = "native"
	TransportRequirementTransparent TransportRequirement = "transparent"
)

type UsageHook 

type UsageHook interface {
	RecordUsage(ctx context.Context, record UsageRecord)
}

UsageHook receives telemetry records after each completed request.

type UsageHookFunc 

type UsageHookFunc func(ctx context.Context, record UsageRecord)

UsageHookFunc is a functional adapter for UsageHook.

func (UsageHookFunc) RecordUsage 

func (f UsageHookFunc) RecordUsage(ctx context.Context, record UsageRecord)

type UsageRecord 

type UsageRecord struct {
	RequestID    string        `json:"request_id"`
	PrincipalID  string        `json:"principal_id,omitempty"`
	ProjectID    string        `json:"project_id,omitempty"`
	Provider     string        `json:"provider"`
	Model        string        `json:"model"`
	InputTokens  int           `json:"input_tokens"`
	OutputTokens int           `json:"output_tokens"`
	Duration     time.Duration `json:"duration"`
	Stream       bool          `json:"stream"`
	StatusCode   int           `json:"status_code"`
	Error        string        `json:"error,omitempty"`
}

UsageRecord holds metrics and token counts from a completed LLM call.

Source Files

View all Source files

Directories

Path Synopsis
zen
Package zen implements the reusable, storage-neutral OpenCode Zen anonymous admission contract.
 Package zen implements the reusable, storage-neutral OpenCode Zen anonymous admission contract.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.