Documentation
¶
Overview ¶
Card helpers keep every provider quota card close to the official provider card style (one short plan chip, one row per real limit window, no group-specific chrome, dollar balances as text or chips, never bars).
The management panel renders plugin cards generically: every summary metric becomes a chip, each group gets a header, and a bucket's Description becomes its row label (the SDK QuotaBucket has no display name). So card code must:
- leave Description empty and put the short row label in Window;
- emit at most MaxCardSummaryChips summary metrics, none repeating a bar;
- format dollar amounts in text with USD (exactly 2 decimals).
Package providerkit implements the CLIProxy provider plugin RPC contract. Provider implementations use the pinned SDK's interfaces directly.
Index ¶
- Constants
- func APIKey(storage []byte, metadata map[string]any) (string, error)
- func AssertModelNamespace(t TestReporter, ns string, models []pluginapi.ModelInfo)
- func AuthFields(storage []byte, metadata map[string]any) (map[string]any, error)
- func AuthProxyURL(storage []byte, metadata map[string]any, hostProxyURL string) string
- func CheckCard(resp pluginapi.QuotaFetchResponse) []string
- func CheckModelNamespace(ns string, models []pluginapi.ModelInfo) error
- func MapError(err error) *pluginabi.Error
- func NewHostHTTPClient(h Caller, callbackID string) pluginapi.HostHTTPClient
- func NewQuotaBucket(window string, remainingFraction float64, resetTime string) pluginapi.QuotaBucket
- func NewQuotaTransport() *http.Transport
- func NormalizePrefix(prefix string) (string, error)
- func ProvisionalParseExpiry(ctx context.Context) (time.Time, bool)
- func QuotaProxy(r *http.Request) (*url.URL, error)
- func QuotaWindowLabel(window string) string
- func RetryAfterSeconds(value string, now time.Time) int
- func USD(v float64) string
- func ValidateMetadata(m pluginapi.Metadata) error
- func ValidateModelNamespace(ns string) error
- func WithProxyURL(ctx context.Context, proxyURL string) context.Context
- type APIKeyAuth
- func (p APIKeyAuth) Identifier() string
- func (p APIKeyAuth) ParseAuth(_ context.Context, req pluginapi.AuthParseRequest) (pluginapi.AuthParseResponse, error)
- func (p APIKeyAuth) PollLogin(_ context.Context, _ pluginapi.AuthLoginPollRequest) (pluginapi.AuthLoginPollResponse, error)
- func (p APIKeyAuth) RefreshAuth(_ context.Context, req pluginapi.AuthRefreshRequest) (pluginapi.AuthRefreshResponse, error)
- func (p APIKeyAuth) StartLogin(_ context.Context, _ pluginapi.AuthLoginStartRequest) (pluginapi.AuthLoginStartResponse, error)
- type CachedQuota
- func (q *CachedQuota) DescribeQuota(context.Context, pluginapi.QuotaDescribeRequest) (pluginapi.QuotaDescribeResponse, error)
- func (q *CachedQuota) FetchQuota(ctx context.Context, r pluginapi.QuotaFetchRequest) (pluginapi.QuotaFetchResponse, error)
- func (q *CachedQuota) Identifier() string
- func (q *CachedQuota) Invalidate(authID string)
- func (q *CachedQuota) ResetQuota(context.Context, pluginapi.QuotaResetRequest) (pluginapi.QuotaResetResponse, error)
- func (q *CachedQuota) Store(r pluginapi.QuotaFetchRequest, out pluginapi.QuotaFetchResponse) error
- type Caller
- type CallerFunc
- type Definition
- type FetchModelsFunc
- type FetchQuotaFunc
- type KeyOwnership
- type ModelCatalog
- func (c *ModelCatalog) Invalidate(authID string)
- func (c *ModelCatalog) ModelsForAuth(ctx context.Context, req pluginapi.AuthModelRequest) (pluginapi.ModelResponse, error)
- func (c *ModelCatalog) RegisterModels(ctx context.Context, req pluginapi.ModelRegistrationRequest) (pluginapi.ModelRegistrationResponse, error)
- func (c *ModelCatalog) StaticModels(ctx context.Context, req pluginapi.StaticModelRequest) (pluginapi.ModelResponse, error)
- type ModelCatalogConfig
- type ModelOverride
- type Passthrough
- func (p Passthrough) CountTokens(ctx context.Context, r pluginapi.ExecutorRequest) (pluginapi.ExecutorResponse, error)
- func (p Passthrough) Execute(ctx context.Context, r pluginapi.ExecutorRequest) (pluginapi.ExecutorResponse, error)
- func (p Passthrough) ExecuteStream(ctx context.Context, r pluginapi.ExecutorRequest) (pluginapi.ExecutorStreamResponse, error)
- func (p Passthrough) HttpRequest(ctx context.Context, r pluginapi.ExecutorHTTPRequest) (pluginapi.ExecutorHTTPResponse, error)
- func (p Passthrough) Identifier() string
- type PrefixClaims
- type QuotaHTTPClient
- type Runtime
- type TestReporter
Constants ¶
const ( CardWindowFiveHour = "5h" CardWindowWeekly = "weekly" CardWindowMonthly = "monthly" CardWindowDaily = "daily" )
Canonical window ids for quota rows, matching the spellings the panel resolves to a countdown timeline and stable sort order ("5h", "weekly"). Window values stay short ids so long labels are never truncated; the panel shows the Window as the row label. Other windows ("daily", "monthly", provider-specific names) render without a countdown but keep their reset text. Keep values lowercase: the panel matches case-insensitively.
const MaxCardSummaryChips = 3
MaxCardSummaryChips bounds the plan-row chip count (plan chip excluded). The panel renders every summary metric as a chip with no limit, so an unbounded summary becomes a wall of chips next to the official cards.
const MaxModelsBody = 1 << 20
MaxModelsBody bounds a discovery response parse (default 1 MiB). The check runs against the buffered HTTPResponse.Body length after the host client returns: an oversized buffered body is treated as a failed fetch (last-good/static fallback) and is not parsed. This is a post-return parse cap, not a transport memory bound: the host may already have buffered an arbitrarily large body in memory before the kit sees it. Redirects and hop policy remain host-owned; the kit cannot disable them through HostHTTPClient and does not bypass host proxy/logging policy to try.
const ModelsFetchTimeout = 15 * time.Second
ModelsFetchTimeout is the kit-owned deadline for one model-discovery fetch. ModelsForAuth derives context.WithTimeout(reqCtx, ModelsFetchTimeout) explicitly; it never assumes a deadline on the incoming callback. When the parent already carries an earlier deadline, the derived context yields the earlier of the two.
const QuotaCacheTTL = 60 * time.Second
QuotaCacheTTL is the maximum age of a successful per-auth quota observation.
const QuotaHTTPTimeout = 15 * time.Second
QuotaHTTPTimeout bounds the entire quota HTTP exchange, including body reads.
Variables ¶
This section is empty.
Functions ¶
func APIKey ¶
APIKey reads a nonempty api_key using host metadata precedence. It never includes the key or raw storage in an error.
func AssertModelNamespace ¶ added in v0.2.0
func AssertModelNamespace(t TestReporter, ns string, models []pluginapi.ModelInfo)
AssertModelNamespace fails the test if any model id would not be registered under ns.
func AuthFields ¶
AuthFields applies the same merge precedence as persisted host auth storage: Metadata overrides StorageJSON. Callers own the returned map.
func AuthProxyURL ¶ added in v0.2.0
AuthProxyURL resolves the outbound proxy for direct (non-host) requests made on behalf of one auth, with the host's precedence: the auth's proxy_url (metadata overrides storage), then the host's global proxy-url. An empty result means "inherit": the environment proxy settings apply.
func CheckCard ¶ added in v0.2.2
func CheckCard(resp pluginapi.QuotaFetchResponse) []string
CheckCard reports official-card-style violations in a quota response:
- any bucket Description (leave it empty so Window stays the row label);
- any bucket with a blank Window (the row would have no label);
- more than MaxCardSummaryChips summary metrics;
- a summary metric repeating a bar (same or overlapping label text).
An empty return means the card follows the shared style. Plugin quota tests assert on this alongside their exact-shape golden tests.
func CheckModelNamespace ¶ added in v0.2.0
CheckModelNamespace is the reusable conformance check for plugin test suites: it reports an error listing every model id that would not be registered under ns. Call it with every id the plugin can emit (static fallback, discovery fixtures and dynamic refresh paths).
func MapError ¶
MapError preserves explicit, public RPC errors. Unclassified errors are intentionally opaque: transport and parsing errors can contain credentials. Providers must put only safe, user-facing text in pluginabi.Error.Message.
func NewHostHTTPClient ¶
func NewHostHTTPClient(h Caller, callbackID string) pluginapi.HostHTTPClient
NewHostHTTPClient creates a callback-scoped SDK client. Runtime injects this automatically into each request. Never retain it past that request/stream. Cancellation uses the host's operation IDs, not abandoned blocking goroutines.
func NewQuotaBucket ¶ added in v0.2.2
func NewQuotaBucket(window string, remainingFraction float64, resetTime string) pluginapi.QuotaBucket
NewQuotaBucket builds one quota bar row: a canonical window label, the remaining fraction, and the reset time. It never sets Description, so the panel shows the short Window label instead of a sentence. remainingFraction is the fraction remaining in [0, 1] (unclamped; NaN fails host decoding), and resetTime is an RFC 3339 instant or "" when the window has no reset (the panel then shows "-").
func NewQuotaTransport ¶ added in v0.2.0
NewQuotaTransport clones the standard transport with QuotaProxy as its proxy (the WithProxyURL setting, else normal environment proxy behavior). It speaks HTTP/1.1 only: Go reuses a cached HTTP/2 connection by destination before it consults Proxy, so one auth's poll could ride another auth's proxy. HTTP/1.1 connections are pooled per proxy.
func NormalizePrefix ¶ added in v0.5.0
NormalizePrefix applies the host filestore's prefix contract to a plugin-returned prefix: surrounding whitespace and "/" are trimmed, and an inner "/" is rejected with invalid_auth. The plugin path pins whatever the plugin returns verbatim, so silently mapping a slashed prefix to unpinned would leave two files sharing one host namespace while the tracker believes both are unpinned. The programme rule is never to silently drop: reject.
func ProvisionalParseExpiry ¶ added in v0.4.1
ProvisionalParseExpiry reports whether ctx belongs to a provisional pre-save validation parse (see withProvisionalParse) and, if so, when the session behind it expires. Parses the host initiates for real auth files are never provisional.
func QuotaProxy ¶ added in v0.2.0
QuotaProxy is an http.Transport Proxy function that applies the setting from WithProxyURL and otherwise falls back to the environment. Custom transports should start from NewQuotaTransport rather than set it alone: a transport shared across proxies must not reuse HTTP/2 connections.
func QuotaWindowLabel ¶ added in v0.2.2
Canonical quota card labels.
The plugin QuotaBucket type has no displayName field, so the panel renders each bucket row with the label description ?? window (group displayName is a separate group header). A bucket Description therefore becomes the row label and tooltip while Window is hidden, so long explanations must never be set as Description. Keep bucket labels short and window-style with QuotaWindowLabel and NewQuotaBucket, and put absolute amounts in the label only where they add information (for example, "Weekly key limit · $5.00 of $5.00 left").
Only the listed spellings canonicalize to the CardWindow* constants; anything else (including typos and the empty string) passes through verbatim after trimming, so callers must spell windows deliberately.
func RetryAfterSeconds ¶
RetryAfterSeconds parses HTTP Retry-After seconds or an HTTP date, rounding a future date up. Invalid or past values yield zero. No wall clock is hidden.
func USD ¶ added in v0.2.2
USD formats dollars with exactly 2 decimals, sign first: 25 -> $25.00, -0.02 -> -$0.02. Amounts in bucket text must use this, never a higher-precision debug format; chip currency metrics are formatted panel-side from the numeric value. Non-finite inputs render as $0.00: JSON-decoded amounts are always finite, so this branch is unreachable defense so a card can never show $NaN.
func ValidateMetadata ¶
ValidateMetadata checks the metadata required by the host and public registry.
func ValidateModelNamespace ¶ added in v0.2.0
ValidateModelNamespace reports whether ns is usable as a fixed model namespace: a bare segment such as "synthetic", without the trailing slash.
func WithProxyURL ¶ added in v0.2.0
WithProxyURL attaches a proxy setting to ctx for QuotaHTTPClient requests. It accepts the host's proxy syntax: "" inherits the environment, "direct" or "none" bypasses proxies, otherwise an http, https, socks5 or socks5h URL.
Types ¶
type APIKeyAuth ¶
APIKeyAuth implements the SDK auth provider for API-key-only providers. Validate may validate additional provider-owned fields; its errors are redacted by Runtime unless explicitly represented as public pluginabi.Error values. It is stateless: duplicate-file/account policy belongs to the provider.
There is no login flow: StartLogin and PollLogin always fail with api_key_only, so API-key providers show no login card in the management UI and auth files load through the host's generic file path. The kit's script-based API-key login page was removed; OpenRouter-style OAuth/PKCE logins live in the provider, not in the kit.
func (APIKeyAuth) Identifier ¶
func (p APIKeyAuth) Identifier() string
func (APIKeyAuth) ParseAuth ¶
func (p APIKeyAuth) ParseAuth(_ context.Context, req pluginapi.AuthParseRequest) (pluginapi.AuthParseResponse, error)
func (APIKeyAuth) PollLogin ¶
func (p APIKeyAuth) PollLogin(_ context.Context, _ pluginapi.AuthLoginPollRequest) (pluginapi.AuthLoginPollResponse, error)
func (APIKeyAuth) RefreshAuth ¶
func (p APIKeyAuth) RefreshAuth(_ context.Context, req pluginapi.AuthRefreshRequest) (pluginapi.AuthRefreshResponse, error)
func (APIKeyAuth) StartLogin ¶
func (p APIKeyAuth) StartLogin(_ context.Context, _ pluginapi.AuthLoginStartRequest) (pluginapi.AuthLoginStartResponse, error)
type CachedQuota ¶
type CachedQuota struct {
// contains filtered or unexported fields
}
CachedQuota implements a non-resettable SDK quota provider with per-account single-flight fetching, copy isolation, and a configurable cache TTL (default 60 seconds). Use Store to incorporate normalized inference response headers by auth identity without another HTTP poll or access to the host's credential context.
func NewCachedQuota ¶
func NewCachedQuota(provider, display string, fetch FetchQuotaFunc) *CachedQuota
NewCachedQuota creates an empty quota cache with the default 60-second TTL. The identity and callback are fixed for its lifetime. No background polls or provider requests are started.
func NewCachedQuotaWithTTL ¶ added in v0.5.0
func NewCachedQuotaWithTTL(provider, display string, fetch FetchQuotaFunc, ttl time.Duration) *CachedQuota
NewCachedQuotaWithTTL creates an empty quota cache with a per-plugin TTL for successful per-auth observations. A non-positive TTL falls back to QuotaCacheTTL. Errors are never cached regardless of the TTL. Polls that are expensive or rate-limited (for example a key-envelope read behind a 5-minute connector poll) use a longer TTL so steady state stays within the provider's budget; header observations share the same TTL.
func (*CachedQuota) DescribeQuota ¶
func (q *CachedQuota) DescribeQuota(context.Context, pluginapi.QuotaDescribeRequest) (pluginapi.QuotaDescribeResponse, error)
func (*CachedQuota) FetchQuota ¶
func (q *CachedQuota) FetchQuota(ctx context.Context, r pluginapi.QuotaFetchRequest) (pluginapi.QuotaFetchResponse, error)
func (*CachedQuota) Identifier ¶
func (q *CachedQuota) Identifier() string
func (*CachedQuota) Invalidate ¶
func (q *CachedQuota) Invalidate(authID string)
Invalidate discards header observations and all polled credential versions for the given auth ID (or AuthIndex when used instead of AuthID).
func (*CachedQuota) ResetQuota ¶
func (q *CachedQuota) ResetQuota(context.Context, pluginapi.QuotaResetRequest) (pluginapi.QuotaResetResponse, error)
func (*CachedQuota) Store ¶
func (q *CachedQuota) Store(r pluginapi.QuotaFetchRequest, out pluginapi.QuotaFetchResponse) error
Store replaces an auth-identity observation (for example, from a provider quota header). Only AuthID (or AuthIndex when AuthID is absent) and an optional matching Provider are required. When the caller knows the credential context (StorageJSON/Metadata/Attributes), include it: the observation is then bound to that credential generation and is never served to a fetch carrying a different credential, which closes the rotated-key-under-reused-identity staleness window (G-K3). Callers without credential context (UsageRecords) store unbound observations, served to any credential of the identity until the TTL expires. Fresh observations take precedence over polls, including older in-flight polls. Data is cloned on store and read. Invalidate when reusing an identity for a different credential without binding context, or recreate the cache on reconfiguration.
type Caller ¶
Caller is the raw host callback boundary. Results are pluginabi.Envelope JSON. Implementations must support concurrent calls. Do not log request/response data.
type CallerFunc ¶
CallerFunc adapts a callback function to Caller.
type Definition ¶
type Definition struct {
Metadata pluginapi.Metadata
Capabilities pluginapi.Capabilities
// ModelNamespace declares the fixed vendor namespace for every model id
// this plugin registers or serves.
ModelNamespace string
Configure func(context.Context, []byte) error
OnQuiesce func(context.Context) error
OnShutdown func(context.Context) error
}
Definition wires SDK capabilities and lifecycle hooks. Configure receives only this plugin's configuration subtree, on both registration and reconfiguration. Hooks run serially after active calls/streams are canceled and drained. Capabilities and metadata must not be mutated after New. Capability methods must be concurrency-safe and honor context cancellation.
ModelNamespace is the plugin's fixed vendor namespace (a bare segment such as "synthetic"; the kit appends the slash). Any provider that serves models made by another vendor must namespace its model ids so they never clash when the official provider is added later. A definition that registers or serves models (ModelRegistrar or ModelProvider set) must declare it; an empty, missing or malformed namespace is a startup error. Every static, discovered and dynamic model list is filtered through it: ids outside "<namespace>/" (including doubled "<namespace>/<namespace>/…") are dropped with a logged error and never served bare. The host auth-file `prefix` is an outer host clone and does not replace this namespace.
type FetchModelsFunc ¶ added in v0.5.0
type FetchModelsFunc func(ctx context.Context, req pluginapi.AuthModelRequest) ([]pluginapi.ModelInfo, error)
FetchModelsFunc returns candidate models for one auth. Provider-owned protocol parsing stays outside the kit: custom Fetch implementations that use req.HTTPClient directly are responsible for their own post-return parse-size checks (see MaxModelsBody) and must honor context cancellation. Fetch runs foreground-only inside ModelsForAuth with the request-scoped host client; the kit starts no goroutine and retains no client past the callback.
func OpenAIModelsFetcher ¶ added in v0.5.0
func OpenAIModelsFetcher(baseURL, path string, headers func(pluginapi.AuthModelRequest) http.Header) FetchModelsFunc
OpenAIModelsFetcher fetches an OpenAI-style {"data":[{"id":…}]} list through the host HTTP client (foreground use inside ModelsForAuth, proxy/logging host-owned). baseURL plus path form the request URL (a single slash joins the parts; an empty path requests baseURL alone). headers, when non-nil, supplies per-request headers such as the auth bearer derived from the request; the kit never derives credentials itself. A non-2xx status, a malformed body, or a body larger than MaxModelsBody is a fetch error (never parsed, never cached); the catalog falls back to last-good/static per the uniform pipeline.
type FetchQuotaFunc ¶
type FetchQuotaFunc func(context.Context, pluginapi.QuotaFetchRequest) (pluginapi.QuotaFetchResponse, error)
FetchQuotaFunc fetches and normalizes one account. Provider-owned protocol parsing stays outside the kit. Errors are never cached; stale data is not silently returned. Honor context cancellation.
type KeyOwnership ¶ added in v0.4.1
type KeyOwnership struct {
// contains filtered or unexported fields
}
KeyOwnership is the shared filename-ownership duplicate guard for ParseAuth pipelines: one credential digest may be configured in only one auth filename at a time, so a duplicate file is rejected while a watcher's repeated parse of the same file is safe. The caller computes the digest over its full credential identity — just the key when keys are unique per account, or a composite such as account ID plus token when one key may serve several accounts (the same split FilenameFields makes for login file names). Only digests and filenames are retained, never keys.
A committed (host-initiated, non-provisional) parse records ownership until the same filename is parsed with a different digest or the plugin restarts; deleting or renaming an auth file therefore still needs a plugin restart to free its key, as with the hand-rolled guards this replaces. A provisional (login-validation) parse instead reserves the digest only until the login session expires: it rejects a key another filename already owns, and blocks other filenames while the login is in flight, but the reservation is dropped lazily once expired. An abandoned login, an evicted session, or a login the host fails to persist can thus hold a key for at most the login TTL, never until restart. Ownership of a completed login commits when the host parses the auth file it saved.
The zero value is ready to use. KeyOwnership is safe for concurrent use.
func (*KeyOwnership) Claim ¶ added in v0.4.1
Claim checks fileName's ownership of digest and records it. digest is caller-chosen: hash the full credential identity (key alone, or account plus key for providers whose keys are shared across accounts). For a provisional parse (ProvisionalParseExpiry reports true for ctx) the claim is only reserved until the login session expires; any other parse commits ownership. Claim returns a duplicate_auth error when another filename owns or has reserved the digest.
type ModelCatalog ¶ added in v0.5.0
type ModelCatalog struct {
// contains filtered or unexported fields
}
ModelCatalog implements pluginapi.ModelProvider and pluginapi.ModelRegistrar: RegisterModels and StaticModels serve Static through the uniform pipeline, and ModelsForAuth runs fetch → glob → overrides → Validate hook → cache → serve with req.HTTPClient.
Every ModelsForAuth invocation attempts a fresh fetch (no TTL skip); only concurrent invocations sharing one key coalesce through single-flight. A waiter whose shared leader failed only because the leader's own context was cancelled retries as a new leader rather than inheriting another caller's cancellation. Results fall back fresh → durable last-good → static, so a transient fetch error never drops a known model: an empty upstream result (before glob filtering) is a failed fetch, while a filtered-empty result is a legitimate policy outcome that is served and cached. A refresh whose candidates are all rejected by the namespace filter is likewise a failed fetch, so an incompatible upstream payload never erases last-good. Errors are never cached and a failed fetch, failed Validate or oversized body never replaces last-good. The last-good store has no TTL eviction and persists until a newer successful fetch replaces it or Invalidate runs; it is still bounded at maxLastGoodGenerations credential generations (FIFO eviction), so plugins sharing one catalog across many auths should call Invalidate on credential rotation. Fetch verification is catalog availability only: it does not bypass credential validity, auth disablement, provider entitlements or executor authorization, which the host and executor still enforce per-request.
Concurrency-safe; the constructor starts no goroutine and there is no background fetch. A catalog is fixed once installed; recreate it on reconfiguration, exactly like CachedQuota.
func NewModelCatalog ¶ added in v0.5.0
func NewModelCatalog(config ModelCatalogConfig) *ModelCatalog
NewModelCatalog creates an empty model catalog for its lifetime.
func NewModelCatalogWithTTL ¶ added in v0.5.0
func NewModelCatalogWithTTL(config ModelCatalogConfig, ttl time.Duration) *ModelCatalog
NewModelCatalogWithTTL records a per-plugin TTL for the freshness markers. Like NewCachedQuotaWithTTL it mirrors the old constructor, but it does not introduce a skip gate: ModelsForAuth fetches on every invocation regardless of the TTL (see ModelCatalogConfig.TTL).
func (*ModelCatalog) Invalidate ¶ added in v0.5.0
func (c *ModelCatalog) Invalidate(authID string)
Invalidate discards the last-good entry (and any in-flight fetch) for the given auth ID. A fetch completing after Invalidate returns its result to its own callback but never resurrects the invalidated generation: its last-good write checks the in-flight entry is still current under lock. It fires no host callback (see the package freshness notes: recovery waits for the next host event).
func (*ModelCatalog) ModelsForAuth ¶ added in v0.5.0
func (c *ModelCatalog) ModelsForAuth(ctx context.Context, req pluginapi.AuthModelRequest) (pluginapi.ModelResponse, error)
ModelsForAuth fetches fresh models for one auth and serves the uniform pipeline result. Concurrent calls for the same credential generation share one fetch; sequential calls each fetch.
func (*ModelCatalog) RegisterModels ¶ added in v0.5.0
func (c *ModelCatalog) RegisterModels(ctx context.Context, req pluginapi.ModelRegistrationRequest) (pluginapi.ModelRegistrationResponse, error)
RegisterModels serves the static list through the uniform pipeline, for development-time registration.
func (*ModelCatalog) StaticModels ¶ added in v0.5.0
func (c *ModelCatalog) StaticModels(ctx context.Context, req pluginapi.StaticModelRequest) (pluginapi.ModelResponse, error)
StaticModels serves the static list through the uniform pipeline.
type ModelCatalogConfig ¶ added in v0.5.0
type ModelCatalogConfig struct {
// Namespace is the kit vendor namespace; upstream ids are prefixed
// with it before serving (the same rule the runtime enforces with
// filterModels, so it must match Definition.ModelNamespace). An empty
// namespace serves ids unchanged.
Namespace string
// Static is the fallback served through the same glob/override/
// namespace pipeline as fresh and last-good models, and the list
// served by RegisterModels and StaticModels.
Static []pluginapi.ModelInfo
// Fetch fetches fresh candidates per auth. Nil means static-only: no
// fetch is attempted and ModelsForAuth serves the static list.
Fetch FetchModelsFunc
// TTL is a compatibility field. It does NOT gate ModelsForAuth
// fetches: every invocation attempts a fresh fetch, so a large TTL
// cannot lose a one-shot catalog-change callback. It bounds the
// freshness-marker store for diagnostics only and never erases the
// durable last-good store.
TTL time.Duration
// Overrides keys by upstream (pre-namespace) id.
Overrides map[string]ModelOverride
// Allow and Deny are path globs matched against the upstream id;
// deny wins and an empty allowlist allows everything else. An invalid
// pattern simply never matches.
Allow, Deny []string
// Validate is the post-fetch hook (glob and overrides applied, not
// yet namespace-prefixed); e.g. probe validation that stays
// plugin-side. It runs only on the fresh path: last-good and static
// results were already validated or are trusted as configured. An
// error is a failed fetch and never replaces last-good. Its
// returned ids preserve the upstream glob decision: ids the
// pipeline already approved survive (including ContributorID
// rewrites whose source was allowed), deny always wins, and ids
// Validate invented face the full allow/deny check, so Validate
// cannot bypass the glob policy.
Validate func(ctx context.Context, req pluginapi.AuthModelRequest, models []pluginapi.ModelInfo) ([]pluginapi.ModelInfo, error)
}
ModelCatalogConfig configures a ModelCatalog. Do not mutate the fields after NewModelCatalog; copy isolation covers the static list, overrides and globs taken at construction.
type ModelOverride ¶ added in v0.5.0
type ModelOverride struct {
DisplayName string
ContextLength int64
MaxCompletionTokens int64
ContributorID string
Thinking *pluginapi.ThinkingSupport
}
ModelOverride tunes one upstream model. Zero/empty fields keep the discovered value; it never deletes a model. ContributorID, when non-empty, makes the emitted id that value (owner rule: the contributor id is preferred whenever a model has one). Overrides key by upstream (pre-namespace) id.
type Passthrough ¶
type Passthrough struct {
Provider string
BuildRequest func(context.Context, pluginapi.ExecutorRequest) (pluginapi.HTTPRequest, error)
PrepareHTTP func(context.Context, pluginapi.ExecutorHTTPRequest) (pluginapi.HTTPRequest, error)
ClassifyError func(pluginapi.HTTPResponse) error
TokenCount func(context.Context, pluginapi.ExecutorRequest) (pluginapi.ExecutorResponse, error)
}
Passthrough is a thin executor. BuildRequest selects the provider protocol, endpoint and authentication; it must preserve the host's final business payload (the host already applied payload rules). No protocol translation is done here. ExecuteStream re-frames SSE upstream bytes into one host chunk per data event: data lines for translators, bare JSON for OpenAI chat/Gemini clients, full named events and standalone comment lines for native SSE clients. Other Content-Types pass through unchanged; they are not parsed as SSE. Callers must not frame again. HttpRequest copies the supplied HTTP request unless PrepareHTTP is provided. Every network operation uses the injected HostHTTPClient; there is no fallback. Provider-specific quota/exhaustion classification belongs in ClassifyError. Hooks are immutable after construction and must support concurrent calls.
func (Passthrough) CountTokens ¶
func (p Passthrough) CountTokens(ctx context.Context, r pluginapi.ExecutorRequest) (pluginapi.ExecutorResponse, error)
CountTokens uses TokenCount when supplied. Otherwise it returns an explicitly estimated one-token-per-four-Unicode-codepoints upper-rounded payload count, not a model tokenizer or a billable upstream token-count request.
func (Passthrough) Execute ¶
func (p Passthrough) Execute(ctx context.Context, r pluginapi.ExecutorRequest) (pluginapi.ExecutorResponse, error)
func (Passthrough) ExecuteStream ¶
func (p Passthrough) ExecuteStream(ctx context.Context, r pluginapi.ExecutorRequest) (pluginapi.ExecutorStreamResponse, error)
func (Passthrough) HttpRequest ¶
func (p Passthrough) HttpRequest(ctx context.Context, r pluginapi.ExecutorHTTPRequest) (pluginapi.ExecutorHTTPResponse, error)
func (Passthrough) Identifier ¶
func (p Passthrough) Identifier() string
type PrefixClaims ¶ added in v0.5.0
type PrefixClaims struct {
// contains filtered or unexported fields
}
PrefixClaims tracks single-segment host prefixes claimed by auth files so two credentials cannot silently share one pinning namespace (G-K1). The host strips a credential's prefix before upstream dispatch, so two accounts sharing a prefix merge into one namespace and defeat per-account pinning. APIKeyAuth stays stateless by design; providers own this optional tracker and the lifecycle that applies or removes their auths. Claim is an immediate update, not a reservation tied to a host parse/install transaction. Providers must serialize claims with committed auth changes; speculative ParseAuth calls alone cannot guarantee uniqueness among installed host auths.
Only non-empty, slash-free prefixes are claimed. The host pins the returned prefix verbatim (TrimSpace only). Use APIKeyAuth's parsed Prefix, or call NormalizePrefix and use its result both for Claim and the returned auth. Claim normalizes for comparison but cannot rewrite a caller's auth value. Empty prefixes are the normal unpinned state shared by any number of files. Credential identity is the SHA-256 of the provider's credential bytes (for example the trimmed API key); only digests are retained, never key material.
func (*PrefixClaims) Claim ¶ added in v0.5.0
func (t *PrefixClaims) Claim(file, prefix string, credential []byte) error
Claim records file's use of prefix for the given credential. It returns a duplicate_prefix *pluginabi.Error (via rpcError, HTTP 400) naming both files when a different credential already claims prefix, and duplicate_auth when the same credential is already configured in another file. Reparsing the same file, changing a file's key, or moving a file between prefixes updates the tracker instead of erroring. A prefix containing "/" is rejected with invalid_auth: the host pins it verbatim on the plugin path, so it can never be treated as unpinned.
func (*PrefixClaims) Release ¶ added in v0.5.0
func (t *PrefixClaims) Release(file string)
Release forgets every claim held by file. Providers call it after an auth has been removed from active use, not merely when a speculative parse fails, so a removed auth cannot squat on a prefix or mask a genuine duplicate.
type QuotaHTTPClient ¶
type QuotaHTTPClient struct {
// contains filtered or unexported fields
}
QuotaHTTPClient is a bounded, secret-redacting polling helper for quota/discovery reads. It performs no logging. Errors omit URLs, headers, bodies and transport text. This is deliberately separate from HostHTTPClient: ABI 1 / schema 10 has no no-redirect setting. Executors must always use the host client instead. The transport may implement a provider's deployment proxy policy. Its RoundTrip method must NOT follow redirects itself (the standard transports do not).
func NewQuotaHTTPClient ¶
func NewQuotaHTTPClient(transport http.RoundTripper) *QuotaHTTPClient
NewQuotaHTTPClient sets a 15s timeout, no redirects and a 1MiB response cap. With nil transport it uses NewQuotaTransport.
func (*QuotaHTTPClient) CloseIdleConnections ¶
func (c *QuotaHTTPClient) CloseIdleConnections()
CloseIdleConnections releases pooled polling connections during plugin shutdown.
func (*QuotaHTTPClient) Do ¶
func (c *QuotaHTTPClient) Do(ctx context.Context, r pluginapi.HTTPRequest) (pluginapi.HTTPResponse, error)
Do reads a quota/discovery resource. Only GET and POST are allowed: quota endpoints that mint or refresh a read-only view (GraphQL, key envelope reads) use POST without becoming an inference or mutation path, and every other method stays rejected. Successful bodies are returned to the provider parser; callers must never log them or credential-bearing headers.
type Runtime ¶
type Runtime struct {
// contains filtered or unexported fields
}
Runtime owns registration, dispatch and the lifetime of asynchronous work.
func New ¶
func New(d Definition, h Caller) (*Runtime, error)
New validates a definition. The runtime starts inactive until plugin.register. Supported capabilities are auth, models, executor, quota, usage and management; unrelated SDK capabilities are rejected instead of silently advertised without dispatch.
type TestReporter ¶ added in v0.2.0
TestReporter is the subset of testing.T used by AssertModelNamespace, so plugin suites in any harness can use the conformance check.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cabi exports the ABI 1 function table for Go c-shared provider plugins.
|
Package cabi exports the ABI 1 function table for Go c-shared provider plugins. |
|
examples
|
|
|
echo
command
Echo is a mock-upstream-only provider example, not a production integration.
|
Echo is a mock-upstream-only provider example, not a production integration. |