providerkit

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: MIT Imports: 18 Imported by: 0

README

CLIProxy provider kit

Shared Go building blocks for first-class CLIProxyAPI provider plugins. This repository is a library, not an installable provider or a scheduler. The echo plugin is a loopback-only example for tests.

  • SDK: github.com/router-for-me/CLIProxyAPI/v8 v8.0.48
  • Native C ABI 1, RPC schema 9; Go 1.26.10+, CGO and a C compiler
  • Native libraries built with -buildmode=c-shared, not Go's plugin package
  • Linux is the CI-tested target. Match the deployment architecture and libc; build provider release artifacts in golang:1.26-bookworm for a Debian-bookworm host. The exact host/plugin Go patch versions need not match across the C ABI.

Provider repository usage

Add the kit at a reviewed commit (or a released version when available), and pin the SDK to v8.0.48. No kit release/tag is required to use a commit through Go's pseudo-version resolution.

go get github.com/TogetherWeOwn/cliproxy-provider-kit@<reviewed-commit>
go get github.com/router-for-me/CLIProxyAPI/v8@v8.0.48

Implement the SDK interfaces for your provider and pass them to providerkit.New. In the provider's package main, export a runtime factory from init. A c-shared library does not run main.

package main

import (
    providerkit "github.com/TogetherWeOwn/cliproxy-provider-kit"
    "github.com/TogetherWeOwn/cliproxy-provider-kit/cabi"
    "github.com/router-for-me/CLIProxyAPI/v8/sdk/pluginapi"
)

func main() {}

func init() {
    if err := cabi.Export(func(host providerkit.Caller) (*providerkit.Runtime, error) {
        return providerkit.New(providerkit.Definition{
            Metadata: pluginapi.Metadata{
                Name: "example-provider", Version: "0.1.0",
                Author: "Example Organization",
                GitHubRepository: "https://github.com/example/provider",
            },
            Capabilities: pluginapi.Capabilities{
                AuthProvider: providerkit.APIKeyAuth{Provider: "example-provider"},
                // Add ModelRegistrar / ModelProvider, Executor, QuotaProvider,
                // and optionally UsagePlugin implemented by your provider.
            },
        }, host)
    }); err != nil {
        panic("cannot install provider factory")
    }
}

Use examples/echo/main.go for a complete executable example, including lifecycle configuration, static/per-auth models, auth headers, host HTTP execution and a cached quota provider. Its only permitted upstream is a literal loopback address supplied by the test harness. It has no production endpoint or real credentials.

Public Go API
API Provider responsibility / behavior
cabi.Export(cabi.Factory) error Install once during main.init; the factory receives the host callback boundary and constructs a fresh runtime per native initialization.
New(Definition, Caller) (*Runtime, error) Supply validated SDK metadata, capabilities, and optional Configure, OnQuiesce, OnShutdown hooks.
Runtime.Call(ctx, method, raw) []byte / Shutdown(ctx) RPC envelopes and logical shutdown; normally used by the C shim, not provider code.
ValidateMetadata(pluginapi.Metadata) error Require nonempty Name/Version/Author/GitHubRepository and an HTTPS GitHub owner/repository URL.
APIKeyAuth Stateless SDK AuthProvider; top-level type dispatch, validation, stable IDs, API-key-only login errors, no-op refresh. Optional Validate checks provider fields.
AuthFields(storage, metadata) / APIKey(storage, metadata) Decode object storage with host-compatible metadata precedence; never include credential values in errors.
Passthrough SDK executor using only the injected HostHTTPClient. Supply BuildRequest; optionally PrepareHTTP, ClassifyError, and TokenCount.
MapError(error) *pluginabi.Error Preserve explicit code/status/retry fields; redact unclassified errors.
RetryAfterSeconds(header, now) int Parse HTTP seconds/date retry guidance without a hidden clock.
NewCachedQuota(provider, display, FetchQuotaFunc) *CachedQuota Non-resettable SDK QuotaProvider; successful per-auth cache for QuotaCacheTTL = 60 seconds, coalesced concurrent fetches, copy isolation.
CachedQuota.Store(request, response) error / Invalidate(authID) Incorporate provider quota headers by auth identity, or invalidate header observations and all polled credential versions for an auth.
NewQuotaHTTPClient(http.RoundTripper) *QuotaHTTPClient Read-only polling, 15-second total timeout, 1 MiB body cap, no redirects, redacted errors; CloseIdleConnections for teardown.
NewHostHTTPClient(Caller, callbackID) pluginapi.HostHTTPClient Scoped callback adapter, normally injected by Runtime. Never retain it beyond its request/stream.
Caller / CallerFunc Raw concurrent host callback boundary; useful for isolated contract tests.

Definition.Capabilities supports auth, model registration, model discovery, executor, quota and usage. Other SDK capabilities are rejected rather than advertised without handlers. Executor definitions must declare input/output formats and may set scope both, static, or oauth (default both). Let the host translate protocols to those formats; the passthrough helper does not translate or rewrite business payloads after host payload rules.

Auth files and persistence

One object per account, using the provider key as top-level type:

{"type":"example-provider","api_key":"example-key","label":"Example","prefix":"example-provider","weight":2,"priority":0,"disabled":false}

Create real auth files privately with mode 0600; never commit them. The default ID is provider: plus the first 12 hex characters of SHA-256 of the trimmed API key, or an explicit string id. Changing filename does not change identity; duplicate files with the same key derive the same identity. Account/file deduplication and rejection policy belong to provider repos, not this stateless parser.

Storage remains a JSON object with provider fields; host-owned id/label/prefix/proxy_url/disabled/weight/priority move to SDK metadata/attributes. The host merges metadata over storage and forces type. APIKey and AuthFields preserve that precedence on execute/refresh/quota calls. Wrong provider type returns Handled:false. Malformed objects, duplicate top-level fields, missing keys, or wrongly typed routing fields return an error without a half-auth.

Executor, errors and streaming

BuildRequest must choose a trusted endpoint and add the provider's auth headers. Preserve the final ExecutorRequest.Payload; CLIProxy has already applied its payload policy. PrepareHTTP can inject auth and restrict the URL for SDK HttpRequest; without it that method forwards the supplied HTTP request unchanged. Do not copy client credentials blindly into upstream headers. The kit never constructs an independent executor transport.

ExecuteStream opens HTTP through host.http.do_stream, reads raw chunks incrementally, and returns headers before completion. Runtime forwards chunks asynchronously with host.stream.emit and exactly one host.stream.close. It does not buffer the entire stream. Upstream setup failures return a normal RPC error; late stream failures carry structured error_detail. Host operation IDs propagate cancellation to blocked HTTP setup/read calls. Providers must honor contexts, and consumers must drain a stream or cancel its context.

Classify provider errors explicitly, including exhaustion versus transient throttling, using safe public text:

return &pluginabi.Error{
    Code: "quota_exhausted", Message: "subscription window exhausted",
    HTTPStatus: 429, Retryable: true, RetryAfterSeconds: 60,
}

These fields drive the host's native per-auth cooldown. A non-retryable provider error can still carry a status and cooldown hint. Do not relabel every 403 as bad credentials or every 429 as exhaustion. Never put raw upstream bodies, keys, URLs with credentials, or transport errors in public messages. Unclassified errors and recovered panic values are redacted by the kit. Panics in provider-created goroutines must be recovered by that provider; Go cannot recover another goroutine's panic.

Default CountTokens is an explicitly marked estimate (ceil of Unicode payload codepoints / 4), not a model tokenizer or an upstream call. Supply TokenCount for a real tokenizer or a protocol-specific result. No helper automatically retries inference, rotates accounts, or bypasses provider entitlements. Multiple auth files use CLIProxy's native selection and cooldown; the kit does not claim a global Scheduler capability.

Quota polling and cache

FetchQuotaFunc fetches and parses the provider's official quota format, returning normalized SDK groups/buckets/summary/subscription. The kit does not invent plan limits, reset support, or dollars. Errors are not cached and stale observations are not silently returned. Polled results are keyed by provider, auth identity, and a hash of storage/metadata/attributes, preventing credential changes from reusing old polled quota. Header observations are a separate auth-identity tier, preferred while fresh. Both tiers expire after 60 seconds and are pruned on fetch; the cache is in-process and starts empty after reload.

A provider UsagePlugin can normalize a quota response header and call CachedQuota.Store using only the identity from pluginapi.UsageRecord:

err := cache.Store(pluginapi.QuotaFetchRequest{
    AuthID: record.AuthID, AuthIndex: record.AuthIndex, Provider: record.Provider,
}, normalizedQuota)

Store and FetchQuota use AuthID, falling back to AuthIndex only when AuthID is absent. Supply AuthID whenever available in the usage record so it matches the host-enriched fetch. Store does not need or key on storage/metadata/attributes: the host may supply physical auth-file bytes and additional metadata only on FetchQuota. An older in-flight poll cannot overwrite the stored header observation. Use one cache instance for both interfaces. Invalidate an identity when reusing it for a different credential (header observations cannot detect credential changes), and recreate/invalidate the cache when configuration changes endpoint or quota semantics.

Transport distinction: v8.0.48's HostHTTPClient follows redirects, without a per-request flag to prohibit them or cap reads. Therefore the strict polling helper is separate: QuotaHTTPClient.Do allows GET only, uses a 15-second total timeout, never follows redirects, caps successful bodies at 1 MiB, and discards error bodies/URL/transport text. Pass a configured http.RoundTripper when deployment proxy policy requires it; nil clones the standard transport (normal environment proxies). This helper is not for inference. If using the host client for quota instead, the host's redirect semantics apply and strict no-redirect polling is not guaranteed. Neither helper logs keys; host request logging remains controlled by the host operator.

Lifecycle and ABI ownership

plugin.register / plugin.reconfigure accept schema 9 and return registration metadata/capability flags. Configure gets the plugin config subtree as YAML bytes, not the whole host config. Reconfiguration first cancels and drains active work; a failed Configure leaves the runtime inactive until a successful configuration. plugin.quiesce stops admission and drains work. plugin.shutdown and the native shutdown function perform idempotent logical shutdown. Do not call lifecycle methods recursively from a capability/hook.

Capabilities must be concurrency-safe, immutable after construction, and cancelable. Quiesce waits for admitted work; a provider ignoring cancellation can prevent draining. Shutdown cancels streams, but does not unload a resident Go runtime. A native reinitialization uses the factory again.

The shim exports cliproxy_plugin_init, cliproxy_plugin_call, cliproxy_plugin_free_buffer, and cliproxy_plugin_shutdown; the init function populates ABI 1's call/free_buffer/shutdown table. JSON is copied at the C boundary. Plugin-allocated response buffers are freed by the plugin allocator, host callback buffers by the host allocator, exactly once. No Go pointers cross the ABI.

Build and verify

scratch="$(mktemp -d)"
trap 'rm -rf "$scratch"' EXIT
export CGO_ENABLED=1

go vet ./...
go test -race -count=1 ./...
go build -trimpath -buildmode=c-shared -o "$scratch/echo.so" ./examples/echo
# Separate module: root ./... deliberately does not include this harness.
go -C integration/hostcompat vet ./...
go -C integration/hostcompat test -race -count=1 -v ./...

CI runs formatting, module verification, both vet/race suites, a c-shared build, gitleaks, and a generic credential-pattern scan, without provider credentials or repository secrets. All network behavior under test uses httptest loopback upstreams; module/tool downloads use public Go infrastructure only. Tests never call provider inference or quota services.

Why the host compatibility harness is a separate module

The real loader is unexported inside internal/pluginhost. The test-only module at integration/hostcompat intentionally uses import path github.com/router-for-me/CLIProxyAPI/v8/providerkitcompat so Go permits imports of the upstream internal host/config/registry packages. This is an internal-package testing harness, not a fork, an upstream package, or a replacement loader. Its module requires exactly v8.0.48 with normal Go checksum verification; there is no local replace or patched core source.

The test builds the local echo c-shared library, then calls the pinned host's Host.ApplyConfig: this runs the original dynamicLibraryLoader.Open (dlopen/dlsym), registerRPCPlugin, HTTP callbacks, streaming bridge and executor/auth/quota adapters. Assertions cover metadata/capabilities, host-merged auth persistence, quota, mock execution, incremental streaming, retry-after mapping, reconfiguration, shutdown and resident-image reinitialization. The host test process is disposable; it is not a running proxy service and opens no public listener.

Source contracts pinned to v8.0.48:

  • sdk/pluginabi/types.go: ABI/schema constants, method names, envelope/error contract.
  • sdk/pluginapi/types.go: auth/model/executor/host HTTP/quota/usage interfaces.
  • internal/pluginhost/loader_unix.go: C layout, buffer allocation/freeing, logical unload.
  • internal/pluginhost/rpc_schema.go, rpc_client.go: registration and scoped requests.
  • internal/pluginhost/host_callbacks.go, rpc_client_stream.go, stream_bridge.go: host HTTP operations and stream emission.
  • internal/pluginhost/auth_provider.go: storage/metadata merge and auth conversion.
  • internal/pluginhost/http_bridge.go: redirect behavior and host request logging.

Provider releases (not kit releases)

This library has no installable .so, registry.json, migration command, or plugin-store entry of its own. Provider repositories own those artifacts and credential migration. For the v8.0.48 GitHub store contract, release <plugin-id>_<version>_linux_amd64.zip with <plugin-id>.so and checksums, and publish a schema-1 registry.json with the provider's GitHub-release installation metadata. Verify the exact contract in internal/pluginstore/github.go before releasing a provider. Keep all credentials out of artifacts and fixtures. Stage each provider before any production cutover; native compatibility executors may take precedence over plugin executors.

The C ABI layout is adapted from CLIProxyAPI's MIT-licensed simple Go plugin example and loader. The original copyright notices are retained in LICENSE, alongside this kit's MIT license.

Documentation

Overview

Package providerkit implements the CLIProxy provider plugin RPC contract. Provider implementations use the pinned SDK's interfaces directly.

Index

Constants

View Source
const QuotaCacheTTL = 60 * time.Second

QuotaCacheTTL is the maximum age of a successful per-auth quota observation.

View Source
const QuotaHTTPTimeout = 15 * time.Second

QuotaHTTPTimeout bounds the entire quota HTTP exchange, including body reads.

Variables

This section is empty.

Functions

func APIKey

func APIKey(storage []byte, metadata map[string]any) (string, error)

APIKey reads a nonempty api_key using host metadata precedence. It never includes the key or raw storage in an error.

func AuthFields

func AuthFields(storage []byte, metadata map[string]any) (map[string]any, error)

AuthFields applies the same merge precedence as persisted host auth storage: Metadata overrides StorageJSON. Callers own the returned map.

func MapError

func MapError(err error) *pluginabi.Error

MapError preserves explicit, public RPC errors. Unclassified errors are intentionally opaque: transport and parsing errors can contain credentials. Providers must put only safe, user-facing text in pluginabi.Error.Message.

func NewHostHTTPClient

func NewHostHTTPClient(h Caller, callbackID string) pluginapi.HostHTTPClient

NewHostHTTPClient creates a callback-scoped SDK client. Runtime injects this automatically into each request. Never retain it past that request/stream. Cancellation uses the host's operation IDs, not abandoned blocking goroutines.

func RetryAfterSeconds

func RetryAfterSeconds(value string, now time.Time) int

RetryAfterSeconds parses HTTP Retry-After seconds or an HTTP date, rounding a future date up. Invalid or past values yield zero. No wall clock is hidden.

func ValidateMetadata

func ValidateMetadata(m pluginapi.Metadata) error

ValidateMetadata checks the metadata required by the host and public registry.

Types

type APIKeyAuth

type APIKeyAuth struct {
	Provider string
	Validate func(map[string]any) error
}

APIKeyAuth implements the SDK auth provider for API-key-only providers. Validate may validate additional provider-owned fields; its errors are redacted by Runtime unless explicitly represented as public pluginabi.Error values. It is stateless: duplicate-file/account policy belongs to the provider.

func (APIKeyAuth) Identifier

func (p APIKeyAuth) Identifier() string

type CachedQuota

type CachedQuota struct {
	// contains filtered or unexported fields
}

CachedQuota implements a non-resettable SDK quota provider with per-account single-flight fetching, copy isolation, and a 60-second cache. Use Store to incorporate normalized inference response headers by auth identity without another HTTP poll or access to the host's credential context.

func NewCachedQuota

func NewCachedQuota(provider, display string, fetch FetchQuotaFunc) *CachedQuota

NewCachedQuota creates an empty quota cache. The identity and callback are fixed for its lifetime. No background polls or provider requests are started.

func (*CachedQuota) Identifier

func (q *CachedQuota) Identifier() string

func (*CachedQuota) Invalidate

func (q *CachedQuota) Invalidate(authID string)

Invalidate discards header observations and all polled credential versions for the given auth ID (or AuthIndex when used instead of AuthID).

func (*CachedQuota) Store

Store replaces an auth-identity observation (for example, from a provider quota header). Only AuthID (or AuthIndex when AuthID is absent) and an optional matching Provider are used; credential context is neither needed nor keyed. Fresh observations take precedence over polls, including older in-flight polls. Data is cloned on store and read. Invalidate when reusing an identity for a different credential, or recreate the cache on reconfiguration.

type Caller

type Caller interface {
	Call(context.Context, string, []byte) ([]byte, error)
}

Caller is the raw host callback boundary. Results are pluginabi.Envelope JSON. Implementations must support concurrent calls. Do not log request/response data.

type CallerFunc

type CallerFunc func(context.Context, string, []byte) ([]byte, error)

CallerFunc adapts a callback function to Caller.

func (CallerFunc) Call

func (f CallerFunc) Call(ctx context.Context, m string, b []byte) ([]byte, error)

type Definition

type Definition struct {
	Metadata     pluginapi.Metadata
	Capabilities pluginapi.Capabilities
	Configure    func(context.Context, []byte) error
	OnQuiesce    func(context.Context) error
	OnShutdown   func(context.Context) error
}

Definition wires SDK capabilities and lifecycle hooks. Configure receives only this plugin's configuration subtree, on both registration and reconfiguration. Hooks run serially after active calls/streams are canceled and drained. Capabilities and metadata must not be mutated after New. Capability methods must be concurrency-safe and honor context cancellation.

type FetchQuotaFunc

FetchQuotaFunc fetches and normalizes one account. Provider-owned protocol parsing stays outside the kit. Errors are never cached; stale data is not silently returned. Honor context cancellation.

type Passthrough

Passthrough is a thin executor. BuildRequest selects the provider protocol, endpoint and authentication; it must preserve the host's final business payload (the host already applied payload rules). No protocol translation is done here. HttpRequest copies the supplied HTTP request unless PrepareHTTP is provided. Every network operation uses the injected HostHTTPClient; there is no fallback. Provider-specific quota/exhaustion classification belongs in ClassifyError. Hooks are immutable after construction and must support concurrent calls.

func (Passthrough) CountTokens

CountTokens uses TokenCount when supplied. Otherwise it returns an explicitly estimated one-token-per-four-Unicode-codepoints upper-rounded payload count, not a model tokenizer or a billable upstream token-count request.

func (Passthrough) Identifier

func (p Passthrough) Identifier() string

type QuotaHTTPClient

type QuotaHTTPClient struct {
	// contains filtered or unexported fields
}

QuotaHTTPClient is a bounded, secret-redacting read-only polling helper. It performs no logging. Errors omit URLs, headers, bodies and transport text. This is deliberately separate from HostHTTPClient: ABI 1 / schema 9 has no no-redirect setting. Executors must always use the host client instead. The transport may implement a provider's deployment proxy policy. Its RoundTrip method must NOT follow redirects itself (the standard transports do not).

func NewQuotaHTTPClient

func NewQuotaHTTPClient(transport http.RoundTripper) *QuotaHTTPClient

NewQuotaHTTPClient sets a 15s timeout, no redirects and a 1MiB response cap. With nil transport it uses a clone of the standard transport (including normal environment proxy behavior), not a mutable shared http.Client.

func (*QuotaHTTPClient) CloseIdleConnections

func (c *QuotaHTTPClient) CloseIdleConnections()

CloseIdleConnections releases pooled polling connections during plugin shutdown.

func (*QuotaHTTPClient) Do

Do reads a quota/discovery resource. Only GET is allowed, to keep this helper out of the inference execution path. Successful bodies are returned to the provider parser; callers must never log them or credential-bearing headers.

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime owns registration, dispatch and the lifetime of asynchronous work.

func New

func New(d Definition, h Caller) (*Runtime, error)

New validates a definition. The runtime starts inactive until plugin.register. Supported capabilities are auth, models, executor, quota and usage; unrelated SDK capabilities are rejected instead of silently advertised without dispatch.

func (*Runtime) Call

func (r *Runtime) Call(ctx context.Context, method string, raw []byte) (out []byte)

Call always returns a valid RPC envelope. Panics on the synchronous dispatch path are contained and redacted. Provider-created goroutines must contain their own panics (Go cannot recover another goroutine's panic).

func (*Runtime) Shutdown

func (r *Runtime) Shutdown(ctx context.Context)

Shutdown cancels and drains work and runs OnShutdown once. Logical shutdown does not unload the c-shared Go runtime from the host address space.

Directories

Path Synopsis
Package cabi exports the ABI 1 function table for Go c-shared provider plugins.
Package cabi exports the ABI 1 function table for Go c-shared provider plugins.
examples
echo command
Echo is a mock-upstream-only provider example, not a production integration.
Echo is a mock-upstream-only provider example, not a production integration.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL