cincai

README

Cincai

Model-oriented AI gateway — pick a model, not a provider.

Point your SDK's base_url at Cincai and ask for a model by name. Cincai works out which of your configured providers can serve it — API-key providers and subscription logins alike (Grok OAuth ships in the box) — and handles the rest: load-balancing, failover, per-provider model-name mapping, and OpenAI↔Anthropic wire translation. One endpoint and one model id, for chat, image, video, and speech.

client = OpenAI(base_url="http://127.0.0.1:9420/v1", api_key="sk-dg-...")
client.chat.completions.create(model="example-model", ...)

The client names a model. Which provider serves it, under which upstream name, with which credential — that's catalog policy in providers.yaml, not client code.

Why

Every provider has its own endpoint, its own auth, and its own spelling of the same model (example-model vs vendor/example-model vs accounts/vendor/models/example-model). Baking those differences into every client means touching every client each time a provider changes, rate-limits, or goes down. Cincai moves that knowledge into one place: a catalog that maps a canonical model id to a pool of providers with a failover or round-robin strategy. Swap a provider, add a fallback, or rotate a credential by editing config — your clients never notice.

A model id can also be a composite: ordered hops to other catalog models (modality.models) so a cheap default can fall through luna → gemini → deepseek on retryable failures. Same models: root, listed on GET /v1/models with a models array; ingress logs use model for the hop that served and alias for the composite id.

Groups (groups: root) are named sets for client pickers (object: "model_group" on GET /v1/models). They are not request model ids — the client still chooses a member. See docs/routing.md.

Quick start

You need Go 1.26.4+ (devenv shell provides it, but any recent system Go works).

just build          # → bin/cincai
cincai init         # scaffold config/ and dev secrets from the tracked *.example templates
set -a && source config/cincai.dev.env && set +a

That last line loads CINCAI_BROKER_KEY, the key that encrypts the credential broker — every credential/keys/serve command needs it in the environment.

Store the upstream credentials your catalog references (see config/providers.yaml), and mint a gateway key for your app:

cincai credential import deepseek-api --api-key "$DEEPSEEK_API_KEY"
cincai credential login xai-oauth     # subscription OAuth; on a remote box see docs/oauth.md
cincai keys create                    # prints the sk-dg-... key your client sends as Bearer

Then check the catalog and serve:

cincai catalog validate
cincai serve

Point your SDK at http://127.0.0.1:9420/v1 with the sk-dg-... key. All commands default to --config config/cincai.yaml; pass the flag to use another path. just verify-smoke runs the offline routing smokes (chat, image, speech, video) — no network or real credentials needed.

Docs

File	Contents
docs/architecture.md	Architecture — request pipeline, packages, extension points
docs/cli.md	CLI reference — init, serve, catalog, credential, keys
docs/configuration.md	Configuration — cincai.yaml, env vars, adapters
docs/credential.md	Credentials — broker, profiles, API keys vs OAuth
docs/routing.md	Model-oriented routing — pools, composite hops, model groups, failover, proxy, efforts
AGENTS.md	Contributor / agent working notes
docs/media.md	Media routing — image, speech, video
docs/catalog-capabilities-modalities.md	capabilities vs modalities naming
docs/catalog-inject.md	inject: map + inject_preset
docs/oauth.md	OAuth login — SSH port-forward on remote servers
docs/observability.md	Logs, OTLP metrics/traces, embedding

Use as a library / extend

Cincai is also a Go library. All integration points are public packages — you never import anything under internal/:

• Embed the gateway in your own binary: cincai.EmbedRun (or the lower-level gateway package). See docs/observability.md.
• Write a vendor adapter: implement adaptersdk.Adapter, register with pack.RegisterAdapter, compose with compose.FromConfig.
• Add an OAuth provider: register with credential/oauth/pack.
• Custom credential backend: gateway.RegisterCredentialBackend.

import "github.com/subosito/cincai"

func main() {
    _ = cincai.Run(context.Background(), cincai.Options{ConfigPath: "config/cincai.yaml"})
}

Project status

Early and moving. The routing paths are covered by unit tests and offline smokes (just verify-smoke, go test -race ./... in CI), not yet by years of production mileage. Secure defaults are in place — loopback listener, encrypted credential broker, hashed gateway keys — but read SECURITY.md before exposing it beyond localhost.

Contributing

See CONTRIBUTING.md. Security reports: SECURITY.md.

License

MIT — see LICENSE.

Documentation

Overview

Package cincai is the batteries-included library entry point. Run and EmbedRun wire the curated catalog loader, adapter pack, and OAuth providers into the gateway engine — everything a self-hoster gets from `cincai serve`. For lower- level control, use the gateway package directly.

Product binaries and hosts (host-app-router) depend on this module via go.mod — they do not fork the engine sources.

Index

• func Adapters() []adaptersdk.Adapter
• func EmbedRun(ctx context.Context, opts EmbedOptions) error
• func LoadCatalog(path string) (*corecatalog.Catalog, error)
• func Run(ctx context.Context, opts Options) error
• type EmbedOptions
• type Options

Functions

func Adapters

func Adapters() []adaptersdk.Adapter

Adapters returns every adapter linked into this binary: the stock defaults plus whatever registered with pack (the link package's blank imports, or your own pack.RegisterAdapter calls). This is the set adapters.enable selects from.

func EmbedRun

func EmbedRun(ctx context.Context, opts EmbedOptions) error

EmbedRun serves without calling OTel Boot. For host binaries that embed the gateway.

func LoadCatalog

func LoadCatalog(path string) (*corecatalog.Catalog, error)

LoadCatalog loads providers.yaml with the same capabilities→surfaces normalization used by Run/EmbedRun. Product CLIs use this for catalog validate without importing internal/catalog.

func Run

func Run(ctx context.Context, opts Options) error

Run serves until ctx is cancelled. For `cincai serve` / product binaries.

Types

type EmbedOptions

type EmbedOptions struct {
	ConfigPath      string
	ServiceName     string
	MetricPrefix    string
	WrapDataHandler func(http.Handler) http.Handler
	// AuxServers lets the host run extra HTTP servers alongside the data plane, sharing
	// the credential store, keyring, and shutdown lifecycle (e.g. a management/admin API).
	AuxServers gateway.AuxServersFunc
}

EmbedOptions runs the gateway inside a host process (host owns OTel Boot).

type Options

type Options struct {
	ConfigPath   string
	ServiceName  string
	MetricPrefix string
	// WrapDataHandler optionally wraps the data-plane handler (e.g. host
	// attribution for usage). Does not affect routing.
	WrapDataHandler func(http.Handler) http.Handler
}

Options boots a standalone Cincai gateway (owns OTel via Boot).