README
¶
Polypus: one OpenAI-compatible face for every model your apps call
Polypus is an inference gateway. Your apps, agents and n8n flows speak the OpenAI API to one address, and Polypus routes each call to the right arm. That arm can be local MLX speech, LM Studio, OpenRouter, Claude, Gemini or Cloudflare Workers AI. The gateway holds every provider URL and key, so your apps never hold one.
Today a Go binary, polypus, serves the gateway, and an NVIDIA Switchyard sidecar handles the routers that pick a model per turn. Phoenix and HyperDX show every call.
- Website: polypus.dev.xynova.com
- Run your own on your machine and tailnet: docs/setup.md
- Documentation: docs/, covering install, local development, routers and releases
- Agents start at AGENTS.md
💡 One gateway learns the providers, so each app doesn't have to
Each provider has its own API shape, keys, limits and failure modes. When every app learns every provider, teams end up copying keys into each app, rewriting clients when a model moves, and guessing why a call failed.
Polypus gathers that work into one place, in four layers:
| Layer | Role |
|---|---|
| Gateway | One OpenAI-compatible surface; picks a backend by model prefix or capability default |
| Backends | OpenAI-compatible servers, plus extensions for Claude, Gemini and Cloudflare |
| Switchyard | Named routers (router/<name>) that choose a cheap or strong model per turn |
| Observability | OTLP traces to Phoenix and HyperDX, and a JSON dump for each failed call |
Swapping a model, a provider or a key then becomes a gateway config change, and the apps stay the same.
Licence
Polypus is licensed under Apache 2.0; see LICENSE. The Switchyard image also carries NVIDIA's Switchyard licence and notice under /usr/share/doc/polypus-switchyard/.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
polypus
command
|
|
|
polypus-smoke
command
polypus-smoke: multi-channel L1 probes against a Polypus gateway.
|
polypus-smoke: multi-channel L1 probes against a Polypus gateway. |
|
internal
|
|
|
batch/service
Package service runs OpenAI-shaped batches on the providers that support them (Anthropic Message Batches, OpenRouter's Batch API, Workers AI async batch).
|
Package service runs OpenAI-shaped batches on the providers that support them (Anthropic Message Batches, OpenRouter's Batch API, Workers AI async batch). |
|
clients/catalog
Package catalog holds what the provider model-catalog clients share: the OpenAI Models API shapes, a TTL cache, a strict list with stale-cache fallback, and a process-scoped client registry.
|
Package catalog holds what the provider model-catalog clients share: the OpenAI Models API shapes, a TTL cache, a strict list with stale-cache fallback, and a process-scoped client registry. |
|
clients/openaicompat
Package openaicompat lists models from, and pings, OpenAI-compatible backends: local servers, hosted APIs such as DeepInfra, OpenRouter's models API and Switchyard.
|
Package openaicompat lists models from, and pings, OpenAI-compatible backends: local servers, hosted APIs such as DeepInfra, OpenRouter's models API and Switchyard. |
|
clients/openrouter
Package openrouter is the OpenRouter Batch API client.
|
Package openrouter is the OpenRouter Batch API client. |
|
config
Package config holds Polypus gateway defaults and runtime options.
|
Package config holds Polypus gateway defaults and runtime options. |
|
deploy/clientkey
Package clientkey creates the gateway client key for a deploy environment: it runs polypus admin-key inside the environment's running gateway container, stores the key in the environment's keyring service, and checks the gateway now needs it.
|
Package clientkey creates the gateway client key for a deploy environment: it runs polypus admin-key inside the environment's running gateway container, stores the key in the environment's keyring service, and checks the gateway now needs it. |
|
deploy/envconfig
Package envconfig loads a polypus deploy environment: the config.yaml that names the environment and points at its gateway YAML.
|
Package envconfig loads a polypus deploy environment: the config.yaml that names the environment and points at its gateway YAML. |
|
deploy/stack
Package stack holds the deploy's Docker Compose stack, embedded in the binary so a release binary deploys without a Polypus checkout.
|
Package stack holds the deploy's Docker Compose stack, embedded in the binary so a release binary deploys without a Polypus checkout. |
|
deploy/tailnet
Package tailnet checks that the machine running a deploy is on the environment's tailnet, so health and smoke checks can reach the gateway's tailnet address.
|
Package tailnet checks that the machine running a deploy is on the environment's tailnet, so health and smoke checks can reach the gateway's tailnet address. |
|
deploy/tsserve
Package tsserve renders the Tailscale serve config for a deploy: HTTPS on 443 to the gateway, and the observability ports forwarded only when those services run.
|
Package tsserve renders the Tailscale serve config for a deploy: HTTPS on 443 to the gateway, and the observability ports forwarded only when those services run. |
|
errors
Package errors defines typed domain errors for Polypus.
|
Package errors defines typed domain errors for Polypus. |
|
gateway/upstream
Package upstream provides per-target circuit breakers for Polypus dials (Switchyard, OpenAI-shaped leaves, Cloudflare).
|
Package upstream provides per-target circuit breakers for Polypus dials (Switchyard, OpenAI-shaped leaves, Cloudflare). |
|
ratelimit
Package ratelimit is the provider-neutral shape of an upstream rate limit.
|
Package ratelimit is the provider-neutral shape of an upstream rate limit. |
|
smoke
Package smoke runs multi-channel L1 probes against a Polypus gateway.
|
Package smoke runs multi-channel L1 probes against a Polypus gateway. |
|
pkg
|
|
|
polypus
Package polypus is the public product façade for the Polypus inference gateway.
|
Package polypus is the public product façade for the Polypus inference gateway. |
Click to show internal directories.
Click to hide internal directories.