README
¶
Polypus: one OpenAI-compatible face for every model your apps call
Polypus is an inference gateway. Your apps, agents and n8n flows speak the OpenAI API to one address, and Polypus routes each call to the right arm. That arm can be local MLX speech, LM Studio, OpenRouter, Claude, Gemini or Cloudflare Workers AI. The gateway holds every provider URL and key, so your apps never hold one.
Today a Go binary, polypus, serves the gateway, and an NVIDIA Switchyard sidecar handles the routers that pick a model per turn. Phoenix and HyperDX show every call.
- Website: polypus.dev.xynova.com
- Run your own on your machine and tailnet: docs/setup.md
- Documentation: docs/, covering install, local development, routers and releases
- Agents start at AGENTS.md
💡 One gateway learns the providers, so each app doesn't have to
Each provider has its own API shape, keys, limits and failure modes. When every app learns every provider, teams end up copying keys into each app, rewriting clients when a model moves, and guessing why a call failed.
Polypus gathers that work into one place, in four layers:
| Layer | Role |
|---|---|
| Gateway | One OpenAI-compatible surface; picks a backend by model prefix or capability default |
| Backends | OpenAI-compatible servers, plus extensions for Claude, Gemini and Cloudflare |
| Switchyard | Named routers (router/<name>) that choose a cheap or strong model per turn |
| Observability | OTLP traces to Phoenix and HyperDX, and a JSON dump for each failed call |
Swapping a model, a provider or a key then becomes a gateway config change, and the apps stay the same.
Licence
Polypus is licensed under Apache 2.0; see LICENSE. The Switchyard image also carries NVIDIA's Switchyard licence and notice under /usr/share/doc/polypus-switchyard/.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
polypus
command
|
|
|
polypus-smoke
command
polypus-smoke: multi-channel L1 probes against a Polypus gateway.
|
polypus-smoke: multi-channel L1 probes against a Polypus gateway. |
|
internal
|
|
|
admin/keys
Package keys stores the gateway's named access keys (ppk.<id>.<secret>) as SHA-256 digests in a locked JSON file and verifies presented tokens.
|
Package keys stores the gateway's named access keys (ppk.<id>.<secret>) as SHA-256 digests in a locked JSON file and verifies presented tokens. |
|
buildinfo
Package buildinfo holds the build identity the binaries print and the gateway reports.
|
Package buildinfo holds the build identity the binaries print and the gateway reports. |
|
cli
Package cli is the Cobra command tree for the polypus and polypus-smoke binaries: bare invoke prints the agent guide, and each command calls one service package.
|
Package cli is the Cobra command tree for the polypus and polypus-smoke binaries: bare invoke prints the agent guide, and each command calls one service package. |
|
clients/catalog
Package catalog holds what the provider clients share: the OpenAI Models API shapes, a TTL cache, a strict list with stale-cache fallback, a process-scoped client registry, one capped JSON GET with status mapping, and the small text helpers their error messages use.
|
Package catalog holds what the provider clients share: the OpenAI Models API shapes, a TTL cache, a strict list with stale-cache fallback, a process-scoped client registry, one capped JSON GET with status mapping, and the small text helpers their error messages use. |
|
clients/github
Package github is the GitHub REST client the deploy uses to follow a release's image builds: the commit a tag points at, workflow runs and their jobs.
|
Package github is the GitHub REST client the deploy uses to follow a release's image builds: the commit a tag points at, workflow runs and their jobs. |
|
clients/openaicompat
Package openaicompat lists models from, and pings, OpenAI-compatible backends: local servers, hosted APIs such as DeepInfra, OpenRouter's models API and Switchyard.
|
Package openaicompat lists models from, and pings, OpenAI-compatible backends: local servers, hosted APIs such as DeepInfra, OpenRouter's models API and Switchyard. |
|
clients/openrouter
Package openrouter is the OpenRouter Batch API client.
|
Package openrouter is the OpenRouter Batch API client. |
|
clients/registry
Package registry is the OCI registry client the deploy uses to check that a pinned image manifest exists.
|
Package registry is the OCI registry client the deploy uses to check that a pinned image manifest exists. |
|
config
Package config holds Polypus gateway defaults and runtime options.
|
Package config holds Polypus gateway defaults and runtime options. |
|
deploy
Package deploy deploys one Polypus environment with Docker Compose: it resolves the plan from the environment's config and gateway YAML, writes the stack, pins and secrets, starts the containers, then checks health and smoke.
|
Package deploy deploys one Polypus environment with Docker Compose: it resolves the plan from the environment's config and gateway YAML, writes the stack, pins and secrets, starts the containers, then checks health and smoke. |
|
deploy/clientkey
Package clientkey creates the gateway client key for a deploy environment: it runs polypus admin-key inside the environment's running gateway container, stores the key in the environment's keyring service, and checks the gateway now needs it.
|
Package clientkey creates the gateway client key for a deploy environment: it runs polypus admin-key inside the environment's running gateway container, stores the key in the environment's keyring service, and checks the gateway now needs it. |
|
deploy/compose
Package compose runs docker compose for one deploy environment's project: pull, up, recreate and remove services, under the outbound retry policy.
|
Package compose runs docker compose for one deploy environment's project: pull, up, recreate and remove services, under the outbound retry policy. |
|
deploy/envconfig
Package envconfig loads a polypus deploy environment: the config.yaml that names the environment and points at its gateway YAML.
|
Package envconfig loads a polypus deploy environment: the config.yaml that names the environment and points at its gateway YAML. |
|
deploy/execx
Package execx runs the deploy's docker and tailscale commands under the caller's context, streaming or capturing their output.
|
Package execx runs the deploy's docker and tailscale commands under the caller's context, streaming or capturing their output. |
|
deploy/gatewayconfig
Package gatewayconfig reads what a deploy needs from an environment's gateway YAML: the secrets its backends name, its routers, and the smoke probes to run.
|
Package gatewayconfig reads what a deploy needs from an environment's gateway YAML: the secrets its backends name, its routers, and the smoke probes to run. |
|
deploy/health
Package health waits for a deployed gateway to answer its health endpoints, and dumps the gateway's logs and state when a deploy fails.
|
Package health waits for a deployed gateway to answer its health endpoints, and dumps the gateway's logs and state when a deploy fails. |
|
deploy/images
Package images holds the deploy's image policy: verify the pins file, check each pinned image is in its registry, wait for the release's image build on GitHub Actions when it is not, and gate on the release's publish-complete run.
|
Package images holds the deploy's image policy: verify the pins file, check each pinned image is in its registry, wait for the release's image build on GitHub Actions when it is not, and gate on the release's publish-complete run. |
|
deploy/overlay
Package overlay prunes the gateway's models allow overlay inside the running gateway container: slugs the gateway YAML already allows are dropped from it.
|
Package overlay prunes the gateway's models allow overlay inside the running gateway container: slugs the gateway YAML already allows are dropped from it. |
|
deploy/paths
Package paths resolves the host paths a deploy bind-mounts into its containers, in the form Docker accepts on each operating system.
|
Package paths resolves the host paths a deploy bind-mounts into its containers, in the form Docker accepts on each operating system. |
|
deploy/progress
Package progress writes a deploy's operator-facing progress lines to one writer and keeps the first write error, so a caller returns it instead of dropping it.
|
Package progress writes a deploy's operator-facing progress lines to one writer and keeps the first write error, so a caller returns it instead of dropping it. |
|
deploy/secrets
Package secrets resolves a deploy environment's secrets from the process environment and its keyring service, writes them as dotenv files for compose, and checks their shape.
|
Package secrets resolves a deploy environment's secrets from the process environment and its keyring service, writes them as dotenv files for compose, and checks their shape. |
|
deploy/smoke
Package smoke runs a deploy's post-health checks against a running gateway: one chat per backend whose secrets are set, each chat router, and SystemOne, plus the OTLP collector fan-out smoke.
|
Package smoke runs a deploy's post-health checks against a running gateway: one chat per backend whose secrets are set, each chat router, and SystemOne, plus the OTLP collector fan-out smoke. |
|
deploy/stack
Package stack holds the deploy's Docker Compose stack, embedded in the binary so a release binary deploys without a Polypus checkout.
|
Package stack holds the deploy's Docker Compose stack, embedded in the binary so a release binary deploys without a Polypus checkout. |
|
deploy/tailnet
Package tailnet checks that the machine running a deploy is on the environment's tailnet, so health and smoke checks can reach the gateway's tailnet address.
|
Package tailnet checks that the machine running a deploy is on the environment's tailnet, so health and smoke checks can reach the gateway's tailnet address. |
|
deploy/tsserve
Package tsserve renders the Tailscale serve config for a deploy: HTTPS on 443 to the gateway, and the observability ports forwarded only when those services run.
|
Package tsserve renders the Tailscale serve config for a deploy: HTTPS on 443 to the gateway, and the observability ports forwarded only when those services run. |
|
errors
Package errors defines typed domain errors for Polypus.
|
Package errors defines typed domain errors for Polypus. |
|
gateway/batches
Package batches runs OpenAI-shaped batches on the providers that support them (Anthropic Message Batches, OpenRouter's Batch API, Workers AI async batch).
|
Package batches runs OpenAI-shaped batches on the providers that support them (Anthropic Message Batches, OpenRouter's Batch API, Workers AI async batch). |
|
gateway/upstream
Package upstream provides per-target circuit breakers for Polypus dials (Switchyard, OpenAI-shaped leaves, Cloudflare).
|
Package upstream provides per-target circuit breakers for Polypus dials (Switchyard, OpenAI-shaped leaves, Cloudflare). |
|
ratelimit
Package ratelimit is the provider-neutral shape of an upstream rate limit.
|
Package ratelimit is the provider-neutral shape of an upstream rate limit. |
|
smoke
Package smoke runs multi-channel L1 probes against a Polypus gateway.
|
Package smoke runs multi-channel L1 probes against a Polypus gateway. |
|
statefile
Package statefile writes small state files atomically and serialises writers with an advisory lock beside each file.
|
Package statefile writes small state files atomically and serialises writers with an advisory lock beside each file. |
|
pkg
|
|
|
polypus
Package polypus is the public product façade for the Polypus inference gateway.
|
Package polypus is the public product façade for the Polypus inference gateway. |
Click to show internal directories.
Click to hide internal directories.