README
¶
Polypus: one OpenAI-compatible face for every model your apps call
Polypus is an inference gateway. Your apps, agents and n8n flows speak the OpenAI API to one address, and Polypus routes each call to the right arm. That arm can be local MLX speech, LM Studio, OpenRouter, Claude, Gemini or Cloudflare Workers AI. The gateway holds every provider URL and key, so your apps never hold one.
Today a Go binary, polypus, serves the gateway, and an NVIDIA Switchyard sidecar handles the routers that pick a model per turn. Phoenix and HyperDX show every call.
- Website: polypus.dev.xynova.com
- Run your own on your machine and tailnet: docs/setup.md
- Documentation: docs/, covering install, local development, routers and releases
- Agents start at AGENTS.md
💡 One gateway learns the providers, so each app doesn't have to
Each provider has its own API shape, keys, limits and failure modes. When every app learns every provider, teams end up copying keys into each app, rewriting clients when a model moves, and guessing why a call failed.
Polypus gathers that work into one place, in four layers:
| Layer | Role |
|---|---|
| Gateway | One OpenAI-compatible surface; picks a backend by model prefix or capability default |
| Backends | OpenAI-compatible servers, plus extensions for Claude, Gemini and Cloudflare |
| Switchyard | Named routers (router/<name>) that choose a cheap or strong model per turn |
| Observability | OTLP traces to Phoenix and HyperDX, and a JSON dump for each failed call |
Swapping a model, a provider or a key then becomes a gateway config change, and the apps stay the same.
Licence
Polypus is licensed under Apache 2.0; see LICENSE. The Switchyard image also carries NVIDIA's Switchyard licence and notice under /usr/share/doc/polypus-switchyard/.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
polypus
command
Command polypus is the Polypus gateway and operator CLI; bare invoke prints the agent guide, and serve starts the gateway.
|
Command polypus is the Polypus gateway and operator CLI; bare invoke prints the agent guide, and serve starts the gateway. |
|
polypus-smoke
command
Command polypus-smoke runs multi-channel L1 probes against a Polypus gateway; bare invoke prints the agent guide, and run sends the probes.
|
Command polypus-smoke runs multi-channel L1 probes against a Polypus gateway; bare invoke prints the agent guide, and run sends the probes. |
|
internal
|
|
|
admin/keys
Package keys stores the gateway's named access keys (ppk.<id>.<secret>) as SHA-256 digests in a locked JSON file and verifies presented tokens.
|
Package keys stores the gateway's named access keys (ppk.<id>.<secret>) as SHA-256 digests in a locked JSON file and verifies presented tokens. |
|
batch
Package batch is the provider-neutral core of the OpenAI Batch and Files APIs: the stored file and batch records, the on-disk store, JSONL input parsing, completion windows, and the output and error files built from finished lines.
|
Package batch is the provider-neutral core of the OpenAI Batch and Files APIs: the stored file and batch records, the on-disk store, JSONL input parsing, completion windows, and the output and error files built from finished lines. |
|
buildinfo
Package buildinfo holds the build identity the binaries print and the gateway reports.
|
Package buildinfo holds the build identity the binaries print and the gateway reports. |
|
cli
Package cli is the Cobra command tree for the polypus and polypus-smoke binaries: bare invoke prints the agent guide, and each command calls one service package.
|
Package cli is the Cobra command tree for the polypus and polypus-smoke binaries: bare invoke prints the agent guide, and each command calls one service package. |
|
clients/catalog
Package catalog holds what the provider model catalogues share: the OpenAI Models API shapes, a TTL cache and a strict list with stale-cache fallback.
|
Package catalog holds what the provider model catalogues share: the OpenAI Models API shapes, a TTL cache and a strict list with stale-cache fallback. |
|
clients/clientcache
Package clientcache keeps one provider client per backend for the life of its owner (each gateway's Clients value holds one Registry), and rebuilds a client when the token or URL it was built from changes.
|
Package clientcache keeps one provider client per backend for the life of its owner (each gateway's Clients value holds one Registry), and rebuilds a client when the token or URL it was built from changes. |
|
clients/github
Package github is the GitHub REST client the deploy uses to follow a release's image builds: the commit a tag points at, workflow runs and their jobs.
|
Package github is the GitHub REST client the deploy uses to follow a release's image builds: the commit a tag points at, workflow runs and their jobs. |
|
clients/httpx
Package httpx holds what the provider clients share for reading an upstream HTTP answer: one capped JSON GET through the outbound seam, a body read that fails rather than truncates, the status to domain code map, and the rune-safe text cut their error messages use.
|
Package httpx holds what the provider clients share for reading an upstream HTTP answer: one capped JSON GET through the outbound seam, a body read that fails rather than truncates, the status to domain code map, and the rune-safe text cut their error messages use. |
|
clients/openaicompat
Package openaicompat lists models from, and pings, OpenAI-compatible backends: local servers, hosted APIs such as DeepInfra, OpenRouter's models API and Switchyard.
|
Package openaicompat lists models from, and pings, OpenAI-compatible backends: local servers, hosted APIs such as DeepInfra, OpenRouter's models API and Switchyard. |
|
clients/openrouter
Package openrouter is the OpenRouter Batch API client.
|
Package openrouter is the OpenRouter Batch API client. |
|
clients/registry
Package registry is the OCI registry client the deploy uses to check that a pinned image manifest exists.
|
Package registry is the OCI registry client the deploy uses to check that a pinned image manifest exists. |
|
config
Package config holds Polypus gateway defaults and runtime options.
|
Package config holds Polypus gateway defaults and runtime options. |
|
deploy
Package deploy deploys one Polypus environment with Docker Compose: it resolves the plan from the environment's config and gateway YAML, writes the stack, pins and secrets, starts the containers, then checks health and smoke.
|
Package deploy deploys one Polypus environment with Docker Compose: it resolves the plan from the environment's config and gateway YAML, writes the stack, pins and secrets, starts the containers, then checks health and smoke. |
|
deploy/clientkey
Package clientkey creates the gateway client key for a deploy environment: it runs polypus admin-key inside the environment's running gateway container, stores the key in the environment's keyring service, and checks the gateway now needs it.
|
Package clientkey creates the gateway client key for a deploy environment: it runs polypus admin-key inside the environment's running gateway container, stores the key in the environment's keyring service, and checks the gateway now needs it. |
|
deploy/compose
Package compose runs docker compose for one deploy environment's project: pull, up, recreate and remove services, under the outbound retry policy.
|
Package compose runs docker compose for one deploy environment's project: pull, up, recreate and remove services, under the outbound retry policy. |
|
deploy/envconfig
Package envconfig loads a polypus deploy environment: the config.yaml that names the environment and points at its gateway YAML.
|
Package envconfig loads a polypus deploy environment: the config.yaml that names the environment and points at its gateway YAML. |
|
deploy/execx
Package execx runs the deploy's docker and tailscale commands under the caller's context, streaming or capturing their output.
|
Package execx runs the deploy's docker and tailscale commands under the caller's context, streaming or capturing their output. |
|
deploy/gatewayconfig
Package gatewayconfig reads what a deploy needs from an environment's gateway YAML: the secrets its backends name, its routers, and the smoke probes to run.
|
Package gatewayconfig reads what a deploy needs from an environment's gateway YAML: the secrets its backends name, its routers, and the smoke probes to run. |
|
deploy/health
Package health waits for a deployed gateway to answer its health endpoints, and dumps the gateway's logs and state when a deploy fails.
|
Package health waits for a deployed gateway to answer its health endpoints, and dumps the gateway's logs and state when a deploy fails. |
|
deploy/images
Package images holds the deploy's image policy: verify the pins file, check each pinned image is in its registry, wait for the release's image build on GitHub Actions when it is not, and gate on the release's publish-complete run.
|
Package images holds the deploy's image policy: verify the pins file, check each pinned image is in its registry, wait for the release's image build on GitHub Actions when it is not, and gate on the release's publish-complete run. |
|
deploy/overlay
Package overlay prunes the gateway's models allow overlay inside the running gateway container: slugs the gateway YAML already allows are dropped from it.
|
Package overlay prunes the gateway's models allow overlay inside the running gateway container: slugs the gateway YAML already allows are dropped from it. |
|
deploy/paths
Package paths resolves the host paths a deploy bind-mounts into its containers, in the form Docker accepts on each operating system.
|
Package paths resolves the host paths a deploy bind-mounts into its containers, in the form Docker accepts on each operating system. |
|
deploy/progress
Package progress writes a deploy's operator-facing progress lines to one writer and keeps the first write error, so a caller returns it instead of dropping it.
|
Package progress writes a deploy's operator-facing progress lines to one writer and keeps the first write error, so a caller returns it instead of dropping it. |
|
deploy/secrets
Package secrets resolves a deploy environment's secrets from the shell, its keyring service and its optional SOPS secrets file, writes them as dotenv files for compose, and checks their shape.
|
Package secrets resolves a deploy environment's secrets from the shell, its keyring service and its optional SOPS secrets file, writes them as dotenv files for compose, and checks their shape. |
|
deploy/smoke
Package smoke runs a deploy's post-health checks against a running gateway: one chat per backend whose secrets are set, each chat router, and SystemOne, plus the OTLP collector fan-out smoke.
|
Package smoke runs a deploy's post-health checks against a running gateway: one chat per backend whose secrets are set, each chat router, and SystemOne, plus the OTLP collector fan-out smoke. |
|
deploy/stack
Package stack holds the deploy's Docker Compose stack, embedded in the binary so a release binary deploys without a Polypus checkout.
|
Package stack holds the deploy's Docker Compose stack, embedded in the binary so a release binary deploys without a Polypus checkout. |
|
deploy/tailnet
Package tailnet checks that the machine running a deploy is on the environment's tailnet, so health and smoke checks can reach the gateway's tailnet address.
|
Package tailnet checks that the machine running a deploy is on the environment's tailnet, so health and smoke checks can reach the gateway's tailnet address. |
|
deploy/tsserve
Package tsserve renders the Tailscale serve config for a deploy: HTTPS on 443 to the gateway, and the observability ports forwarded only when those services run.
|
Package tsserve renders the Tailscale serve config for a deploy: HTTPS on 443 to the gateway, and the observability ports forwarded only when those services run. |
|
errors
Package errors defines typed domain errors for Polypus.
|
Package errors defines typed domain errors for Polypus. |
|
gateway/assets
Package assets embeds the static files the gateway landing page serves.
|
Package assets embeds the static files the gateway landing page serves. |
|
gateway/batches
Package batches runs OpenAI-shaped batches on the providers that support them (Anthropic Message Batches, OpenRouter's Batch API, Workers AI async batch).
|
Package batches runs OpenAI-shaped batches on the providers that support them (Anthropic Message Batches, OpenRouter's Batch API, Workers AI async batch). |
|
gateway/router
Package router is the gateway's Bifrost seam: it resolves a public model id to a backend and downstream model, registers every backend (and the optional Switchyard hop) with Bifrost, and sends chat, embeddings, speech, SystemOne decisions and Claude Message Batches through it, mapping provider errors to domain codes.
|
Package router is the gateway's Bifrost seam: it resolves a public model id to a backend and downstream model, registers every backend (and the optional Switchyard hop) with Bifrost, and sends chat, embeddings, speech, SystemOne decisions and Claude Message Batches through it, mapping provider errors to domain codes. |
|
gateway/upstream
Package upstream provides per-target circuit breakers for Polypus dials (Switchyard, OpenAI-shaped leaves, Cloudflare).
|
Package upstream provides per-target circuit breakers for Polypus dials (Switchyard, OpenAI-shaped leaves, Cloudflare). |
|
outbound
Package outbound is the failsafe-go seam for outbound HTTP and exec: retry with backoff and jitter for classified transients, and one circuit breaker per remote dependency.
|
Package outbound is the failsafe-go seam for outbound HTTP and exec: retry with backoff and jitter for classified transients, and one circuit breaker per remote dependency. |
|
ratelimit
Package ratelimit is the provider-neutral shape of an upstream rate limit.
|
Package ratelimit is the provider-neutral shape of an upstream rate limit. |
|
redact
Package redact masks credentials in URLs and in text that carries URLs.
|
Package redact masks credentials in URLs and in text that carries URLs. |
|
smoke
Package smoke runs multi-channel L1 probes against a Polypus gateway.
|
Package smoke runs multi-channel L1 probes against a Polypus gateway. |
|
statefile
Package statefile writes small state files atomically and serialises writers with an advisory lock beside each file.
|
Package statefile writes small state files atomically and serialises writers with an advisory lock beside each file. |
|
switchyard
Package switchyard renders the routes.toml that Switchyard, Polypus's own router process, reads: one route per composed router in the gateway's router config, with every leaf hop pointed back at the Polypus gateway.
|
Package switchyard renders the routes.toml that Switchyard, Polypus's own router process, reads: one route per composed router in the gateway's router config, with every leaf hop pointed back at the Polypus gateway. |
|
pkg
|
|
|
polypus
Package polypus is the public product façade for the Polypus inference gateway.
|
Package polypus is the public product façade for the Polypus inference gateway. |
Click to show internal directories.
Click to hide internal directories.