gateway

README

Inja LLM Gateway

Small, dependency-free LLM API gateway
OpenAI · Anthropic · Gemini dialects · multi-provider routing · usage hooks
One static binary — laptop, Docker, or Kubernetes

Documentation · Quickstart · HTTP API · Config · Deploy · Contributing · Security · License

---

Clients speak OpenAI, Anthropic, or native Gemini. The gateway routes to any configured upstream (OpenAI, Anthropic, Google, DeepSeek, xAI, Moonshot, OpenRouter, vLLM, …), passthroughs same-family traffic, translates cross-dialect chat when needed, and emits one usage event per request (JSONL / webhook / Go hook). Stateless — no database.

  OpenAI SDK / Anthropic SDK / Gemini client / Claude Code / curl
                            │
                            ▼
                    ┌───────────────┐
                    │  llm-gateway  │──► usage hooks (JSONL / webhook)
                    └───────┬───────┘
                            │
         ┌──────────┬───────┼────────┬────────────┐
         ▼          ▼       ▼        ▼            ▼
      OpenAI   Anthropic  Google  OpenAI-compat  …
                              native   (xAI, DeepSeek, …)

Stateless	No DB, sessions, or sticky routing — scale identical replicas
Cloud-native	Distroless Docker, K8s sample, SIGTERM drain, env overrides
Local-first	Single binary on macOS, Linux, Windows; docker compose up
Deps	Runtime: gopkg.in/yaml.v3 only
Module	github.com/inja-online/llm-gateway
License	AGPL-3.0

Docs site: inja-online.github.io/llm-gateway ·
Also in-repo: compatibility matrix · SDK hermetic matrix · deprecation policy · Claude Code checklist · multipart security · CHANGELOG

---

Table of contents

• Quickstart
• Features
• HTTP API
• Model routing
• Configuration
• Auth & keys
• Provider notes
• Passthrough vs translation
• Hooks & usage events
• Claude Code
• Library use
• Architecture
• Deploy
• Development & CI
• Roadmap
• Contributing
• License

---

Quickstart

1. Binary

git clone https://github.com/inja-online/llm-gateway.git
cd llm-gateway
go build -o llm-gateway ./cmd/gateway

cp gateway.example.yaml gateway.yaml
# edit providers / keys / hooks
./llm-gateway -config gateway.yaml
# override: GATEWAY_CONFIG=…  GATEWAY_LISTEN=0.0.0.0:8787

curl -s http://localhost:8787/healthz
# {"status":"ok"}

2. Docker

docker compose up --build
# or
docker build -t llm-gateway:local .
docker run --rm -p 8787:8787 \
  -e OPENAI_API_KEY -e ANTHROPIC_API_KEY -e GEMINI_API_KEY \
  -v "$PWD/gateway.yaml:/config/gateway.yaml:ro" \
  llm-gateway:local

3. Kubernetes

kubectl apply -f deploy/k8s/gateway.yaml
# point the Deployment image at your registry build

Minimal config

listen: ":8787"

providers:
  openai:
    kind: openai
    base_url: "https://api.openai.com/v1"
    api_key_env: OPENAI_API_KEY
  anthropic:
    kind: anthropic
    base_url: "https://api.anthropic.com/v1"
    api_key_env: ANTHROPIC_API_KEY
  google:
    kind: google
    base_url: "https://generativelanguage.googleapis.com/v1beta"
    api_key_env: GEMINI_API_KEY
  deepseek:
    kind: openai_compat
    base_url: "https://api.deepseek.com"

defaults:
  openai_dialect: openai
  anthropic_dialect: anthropic
  google_dialect: google

aliases:
  fast: deepseek/deepseek-chat

hooks:
  jsonl:
    output: stdout

Full commented sample: gateway.example.yaml.

---

Features

Area	What you get
Triple ingress	OpenAI Chat Completions, Anthropic Messages (Claude Code), native Gemini generateContent
Multi-provider egress	openai, openai_compat, anthropic, google
Passthrough-first	Same dialect → near-verbatim bytes
Cross-dialect chat	OpenAI ↔ Anthropic ↔ Google translation (structured outputs, thinking, tools, …)
Media	Images, video jobs, TTS/STT (OpenAI paths + Anthropic-version gate + Google speech)
Agent surface	Responses, Files (OpenAI + Anthropic), Message Batches, Moderations
Realtime	OpenAI Realtime WS + Google Live passthrough; cross-protocol bridge not implemented (fail-closed)
Usage metering	JSONL, async webhook, or in-process Go hook — one event per proxied request
Ops	One YAML file, /healthz, body limits, optional edge auth, multi-arch releases

---

HTTP API

Chat

Method	Path	Notes
POST	/v1/chat/completions	OpenAI dialect (also Gemini OpenAI-compat clients)
GET / POST / DELETE	/v1/chat/completions · /{id}	Stored completions (store=true) openai-family proxy
POST	/v1/messages	Anthropic dialect (anthropic-version required by clients)
POST	/v1/messages/count_tokens	Anthropic proxy, Google :countTokens map, or local estimate
POST	/v1beta/models/{model}:generateContent	Native Gemini
POST	/v1beta/models/{model}:streamGenerateContent	Native Gemini SSE (?alt=sse upstream)
POST	/v1beta/models/{model}:countTokens	Native Gemini; no usage event

Discovery & health

Method	Path	Notes
GET	/v1/models	Config-derived catalog + capabilities; live Anthropic when anthropic-version or ?live=1
GET	/v1/models/{id}	Single entry (provider/model ids); live Anthropic with same triggers
GET	/v1beta/models	Gemini models list passthrough (?provider= or defaults.google_dialect)
POST / GET / DELETE	/v1beta/fileSearchStores…	Gemini File Search stores (kind:google)
POST / GET / PATCH / DELETE	/v1beta/cachedContents…	Gemini context cache CRUD (kind:google); reference via cachedContent on generateContent
POST / GET / DELETE	/v1beta/tunedModels…	Gemini tuned model lifecycle (kind:google)
GET	/v1beta/models/{model}	Gemini model get / Live upgrade when :bidiGenerateContent
GET	/healthz	{"status":"ok"} — process liveness only
GET	/v1/health/providers	Optional upstream probes when health_checks.enabled (default off)
GET	/metrics	Prometheus via client_golang/promhttp (counters + histogram + Go/process; open with edge auth)

Embeddings, Responses, Files, Batches

Method	Path	Notes
POST	/v1/embeddings	OpenAI-family passthrough; OpenAI→Google embedContent / batch map
POST	/v1beta/models/{m}:embedContent	Native Gemini embeddings
POST	/v1beta/models/{m}:batchEmbedContents	Native Gemini batch embeddings
POST	/v1/responses	OpenAI Responses (stream SSE supported)
GET / DELETE	/v1/responses/{id}	?provider= or default OpenAI dialect
POST / GET / DELETE	/v1/files…	OpenAI Files or Anthropic Files when anthropic-version is set
POST / GET / DELETE	/v1/vector_stores…	OpenAI Vector stores (upstream-owned)
POST / GET	/v1/uploads…	OpenAI Uploads (multipart resume; upstream-owned)
POST / GET / DELETE	/v1/containers…	OpenAI Containers (upstream-owned)
GET	/v1/files/{id}/content	Streamed download
POST / GET	/v1/messages/batches…	Anthropic Message Batches (kind: anthropic only)
POST / GET / DELETE	/v1/skills…	Anthropic Skills Management (beta headers forwarded)
POST / GET / DELETE	/v1/tunnels…	Anthropic MCP tunnels (beta headers forwarded)
POST / GET / DELETE	/v1/memory_stores…	Anthropic agent memory stores
POST / GET	/v1/batches…	OpenAI Batches (openai / openai_compat; cancel via POST …/cancel)
POST / GET	/v1/fine_tuning/jobs…	OpenAI Fine-tuning jobs, cancel, events, checkpoints
POST	/v1/moderations	OpenAI-family passthrough
POST	/v1/tokenizers/estimate-token-count	Moonshot helper (openai_compat; ?provider= / default)
GET	/v1/users/me/balance	Moonshot balance helper (openai_compat)
GET	/v1/credits · /v1/key · /v1/generation	OpenRouter account/generation helpers
POST / GET / DELETE	/v1/assistants… · /v1/threads…	Legacy Assistants v2 (prefer Responses; send OpenAI-Beta)

Files and batches are upstream-owned (no gateway disk store). Body cap: max_body_bytes (default 32 MiB).

Anthropic Files use the same /v1/files* paths as OpenAI; presence of anthropic-version selects the Anthropic path. Client anthropic-beta (including unknown values) is forwarded.

Media & audio

Method	Path	Notes
POST	/v1/images/generations · /edits · /variations	OpenAI-shaped
POST	/v1/images · /v1/images/edits	Anthropic-gateway when anthropic-version set
POST	/v1/videos · GET /v1/videos/{id} · /content	Video jobs
POST	/v1/audio/speech	TTS (OpenAI path; Anthropic-version → Anthropic-gateway contract)
POST	/v1/audio/transcriptions · /translations	STT multipart/JSON
POST	/v1beta/models/{m}:generateSpeech	Google-shaped TTS → Gemini AUDIO generateContent

openai_compat media/realtime defaults off — opt in with capabilities in YAML.

Realtime (WebSocket)

Path	Notes
GET /v1/realtime	OpenAI Realtime upgrade; requires capabilities.realtime
GET /v1beta/models/{m}:bidiGenerateContent	Google Live; kind: google + realtime capability

Same-protocol passthrough only. Cross-protocol Realtime↔Live attempts return unsupported_realtime_bridge. Session limits: realtime.max_sessions (default 1024), realtime.max_session_minutes (default 60).

TLS / wss: production upstreams work. Provider base_url may be https://… (or wss://…); the gateway dials TLS with system roots (TLS 1.2+) and HTTP Upgrade. Example:

providers:
  openai:
    kind: openai
    base_url: "https://api.openai.com/v1"   # dials wss via TLS
    api_key_env: OPENAI_API_KEY
    # capabilities.realtime defaults on for kind: openai

TCP keepalive (30s) is enabled on the upstream socket. Application WebSocket ping/pong frames from either peer are passed through raw. Auth uses the same modes as HTTP (api_key_env, oauth2, client_bearer, ADC/SA TokenSource).

Completions (experimental)

Method	Path	Notes
POST	/v1/completions	OpenAI-family Completions passthrough
POST	/beta/completions	Rewrites base …/v1 → …/beta (DeepSeek FIM)

Not multi-dialect translated. Prefer chat for normal use.

Conversations (not supported)

/v1/conversations* (including nested paths such as /{id}/items) returns HTTP 501 with an OpenAI-shaped error:

Field	Value
status	501
error.type / code	not_implemented
guidance	Prefer POST /v1/responses with client-side conversation/history state; use Files for durable assets

Decision: permanent skip of gateway-side conversation storage (stateless). Routes are registered so SDKs get a structured 501 instead of a bare 404. Do not add a gateway database or Redis thread store.

Alternatives:

1. POST /v1/responses (+ get/delete by id on upstream when supported)
2. Client-owned message history on subsequent chat/Responses calls
3. Files / vector-store workstreams for stored assets (upstream-owned)

Formal product decision (Option A permanent 501): docs/conversations-decision.md (#118).

Dialect pairing (chat)

Client dialect	Upstream	Path
OpenAI	openai / openai_compat	passthrough
OpenAI	anthropic / google	translated
Anthropic	anthropic	passthrough
Anthropic	openai / openai_compat / google	translated
Google	google	passthrough
Google	openai / openai_compat / anthropic	translated

Limits & timeouts

Limit	Default	Config / code
Request/response body	32 MiB	max_body_bytes (bytes); oversize → 413
HTTP ReadHeaderTimeout	10s	server
Upstream ResponseHeaderTimeout	60s	HTTP client
Idle conn	90s	HTTP client
count_tokens upstream	15s	request context
Realtime max sessions	1024	realtime.max_sessions
Realtime max duration	60 min	realtime.max_session_minutes
SIGTERM drain	30s	process
Webhook hook timeout	3s	hooks.webhook.timeout

---

Model routing

Public model resolves in order:

1. aliases — exact match (fast → deepseek/deepseek-chat)
2. provider/model — first segment is a configured provider name
3. Bare id — dialect default (defaults.openai_dialect / anthropic_dialect / google_dialect)

Missing default or unknown provider → 404 (dialect error envelope).

GET /v1/models is built from config only (no upstream fan-out). Each entry may include:

{
  "id": "fast",
  "object": "model",
  "created": 0,
  "owned_by": "llm-gateway",
  "capabilities": {
    "chat": true,
    "image_gen": false,
    "video_gen": false,
    "audio_speech": false,
    "audio_transcribe": false,
    "realtime": false
  }
}

Flags come from provider kind defaults + optional YAML capabilities (text maps to JSON chat).

---

Configuration

Single YAML file. Unknown fields are rejected.

Field	Required	Description
listen	no	Bind address; default :8787 (GATEWAY_LISTEN)
providers	yes	Map of name → provider (≥1)
providers.<n>.kind	yes	openai | openai_compat | anthropic | google
providers.<n>.base_url	yes	Origin with version prefix; trailing / trimmed
providers.<n>.api_key_env	no	Env var; when set & non-empty, replaces client key
providers.<n>.capabilities	no	Override modality flags; nil → kind defaults (openai_compat = text only)
providers.<n>.auth	no	api_key (default) | adc | service_account | oauth2 | client_bearer | bearer
providers.<n>.oauth	when auth: oauth2	Token URL + client/refresh credentials (see OAuth / token sources)
providers.<n>.service_account_file	no	GCP SA JSON path; auto TokenSource with auth: service_account / adc
defaults.openai_dialect	no	Bare models on OpenAI ingress
defaults.anthropic_dialect	no	Bare models on Anthropic ingress
defaults.google_dialect	no	Bare models on Gemini ingress
aliases	no	Public id → provider/upstream-model
max_body_bytes	no	Default 33554432 (32 MiB)
observe_dropped_fields	no	Default false. When true, translate responses set X-Gateway-Dropped-Fields (names only) and usage dropped_fields
health_checks.enabled	no	Default false. Enables GET /v1/health/providers upstream probes
health_checks.timeout	no	Per-provider probe timeout (default 2s)
edge_auth	no	Optional shared-secret gate (see Auth)
realtime.*	no	Session caps
hooks.jsonl / hooks.webhook	no	Usage sinks

Provider kinds

Kind	Typical base	Auth
openai	https://api.openai.com/v1	Authorization: Bearer
openai_compat	DeepSeek, xAI, Moonshot, OpenRouter, Gemini …/v1beta/openai, vLLM	Bearer
anthropic	https://api.anthropic.com/v1	x-api-key + anthropic-version
google	https://generativelanguage.googleapis.com/v1beta	x-goog-api-key (or Bearer via auth: adc)

---

Auth & keys

Upstream credentials

The gateway reads a client credential from:

1. Authorization: Bearer <key>, or
2. x-api-key: <key>, or
3. x-goog-api-key: <key>

…and forwards it using the provider’s scheme, unless a server-held mode replaces it:

Mode	Behavior
api_key (default)	Client key, or api_key_env when set and non-empty
bearer	Always Authorization: Bearer (client or api_key_env)
client_bearer	Always forward client Bearer; never replace with api_key_env (multi-tenant / user OAuth)
oauth2	Built-in OAuth2 TokenSource from oauth: block (client_credentials or refresh_token)
adc / service_account	Bearer from TokenSource: inject via SetTokenSource, or auto from service_account_file / GOOGLE_APPLICATION_CREDENTIALS

Usage events include key_hash (12 hex chars of SHA-256 of the upstream credential) — correlate without storing secrets.

OAuth / token sources

Server-held OAuth2 (no cloud SDK; form POST + optional SA JWT):

providers:
  openai:
    kind: openai
    base_url: "https://api.openai.com/v1"
    auth: oauth2
    oauth:
      token_url: "https://oauth.example.com/token"
      client_id_env: OPENAI_OAUTH_CLIENT_ID
      client_secret_env: OPENAI_OAUTH_CLIENT_SECRET
      # or refresh_token_env for refresh grant (auto when refresh is set)
      scopes: ["api"]

  vertex:
    kind: google
    base_url: "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT/locations/us-central1/publishers/google"
    auth: service_account
    service_account_file: /secrets/vertex-sa.json   # auto JWT → access token

• Tokens are cached until expires_in (30s skew). Concurrent refresh is single-flight.
• Prefer *_env for secrets; inline client_id / client_secret / refresh_token are for tests only.
• Edge auth is orthogonal to upstream OAuth — do not put provider refresh tokens in edge_auth.keys.
• ToS: consumer subscription OAuth (e.g. ChatGPT/Claude consumer) may forbid multi-user products; use operator-held credentials and vendor docs.

Multi-tenant pattern: edge_auth + auth: client_bearer so each client’s OAuth access token is forwarded upstream.

See SECURITY.md, docs/oauth-token-sources.md, and docs/wif-recipes.md (WIF / token_file).

Optional edge auth

By default the gateway does not authenticate callers (trusted network / external auth). To require a shared secret:

edge_auth:
  enabled: true
  keys_env: GATEWAY_EDGE_KEYS   # comma-separated

When enabled, every route except GET /healthz requires a matching key. Missing/invalid → 401. Constant-time compare; keys never logged. With api_key_env on providers, clients only need the edge key.

See SECURITY.md.

Forwarded client headers

When present: HTTP-Referer, Referer, X-Title, OpenAI-Organization, OpenAI-Project, anthropic-beta, client anthropic-version.
anthropic-beta is not allowlisted — unknown / future beta strings are forwarded unchanged.

---

Provider notes

Full comments: gateway.example.yaml. Matrices: docs/compatibility-matrix.md, docs/sdk-compatibility-matrix.md.

Provider	Kind	Notes
OpenAI	openai	Chat, Responses, Files, Moderations, images, video, audio, Realtime
Anthropic	anthropic	Messages, count_tokens, Files (+ beta), Batches
Google native	google	generateContent, embeddings, models list, Live, speech
Gemini OpenAI-compat	openai_compat	…/v1beta/openai base; opt-in media capabilities
DeepSeek	openai_compat	Chat + experimental Completions/FIM (/v1 or /beta)
OpenRouter / xAI / Moonshot / Groq / Qwen / …	openai_compat	Passthrough; set capabilities for media/realtime
Z.AI / Zhipu (GLM)	openai_compat	Regional bases (intl vs CN) — docs/providers/zai.md; wrong region ⇒ auth fail
Qwen (DashScope)	openai_compat	Regional bases + compatible-mode path — docs/providers/qwen.md; aliases qwen-turbo / qwen-plus
xAI (Grok)	openai_compat	Chat + Responses; Imagine images need image_gen — docs/providers/xai.md; alias grok
Groq	openai_compat	STT-first split routing — docs/providers/groq-stt.md; audio_transcribe + alias whisper-fast
Moonshot helpers	openai_compat	POST /v1/tokenizers/estimate-token-count, GET /v1/users/me/balance via ?provider= / default OpenAI dialect (regional base)
Vertex	google + auth: adc / service_account	Auto SA JWT from service_account_file, or inject TokenSource; no Google SDK bundled

---

Passthrough vs translation

Passthrough (same family)

Client dialect matches provider kind → near-verbatim proxy: model rewrite, auth, headers, one usage event. Highest fidelity.

Translation (cross family)

Client and upstream disagree → parse to canonical IR, rebuild wire, stream map. Chat fidelity includes tools, structured outputs, thinking/reasoning, sampling knobs, document/audio blocks where mapped. Some vendor-only fields are still dropped — see docs/deprecation-policy.md (passthrough never drops; translation drop lists + semver) and testdata/fixtures/chat_translate/drops/. Non-function OpenAI tools: error on translate, forward on passthrough — docs/tools-policy.md.

Prompt caching: Anthropic cache_control, OpenAI prompt_cache_*, and Google cachedContent are preserved within family (including Anthropic translate rebuild). Cross-family hops drop foreign cache directives. Optional caching.auto_breakpoints can inject Anthropic breakpoints on OpenAI/Google → Anthropic translate (default off) — docs/cache-control-policy.md.

Not multi-dialect translated: Completions/FIM, most media jobs (family passthrough + limited speech translate), Files, Batches.

---

Hooks & usage events

Exactly one UsageEvent per proxied chat, media, embeddings, audio, responses, files, batches create, or realtime session (including errors). Not emitted for count_tokens, Gemini :countTokens, models discovery, or healthz.

Sink	Config
JSONL	hooks.jsonl.output: stdout | stderr | file path
Webhook	hooks.webhook.url (+ optional timeout, default 3s)
Go	gateway.WithHook(...) in library mode

Metrics / Prometheus

GET /metrics is served by prometheus/client_golang (promhttp) on a per-server registry:

Metric	Type	Labels
llm_gateway_requests_total	counter	status
llm_gateway_tokens_in_total / _out_total	counter	—
llm_gateway_request_duration_seconds	histogram	status
Go + process collectors	—	standard

Open when edge auth is on (like /healthz). Prefer hooks for high-cardinality labels (model/provider) / full billing detail.

Provider health

/healthz is process liveness only. Optional GET /v1/health/providers probes configured upstreams when health_checks.enabled: true (timeouts; no key logging).

Event shape (JSON)

Typical fields: request_id, time, dialect_in, provider, model, upstream_model, modality, transport, token counts, optional media, stream, status, http_status, latency_ms, key_hash. See hooks package docs.

---

Claude Code

Point Claude Code at the gateway (Anthropic Messages ingress). Same-family Anthropic is passthrough; OpenAI / xAI aliases are translated.

export ANTHROPIC_BASE_URL=http://localhost:8787
export ANTHROPIC_API_KEY=sk-…   # or edge key when edge_auth is on
# optional: ANTHROPIC_MODEL=deepseek/deepseek-chat
claude

Multi-provider with subscription OAuth (ChatGPT Plus/Pro, Claude Pro/Max, SuperGrok — not API keys):

# one-time: install helpers from the binary (or source examples/shell/… from a checkout)
llm-gateway helpers install && eval "$(llm-gateway helpers source)"

llm-gateway auth login chatgpt   # and/or claude, grok
export KEY=local-dev
cc-gateway-up
cc-gpt                           # or cc-grok / cc-gpt-grok / cc-multi
# /model grok-4.5 | /model composer-2.5 | /model gpt | /model sonnet
cc-gateway-logs -f               # usage + HTTP access

From a git checkout you can still use KEY=local-dev ./examples/claude-code-multi.sh gpt+grok or source examples/shell/claude-code-helpers.sh.

API-key recipe (alternative): examples/configs/claude-code-multi.yaml.

Guide: docs/claude-code-multi.md. Checklist: docs/claude-code-checklist.md.

Same subscriptions in Claude Desktop, Codex, Cursor, …

One gateway backs any OpenAI- or Anthropic-compatible app.

From a release binary (no git checkout):

llm-gateway helpers install    # or: llm-gateway load-helpers
# add to ~/.zshrc the "source …" lines it prints, then:
source ~/.config/inja-gateway/shell/claude-code-helpers.sh
source ~/.config/inja-gateway/shell/cursor-helpers.sh
source ~/.config/inja-gateway/shell/apps-helpers.sh
export KEY=local-dev && cc-gateway-up

From a git checkout:

source examples/shell/claude-code-helpers.sh
export KEY=local-dev && cc-gateway-up
source examples/shell/apps-helpers.sh
apps-setup                    # Claude Desktop, Codex, Continue, Cline, Aider, …
apps-use-gateway              # write gateway configs; snapshot "default" for rollback
apps-use-default              # restore pre-gateway settings
# apps-status · apps-list-backups · apps-switch gateway|default

Templates: examples/apps/. Docs: Use subscriptions in any app.

---

Library use

import (
    gateway "github.com/inja-online/llm-gateway"
    "github.com/inja-online/llm-gateway/config"
    "github.com/inja-online/llm-gateway/hooks/jsonl"
)

cfg, err := config.Load("gateway.yaml")
// ...
hook, _ := jsonl.New(cfg.Hooks.JSONL.Output)
h, err := gateway.New(cfg, gateway.WithHook(hook))
// http.ListenAndServe(cfg.Listen, h)

Inject Vertex/ADC tokens with proxy.Server.SetTokenSource when embedding the package.

---

Architecture

cmd/gateway          → binary, flags, graceful shutdown
config/              → YAML, capabilities, edge_auth, body limit
proxy/               → HTTP/WS routing, passthrough, translation orchestration
canonical/           → dialect-neutral chat/image/video/audio/realtime types
ingress/{openai,anthropic,google}/
egress/{openai,anthropic,google}/
hooks/{jsonl,webhook}/

Invariant: same-family traffic prefers byte passthrough; translation only when dialects differ.

---

Deploy

Path	Use
Dockerfile	Distroless multi-stage build
docker-compose.yml	Local stack
deploy/k8s/gateway.yaml	Sample Deployment/Service
Release workflow	Tag v* → multi-arch binaries

Set secrets via env (api_key_env / keys_env); mount config read-only.

---

Development & CI

go test ./...
go test -race ./...
go test ./... -coverprofile=coverage.out && go tool cover -func=coverage.out | tail -1
go vet ./...
docker build -t llm-gateway:dev .

CI (push/PR): build, vet, go test -race -count=1 ./... (air-gapped httptest only — no -tags live), coverage ≥ 90%, binary smoke, Docker healthz.

Hermetic dialect anchors: docs/sdk-compatibility-matrix.md.

Docs site (Nimbus + Astro 7, GitHub Pages):

cd website && npm install && npm run dev   # http://localhost:4321/llm-gateway/

Product landing + docs tree under website/ · workflow: .github/workflows/docs.yml. Agent index: /llms.txt.

Release: git tag vX.Y.Z && git push origin vX.Y.Z

---

Roadmap

Shipped: multi-dialect chat fidelity, media/audio, Responses/Files/Batches, Realtime/Live passthrough, models capabilities, edge auth, AGPL-3.0.

Possible follow-ups:

• Optional full Realtime ↔ Live IR bridge (today: fail-closed)
• Deeper cross-dialect image/video generation translation
• Richer Prometheus histograms/labels beyond low-cardinality counters

---

Contributing

See CONTRIBUTING.md (modality checklist + layout).
Security: SECURITY.md · docs/security-multipart-review.md.
Changelog: CHANGELOG.md.

---

License

GNU Affero General Public License v3.0 (AGPL-3.0) © 2026 inja-online

This project is free software under the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

If you run a modified version of this software as a network service (for example a hosted LLM gateway), the AGPL requires that you offer the corresponding source code of that modified version to users of the service. See LICENSE for the full terms.

Documentation

Overview

Package gateway is the library entrypoint. Embed the gateway in your own process with New, or run the standalone binary in cmd/gateway.

Index

• func New(cfg *config.Config, opts ...Option) (http.Handler, error)
• type Option
  • func WithHook(h hooks.Hook) Option

Functions

func New

func New(cfg *config.Config, opts ...Option) (http.Handler, error)

New builds the gateway HTTP handler from config. Hooks configured in the YAML (jsonl, webhook) are wired automatically; WithHook adds more.

Types

type Option

type Option func(*options)

func WithHook

func WithHook(h hooks.Hook) Option

WithHook registers an additional in-process usage hook.