README
¶
llm-mux
Free multi-format LLM gateway - access any model through your preferred API format, without API keys.
Authenticate once with OAuth, then use OpenAI/Gemini/Claude/Ollama API formats to call any provider.
Why llm-mux?
| Traditional API Access | llm-mux |
|---|---|
| Requires API keys | Uses OAuth from CLI tools |
| Pay per token | Free (within CLI quotas) |
| One provider per key | All providers, one gateway |
| Learn each provider's API | Use your preferred format |
Quick Start
# Install
brew tap nghyane/tap && brew install llm-mux
# Authenticate with any provider
llm-mux --login # Gemini CLI
llm-mux --antigravity-login # Antigravity (Gemini + Claude + GPT-OSS)
llm-mux --copilot-login # GitHub Copilot
# Start service
brew services start llm-mux
# Call Gemini using OpenAI format
curl http://localhost:8318/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "gemini-2.5-flash", "messages": [{"role": "user", "content": "Hello!"}]}'
Supported Providers
| Provider | Login | Models |
|---|---|---|
| Gemini CLI | --login |
gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-3-pro-preview |
| Antigravity | --antigravity-login |
Gemini + Claude Sonnet/Opus 4.5 + GPT-OSS + Computer Use |
| AI Studio | --login |
Gemini models + image generation |
| Vertex AI | API Key | Gemini models |
Anthropic
| Provider | Login | Models |
|---|---|---|
| Claude | --claude-login |
claude-sonnet-4-5, claude-opus-4-5 |
| Kiro | --kiro-login |
Claude models via Amazon Q |
OpenAI
| Provider | Login | Models |
|---|---|---|
| Codex | --codex-login |
gpt-5.1, gpt-5.1-codex, gpt-5.1-codex-max |
| GitHub Copilot | --copilot-login |
gpt-4.1, gpt-4o, gpt-5-mini, gpt-5.1-codex-max |
Others
| Provider | Login | Models |
|---|---|---|
| iFlow | --iflow-login |
qwen3-coder-plus, deepseek-r1, kimi-k2, glm-4.6 |
| Cline | --cline-login |
minimax-m2, grok-code-fast-1 |
| Qwen | --qwen-login |
qwen3-coder-plus, qwen3-coder-flash |
API Formats
Choose your preferred format - all formats can access all providers:
| Format | Endpoints |
|---|---|
| OpenAI | /v1/chat/completions, /v1/completions, /v1/models |
| Gemini | /v1beta/models/{model}:generateContent, /v1beta/models/{model}:streamGenerateContent |
| Claude | /v1/messages |
| Ollama | /api/chat, /api/generate, /api/tags |
# Same model, different formats:
# OpenAI format
curl localhost:8318/v1/chat/completions -d '{"model":"gemini-2.5-flash",...}'
# Gemini format
curl localhost:8318/v1beta/models/gemini-2.5-flash:generateContent -d '{"contents":[...]}'
# Ollama format
curl localhost:8318/api/chat -d '{"model":"gemini-2.5-flash",...}'
Architecture
API Formats Providers
─────────── ─────────
OpenAI ───┐ ┌─── Gemini CLI
Claude ───┼─── Unified ─────┼─── Antigravity
Gemini ───┤ IR ├─── Claude/Kiro
Ollama ───┘ └─── Codex/Copilot/iFlow
- IR Translation: Each format converts to/from IR (2n translations instead of n²)
- Tool Call Normalization: Auto-fixes parameter mismatches (
filePath↔file_path) - Dynamic Registry: Tracks OAuth sessions, auto-hides models when quota exceeded
Installation
Homebrew
brew tap nghyane/tap
brew install llm-mux
brew services start llm-mux
Docker
docker pull nghyane/llm-mux
docker run -p 8318:8318 -v ~/.config/llm-mux:/root/.config/llm-mux nghyane/llm-mux
From Source
go build -o llm-mux ./cmd/server/
./llm-mux -config config.yaml
Configuration
port: 8318
auth-dir: "~/.config/llm-mux/auth"
use-canonical-translator: true
Tokens stored in ~/.config/llm-mux/auth/ with auto-refresh.
How It Works
- OAuth Capture: Performs same OAuth flow as official CLI tools
- Token Management: Stores and auto-refreshes tokens
- Format Translation: Parses request (any format) → IR → provider-native format
- Response Translation: Provider response → IR → original request format
- Load Balancing: Routes to available OAuth sessions, handles quota limits
License
MIT License - see LICENSE
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
server
command
Package main provides the entry point for the CLI Proxy API server.
|
Package main provides the entry point for the CLI Proxy API server. |
|
examples
|
|
|
custom-provider
command
Package main demonstrates how to create a custom AI provider executor and integrate it with the CLI Proxy API server.
|
Package main demonstrates how to create a custom AI provider executor and integrate it with the CLI Proxy API server. |
|
internal
|
|
|
api
Package api provides the HTTP API server implementation for the CLI Proxy API.
|
Package api provides the HTTP API server implementation for the CLI Proxy API. |
|
api/handlers/management
Package management provides the management API handlers and middleware for configuring the server and managing auth files.
|
Package management provides the management API handlers and middleware for configuring the server and managing auth files. |
|
api/middleware
Package middleware provides HTTP middleware components for the CLI Proxy API server.
|
Package middleware provides HTTP middleware components for the CLI Proxy API server. |
|
api/modules
Package modules provides a pluggable routing module system for extending the API server with optional features without modifying core routing logic.
|
Package modules provides a pluggable routing module system for extending the API server with optional features without modifying core routing logic. |
|
api/modules/amp
Package amp implements the Amp CLI routing module, providing OAuth-based integration with Amp CLI for ChatGPT and Anthropic subscriptions.
|
Package amp implements the Amp CLI routing module, providing OAuth-based integration with Amp CLI for ChatGPT and Anthropic subscriptions. |
|
auth
Package auth provides authentication functionality for various AI service providers.
|
Package auth provides authentication functionality for various AI service providers. |
|
auth/claude
Package claude provides OAuth2 authentication functionality for Anthropic's Claude API.
|
Package claude provides OAuth2 authentication functionality for Anthropic's Claude API. |
|
auth/codex
Package codex provides authentication and token management for OpenAI's Codex API.
|
Package codex provides authentication and token management for OpenAI's Codex API. |
|
auth/copilot
Package copilot provides authentication for GitHub Copilot API.
|
Package copilot provides authentication for GitHub Copilot API. |
|
auth/gemini
Package gemini provides authentication and token management functionality for Google's Gemini AI services.
|
Package gemini provides authentication and token management functionality for Google's Gemini AI services. |
|
auth/qwen
Package qwen provides authentication and token management functionality for Alibaba's Qwen AI services.
|
Package qwen provides authentication and token management functionality for Alibaba's Qwen AI services. |
|
auth/vertex
Package vertex provides token storage for Google Vertex AI Gemini via service account credentials.
|
Package vertex provides token storage for Google Vertex AI Gemini via service account credentials. |
|
browser
Package browser provides cross-platform functionality for opening URLs in the default web browser.
|
Package browser provides cross-platform functionality for opening URLs in the default web browser. |
|
buildinfo
Package buildinfo exposes compile-time metadata shared across the server.
|
Package buildinfo exposes compile-time metadata shared across the server. |
|
cmd
Package cmd provides command-line interface functionality for the CLI Proxy API server.
|
Package cmd provides command-line interface functionality for the CLI Proxy API server. |
|
config
Package config provides configuration management for the CLI Proxy API server.
|
Package config provides configuration management for the CLI Proxy API server. |
|
constant
Package constant defines provider name constants used throughout the CLI Proxy API.
|
Package constant defines provider name constants used throughout the CLI Proxy API. |
|
interfaces
Package interfaces defines the core interfaces and shared structures for the CLI Proxy API server.
|
Package interfaces defines the core interfaces and shared structures for the CLI Proxy API server. |
|
logging
Package logging provides Gin middleware for HTTP request logging and panic recovery.
|
Package logging provides Gin middleware for HTTP request logging and panic recovery. |
|
misc
Package misc provides miscellaneous utility functions and embedded data for the CLI Proxy API.
|
Package misc provides miscellaneous utility functions and embedded data for the CLI Proxy API. |
|
registry
Package registry provides model definitions for various AI service providers.
|
Package registry provides model definitions for various AI service providers. |
|
runtime/executor
Package executor provides runtime execution capabilities for various AI service providers.
|
Package executor provides runtime execution capabilities for various AI service providers. |
|
translator/from_ir
Package from_ir converts unified request format to provider-specific formats.
|
Package from_ir converts unified request format to provider-specific formats. |
|
translator/ir
Package ir provides memory pools for the translator layer.
|
Package ir provides memory pools for the translator layer. |
|
translator/to_ir
Package to_ir converts provider-specific API formats into unified format.
|
Package to_ir converts provider-specific API formats into unified format. |
|
usage
Package usage provides usage tracking and logging functionality for the CLI Proxy API server.
|
Package usage provides usage tracking and logging functionality for the CLI Proxy API server. |
|
util
Package util provides utility functions used across the llm-mux application.
|
Package util provides utility functions used across the llm-mux application. |
|
watcher
Package watcher provides file system monitoring functionality for the CLI Proxy API.
|
Package watcher provides file system monitoring functionality for the CLI Proxy API. |
|
sdk
|
|
|
cliproxy
Package cliproxy provides the core service implementation for the CLI Proxy API.
|
Package cliproxy provides the core service implementation for the CLI Proxy API. |
|
config
Package config provides configuration management for the CLI Proxy API server.
|
Package config provides configuration management for the CLI Proxy API server. |
|
translator
Package translator provides format definitions for request/response translation.
|
Package translator provides format definitions for request/response translation. |
Click to show internal directories.
Click to hide internal directories.