llm-mux

module

v1.0.10 Latest Latest Go to latest Published: Dec 17, 2025 License: MIT

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/nghyane/llm-mux

Links

README ¶

llm-mux

A unified gateway to access multiple LLM providers through any API format, without managing API keys.

Authenticate once via OAuth, then use OpenAI, Gemini, Claude, or Ollama API formats to call any provider. Free within CLI tool quotas.

Quick Start

# Install and start service
curl -fsSL https://raw.githubusercontent.com/nghyane/llm-mux/main/install.sh | bash

# Authenticate with a provider
llm-mux --login  # or --copilot-login, --antigravity-login, etc.

# Call any model using your preferred API format
curl http://localhost:8318/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model": "gemini-2.5-flash", "messages": [{"role": "user", "content": "Hello"}]}'

Features

Traditional API Access	llm-mux
Requires API keys per provider	Single OAuth authentication
Pay per token	Free (within CLI quotas)
Learn each provider's API	Use any API format with any provider
Manual token management	Auto-refresh tokens

Key Capabilities:

Multi-format API support (OpenAI, Gemini, Claude, Ollama)
Dynamic model registry with quota tracking
Automatic tool call normalization
OAuth session management with token refresh
Load balancing across providers

Installation

Script (Recommended)

# Install with auto-start service
curl -fsSL https://raw.githubusercontent.com/nghyane/llm-mux/main/install.sh | bash

Options:

# Binary only (no service)
bash -s -- --no-service

# Specific version
bash -s -- --version v1.0.0

# Custom directory
bash -s -- --dir /custom/path

The installer:

Downloads the binary for your platform
Installs to /usr/local/bin (or ~/.local/bin)
Creates default config at ~/.config/llm-mux/config.yaml
Sets up auto-start service (launchd/systemd)

Homebrew

brew tap nghyane/tap
brew install llm-mux
brew services start llm-mux

Docker

docker pull nghyane/llm-mux
docker run -p 8318:8318 -v ~/.config/llm-mux:/root/.config/llm-mux nghyane/llm-mux

From Source

git clone https://github.com/nghyane/llm-mux.git
cd llm-mux
go build -o llm-mux ./cmd/server/
./llm-mux --init
./llm-mux

Authentication

Login to providers using their respective CLI OAuth flows:

llm-mux --login              # Gemini CLI
llm-mux --antigravity-login  # Antigravity (multi-model)
llm-mux --claude-login       # Claude
llm-mux --codex-login        # OpenAI Codex
llm-mux --copilot-login      # GitHub Copilot
llm-mux --qwen-login         # Qwen
llm-mux --iflow-login        # iFlow
llm-mux --cline-login        # Cline
llm-mux --kiro-login         # Amazon Q/Kiro

Tokens are stored in ~/.config/llm-mux/auth/ and automatically refreshed.

Providers and Models

Google

Provider	Login Flag	Available Models
Gemini CLI	`--login`	gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-3-pro-preview
Antigravity	`--antigravity-login`	Gemini + Claude Sonnet/Opus 4.5 + GPT-OSS + Computer Use
AI Studio	`--login`	Gemini models + image generation

Anthropic

Provider	Login Flag	Available Models
Claude	`--claude-login`	claude-sonnet-4-5, claude-opus-4-5
Kiro	`--kiro-login`	Claude models via Amazon Q

OpenAI

Provider	Login Flag	Available Models
Codex	`--codex-login`	gpt-5.1, gpt-5.1-codex, gpt-5.1-codex-max
GitHub Copilot	`--copilot-login`	gpt-4.1, gpt-4o, gpt-5-mini, gpt-5.1-codex-max

Other Providers

Provider	Login Flag	Available Models
iFlow	`--iflow-login`	qwen3-coder-plus, deepseek-r1, kimi-k2, glm-4.6
Cline	`--cline-login`	minimax-m2, grok-code-fast-1
Qwen	`--qwen-login`	qwen3-coder-plus, qwen3-coder-flash

API Reference

llm-mux supports multiple API formats. Use any format with any provider.

Supported Formats

Format	Endpoints
OpenAI	`/v1/chat/completions`, `/v1/completions`, `/v1/models`
Gemini	`/v1beta/models/{model}:generateContent`, `/v1beta/models/{model}:streamGenerateContent`
Claude	`/v1/messages`
Ollama	`/api/chat`, `/api/generate`, `/api/tags`

Examples

OpenAI format:

curl http://localhost:8318/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model": "gemini-2.5-flash", "messages": [{"role": "user", "content": "Hello"}]}'

Gemini format:

curl http://localhost:8318/v1beta/models/gemini-2.5-flash:generateContent -H "Content-Type: application/json" \
  -d '{"contents": [{"role": "user", "parts": [{"text": "Hello"}]}]}'

Ollama format:

curl http://localhost:8318/api/chat -H "Content-Type: application/json" \
  -d '{"model": "gemini-2.5-flash", "messages": [{"role": "user", "content": "Hello"}]}'

Claude format:

curl http://localhost:8318/v1/messages -H "Content-Type: application/json" \
  -d '{"model": "claude-sonnet-4-5", "messages": [{"role": "user", "content": "Hello"}]}'

List Available Models

# OpenAI format
curl http://localhost:8318/v1/models

# Ollama format
curl http://localhost:8318/api/tags

Configuration

Initialize default config:

llm-mux --init

Config location: ~/.config/llm-mux/config.yaml Token storage: ~/.config/llm-mux/auth/

Service Management

macOS (launchd)

launchctl stop com.llm-mux     # Stop service
launchctl start com.llm-mux    # Start service
launchctl unload ~/Library/LaunchAgents/com.llm-mux.plist  # Disable auto-start

Service file: ~/Library/LaunchAgents/com.llm-mux.plist Logs: ~/.local/var/log/llm-mux.log

Linux (systemd)

systemctl --user stop llm-mux      # Stop service
systemctl --user start llm-mux     # Start service
systemctl --user status llm-mux    # Check status
systemctl --user disable llm-mux   # Disable auto-start

Service file: ~/.config/systemd/user/llm-mux.service Logs: journalctl --user -u llm-mux

Architecture

  Client API Formats              Providers
  ──────────────────              ─────────
    OpenAI ─────┐                 ┌─── Gemini CLI
    Claude ─────┤                 ├─── Antigravity
    Gemini ─────┼── Unified IR ───┼─── Claude/Kiro
    Ollama ─────┘                 ├─── Codex/Copilot
                                  └─── iFlow/Cline/Qwen

Core Components:

Format Translation Layer: Converts any API format to/from unified Intermediate Representation (IR), enabling 2n translations instead of n²
Provider Adapters: OAuth session management and provider-specific request handling
Dynamic Model Registry: Tracks available models and quota status across all authenticated providers
Tool Call Normalizer: Auto-fixes parameter name mismatches between formats (e.g., filePath ↔ file_path)
Load Balancer: Routes requests to available OAuth sessions, handles quota exhaustion

How It Works

OAuth Flow: Performs the same OAuth authentication as official CLI tools
Token Management: Stores credentials securely, auto-refreshes expired tokens
Request Processing:
- Client sends request in any supported format
- Translates to unified IR
- Converts to provider-native format
Response Processing:
- Receives provider response
- Translates to IR
- Converts back to original request format
Quota Handling: Automatically routes to alternative providers when quotas are exceeded

Credits

Refactored from CLIProxyAPI with redesigned IR translator, improved load balancing, and enhanced error handling.

License

MIT License - see LICENSE

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
cmd
server command Package main provides the entry point for the CLI Proxy API server.	Package main provides the entry point for the CLI Proxy API server.
examples
custom-provider command Package main demonstrates how to create a custom AI provider executor and integrate it with the CLI Proxy API server.	Package main demonstrates how to create a custom AI provider executor and integrate it with the CLI Proxy API server.
internal
access
access/config_access
api Package api provides the HTTP API server implementation for the CLI Proxy API.	Package api provides the HTTP API server implementation for the CLI Proxy API.
api/handlers/management Package management provides the management API handlers and middleware for configuring the server and managing auth files.	Package management provides the management API handlers and middleware for configuring the server and managing auth files.
api/middleware Package middleware provides HTTP middleware components for the CLI Proxy API server.	Package middleware provides HTTP middleware components for the CLI Proxy API server.
api/modules Package modules provides a pluggable routing module system for extending the API server with optional features without modifying core routing logic.	Package modules provides a pluggable routing module system for extending the API server with optional features without modifying core routing logic.
api/modules/amp Package amp implements the Amp CLI routing module, providing OAuth-based integration with Amp CLI for ChatGPT and Anthropic subscriptions.	Package amp implements the Amp CLI routing module, providing OAuth-based integration with Amp CLI for ChatGPT and Anthropic subscriptions.
auth Package auth provides authentication functionality for various AI service providers.	Package auth provides authentication functionality for various AI service providers.
auth/claude Package claude provides OAuth2 authentication functionality for Anthropic's Claude API.	Package claude provides OAuth2 authentication functionality for Anthropic's Claude API.
auth/cline
auth/codex Package codex provides authentication and token management for OpenAI's Codex API.	Package codex provides authentication and token management for OpenAI's Codex API.
auth/copilot Package copilot provides authentication for GitHub Copilot API.	Package copilot provides authentication for GitHub Copilot API.
auth/gemini Package gemini provides authentication and token management functionality for Google's Gemini AI services.	Package gemini provides authentication and token management functionality for Google's Gemini AI services.
auth/iflow
auth/kiro
auth/qwen Package qwen provides authentication and token management functionality for Alibaba's Qwen AI services.	Package qwen provides authentication and token management functionality for Alibaba's Qwen AI services.
auth/vertex Package vertex provides token storage for Google Vertex AI Gemini via service account credentials.	Package vertex provides token storage for Google Vertex AI Gemini via service account credentials.
browser Package browser provides cross-platform functionality for opening URLs in the default web browser.	Package browser provides cross-platform functionality for opening URLs in the default web browser.
buildinfo Package buildinfo exposes compile-time metadata shared across the server.	Package buildinfo exposes compile-time metadata shared across the server.
cmd Package cmd provides command-line interface functionality for the CLI Proxy API server.	Package cmd provides command-line interface functionality for the CLI Proxy API server.
config Package config provides configuration management for the CLI Proxy API server.	Package config provides configuration management for the CLI Proxy API server.
constant Package constant defines provider name constants used throughout the CLI Proxy API.	Package constant defines provider name constants used throughout the CLI Proxy API.
embedded
interfaces Package interfaces defines the core interfaces and shared structures for the CLI Proxy API server.	Package interfaces defines the core interfaces and shared structures for the CLI Proxy API server.
logging Package logging provides Gin middleware for HTTP request logging and panic recovery.	Package logging provides Gin middleware for HTTP request logging and panic recovery.
managementasset
misc Package misc provides miscellaneous utility functions and embedded data for the CLI Proxy API.	Package misc provides miscellaneous utility functions and embedded data for the CLI Proxy API.
oauth Package oauth provides OAuth HTML response generation.	Package oauth provides OAuth HTML response generation.
registry Package registry provides model definitions for various AI service providers.	Package registry provides model definitions for various AI service providers.
runtime/executor Package executor provides runtime execution capabilities for various AI service providers.	Package executor provides runtime execution capabilities for various AI service providers.
runtime/geminicli
store
translator/from_ir Package from_ir converts unified request format to provider-specific formats.	Package from_ir converts unified request format to provider-specific formats.
translator/ir Package ir provides memory pools for the translator layer.	Package ir provides memory pools for the translator layer.
translator/to_ir Package to_ir converts provider-specific API formats into unified format.	Package to_ir converts provider-specific API formats into unified format.
usage Package usage provides usage tracking and logging functionality for the CLI Proxy API server.	Package usage provides usage tracking and logging functionality for the CLI Proxy API server.
util Package util provides utility functions used across the llm-mux application.	Package util provides utility functions used across the llm-mux application.
watcher Package watcher provides file system monitoring functionality for the CLI Proxy API.	Package watcher provides file system monitoring functionality for the CLI Proxy API.
wsrelay
sdk
access
api/handlers
api/handlers/claude
api/handlers/gemini
api/handlers/ollama
api/handlers/openai
auth
cliproxy Package cliproxy provides the core service implementation for the CLI Proxy API.	Package cliproxy provides the core service implementation for the CLI Proxy API.
cliproxy/auth
cliproxy/executor
cliproxy/pipeline
cliproxy/usage
config Package config provides configuration management for the CLI Proxy API server.	Package config provides configuration management for the CLI Proxy API server.
translator Package translator provides format definitions for request/response translation.	Package translator provides format definitions for request/response translation.