llm-mux

module
v1.0.20 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 20, 2025 License: MIT

README

llm-mux

Platform Go Version

The Universal AI Gateway. Access Gemini, Claude, OpenAI, and GitHub Copilot models through a single local API. No API keys required—just OAuth.

graph LR
    User[Your App/CLI] -->|OpenAI/Claude Format| Mux[llm-mux]
    Mux -->|OAuth| Google[Gemini]
    Mux -->|OAuth| Anthropic[Claude]
    Mux -->|OAuth| GitHub[Copilot]
    Mux -->|OAuth| Others[...]

⚡️ Quick Install

Get up and running in seconds. The installer sets up the binary and a background service automatically.

OS Command
macOS / Linux curl -fsSL https://raw.githubusercontent.com/nghyane/llm-mux/main/install.sh | bash
Windows (PowerShell) irm https://raw.githubusercontent.com/nghyane/llm-mux/main/install.ps1 | iex
Docker docker run -p 8318:8318 -v ~/.config/llm-mux:/root/.config/llm-mux nghyane/llm-mux

🔐 Authentication

Authenticate once, use forever. Tokens are auto-refreshed.

# Most Popular
llm-mux --login          # Google (Gemini Pro/Flash)
llm-mux --claude-login   # Anthropic (Claude 3.5 Sonnet/Opus)
llm-mux --copilot-login  # GitHub (GPT-4o, GPT-4.1)

# Others
llm-mux --codex-login    # OpenAI Codex
llm-mux --qwen-login     # Qwen / Alibaba

🚀 Usage

llm-mux runs on http://localhost:8318. Point any AI client (Cursor, VS Code, Python, LangChain) to this URL.

Example: Using OpenAI Format (Universal)

You can call any model using the standard OpenAI format:

curl http://localhost:8318/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-2.5-flash",
    "messages": [{"role": "user", "content": "Explain quantum computing in one sentence."}]
  }'
Supported API Endpoints
Standard Endpoint URL Use Case
OpenAI /v1/chat/completions Most apps, LangChain, AutoGen
Anthropic /v1/messages Cursor, Claude Dev, Aider
Gemini /v1beta/models/... Google ecosystem tools
Ollama /api/chat Local-first tools

Tip: List all available models with curl http://localhost:8318/v1/models

🛠️ Advanced & Services

🖥️ Service Management (Start/Stop)
Action macOS (launchctl) Linux (systemd) Windows (Task Scheduler)
Start launchctl start com.llm-mux systemctl --user start llm-mux Start-ScheduledTask "Start llm-mux"
Stop launchctl stop com.llm-mux systemctl --user stop llm-mux Stop-ScheduledTask "Start llm-mux"
Logs ~/.local/var/log/llm-mux.log journalctl --user -u llm-mux Windows Event Viewer
☁️ Sync Config with GitStore (Pro)

Sync your tokens and config across multiple machines using a private Git repo.

  1. Create a private empty repo on GitHub.
  2. Set environment variables (in .bashrc or Windows Env): 
    export GITSTORE_GIT_URL=https://github.com/username/my-mux-store.git
export GITSTORE_GIT_USERNAME=your_user
export GITSTORE_GIT_TOKEN=your_pat_token
  3. Restart llm-mux. It will auto-sync!
🏗️ Architecture
  • Intermediate Representation (IR): Translates requests 2N times instead of N² times.
  • Provider Adapters: Handles specific OAuth flows and token rotation.
  • Load Balancer: Smart routing based on quota availability.

License

MIT License - see LICENSE

Directories

Path Synopsis
cmd
server command
Package main provides the entry point for the CLI Proxy API server.
 Package main provides the entry point for the CLI Proxy API server.
examples
custom-provider command
Package main demonstrates how to create a custom AI provider executor and integrate it with the CLI Proxy API server.
 Package main demonstrates how to create a custom AI provider executor and integrate it with the CLI Proxy API server.
internal
access
access/config_access
api
Package api provides the HTTP API server implementation for the CLI Proxy API.
 Package api provides the HTTP API server implementation for the CLI Proxy API.
api/handlers/management
Package management provides the management API handlers and middleware for configuring the server and managing auth files.
 Package management provides the management API handlers and middleware for configuring the server and managing auth files.
api/middleware
Package middleware provides HTTP middleware components for the CLI Proxy API server.
 Package middleware provides HTTP middleware components for the CLI Proxy API server.
api/modules
Package modules provides a pluggable routing module system for extending the API server with optional features without modifying core routing logic.
 Package modules provides a pluggable routing module system for extending the API server with optional features without modifying core routing logic.
api/modules/amp
Package amp implements the Amp CLI routing module, providing OAuth-based integration with Amp CLI for ChatGPT and Anthropic subscriptions.
 Package amp implements the Amp CLI routing module, providing OAuth-based integration with Amp CLI for ChatGPT and Anthropic subscriptions.
auth
Package auth provides authentication functionality for various AI service providers.
 Package auth provides authentication functionality for various AI service providers.
auth/claude
Package claude provides OAuth2 authentication functionality for Anthropic's Claude API.
 Package claude provides OAuth2 authentication functionality for Anthropic's Claude API.
auth/cline
auth/codex
Package codex provides authentication and token management for OpenAI's Codex API.
 Package codex provides authentication and token management for OpenAI's Codex API.
auth/copilot
Package copilot provides authentication for GitHub Copilot API.
 Package copilot provides authentication for GitHub Copilot API.
auth/gemini
Package gemini provides authentication and token management functionality for Google's Gemini AI services.
 Package gemini provides authentication and token management functionality for Google's Gemini AI services.
auth/iflow
auth/kiro
auth/qwen
Package qwen provides authentication and token management functionality for Alibaba's Qwen AI services.
 Package qwen provides authentication and token management functionality for Alibaba's Qwen AI services.
auth/vertex
Package vertex provides token storage for Google Vertex AI Gemini via service account credentials.
 Package vertex provides token storage for Google Vertex AI Gemini via service account credentials.
browser
Package browser provides cross-platform functionality for opening URLs in the default web browser.
 Package browser provides cross-platform functionality for opening URLs in the default web browser.
buildinfo
Package buildinfo exposes compile-time metadata shared across the server.
 Package buildinfo exposes compile-time metadata shared across the server.
cmd
Package cmd provides command-line interface functionality for the CLI Proxy API server.
 Package cmd provides command-line interface functionality for the CLI Proxy API server.
config
Package config provides configuration management for the CLI Proxy API server.
 Package config provides configuration management for the CLI Proxy API server.
constant
Package constant defines provider name constants used throughout the CLI Proxy API.
 Package constant defines provider name constants used throughout the CLI Proxy API.
embedded
interfaces
Package interfaces defines the core interfaces and shared structures for the CLI Proxy API server.
 Package interfaces defines the core interfaces and shared structures for the CLI Proxy API server.
logging
Package logging provides Gin middleware for HTTP request logging and panic recovery.
 Package logging provides Gin middleware for HTTP request logging and panic recovery.
managementasset
misc
Package misc provides miscellaneous utility functions and embedded data for the CLI Proxy API.
 Package misc provides miscellaneous utility functions and embedded data for the CLI Proxy API.
oauth
Package oauth provides OAuth HTML response generation.
 Package oauth provides OAuth HTML response generation.
registry
Package registry provides model definitions for various AI service providers.
 Package registry provides model definitions for various AI service providers.
runtime/executor
Package executor provides runtime execution capabilities for various AI service providers.
 Package executor provides runtime execution capabilities for various AI service providers.
runtime/geminicli
store
translator/from_ir
Package from_ir converts unified request format to provider-specific formats.
 Package from_ir converts unified request format to provider-specific formats.
translator/ir
Package ir provides memory pools for the translator layer.
 Package ir provides memory pools for the translator layer.
translator/to_ir
Package to_ir converts provider-specific API formats into unified format.
 Package to_ir converts provider-specific API formats into unified format.
usage
Package usage provides usage tracking and logging functionality for the CLI Proxy API server.
 Package usage provides usage tracking and logging functionality for the CLI Proxy API server.
util
Package util provides utility functions used across the llm-mux application.
 Package util provides utility functions used across the llm-mux application.
watcher
Package watcher provides file system monitoring functionality for the CLI Proxy API.
 Package watcher provides file system monitoring functionality for the CLI Proxy API.
wsrelay
sdk
access
api/handlers
api/handlers/claude
api/handlers/gemini
api/handlers/ollama
api/handlers/openai
auth
cliproxy
Package cliproxy provides the core service implementation for the CLI Proxy API.
 Package cliproxy provides the core service implementation for the CLI Proxy API.
cliproxy/auth
cliproxy/executor
cliproxy/pipeline
cliproxy/usage
config
Package config provides configuration management for the CLI Proxy API server.
 Package config provides configuration management for the CLI Proxy API server.
translator
Package translator provides format definitions for request/response translation.
 Package translator provides format definitions for request/response translation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.