gomodel

module
v0.1.56 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 21, 2026 License: MIT

README

GoModel logo

GoModel - AI Gateway in Go

CI GO Version Docker Pulls Discord

Hacker News docs GoModel

GoModel on Hacker News

A fast and lightweight AI gateway written in Go, providing unified OpenAI-compatible and Anthropic-compatible APIs for OpenAI, Anthropic, Gemini, DeepSeek, xAI, Groq, OpenRouter, Z.ai, Azure OpenAI, Oracle, Ollama, and more.

GoModel AI gateway dashboard showing AI usage analytics, observability panel, token and costs tracking, and estimated cost monitoring

Quick Start

Step 1: Install and start GoModel

macOS / Linux

curl -fsSL https://gomodel.enterpilot.io/install.sh | sh
OPENAI_API_KEY="your-openai-key" gomodel

Windows (PowerShell)

irm https://gomodel.enterpilot.io/install.ps1 | iex
$env:OPENAI_API_KEY = "your-openai-key"; gomodel

Docker

docker run --rm -p 8080:8080 \
  -e LOG_FORMAT=text \
  -e OPENAI_API_KEY="your-openai-key" \
  enterpilot/gomodel

Full list of environment variables (including all available providers): .env.template

⚠️ Avoid passing secrets with -e on the command line in production — they can leak through shell history and process lists. Use docker run --env-file .env to load API keys from a file instead.

Step 2: Make your first API call

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5-chat-latest",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

That's it! GoModel automatically detects which providers are available based on the credentials you supply.

Prefer not to set API keys as env vars? Start GoModel with none configured — gomodel with no provider env vars still boots — then add provider credentials from the Providers page in the admin dashboard (/admin/dashboard). They apply immediately, no restart.

Supported LLM Providers

GoModel supports OpenAI, Anthropic, Google Gemini, Vertex AI, DeepSeek, Groq, Fireworks AI, Meta (Muse Spark), OpenRouter, Z.ai, xAI (Grok), Alibaba Cloud Model Studio (Bailian), Kilo AI, MiniMax, Xiaomi MiMo, OpenCode Go, Azure OpenAI, Oracle, Ollama, vLLM, Amazon Bedrock Runtime, Amazon Bedrock Mantle, and all OpenAI-compatible providers.

See the Providers Overview for the full per-provider feature matrix (chat, /responses, embeddings, files, batches, passthrough), credentials, and configuration notes.


Alternative Setup Methods

Running from Source

Prerequisites: Go 1.26.4+

  1. Create a .env file:

    cp .env.template .env
    
  2. Add your API keys to .env (optional — you can instead add them later from the admin dashboard's Providers page).

  3. Start the server:

    make run
    

Docker Compose

Infrastructure only (Redis, PostgreSQL, MongoDB, Adminer - no image build):

docker compose up -d
# or: make infra

Full stack (adds GoModel + Prometheus; builds the app image):

cp .env.template .env
# Add your API keys to .env
docker compose --profile app up -d
# or: make image
Service URL
GoModel API http://localhost:8080
Adminer (DB UI) http://localhost:8081
Prometheus http://localhost:9090

Building the Docker Image Locally

docker build -t gomodel .
docker run --rm -p 8080:8080 --env-file .env gomodel

API Endpoints

GoModel exposes OpenAI-compatible and Anthropic-compatible APIs, provider-native passthrough, and operations routes. See the API Endpoints reference for the full endpoint tables, and Admin Endpoints for the admin REST API and dashboard.


Gateway Configuration

GoModel is configured through environment variables and an optional config.yaml. Environment variables override YAML values. See the Configuration reference for the full list of settings organized by category, along with .env.template and config/config.example.yaml.

Quick Start - Authentication: By default GOMODEL_MASTER_KEY is unset. Without this key, API endpoints are unprotected and anyone can call them. This is insecure for production. Strongly recommend setting a strong secret before exposing the service. Add GOMODEL_MASTER_KEY to your .env or environment for production deployments.


See DEVELOPMENT.md for testing, linting, and pre-commit setup.


Roadmap

See the Roadmap for commercial features and the public 0.2.0 milestone.

Community

Join our Discord to connect with other GoModel users.

Directories

Path Synopsis
cmd
gomodel command
Package main is the entry point for the LLM gateway server.
Package main is the entry point for the LLM gateway server.
recordapi command
Package main provides a CLI tool to record real API responses for contract tests.
Package main provides a CLI tool to record real API responses for contract tests.
Package config provides configuration management for the application.
Package config provides configuration management for the application.
docs
2026-03-23_benchmark_scripts/gateway-comparison/mock-backend command
Mock OpenAI-compatible backend server for benchmarking AI gateways.
Mock OpenAI-compatible backend server for benchmarking AI gateways.
2026-03-23_benchmark_scripts/gateway-comparison/stream-bench command
Streaming SSE benchmark tool.
Streaming SSE benchmark tool.
Package ext is the public extension API for building custom gateway binaries on top of GoModel.
Package ext is the public extension API for building custom gateway binaries on top of GoModel.
internal
admin
Package admin provides the admin REST API and dashboard for GoModel.
Package admin provides the admin REST API and dashboard for GoModel.
admin/dashboard
Package dashboard provides the embedded admin dashboard UI for GoModel.
Package dashboard provides the embedded admin dashboard UI for GoModel.
anthropicapi
Package anthropicapi translates the Anthropic Messages API dialect to and from GoModel's canonical chat types.
Package anthropicapi translates the Anthropic Messages API dialect to and from GoModel's canonical chat types.
app
Package app provides the main application struct for centralized dependency management and lifecycle control of the GoModel server.
Package app provides the main application struct for centralized dependency management and lifecycle control of the GoModel server.
auditlog
Package auditlog provides audit logging for the AI gateway.
Package auditlog provides audit logging for the AI gateway.
batch
Package batch provides persistence for OpenAI-compatible batch lifecycle endpoints.
Package batch provides persistence for OpenAI-compatible batch lifecycle endpoints.
cache
Package cache provides a generic key-value store abstraction.
Package cache provides a generic key-value store abstraction.
cache/modelcache
Package modelcache provides model-specific cache types and interfaces.
Package modelcache provides model-specific cache types and interfaces.
conversationstore
Package conversationstore provides persistence for the OpenAI-compatible Conversations lifecycle endpoints.
Package conversationstore provides persistence for the OpenAI-compatible Conversations lifecycle endpoints.
core
Package core provides core types and interfaces for the LLM gateway.
Package core provides core types and interfaces for the LLM gateway.
failover
Package failover stores operator-managed manual failover mappings.
Package failover stores operator-managed manual failover mappings.
filestore
Package filestore persists provider ownership for uploaded files.
Package filestore persists provider ownership for uploaded files.
gateway
Package gateway contains transport-independent gateway use cases.
Package gateway contains transport-independent gateway use cases.
guardrails
Package guardrails provides a pluggable pipeline for request-level guardrails.
Package guardrails provides a pluggable pipeline for request-level guardrails.
httpclient
Package httpclient provides a centralized HTTP client factory with unified configuration.
Package httpclient provides a centralized HTTP client factory with unified configuration.
live
Package live provides in-process realtime dashboard event fan-out.
Package live provides in-process realtime dashboard event fan-out.
llmclient
Package llmclient provides a base HTTP client for LLM providers with: - Request marshaling/unmarshaling - Retries with exponential backoff and jitter - Standardized error parsing (429, 502, 503, 504) - Circuit breaking with half-open state protection
Package llmclient provides a base HTTP client for LLM providers with: - Request marshaling/unmarshaling - Retries with exponential backoff and jitter - Standardized error parsing (429, 502, 503, 504) - Circuit breaking with half-open state protection
mcpgateway
Package mcpgateway aggregates upstream MCP servers behind GoModel's authenticated /mcp endpoints.
Package mcpgateway aggregates upstream MCP servers behind GoModel's authenticated /mcp endpoints.
modeldata
Package modeldata provides fetching, parsing, and merging of the external AI model metadata registry (models.json) for enriching GoModel's model data.
Package modeldata provides fetching, parsing, and merging of the external AI model metadata registry (models.json) for enriching GoModel's model data.
modelselectors
Package modelselectors normalizes provider/model selector strings shared by model-level admin features.
Package modelselectors normalizes provider/model selector strings shared by model-level admin features.
observability
Package observability provides instrumentation for metrics, tracing, and logging.
Package observability provides instrumentation for metrics, tracing, and logging.
platformdir
Package platformdir resolves the OS-conventional per-user directories for GoModel's durable data and caches, used when no explicit path is configured.
Package platformdir resolves the OS-conventional per-user directories for GoModel's durable data and caches, used when no explicit path is configured.
providers
Package providers provides a factory for creating provider instances.
Package providers provides a factory for creating provider instances.
providers/anthropic
Package anthropic provides Anthropic API integration for the LLM gateway.
Package anthropic provides Anthropic API integration for the LLM gateway.
providers/bailian
Package bailian provides the Alibaba Cloud Bailian (百炼 / DashScope) provider.
Package bailian provides the Alibaba Cloud Bailian (百炼 / DashScope) provider.
providers/bedrock
Package bedrock provides Amazon Bedrock integration for the LLM gateway.
Package bedrock provides Amazon Bedrock integration for the LLM gateway.
providers/bedrockmantle
Package bedrockmantle provides direct access to Amazon Bedrock's OpenAI-compatible Mantle API.
Package bedrockmantle provides direct access to Amazon Bedrock's OpenAI-compatible Mantle API.
providers/deepseek
Package deepseek provides DeepSeek API integration for the LLM gateway.
Package deepseek provides DeepSeek API integration for the LLM gateway.
providers/fireworks
Package fireworks provides Fireworks AI API integration for the LLM gateway.
Package fireworks provides Fireworks AI API integration for the LLM gateway.
providers/gemini
Package gemini provides Google Gemini API integration for the LLM gateway.
Package gemini provides Google Gemini API integration for the LLM gateway.
providers/googlecommon
Package googlecommon holds infrastructure shared by GoModel's Google-backed providers (Gemini AI Studio + Vertex AI).
Package googlecommon holds infrastructure shared by GoModel's Google-backed providers (Gemini AI Studio + Vertex AI).
providers/groq
Package groq provides Groq API integration for the LLM gateway.
Package groq provides Groq API integration for the LLM gateway.
providers/health
Package health tracks recent request outcomes per provider and model.
Package health tracks recent request outcomes per provider and model.
providers/kilo
Package kilo provides Kilo AI Gateway integration for the LLM gateway.
Package kilo provides Kilo AI Gateway integration for the LLM gateway.
providers/kimicode
Package kimicode provides Kimi Code API integration for the LLM gateway.
Package kimicode provides Kimi Code API integration for the LLM gateway.
providers/meta
Package meta provides Meta Model API integration for the LLM gateway.
Package meta provides Meta Model API integration for the LLM gateway.
providers/minimax
Package minimax provides MiniMax API integration for the LLM gateway.
Package minimax provides MiniMax API integration for the LLM gateway.
providers/ollama
Package ollama provides Ollama API integration for the LLM gateway.
Package ollama provides Ollama API integration for the LLM gateway.
providers/openai
Package openai provides OpenAI API integration for the LLM gateway.
Package openai provides OpenAI API integration for the LLM gateway.
providers/opencodego
Package opencodego provides OpenCode Zen (Go subscription) integration for the LLM gateway.
Package opencodego provides OpenCode Zen (Go subscription) integration for the LLM gateway.
providers/vertex
Package vertex provides Google Vertex AI Gemini integration.
Package vertex provides Google Vertex AI Gemini integration.
providers/vllm
Package vllm provides vLLM OpenAI-compatible API integration for the LLM gateway.
Package vllm provides vLLM OpenAI-compatible API integration for the LLM gateway.
providers/xai
Package xai provides xAI (Grok) API integration for the LLM gateway.
Package xai provides xAI (Grok) API integration for the LLM gateway.
providers/xiaomi
Package xiaomi provides Xiaomi MiMo API integration for the LLM gateway.
Package xiaomi provides Xiaomi MiMo API integration for the LLM gateway.
providers/zai
Package zai provides Z.ai API integration for the LLM gateway.
Package zai provides Z.ai API integration for the LLM gateway.
ratelimit
Package ratelimit enforces request, token, and concurrency limits for the AI gateway.
Package ratelimit enforces request, token, and concurrency limits for the AI gateway.
realtime
Package realtime implements a transparent, provider-agnostic websocket reverse proxy for OpenAI-compatible realtime (speech-to-speech) sessions.
Package realtime implements a transparent, provider-agnostic websocket reverse proxy for OpenAI-compatible realtime (speech-to-speech) sessions.
responsestore
Package responsestore provides persistence for OpenAI-compatible Responses lifecycle endpoints.
Package responsestore provides persistence for OpenAI-compatible Responses lifecycle endpoints.
server
Package server provides HTTP handlers and server setup for the LLM gateway.
Package server provides HTTP handlers and server setup for the LLM gateway.
storage
Package storage provides shared database connections for all features.
Package storage provides shared database connections for all features.
storage/sqlutil
Package sqlutil provides small SQL query-building and value-conversion helpers shared by the SQL-backed stores and readers.
Package sqlutil provides small SQL query-building and value-conversion helpers shared by the SQL-backed stores and readers.
tagging
Package tagging labels requests based on configurable HTTP headers.
Package tagging labels requests based on configurable HTTP headers.
usage
Package usage provides token usage tracking for the AI gateway.
Package usage provides token usage tracking for the AI gateway.
validation
Package validation provides the shared validation error type used by admin-managed resources (aliases, auth keys, guardrails, workflows, model selectors and overrides).
Package validation provides the shared validation error type used by admin-managed resources (aliases, auth keys, guardrails, workflows, model selectors and overrides).
virtualmodels
Package virtualmodels unifies model aliases (redirects) and model access overrides (policies) behind one entity, the virtual model, persisted in a single virtual_models table.
Package virtualmodels unifies model aliases (redirects) and model access overrides (policies) behind one entity, the virtual model, persisted in a single virtual_models table.
Package run exposes the complete GoModel gateway lifecycle as an importable entry point.
Package run exposes the complete GoModel gateway lifecycle as an importable entry point.
tests
contract
Package contract provides contract tests that validate API response structures against recorded golden files.
Package contract provides contract tests that validate API response structures against recorded golden files.
e2e/mockmcp command
Command mockmcp serves two deterministic MCP upstream servers ("alpha" and "beta") over streamable HTTP for the release E2E curl matrix.
Command mockmcp serves two deterministic MCP upstream servers ("alpha" and "beta") over streamable HTTP for the release E2E curl matrix.
integration
Package integration provides integration tests that verify database state after requests.
Package integration provides integration tests that verify database state after requests.
integration/dbassert
Package dbassert provides database assertion helpers for integration tests.
Package dbassert provides database assertion helpers for integration tests.
Package tools pins developer tooling dependencies used by this repository.
Package tools pins developer tooling dependencies used by this repository.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL