millivolt

README

millivolt

See every AI request. Keep your own routing.

A self-hosted, cross-protocol inference proxy with a live dashboard, SQLite history, and controls to pause, limit, or debug traffic. OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages clients can target any configured upstream. Request metadata, timing, tokens, cost, and outcomes; body capture is opt-in.

Run it locally · Documentation site · Dashboard guide · Documentation · Security

millivolt is pre-1.0. Linux is the supported deployment target. The 0.x series can change configuration and API details between releases.

Quickstart

Linux with Docker Engine and the Compose plugin. No Go, Node, or source checkout is needed for the container path.

Docker Compose

curl -fsSL https://raw.githubusercontent.com/LLM-4-People/millivolt/main/compose.yaml -o compose.yaml
export MILLIVOLT_OPERATOR_TOKEN='replace-with-a-long-random-value'
docker compose up -d

Open http://127.0.0.1:8080/ and sign in with that token. If the container exits, run docker compose logs --tail 50.

[!IMPORTANT] One shared operator credential protects the dashboard and operator plane only. Inference is transparent: it forwards your upstream key and does not use the operator token. There are no user accounts. Keep the listener on loopback, or put the network or ingress boundary in front. With the token unset, the dashboard denies every request. Read Security before network exposure.

compose.yaml publishes only on host loopback. Set MILLIVOLT_PORT for an unused host port. Named volumes retain history and private Settings configuration; existing saved settings are not overwritten. The runtime is non-root with a read-only root filesystem. For bind mounts, pre-create directories writable by UID/GID 65532:65532; see persistent state.

Stop with docker compose stop. For image selection, configuration, backups, updates, shutdown, or protected ingress, see container operation and the reverse-proxy guide.

From source

Install the Go toolchain required by go.mod, then:

git clone https://github.com/LLM-4-People/millivolt.git
cd millivolt
test -e proxy.yaml || cp proxy.example.yaml proxy.yaml
export MILLIVOLT_OPERATOR_TOKEN='replace-with-a-long-random-value'
go run ./cmd/proxy -config proxy.yaml -listen 127.0.0.1:8080

For isolated development and checks, see Contributing.

Connect a client

Point an OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages client at the proxy. The upstream URL and key stay yours; the proxy does not register providers or keys. The request path selects the client protocol, and X-Proxy-Format selects the upstream protocol.

• API base URL: http://127.0.0.1:8080/v1
• API key: your upstream key. It is forwarded through Authorization; it does not authenticate dashboard access.
• X-Proxy-Base-URL: your upstream API base URL, including its path, such as /v1.
• X-Proxy-Client: optional label for this client in the dashboard.

curl -sS http://127.0.0.1:8080/v1/chat/completions \
  -H "X-Proxy-Base-URL: $UPSTREAM_BASE_URL" \
  -H "Authorization: Bearer $UPSTREAM_API_KEY" \
  -H 'Content-Type: application/json' \
  -d "{\"model\":\"$UPSTREAM_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}"

A JSON response means the relay worked. The request should appear in the dashboard within a second or two. On a shared Compose network, use http://millivolt:8080/v1 instead of loopback. /v1/chat/completions, /v1/responses and /v1/messages are all available; any of them can target a different upstream protocol, and a Chat client can also target the Cursor bridge. Read upstream compatibility and cross-protocol translation.

Normal relay paths preserve request and response content, with documented exceptions for buffering, retries, SSE keepalives, format translation, and configured overrides. This is not an unconditional byte-for-byte or exactly-once contract. See protocol.

What it looks like

The instrument panel is a dense operator surface in either Black or Clear, not a marketing dashboard. The product screenshots below are static captures from an authorized redacted history snapshot; identifiers are replaced.

Dashboard overview

Global totals stay global; explorer cards, timelines, and the request log share one selected scope. See the visual guide for focused captures of tokens, speed and latency, cost, providers, models, conversations, and Settings.

This capture retains recorded metrics from an authorized history snapshot with replaced identifiers. The offline indicator reflects the capture session, not service health. The guide explains the capture limitations.

What you get

See the request, not just the total.

Watch in-flight requests next to global request, token, and cost totals. Open a row for TTFT, throughput, retries, queue wait, token mix, and provider-reported cost. History stays in SQLite; an in-memory ring is available when you do not need retention. Scope by client, provider, model, conversation, tool, status, error, time, or key without losing the selected filter.

Data interpretation explains timing, scope, and missing measurements.

Act before the retry storm.

Pause new sends and retries by client, provider, or all traffic. Set provider-wide concurrency, request-rate, and token-rate limits, plus an opt-in anti-burst ramp that eases a new provider+key group up to the concurrency target. Capture scoped debug bodies, export exactly the records you filtered, and turn on error-storm protection with recovery probes. These are operator actions, not per-user permissions.

Read operator workflows before changing state. The shared operator token protects the dashboard and operator routes. Inference is transparent and does not use it.

Keep your routing and your keys.

Choose the upstream base URL, credentials, authentication header or prefix, path, and query on each request. Constrain accepted URL prefixes in configuration. Optional adapters translate between OpenAI Chat Completions, OpenAI Responses and Anthropic Messages through one canonical model; the Cursor Connect bridge is a separate direct path that requires a Chat client and is not an IR-translated protocol. Coverage and limits are documented, not implied. There is no separate provider registry or API-key vault in the proxy.

See routing, timing and discovery and adapter support and limitations.

Run one small service.

One Go service with embedded dashboard assets and SQLite. The non-root image needs no separate database server, frontend runtime, or monitoring stack. The same 22-tool MCP server is at /mcp behind the operator token, so an LLM client needs only the endpoint URL and its Bearer credential. See the MCP server guide.

Operations owns configuration, persistence, exports, and upgrades; architecture maps the implementation.

Limits and scope

millivolt is built for a single operator or small team that controls the host. It is not a multi-tenant gateway, a billing system, or a compliance audit log.

• No multi-tenant isolation. One operator credential protects the dashboard and operator plane; inference is transparent and depends on the network or ingress boundary. There are no per-user accounts.
• Durable accounting is best-effort. Sustained overload can drop records even when inference succeeds. Backups, capacity, and storage drops are in storage and accounting.
• The relay is not a byte-for-byte or exactly-once contract. Normal paths preserve content, with documented exceptions. See protocol.
• Observability, not billing or compliance. Cost is provider-reported; this is not an independent invoice or a complete audit log.
• Linux is the supported deployment target. See containers for image and volume details.

Performance and footprint

The performance and resource usage page is the single source for current image and binary size, idle and loaded footprints, fast-response and streaming request throughput and latency, durable writer rates, SQLite growth, projection cost, and query/metadata latency. It carries the exact revision, host, method, reproduce commands and limitations. Small images do not imply fixed RAM or negligible CPU. Active streams, configured buffers, stored-history projections, and identity cardinality contribute separate costs. Short local bursts are regressions, not deployment guarantees.

Contributing and promotion

Work lands on testing first. The full check suite and both native container smoke tests run on all testing pull requests, code-bearing testing pushes, manually dispatched testing gates, and manually pushed v* tags; docs and site-only testing pushes use docs.yml instead. They also run on Dependabot's main security PRs. An ordinary main promotion proves the head is the current testing tip, main is already merged back into testing, and that exact commit passed the full gate, or that only docs paths changed since the closest gated testing commit (the proof has no commit-count bound). For a code-bearing head, let the testing check.yml push gate finish before opening or re-running a promotion PR; pending runs get a short bounded wait and the proof fails closed while they remain pending. A docs/site-only head has no canonical check.yml push run of its own, so wait for docs.yml or only the required docs validation. required does not repeat the suite. main is promotion-only and protected: open a pull request from testing, wait for required, and merge with a merge commit. A code-bearing promotion bumps VERSION above every release tag; a docs/site-only promotion skips that check, validates docs, workflow policy and the site, and does not rebuild images. For a code-bearing promotion, the merged-PR release run creates the release tag and image; a delivered main push is an idempotent fallback, and a manually pushed v* tag still ships exactly as tagged. Never push main or a release tag as part of feature work. See the branch workflow for the full checklist and versions and images for publication behavior.

Configuration and documentation

proxy.example.yaml is the generated, documented example. It includes enabled compatibility profiles without registering provider keys or upstream URLs. proxy.yaml is your ignored local configuration; do not commit it. Defaults, example profiles, and validation are defined in internal/config; Settings uses that same schema. See operations for overrides, reloads, backups, and restart requirements.

VERSION owns the application version. See image and release usage and contribution guidelines before publishing a change.

• Start: documentation index, protocol
• Operate: operations, performance, reverse proxy
• Integrate: adapters, MCP server
• Project: architecture, contributing, security

The application is MIT licensed. Dependency notices are listed in THIRD_PARTY_NOTICES.md.

Documentation

Overview

Package millivolt exposes the release identity shared by every build path.

Index

• func CompareVersions(a, b string) (int, error)
• func ValidateRevision(revision string) error
• func ValidateVersion(version string) error
• func Version() string
• type BuildInfo
  • func CurrentBuild() (BuildInfo, error)
• type ReleaseInfo
  • func ReleaseMetadata(tag, revision string) (ReleaseInfo, error)

Functions

func CompareVersions

func CompareVersions(a, b string) (int, error)

CompareVersions compares two validated release versions using SemVer 2.0.0 precedence. It returns -1 when a sorts before b, 0 when they are equal, and 1 when a sorts after b. Build metadata is not accepted.

func ValidateRevision

func ValidateRevision(revision string) error

ValidateRevision accepts a full lowercase Git SHA-1 or SHA-256 object ID.

func ValidateVersion

func ValidateVersion(version string) error

ValidateVersion enforces the release naming policy, including numeric prerelease identifiers without leading zeroes. It accepts no whitespace, leading v, or build metadata suffix.

func Version

func Version() string

Version returns the embedded application version, without the file newline.

Types

type BuildInfo

type BuildInfo struct {
	Version   string `json:"version"`
	Revision  string `json:"revision"`
	Modified  *bool  `json:"modified"`
	GoVersion string `json:"go_version"`
}

BuildInfo identifies the running binary. A nil Modified means the source tree's cleanliness is unknown, not clean. Revision is "unknown" when absent.

func CurrentBuild

func CurrentBuild() (BuildInfo, error)

CurrentBuild reads the standard Go build metadata. No subprocess, filesystem scan, clock stamp, configuration load, or network access is needed.

type ReleaseInfo

type ReleaseInfo struct {
	Version    string
	Revision   string
	Prerelease bool
}

ReleaseInfo is validated metadata for the container/release workflow. Tag is optional for default-branch builds; when present it must match VERSION exactly.

func ReleaseMetadata

func ReleaseMetadata(tag, revision string) (ReleaseInfo, error)