tokenless

README

Tokenless

Tokenless is a deterministic mock LLM gateway for testing agents without API calls, token costs, or unpredictable model responses. Define predictable conversations in YAML and run them through a mock server that speaks the OpenAI-compatible API and Anthropic's Messages API - your agent runs its real tools, real approval flows, and real output pipeline against a "model" that always says exactly what your test expects.

• Key Features
• How It Works
• Supported Endpoints
• Usage
• Scenario Format
• Packages
• Alternatives
• Contributing
• License

Key Features

• 📜 Open Source: Available under the Apache 2.0 License.
• 🎯 Deterministic: Scenarios are selected by regex and served by turn position - no hidden session state, no cleverness.
• 🌊 Real Streaming: OpenAI SSE chunks and Anthropic-native event streams, including multi-fragment tool-call arguments.
• 🔧 Tool-Call Scripting: Script tool calls turn by turn and exercise your agent's real execution and approval paths.
• 💥 Failure Injection: HTTP errors, stalled streams, and malformed SSE frames for the retry paths a real provider never tests.
• 🧰 Three Consumption Modes: Go import, standalone binary, or an app-level mock switch - any language, any harness.
• 💸 Zero Cost: A scenario costs the same to run one time or one million times: nothing.

How It Works

It is not a stub that returns "Hello, world". It is a scenario engine: a scenario is selected by regex against the first user message, and each turn is served by position - derived from how many assistant messages are already in the request.

# Optional: custom model list for GET /v1/models (omit for built-in list)
models:
  - id: my-custom-model
    object: model
    owned_by: me

fallback:
  content: "Done."

scenarios:
  - name: write-approved
    match: '(?i)create a file named approved\.txt'
    turns:
      - tool_calls:
          - { name: Write, args: { file_path: "approved.txt", content: "hi" } }
      - content: "The file was written."

  - name: model-specific
    match: '(?i)model specific'
    model: gpt-4o-mini  # optional: only match requests for this model
    turns:
      - content: "This is a model-specific response for gpt-4o-mini."

Supported Endpoints

Endpoint	Notes
POST /v1/chat/completions	Sync JSON and SSE streaming; text, reasoning deltas, multi-fragment tool-call chunks, usage frame, [DONE]
POST /v1/messages	Anthropic-native: thinking/text/tool_use blocks, input_json_delta fragments, cache-aware usage
POST /v1/images/generations, POST /v1/images/edits	Canned 1x1 PNG so decode-and-save paths run end to end
GET /v1/models	Model list with pricing and context-window metadata
GET /v1/expect	JSON report of recorded expectation failures; 200 when clean, 412 otherwise
GET /v1/health	{"status":"ok"}

Usage

1. Go import - embed the mock in your tests:

import "github.com/inference-gateway/tokenless"

func TestSomething(t *testing.T) {
    mock := tokenless.StartMock(t)
    orc := tokenless.Orchestrator{Bin: binPath, Env: map[string]string{"MYAPP_GATEWAY_URL": mock.URL}}

    res := orc.Run(t, "agent", "say hello")
    require.Zero(t, res.ExitCode)
    mock.AssertExpectations(t)
    ...
}

2. Standalone binary - for any agent in any language; point its base URL at the mock:

go run github.com/inference-gateway/tokenless/cmd/tokenless@latest \
  --port 8080 --scenarios scenarios.yaml
# or: TOKENLESS_SCENARIOS=scenarios.yaml tokenless --port 8080

3. Via your app's own mock switch - an app can embed tokenless and expose it behind an environment variable, so anything that spawns the app gets the mock without writing a line of Go. The infer CLI does exactly this:

INFER_GATEWAY_MOCK=true \
INFER_GATEWAY_MOCK_SCENARIOS=$PWD/e2e/scenarios.yaml \
infer agent --require-approval -m openai/gpt-4o "create a file named approved.txt"

That is how the inference-gateway Desktop app runs its UI tests: its repo ships its own scenarios.yaml, and the spawned infer children inherit both variables. Yesterday this tested a CLI, today a desktop app, tomorrow your thing.

Already have an app built on an official client SDK? Nothing changes in your code - construct the client with its base URL pointed at the mock. See examples for runnable programs using the official OpenAI Go client (sync and streaming) and the inference-gateway Go SDK.

Testing a CLI end to end? examples/00-cobra-agent is a complete worked example: a small cobra CLI whose gateway URL comes from an environment variable, and a test that builds the real binary once (tokenless.BuildBinary in TestMain), starts the mock (tokenless.StartMock), runs the binary as a subprocess (tokenless.Orchestrator{...}.Run), and asserts on its stdout and exit code - including a custom inline scenario and a failure path. The examples live in their own Go module, so none of their dependencies touch the library.

Scenario Format

See examples/scenarios.yaml for a commented example and gateway/scenarios.yaml for the embedded default library. When a scenario runs out of turns the top-level fallback is served, which lets a headless agent terminate naturally. Runnable Go examples live in gateway/example_test.go.

Expect blocks

Each turn can carry an expect block that validates the incoming request that resolved to that turn. A mismatch is recorded but the turn is still served as normal, keeping the conversation on script.

turns:
  - tool_calls:
      - { name: Write, args: { file_path: "hi.txt", content: "hi" } }
  - expect:
      model: gpt-4o
      endpoint: /v1/chat/completions
      tool_calls:
        - name: Write
          args: { file_path: "hi.txt" }
      messages:
        - role: tool
          content: "fixture content"
    content: "The file was written."

Supported expect fields:

Field	Matching rule
model	Exact string match against the request model
endpoint	Exact string match against the request path
messages	Ordered subsequence: each expected message must appear in order, with exact role and substring content; extra messages allowed
tool_calls	Ordered, deep partial match on args: expected keys must be present and equal, extra keys allowed

Failures are surfaced three ways:

• Mock.ExpectFailures() []ExpectFailure in Go
• Mock.AssertExpectations(t) with readable want/got diffs
• GET /v1/expect returns a JSON report (200 when clean, 412 otherwise)
• X-Tokenless-Expect-Failure header on the mismatched response

Packages

• tokenless (root) - Go test helpers: StartMock (returns *Mock with URL and AssertExpectations), Orchestrator/Orchestrator.Run (hermetic subprocess runs with caller-supplied env), BuildBinary (build once in TestMain), JSONLines/ContentsByRole/StatusOfType (NDJSON assertions), and tmux TUI drivers (SendKeys, CapturePane, WaitForPane).
• gateway - the HTTP server, scenario parser/validator (Load, LoadFile, Default), the hand-written wire types, and the embedded default scenario library. Zero dependencies beyond yaml.
• cmd/tokenless - the standalone binary.

Alternatives

If you want TypeScript-native record-and-replay across many providers, look at aimock; for OpenAI-only JMESPath rules in Node, mock-llm. Tokenless is the Go-embeddable, single-YAML, regex-scenario take on the same problem - built for hermetic go test runs and for harnesses that spawn agents as subprocesses.

Contributing

Contributions are welcome. Open an issue or a pull request on GitHub.

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

Documentation

Overview

Package tokenless provides reusable helpers for hermetic end-to-end tests of agent apps: building a binary once per test run, running it as a subprocess against an in-test mock gateway, and asserting on newline-delimited JSON output and recorded expectation failures.

Index

• func BuildBinary(srcDir, reuseEnvVar string) (string, func(), error)
• func CapturePane(session string) string
• func ContentsByRole(lines []map[string]any, role string) []string
• func JSONLines(t *testing.T, stdout string) []map[string]any
• func SendKeys(t *testing.T, session string, args ...string)
• func StatusOfType(lines []map[string]any, typ string) map[string]any
• func ToolMessages(body gateway.CreateChatCompletionRequest) []gateway.Message
• func WaitForPane(t *testing.T, session, want string, timeout time.Duration) bool
• func WriteFixtures(t *testing.T, dir string, names ...string)
• type Mock
  • func StartMock(t *testing.T, defs ...*gateway.ScenarioFile) *Mock
  • func (m *Mock) AssertExpectations(t TestingT)
• type Orchestrator
  • func (a Orchestrator) Run(t *testing.T, args ...string) Result
• type Result
• type TestingT
• type ToolFunc
• type ToolLoop
  • func (l *ToolLoop) LoadScenarioTools(defs *gateway.ScenarioFile)
  • func (l *ToolLoop) Run(t testing.TB, prompt string) *ToolLoopResult
• type ToolLoopResult

Functions

func BuildBinary

func BuildBinary(srcDir, reuseEnvVar string) (string, func(), error)

BuildBinary builds the Go main package at srcDir into a temp dir and returns the binary path plus a cleanup func. Intended for TestMain: build once, share the path across tests. Honors reuseEnvVar (e.g. "MYAPP_E2E_BINARY"): when that environment variable is set, its value is returned as-is and no build happens.

func CapturePane

func CapturePane(session string) string

CapturePane returns the last 200 lines of the tmux session's pane, or "" if it cannot be read.

func ContentsByRole

func ContentsByRole(lines []map[string]any, role string) []string

ContentsByRole returns the content of every line whose "role" matches.

func JSONLines

func JSONLines(t *testing.T, stdout string) []map[string]any

JSONLines parses every non-empty stdout line into a generic map; headless agents emit newline-delimited JSON only.

func SendKeys

func SendKeys(t *testing.T, session string, args ...string)

SendKeys sends one send-keys call to the session (literal text needs a leading "-l" arg; named keys like Enter are passed bare).

func StatusOfType

func StatusOfType(lines []map[string]any, typ string) map[string]any

StatusOfType returns the first line whose "type" matches, or nil.

func ToolMessages

func ToolMessages(body gateway.CreateChatCompletionRequest) []gateway.Message

ToolMessages returns the tool-role messages of a chat completion request.

func WaitForPane

func WaitForPane(t *testing.T, session, want string, timeout time.Duration) bool

WaitForPane polls the pane until it contains want or timeout elapses.

func WriteFixtures

func WriteFixtures(t *testing.T, dir string, names ...string)

WriteFixtures creates one small fixture file per name inside dir.

Types

type Mock

type Mock struct {
	*gateway.Server
	// URL is the base URL of the running mock server.
	URL string
}

Mock wraps the gateway server with test-friendly assertion methods.

func StartMock

func StartMock(t *testing.T, defs ...*gateway.ScenarioFile) *Mock

StartMock serves a scenario library on an httptest server tied to the test's lifetime and returns the Mock. With no arguments the built-in library is served.

func (*Mock) AssertExpectations

func (m *Mock) AssertExpectations(t TestingT)

AssertExpectations fails the test if any expectation mismatches were recorded, printing a readable want/got diff for each failure.

type Orchestrator

type Orchestrator struct {
	Bin     string
	Dir     string
	Env     map[string]string
	Stdin   string
	Timeout time.Duration
}

Orchestrator describes the binary under test. Zero values work: Dir defaults to a fresh temp dir, Timeout to 30s. Env entries (typically at least the app's gateway-URL variable pointed at the mock) are appended on top of a hermetic base whose HOME is a temp dir, so the user's real config never leaks in.

func (Orchestrator) Run

func (a Orchestrator) Run(t *testing.T, args ...string) Result

Run executes the orchestrator with the given arguments and waits for it to exit.

type Result

type Result struct {
	Stdout   string
	Stderr   string
	ExitCode int
}

Result is one finished Orchestrator run.

type TestingT

type TestingT interface {
	Errorf(format string, args ...interface{})
	Helper()
}

TestingT is the minimal interface for AssertExpectations, mirroring testify/mock's pattern so the library does not import "testing".

type ToolFunc

type ToolFunc func(ctx context.Context, args json.RawMessage) (string, error)

ToolFunc is a real tool implementation: given the tool call's JSON args, it returns the string result to feed back as a role:tool message.

type ToolLoop

type ToolLoop struct {
	BaseURL string
	Model   string
	Tools   map[string]ToolFunc

	// Context is the context passed to each ToolFunc invocation. If nil,
	// context.Background() is used. Set this to inject test helpers (e.g.
	// a *Mock for in-tool assertions) via context.WithValue.
	Context context.Context

	// Approve is an optional callback that decides whether a tool call
	// should be executed. It receives the tool call and returns true to
	// approve or false to reject. A rejected tool call returns a "denied"
	// result to the model and the loop continues. If nil, all tool calls
	// are approved (backward compatible).
	Approve func(ctx context.Context, tc gateway.ChatCompletionMessageToolCall) bool
}

ToolLoop drives a multi-turn conversation against a tokenless mock, executing real tool implementations when the mock returns tool_calls. Unregistered tool names cause a test failure.

func (*ToolLoop) LoadScenarioTools

func (l *ToolLoop) LoadScenarioTools(defs *gateway.ScenarioFile)

LoadScenarioTools reads the tools: block from a ScenarioFile and registers each exec-based tool as a ToolFunc that templates the argv from the tool call's JSON args and runs the resulting command.

func (*ToolLoop) Run

func (l *ToolLoop) Run(t testing.TB, prompt string) *ToolLoopResult

Run sends prompt as a user message and loops until the model responds with content (no more tool_calls). Each tool_calls turn invokes the matching ToolFunc and feeds the real result back as a role:tool message.

type ToolLoopResult

type ToolLoopResult struct {
	FinalContent string
}

ToolLoopResult holds the final assistant content after all tool calls have been resolved.