llmtest

Documentation

Overview

Package llmtest provides a deterministic, offline mock DeepSeek server for exercising the agent loop and the llm client end-to-end without a network connection or API credentials.

The seam it plugs into is the same one production uses: llm.Client talks to whatever BaseURL it is given, so pointing a Client at this package's httptest.Server reproduces the real wire path — SSE framing, the reasoning_content channel, tool_call deltas, the finish-reason override, the two-tier stream timeout, and the trailing usage frame — with no third-party dependency (net/http/httptest is stdlib).

A run is scripted as an ordered list of Turn values, one per model round-trip (one Client.Stream call). The server records every request body it receives so tests can assert on what was actually sent (e.g. that `thinking` serialized as a struct, or that a tool_call paired with its result on the following turn). When a run issues more requests than there are scripted Turns, the server replies with a synthetic terminal "stop" turn so an under-scripted test can never spin the loop to its step cap.

Import note: this package imports internal/llm, so internal/llm's own tests cannot import it (that would be an import cycle). It is meant for internal/agent, cmd/dsc, and other callers above the llm layer.

Index

• type Server
  • func NewServer(turns ...Turn) *Server
  • func (s *Server) Client() *llm.Client
  • func (s *Server) Count() int
  • func (s *Server) LastRequest() []byte
  • func (s *Server) Requests() [][]byte
• type ToolCall
• type Turn
• type Usage

Types

type Server

type Server struct {
	*httptest.Server
	// contains filtered or unexported fields
}

Server is a scripted mock DeepSeek endpoint. Construct it with NewServer and close it with Close (embedded from httptest.Server).

func NewServer

func NewServer(turns ...Turn) *Server

NewServer starts a mock server that serves the given turns in order.

func (*Server) Client

func (s *Server) Client() *llm.Client

Client returns an llm.Client pointed at this server with test-friendly timeouts and retries disabled. Callers may mutate the returned Client's FirstTokenTimeout / ChunkStallTimeout / MaxRetries to drive timeout and retry tests.

func (*Server) Count

func (s *Server) Count() int

Count returns how many requests the server has served.

func (*Server) LastRequest

func (s *Server) LastRequest() []byte

LastRequest returns the most recent request body, or nil if none.

func (*Server) Requests

func (s *Server) Requests() [][]byte

Requests returns a copy of every request body received so far, in order.

type ToolCall

type ToolCall struct {
	ID   string
	Name string
	Args string // raw JSON arguments, streamed verbatim
}

ToolCall is one tool invocation a Turn streams to the agent.

type Turn

type Turn struct {
	// Reasoning is emitted on DeepSeek's reasoning_content channel before
	// any visible content (the thinking-mode output).
	Reasoning string
	// Text is the visible assistant content.
	Text string
	// ToolCalls are streamed as tool_call deltas.
	ToolCalls []ToolCall

	// Finish overrides finish_reason. Empty is inferred: "tool_calls" when
	// ToolCalls is non-empty, otherwise "stop". Set Finish to "stop" while
	// also providing ToolCalls to exercise the finish-reason override.
	Finish string

	// Usage overrides the trailing usage frame; nil yields a default
	// cache-hit usage. OmitUsage suppresses the frame entirely.
	Usage     *Usage
	OmitUsage bool

	// Status, when non-zero, returns this HTTP status code (with Body) in
	// place of a stream — for error/retry-path tests.
	Status int
	Body   string

	// DelayBeforeHeaders delays before the response headers are sent, so the
	// client's HTTP round-trip (Client.Stream → Do) is still blocked waiting
	// for headers when a short request-context deadline or cancellation fires.
	// Context abort at this connect/headers phase is reliable (Do honours the
	// request context), unlike a context deadline that fires mid-body. Use it
	// to exercise per-step-deadline and user-cancellation paths.
	DelayBeforeHeaders time.Duration

	// DelayFirst delays the first SSE chunk (after headers), to trip the
	// client's first-token timeout. DelayChunk delays every chunk after the
	// first, to trip the chunk-stall timeout.
	DelayFirst time.Duration
	DelayChunk time.Duration

	// OmitDone suppresses the trailing "data: [DONE]" sentinel, modelling a
	// truncated stream.
	OmitDone bool

	// MalformedTail emits a malformed SSE data line after the content chunks
	// (reasoning/text/tool calls) and before any finish or usage frame. The
	// client accumulates the content it already received, then fails parsing
	// the bad line — modelling a mid-stream break with partial content
	// already in hand (the case partial-turn persistence must survive).
	MalformedTail bool
}

Turn scripts a single model response — exactly one HTTP round-trip, i.e. one llm.Client.Stream call / one agent step.

type Usage

type Usage struct {
	Prompt     int
	Completion int
	CacheHit   int
	CacheMiss  int
}

Usage overrides the usage frame emitted at the end of a Turn. The zero value of *Usage (nil) yields a realistic cache-hit usage so cache-metric and budget code paths see plausible numbers.