Documentation
¶
Overview ¶
Package testkit is shared test infrastructure for the agent: deterministic fault injection and invariant checks that make rigorous tests cheap to write.
Chaos engineering falls out of the ports architecture - wrap a unit of work or any port (dispatch work, dispatch.EventSink, ...) with a FaultPlan and assert the system degrades and recovers cleanly. Combined with clock.Manual and seeded inputs, the faults are deterministic, so a failing run reproduces exactly.
Index ¶
- func ActionGen() *rapid.Generator[dispatch.Action]
- func AppendInputGen(stream string) *rapid.Generator[spine.AppendInput]
- func CountByType(events []dispatch.Event) map[string]int
- func Diff[T any](t TB, want, got T)
- func DiffEvents(t TB, want, got []dispatch.Event)
- func FaultyBus(inner bus.Bus, plan *FaultPlan) bus.Bus
- func FaultyDoer(inner request.Doer, plan *FaultPlan) request.Doer
- func FaultyExecutor(inner goal.StepExecutor, plan *FaultPlan) goal.StepExecutor
- func FaultyLog(inner spine.Log, plan *FaultPlan) spine.Log
- func FaultyModel(inner llm.Model, plan *FaultPlan) llm.Model
- func FaultyReader(inner io.Reader, plan *FaultPlan) io.Reader
- func FaultySink(inner dispatch.EventSink, plan *FaultPlan) dispatch.EventSink
- func FaultySource(inner secret.Source, plan *FaultPlan) secret.Source
- func FaultyWork(inner func(context.Context) (dispatch.Metering, error), plan *FaultPlan) func(context.Context) (dispatch.Metering, error)
- func FaultyWriter(inner io.Writer, plan *FaultPlan) io.Writer
- func Golden[T any](t TB, name string, got T)
- func MemoryItemGen() *rapid.Generator[state.MemoryItem]
- func RequireLifecycle(t TB, events []dispatch.Event)
- func ScopeGen() *rapid.Generator[state.Scope]
- func SkillGen() *rapid.Generator[state.Skill]
- type FaultPlan
- type TB
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AppendInputGen ¶
func AppendInputGen(stream string) *rapid.Generator[spine.AppendInput]
AppendInputGen generates a spine.AppendInput for a fixed stream.
func CountByType ¶
CountByType tallies events by their Type, for assertions on what was emitted.
func Diff ¶
Diff fails the test with a readable diff when want != got. It is the generic comparison primitive for any type, so packages don't write a bespoke DiffXxx; nil and empty slices/maps compare equal.
func DiffEvents ¶
DiffEvents is Diff specialised to a dispatch-event stream, kept for readability at call sites. Two runs of the same scenario under a clock.Manual produce byte-identical streams, so this is also the determinism/replay check.
func FaultyBus ¶
FaultyBus wraps a bus.Bus, injecting plan's faults on Publish before delegating; Subscribe and Close pass through. It proves a publisher tolerates a flaky or dead signal bus: a failed or dropped publish must never corrupt the durable record a subscriber eventually reads. A nil plan never faults. A nil inner bus models a fully dead bus: Publish is a silent no-op and Subscribe returns an inert subscription that never delivers, so a consumer must make progress some other way (a poll floor).
func FaultyDoer ¶
FaultyDoer wraps a request.Doer, injecting plan's faults before delegating, so a flaky network (a few transient failures then recovery, or a hard outage) is modelled deterministically against the shared HTTP transport. The injected error flows through the transport's classification, so a fault.Transient drives the retry path and a fault.Terminal proves a non-retryable failure is not replayed. A nil inner doer returns a minimal 200 response when not faulting, so a plan alone is enough to drive the transport through its retry and recovery path.
func FaultyExecutor ¶
func FaultyExecutor(inner goal.StepExecutor, plan *FaultPlan) goal.StepExecutor
FaultyExecutor wraps a goal.StepExecutor, injecting plan's faults before delegating, so a flaky model or tool call (a step that fails a few times then succeeds) is modelled deterministically. A nil inner executor performs no work when not failing, so a plan alone is enough to drive a goal through its retry and recovery path. The checkpoint of a faulting call is dropped: a failed step makes no progress.
func FaultyLog ¶
FaultyLog wraps a spine.Log, injecting plan's faults on Append before delegating; Read passes through. It models a flaky durable log: an append that fails records nothing and assigns no Seq, so the stream stays gap-free (a dropped event simply never existed) rather than leaving a hole. A nil plan never faults; a nil inner log returns a zero Event on a non-faulting append and reads back nothing.
func FaultyModel ¶
FaultyModel wraps an llm.Model, injecting plan's faults on Generate before delegating. It models a flaky provider (a transient API error) so a test can drive the conversation loop's retry path, where the fault lands AFTER a turn has already been announced. That is the case the well-formedness invariant guards: a retried turn must not duplicate its turn.started on the event stream.
func FaultyReader ¶
FaultyReader wraps an io.Reader, injecting plan's faults before delegating, so an input stream that fails mid-read (a hung console, a torn multiplexer session) is modelled deterministically against anything that consumes it. A firing fault consumes nothing and returns the error; a nil inner reader reports io.EOF when not faulting.
func FaultySink ¶
FaultySink wraps a dispatch.EventSink, injecting plan's faults before delegating - used to prove that event-sink failures never break a dispatch. A nil inner sink discards events when not failing.
func FaultySource ¶
FaultySource wraps a secret.Source, injecting plan's faults on Lookup before delegating, so a flaky or unreachable credential store (a locked OS keychain, an unreachable vault or broker) is modelled deterministically. It proves a credential consumer fails closed on a resolve failure: a step that needs a secret must stop, never proceed without it or act on a partial value. A nil inner source resolves every reference to an empty value when not faulting, so a plan alone drives the consumer's failure path.
func FaultyWork ¶
func FaultyWork(inner func(context.Context) (dispatch.Metering, error), plan *FaultPlan) func(context.Context) (dispatch.Metering, error)
FaultyWork wraps a unit of dispatch work, injecting plan's faults before delegating. A nil inner returns zero Metering when not failing, so a plan alone is enough to drive a Dispatcher.Govern call through its retry and recovery path.
func FaultyWriter ¶
FaultyWriter wraps an io.Writer, injecting plan's faults before delegating, so a terminal that dies mid-session (a closed pty, a dropped SSH connection) is modelled deterministically against anything that renders. A firing fault writes nothing and returns the error; a nil inner writer discards writes when not faulting, so a plan alone can drive a renderer's error path.
func Golden ¶
Golden compares got against testdata/<name>.golden (pretty JSON). Run the tests with -update to (re)write the file. It lets an entire output - a full mission replay, a rendered spec, a whole event stream - be a single snapshot with no hand-written expected value, which is what keeps large tests small.
func MemoryItemGen ¶
func MemoryItemGen() *rapid.Generator[state.MemoryItem]
MemoryItemGen generates a state.MemoryItem.
func RequireLifecycle ¶
RequireLifecycle asserts a coherent event lifecycle on a recorded stream: every dispatched action is either a start paired with an end, or a single rejection. An imbalance means an action was lost or double-recorded.
Types ¶
type FaultPlan ¶
type FaultPlan struct {
// contains filtered or unexported fields
}
FaultPlan decides, deterministically, when to inject a fault. Each wrapped call advances a counter; the plan returns an error to inject on that call, or nil to pass through. Safe for concurrent use.
func FailFirst ¶
FailFirst injects err on the first k calls, then passes through. Models a flaky dependency that recovers - exercises retry and degrade paths.
func FailOnCall ¶
FailOnCall injects err only on the call with the given 1-based index.
Source Files