lockstep

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 25, 2026 License: MPL-2.0 Imports: 22 Imported by: 0

README

Write actors. Get exact replay.

lockstep is an actor runtime for Go where every message delivery, timer, and I/O event goes through the runtime. The same actor code runs in your program on lockstep.New(), and in go test on a seeded simulation with partitions, dropped and delayed messages, and crashes. Any failure in the simulation replays exactly from its seed.

What "lockstep" means here

Marching in lockstep means moving in exact unison, every step identical. That's the promise: every run with the same seed is identical.

The name also comes from deterministic lockstep, a game-networking technique used by real-time strategy games such as Age of Empires and StarCraft. Instead of sending game state over the network, every machine starts from the same state, applies the same inputs in the same order, and runs a deterministic simulation, so every machine computes the same game.

lockstep's simulation mode has the same properties:

Deterministic lockstep (games) lockstep (this project)
Same inputs, same order → same game state Same seed → same message order → same system state
A replay file is tiny: only the inputs A failing test comes down to a single seed
A desync means nondeterminism leaked in (floats, random numbers) A divergence means nondeterminism leaked into an actor (time.Now, math/rand, a raw goroutine)
The engine must own every source of randomness The runtime must own every scheduling, time, and I/O decision

What it doesn't mean: in games, lockstep peers wait for each other every tick. lockstep is not a game-sync library, and outside simulation it does not run in lockstep: no actor waits for another, and turns follow the order in which messages arrive. The lockstep behavior applies to simulation mode, where a single seeded scheduler steps every actor in a fixed, replayable order.

Relationship with faultline

lockstep's simulation mode, the lockstep/sim package, runs on faultline, deterministic simulation testing for Go. lockstep imports faultline; it isn't part of the faultline repo. Only lockstep/sim imports it, so a program that uses lockstep without sim never links faultline.

github.com/hmdsefi/lockstep          actor runtime
├── lockstep.New()                   local engine for your program
└── lockstep/sim ──imports──▶ github.com/hmdsefi/faultline (v0.1.2)

Install

lockstep needs Go 1.26 or later:

go version
go get github.com/hmdsefi/lockstep@v0.1.0

A first actor

An actor is a type with a Receive method for one message type. The following program is a counter: Add messages change its total, and a Get message asks for it.

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/hmdsefi/lockstep"
)

// Msg is every message the counter accepts.
type Msg interface{ isMsg() }

// Add adds N to the total.
type Add struct{ N int }

// Get asks for the total.
type Get struct{ Reply lockstep.ReplyTo[int] }

func (Add) isMsg() {}
func (Get) isMsg() {}

// Counter is an actor that keeps a running total.
type Counter struct{ total int }

func (c *Counter) Receive(ctx *lockstep.Context[Msg], msg Msg) {
	switch m := msg.(type) {
	case Add:
		c.total += m.N
	case Get:
		m.Reply.Send(ctx, c.total)
	}
}

func newCounter() lockstep.Actor[Msg] { return &Counter{} }

func getTotal(r lockstep.ReplyTo[int]) Msg { return Get{Reply: r} }

func main() {
	sys := lockstep.New()
	counter, err := lockstep.Spawn(sys, "counter", newCounter)
	if err != nil {
		log.Fatal(err)
	}
	for _, n := range []int{1, 2, 3} {
		if err := lockstep.Send[Msg](sys, counter, Add{N: n}); err != nil {
			log.Fatal(err)
		}
	}
	total, err := lockstep.Ask(sys, counter, getTotal).Await(context.Background())
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("total:", total)

	if err := sys.Shutdown(context.Background()); err != nil {
		log.Fatal(err)
	}
}

go run .

total: 6

Spawn returns a PID[Msg], so the compiler rejects a message of the wrong type. Send takes [Msg] explicitly because Add{N: n} alone has type Add. Ask puts a ReplyTo[int] into the Get message and returns a *Reply[int]. Await blocks, so call it outside actors; inside an actor, lockstep.Pipe delivers the reply as a message instead.

The same actor in a simulation test

sim.Simulate runs a test body once per seed. Each run is a simulated world with virtual time and a simulated network. The following test runs the counter on server node n1, sends 20 Add messages from client node c1, and lets faultline crash and restart n1 at random times. A final check runs when the run ends:

package main

import (
	"fmt"
	"testing"
	"time"

	"github.com/hmdsefi/faultline"
	"github.com/hmdsefi/faultline/kernel/fault"
	"github.com/hmdsefi/lockstep"
	"github.com/hmdsefi/lockstep/sim"
)

func TestCounter(t *testing.T) {
	opts := sim.Options{Options: faultline.Options{Seeds: 50, Duration: 10 * time.Second}}
	sim.Simulate(t, opts, func(s *sim.Sim) {
		w := s.World()
		var counter lockstep.PID[Msg]
		s.Node("n1", func(sys *lockstep.System) {
			counter, _ = lockstep.Spawn(sys, "counter", newCounter)
		})
		s.Client("c1", func(*lockstep.System) {})
		w.Plan(&fault.Random{MaxDown: 1, Rules: []fault.Rule{
			{Kind: fault.KindCrash, Every: 8 * time.Second, MaxFor: time.Second},
		}})

		for range 20 {
			w.RunFor(250 * time.Millisecond)
			if err := sim.Send[Msg](s, "c1", counter, Add{N: 1}); err != nil {
				w.T().Fatal(err)
			}
		}
		w.RunFor(4 * time.Second) // faults stop at 7.5s, a quarter before the end
		total, err := sim.Ask(s, "c1", counter, getTotal)
		w.Final("every add counted", func() error {
			if err != nil {
				return err
			}
			if total != 20 {
				return fmt.Errorf("total is %d after 20 adds", total)
			}
			return nil
		})
	})
}

The test imports faultline directly, so add it to go.mod and run the test:

go mod tidy
go test .

--- FAIL: TestCounter (0.01s)
    counter_test.go:16: faultline: running 50 seeds from base 0x64b5629963419063 (test name)
    --- FAIL: TestCounter/seed=0x870af07a512d3866 (0.01s)
        faultline: final check "every add counted" failed after the run at t=10.000000000s
          total is 0 after 20 adds
        replay:    FAULTLINE_SEED=0x870af07a512d3866 go test -v -run '^TestCounter$' example.com/counter
        artifacts: $TMPDIR/faultline-501/example.com_counter/TestCounter/870af07a512d3866/
    counter_test.go:16: faultline: stopping after failing seed 0x870af07a512d3866; 48 of 50 seeds not run (set Options.KeepGoing to run all)
FAIL
FAIL	example.com/counter	0.330s
FAIL

The first seed passes and the second fails. The counter keeps its total in memory, so a crash loses it. The run's schedule.json artifact shows when: n1 crashed at t=4.87s, after 19 adds, and restarted at t=5.71s with a new counter. The 20th add reached a node that was down.

The seeds come from the test's name, so they are the same on every machine. The replay line is the command that runs the failing seed again:

FAULTLINE_SEED=0x870af07a512d3866 go test -v -run '^TestCounter$' example.com/counter

=== RUN   TestCounter
    counter_test.go:16: faultline: FAULTLINE_SEED=0x870af07a512d3866: running 1 seed
=== RUN   TestCounter/seed=0x870af07a512d3866
    faultline: final check "every add counted" failed after the run at t=10.000000000s
      total is 0 after 20 adds
    replay:    FAULTLINE_SEED=0x870af07a512d3866 go test -v -run '^TestCounter$' example.com/counter
    artifacts: $TMPDIR/faultline-501/example.com_counter/TestCounter/870af07a512d3866/
--- FAIL: TestCounter (0.01s)
    --- FAIL: TestCounter/seed=0x870af07a512d3866 (0.01s)
FAIL
FAIL	example.com/counter	0.250s
FAIL

The replay fails the same check at the same virtual time, and its report.json holds the same trace hash (a hash of every event in the run) as the first run: 0x56d752b8033ec3c3. faultline writes the artifacts under os.TempDir(), shown here as $TMPDIR; they include trace.jsonl, timeline.txt, and a self-contained timeline.html.

Besides the checks you register, sim fails a seed when actors deadlock on each other's asks (lockstep: deadlock: A → B → A), when an effect has no handler, and when an actor stops or restarts with an effect still pending or a resource it owns fails to close.

Check actor code with lockstep-vet

A replay is exact only if actor code takes time, randomness, and I/O from the runtime. lockstep-vet reports actor code that doesn't:

  • LS100: nondeterminism and blocking, such as time.Now, global math/rand, crypto/rand, a go statement, select, a range over a map, environment variables, channel operations, locks, and blocking network, file, database, or process calls.
  • LS110: Reply.Await, which blocks the actor; use lockstep.Pipe instead.
  • LS001 and LS002: malformed and unknown //lockstep: directives.

Install it and run it through go vet:

go install github.com/hmdsefi/lockstep/analysis/cmd/lockstep-vet@v0.1.0
go vet -vettool=$(which lockstep-vet) ./...

On the counter it prints nothing. If Receive also recorded time.Now() on every Add and waited on an ask with Await, it would print:

main.go:37:50: LS110: Reply.Await blocks the actor; use lockstep.Pipe to receive the reply as a message (actor code: (*Counter).Receive)
main.go:34:17: LS100: time.Now uses the wall clock; use ctx.Now, ctx.After or ctx.Every (actor code: (*Counter).Receive)

To accept one finding, add //lockstep:allow LS100 <reason> on the reported line, on the line above it, or in the doc comment of the enclosing declaration.

The same checks run in golangci-lint as the module plugin lockstep. Build a custom binary with this .custom-gcl.yml:

version: v2.14.0
plugins:
  - module: github.com/hmdsefi/lockstep/analysis
    import: github.com/hmdsefi/lockstep/analysis/golangci
    version: v0.1.0

Enable the linter in .golangci.yml:

version: "2"
linters:
  enable:
    - lockstep
  settings:
    custom:
      lockstep:
        type: module

Then run golangci-lint custom and ./custom-gcl run ./.... golangci-lint's module plugin guide describes both files.

The Raft demo

examples/raft is a Raft cluster written as lockstep actors. It runs three or five servers and their clients through partitions, crashes, paused servers, and lossy links, checks Raft's safety properties after every event, and replays any failing seed exactly. Three planted bugs show what a failing seed looks like.

Limitations

v0.1.0 runs local actors in one process, plus the multi-node simulation in lockstep/sim. lockstep.New() runs every actor turn on one engine goroutine, so actors interleave but never run in parallel. Effects started with lockstep.Do run on separate goroutines, at most 64 at a time.

v0.1.0 has no pooled production scheduler. It does not read the live heap: WithAdmission, WithWatchdog, and WithGoroutineWarning are accepted and have no effect on lockstep.New(). The actor cap (WithMaxActors) applies everywhere. There is no remoting, clustering, persistence, or OpenTelemetry support, so actors on different nodes talk only in simulation.

The API can change before v1.0. The roadmap tracks what comes next.

Built with

gograph finds the cycles in the wait-for graph that lockstep/sim builds from pending asks, which is how it reports a deadlock.

License

lockstep is licensed under the Mozilla Public License 2.0.

Next steps

  • The Raft demo: a larger lockstep program and its simulation tests.
  • faultline: seeds, fault planners, and failure artifacts.
  • Roadmap: the plan for later releases.

Documentation

Overview

Package lockstep is an actor runtime for Go whose actor code runs unchanged on the local engine of New and on a seeded simulation in tests (package lockstep/sim), where a failing seed replays exactly.

The local engine runs every actor turn on one goroutine, so actors interleave but never run in parallel; effects started with Do run on separate goroutines.

Index

Constants

View Source 
const DefaultEffectTimeout = 30 * time.Second

DefaultEffectTimeout is the timeout of an effect started without WithEffectTimeout.

View Source 
const DefaultGoroutineWarning = 10_000

DefaultGoroutineWarning is the goroutine count above which a production System logs a warning.

View Source 
const DefaultMaxActors = 1_000_000

DefaultMaxActors is the actor cap of a System built without WithMaxActors.

View Source 
const DefaultWatchdogTimeout = 5 * time.Second

DefaultWatchdogTimeout is the stuck-turn timeout of a production System without WithWatchdog.

Variables

View Source 
var (
	ErrStopped       = errors.New("lockstep: stopped")
	ErrNameTaken     = errors.New("lockstep: name taken")
	ErrWrongType     = errors.New("lockstep: wrong message type")
	ErrNotFound      = errors.New("lockstep: not found")
	ErrInvalidOption = errors.New("lockstep: invalid option")
	ErrInvalidName   = errors.New("lockstep: invalid name")
	ErrNotLocal      = errors.New("lockstep: address not local")
)

Errors of package lockstep (ACT §4, §7).

View Source 
var (
	ErrTimeout      = errors.New("lockstep: timeout")
	ErrTerminated   = errors.New("lockstep: actor terminated")
	ErrAwaitInActor = errors.New("lockstep: blocking wait inside an actor")
)

Errors of sending and asking (MSG §4.1, §7).

View Source 
var (
	Normal         = StopReason{Kind: StopNormal}
	ParentStopping = StopReason{Kind: StopParentStopping}
	Shutdown       = StopReason{Kind: StopShutdown}
	// NoActor is the reason of a Terminated for an address that was not alive when it was watched
	// (Erlang's noproc). It never reaches Stopped.
	NoActor = StopReason{Kind: StopNoActor}
)
View Source 
var ErrActorCap = errors.New("lockstep: actor cap reached")

ErrActorCap is the sentinel of architecture §22.1, repeated for its text. ErrOverloaded, its companion, is declared with the engine's errors (ENG-120).

View Source 
var ErrMailboxFull = errors.New("lockstep: mailbox full")

ErrMailboxFull is the error of a message that a full mailbox refuses (MBX-090). A refusal returns fmt.Errorf("%w: %s", ErrMailboxFull, address), so errors.Is matches it and the text names the target.

View Source 
var ErrNoEffectHandler = errors.New("lockstep: no effect handler")

ErrNoEffectHandler is the error that fails a simulation seed when an actor starts an effect and the simulation has no handler for the effect's name. The failure wraps it, so errors.Is(err, ErrNoEffectHandler) holds, and its text names the effect and the actor.

View Source 
var ErrOverloaded = engine.ErrOverloaded

ErrOverloaded is returned when an engine refuses work. It is the same value as engine.ErrOverloaded, so errors.Is(err, ErrOverloaded) holds for errors from any engine (ENG-120).

Functions

func NewReply 

func NewReply[R any](from Sender, opts ...AskOption) (ReplyTo[R], *Reply[R])

NewReply creates an ask without a target: a pending reply on from's system with Ask's timeout and exactly-once resolution, and the ReplyTo that answers it. The caller builds and sends the request itself with Send (cluster.Ask does, VAC). An error from that Send (for example ErrMailboxFull) does not resolve the reply; the caller resolves it, for example with rt.Fail(from, err), as cluster.Ask does. A request that a DropNewest or DropOldest mailbox drops is not reported, so the reply resolves at its timeout (MSG-390). An invalid option value returns the zero ReplyTo and a reply resolved with an error wrapping ErrInvalidOption. NewReply panics if from is nil, an option is nil, or from is a Context outside its turn.

func Pipe 

func Pipe[R, M any](ctx *Context[M], r *Reply[R], to func(R, error) M)

Pipe delivers the resolution of r to the actor of ctx as one of its own messages: when r resolves, to(value, err) is called during a turn of the same incarnation and its result is passed to Receive in that turn. If the incarnation has ended by then, nothing is delivered. Pipe panics if ctx, r or to is nil, if ctx is outside its turn, or if r was created on another system.

func Send 

func Send[M any](from Sender, to PID[M], msg M) error

Send delivers msg to the actor to. It returns nil when msg was enqueued, became a dead letter, or was accepted by the transport for another node. Errors: ErrNotFound for the zero PID, ErrStopped when the sender's system is stopped, the target mailbox's policy error (ErrMailboxFull, MBX), the admission error (ErrOverloaded, OVL), and a synchronous transport error, wrapped (MSG-110). Send panics if from is nil or is a Context outside its turn.

func SendContext 

func SendContext[M any](ctx context.Context, from Sender, to PID[M], msg M) error

SendContext is Send, except that when from is a *System, the target's policy is BlockSender and the message cannot be enqueued yet, it waits until the message is enqueued or ctx ends, and then returns ctx.Err(). Called from an actor's turn it never waits: BlockSender acts as FailSend. Send with a *System sender is SendContext with context.Background().

func Stop 

func Stop(from Sender, addr Address) error

Stop asks the actor at addr to stop with reason Normal and returns without waiting. A parent may stop its child this way; any Sender may stop any local actor. A path address stops the actor registered at its path. Errors: ErrNotFound if addr is not alive or names a guardian, ErrNotLocal for another node's address in every phase (there is no remote stop). Stop panics if from is nil.

func Unwatch 

func Unwatch[M any](ctx *Context[M], addr Address)

Unwatch cancels Watch. After it returns, the incarnation receives no Terminated for addr.

func Watch 

func Watch[M any](ctx *Context[M], addr Address) error

Watch asks for a Terminated signal (SUP) when the actor at addr stops for good. If addr is not alive, the signal is delivered with reason NoActor (SUP). A path address (PIDAt) watches the actor registered at its path at the time of the call. From Phase 3 an address on another node is watched through remoting (ACT-602). Errors: ErrNotFound for the zero address and guardian addresses; ErrNotLocal for another node's address in Phase 1 and 2, and from Phase 3 on a system without a transport; ErrStopped when the calling actor is stopping or restarting (inside a restart's Stopped too); a wrapped transport error (Phase 3).

Types

type Actor 

type Actor[M any] interface {
	// Receive handles one user message. It runs during a turn and must not block.
	Receive(ctx *Context[M], msg M)
}

Actor is implemented by actor values. M is the actor's message type.

type ActorEstimate 

type ActorEstimate struct {
	Addr         Address
	MailboxBytes int64
	StateBytes   int64
}

ActorEstimate is the memory estimate of one live actor.

type Address 

type Address struct {
	// contains filtered or unexported fields
}

Address is the untyped address of one actor: node, path and uid. It is comparable; two addresses are equal when all three parts are equal. The zero Address names no actor.

func (Address) IsZero 

func (a Address) IsZero() bool

IsZero reports whether a is the zero Address.

func (Address) Node 

func (a Address) Node() string

Node returns the node name.

func (Address) Path 

func (a Address) Path() string

Path returns the absolute path, for example "/user/orders/o-17".

func (Address) String 

func (a Address) String() string

String returns "lockstep://<node><path>#<uid in lowercase hex>", or "<zero>" for the zero Address.

func (Address) UID 

func (a Address) UID() uint64

UID returns the uid.

type AskOption 

type AskOption func(*askConfig)

AskOption configures one Ask.

func WithTimeout 

func WithTimeout(d time.Duration) AskOption

WithTimeout sets the ask's timeout. The default is 5 s. It never panics: Ask and NewReply resolve the reply with an error wrapping ErrInvalidOption when d <= 0 (MSG-700).

type ChildFailed 

type ChildFailed struct {
	Child     Address
	Err       error
	Directive Directive // the directive applied; 0 when Cleanup is true
	Cleanup   bool      // the failure happened in Stopped; no directive was applied
}

ChildFailed reports a child's failure after the supervisor's strategy has handled it.

type Context 

type Context[M any] struct {
	// contains filtered or unexported fields
}

Context is an incarnation's handle to the runtime. The runtime passes the same *Context to every turn of one incarnation and a new one to the next incarnation. Self and Parent may be called at any time; every other method panics when called outside a turn of its incarnation. Every method panics on a nil *Context.

func (*Context[M]) After 

func (c *Context[M]) After(d time.Duration, msg M) TimerID

After delivers msg to the actor itself once, d from now on the engine clock. A negative d is treated as 0. The timer is cancelled if the incarnation ends first. Called from Stopped it schedules nothing and returns 0.

func (*Context[M]) CancelEffect 

func (c *Context[M]) CancelEffect(id EffectID) bool

CancelEffect cancels the pending effect id of this incarnation and reports whether it was pending; done is then never called for it. It returns false for 0, for an ID of another incarnation, and for an effect that already has an outcome.

func (*Context[M]) CancelTimer 

func (c *Context[M]) CancelTimer(id TimerID) bool

CancelTimer stops timer id. It reports whether the timer was active. Once it returns, the actor receives no message from that timer, even one already in the mailbox.

func (*Context[M]) Every 

func (c *Context[M]) Every(d time.Duration, msg M) TimerID

Every delivers msg to the actor itself every d, starting d from now, until the timer is cancelled or the incarnation ends. While a previous message of the timer is still in the mailbox, a firing delivers nothing. It panics if d <= 0. Called from Stopped it schedules nothing and returns 0.

func (*Context[M]) Fail 

func (c *Context[M]) Fail(err error)

Fail marks the actor failed with err when the current handler returns. Only the first call per handler counts. It panics if err is nil.

func (*Context[M]) Logf 

func (c *Context[M]) Logf(format string, args ...any)

Logf formats a log line for this actor (ACT-130). It formats with fmt.Sprintf and records lockstep.log when the tracer is enabled. On engines whose goroutines may block (the local and production engines) it also logs the line at slog.LevelInfo through slog.Default() with the attributes node, path and uid (lowercase hex); on the simulation engine the line reaches only the trace.

func (*Context[M]) Now 

func (c *Context[M]) Now() time.Time

Now returns the engine clock: wall time in production, the node's local clock in simulation, where faultline's clock drift and jumps apply.

func (*Context[M]) Own 

func (c *Context[M]) Own(r io.Closer)

Own registers r to be closed after the incarnation's Stopped hook returns. Resources are closed in reverse registration order. It panics if r is nil.

func (*Context[M]) Parent 

func (c *Context[M]) Parent() Address

Parent returns the parent's address: the user guardian's for actors spawned from a System.

func (*Context[M]) Rand 

func (c *Context[M]) Rand() *rand.Rand

Rand returns the random source of the current incarnation. Every call in one incarnation returns the same *rand.Rand; a restart gets a new one. Use it only inside the actor's handlers.

func (*Context[M]) Self 

func (c *Context[M]) Self() PID[M]

Self returns the actor's PID. It is the same for every incarnation of the actor.

func (*Context[M]) Stop 

func (c *Context[M]) Stop()

Stop stops the actor with reason Normal when the current handler returns. No further user message is processed by this incarnation.

func (*Context[M]) SubscribeOverload 

func (c *Context[M]) SubscribeOverload()

SubscribeOverload subscribes the current incarnation to Overloaded signals. Subscribing again has no effect. If the System is shedding, an Overloaded{Active: true} signal is queued at once.

func (*Context[M]) UnsubscribeOverload 

func (c *Context[M]) UnsubscribeOverload()

UnsubscribeOverload cancels the subscription of the current incarnation.

type Directive 

type Directive uint8

Directive is a supervisor's decision about a failed child. 

const (
	DirectiveRestart  Directive = iota + 1 // end the incarnation and start a new one in place
	DirectiveStop                          // stop the child for good with Failed(err)
	DirectiveEscalate                      // fail the supervisor with the child's error
	DirectiveResume                        // keep the incarnation; the failing message is lost
)

func (Directive) String 

func (d Directive) String() string

String returns "restart", "stop", "escalate", "resume", or "Directive(<n>)".

type Effect 

type Effect[T any] interface {
	// Name returns the effect's stable name: 1 to 64 characters from [A-Za-z0-9._-], starting
	// with a letter or a digit. Simulation finds the effect's handler by this name, and traces
	// show it. Names starting with "lockstep." are reserved for lockstep's built-in effects.
	Name() string

	// Run performs the operation and returns its result. The local engine (Phase 1) and the
	// production engine call it on one of their effect workers; simulation never calls it. Run
	// should return soon after ctx is done, which happens when the effect times out or is
	// cancelled.
	Run(ctx context.Context) (T, error)
}

Effect is an operation that waits: a network call, file I/O, a database query, CPU work over a budget. An actor starts it with Do and gets its outcome back as a message.

type EffectID 

type EffectID uint64

EffectID identifies one Do call within a System. IDs start at 1 and increase by one per Do call that starts an effect. 0 means that no effect was started.

func Do 

func Do[T, M any](ctx *Context[M], e Effect[T], done func(T, error) M, opts ...EffectOption) EffectID

Do starts effect e for the actor of ctx and returns its ID. When the effect's outcome is delivered, a later turn of the actor calls done with the result, or with the zero T and ErrTimeout or ErrOverloaded, and passes done's return value to Receive. done runs on the actor's turn, never concurrently with Receive. If the incarnation ends or CancelEffect cancels the effect first, done is never called.

Do panics if e is nil, e.Name() is invalid, done is nil, or an option is invalid.

type EffectOption 

type EffectOption func(*effectConfig)

EffectOption configures one Do call.

func WithEffectTimeout 

func WithEffectTimeout(d time.Duration) EffectOption

WithEffectTimeout sets the effect's timeout, measured on the engine's clock from the Do call. Do panics if d is not positive.

type MailboxPolicy 

type MailboxPolicy uint8

MailboxPolicy selects what happens when a message is sent to a full mailbox. A mailbox is full when it holds as many sent messages as its capacity. Timer messages, piped replies and effect results are never refused and never count toward capacity. 

const (
	// FailSend refuses the message: Send returns an error wrapping ErrMailboxFull. The default.
	FailSend MailboxPolicy = iota
	// DropNewest drops the message being sent and records the drop. Send returns nil.
	DropNewest
	// DropOldest drops the oldest sent message in the mailbox, records the drop and enqueues the
	// new one. Send returns nil.
	DropOldest
	// BlockSender makes an outside sender (a *System) wait until the mailbox has room and the
	// System admits messages again (OVL), or until its context ends. For a sender inside an actor, and
	// on engines that cannot make a caller wait (simulation), it behaves as FailSend.
	BlockSender
)

func (MailboxPolicy) String 

func (p MailboxPolicy) String() string

String returns "fail-send", "drop-newest", "drop-oldest", "block-sender", or "MailboxPolicy(<n>)" for other values. Records, logs and metrics use these spellings (OBS-145).

type MemSizer 

type MemSizer interface{ MemSize() int }

MemSizer is implemented by actors that report the size of their state in bytes.

type Option 

type Option func(*systemConfig)

Option configures New. Options are applied in argument order; a later option overrides an earlier one for the same setting. Option values are validated by New, not by the option constructors: no option constructor panics (R44).

func WithAdmission 

func WithAdmission(high, low uint64) Option

WithAdmission sets absolute admission thresholds in bytes: admission stops when the live heap reaches high and resumes when it falls to low. WithAdmission(0, 0) turns admission off even when a Go memory limit is set. Any other pair needs 0 < low < high, or New panics with an error wrapping ErrInvalidOption. Simulation ignores it; its byte budgets are sim.Options.Budget.

func WithDefaultMailbox 

func WithDefaultMailbox(capacity int, policy MailboxPolicy) Option

WithDefaultMailbox sets the capacity and policy of every actor spawned in the system without WithMailbox. Without it the default is 1024 and FailSend. It never panics: an invalid capacity or policy makes New panic with an error wrapping ErrInvalidOption (ACT).

func WithGoroutineWarning 

func WithGoroutineWarning(bound int) Option

WithGoroutineWarning sets the goroutine count that triggers a warning. 0 turns the warning off (the count is still exported). A bound < 0 makes New panic with an error wrapping ErrInvalidOption. Simulation ignores it.

func WithGuardianStrategy 

func WithGuardianStrategy(s Strategy) Option

WithGuardianStrategy sets the user guardian's strategy for top-level actors. Without it, the strategy is OneForOne(nil). New panics with an error wrapping ErrInvalidOption if s is invalid (SUP-370).

func WithMaxActors 

func WithMaxActors(n int) Option

WithMaxActors sets the maximum number of live actors of the System to n. n/100 of them (at least 1 when n >= 2) are reserved for restart rebuilds and system actors. An n < 1 makes New panic with an error wrapping ErrInvalidOption.

func WithMemoryEstimates 

func WithMemoryEstimates() Option

WithMemoryEstimates turns on per-actor memory estimates (Sizer, MemSizer, System.Estimates).

func WithNodeName 

func WithNodeName(name string) Option

WithNodeName sets the system's node name. The name must be 1-64 bytes from [A-Za-z0-9._-], starting with a letter or digit (the faultline node-name rule). WithNodeName never panics; New panics with an error wrapping ErrInvalidOption on an invalid name. The default is "local". It is ignored on a system built on an engine (simulation), which takes the engine's node name.

func WithWatchdog 

func WithWatchdog(d time.Duration) Option

WithWatchdog sets the stuck-turn timeout. 0 turns the watchdog off. A d < 0 makes New panic with an error wrapping ErrInvalidOption. Simulation ignores it.

type OverloadReason 

type OverloadReason uint8

OverloadReason is the source of an admission change. Its values equal those of engine.AdmissionSource. 

const (
	OverloadMemory   OverloadReason = iota + 1 // production live heap
	OverloadBudget                             // simulation count budget (sim.Options.Budget)
	OverloadPressure                           // simulation memory-pressure fault
)

func (OverloadReason) String 

func (r OverloadReason) String() string

String returns "memory", "budget", "pressure", or "OverloadReason(<n>)".

type Overloaded 

type Overloaded struct {
	Active bool           // true: admission stopped; false: admission resumed
	Reason OverloadReason // the source whose change caused this transition
	Live   uint64         // measured bytes at the change (OverloadBudget: the estimate); 0 for OverloadPressure
	High   uint64         // thresholds in force (OverloadBudget: the byte budget and its resume mark); 0 for OverloadPressure
	Low    uint64
}

Overloaded is the signal sent to subscribed actors when their System starts or stops shedding. It implements Signal.

type PID 

type PID[M any] struct {
	// contains filtered or unexported fields
}

PID is the typed address of an actor whose message type is M. It is comparable and contains only its Address. The zero PID names no actor.

func Lookup 

func Lookup[M any](sys *System, path string) (PID[M], error)

Lookup returns the PID of the actor registered at path on sys, if it was spawned with exactly message type M. It finds an actor whose stop has begun and none whose stop has completed. It returns ErrInvalidName if path is not well-formed, ErrNotFound if no actor is registered there (the guardians /user and /system are not), and ErrWrongType if the actor's message type is not M. Lookup panics if sys is nil.

func PIDAt 

func PIDAt[M any](node, path string) PID[M]

PIDAt returns a PID with uid 0 that addresses whichever actor is registered at path on node when the PID is used, locally and through remoting (Phase 3). A message to it when no actor of type M is registered at path goes to dead letters on that node. path has the form Lookup accepts. PIDAt panics if node is not a valid node name or path is malformed.

func Spawn 

func Spawn[M any](parent Spawner, name string, factory func() Actor[M], opts ...SpawnOption) (PID[M], error)

Spawn creates a child of parent named name and returns its PID. name "" asks for a generated name "$<n>". factory is called at every start and restart, never by Spawn itself.

Errors: ErrInvalidName, ErrInvalidOption, ErrStopped, ErrNameTaken, ErrActorCap and ErrOverloaded (OVL), each wrapped as in §7. Spawn panics if parent, factory or an option is nil. A spawn refused for any of these reasons registers nothing and takes no generated name.

func (PID[M]) Address 

func (p PID[M]) Address() Address

Address returns the PID's untyped address.

func (PID[M]) IsZero 

func (p PID[M]) IsZero() bool

IsZero reports whether p is the zero PID.

func (PID[M]) String 

func (p PID[M]) String() string

String returns p.Address().String().

type PanicError 

type PanicError struct {
	Value any    // the value passed to panic
	Stack []byte // runtime/debug.Stack() at recovery; never recorded in traces
}

PanicError is the error of a failure caused by a panic. errors.Unwrap returns Value when it is an error.

func (*PanicError) Error 

func (e *PanicError) Error() string

Error returns "lockstep: panic: " followed by a description of Value (SUP-145).

func (*PanicError) Unwrap 

func (e *PanicError) Unwrap() error

Unwrap returns Value if it is an error, otherwise nil.

type Reply 

type Reply[R any] struct {
	// contains filtered or unexported fields
}

Reply is the typed future of one ask.

func Ask 

func Ask[M, R any](from Sender, to PID[M], build func(ReplyTo[R]) M, opts ...AskOption) *Reply[R]

Ask sends the message build returns to the actor to and returns the future of its answer. build receives the ask's ReplyTo and is called once, synchronously, before Ask returns, unless the ask fails before sending (MSG-300). With a *System sender outside any actor's turn, a full BlockSender mailbox on a blocking engine makes Ask wait until the request is enqueued or the ask resolves, at its timeout at the latest (MSG-330). An invalid option value resolves the reply with an error wrapping ErrInvalidOption. Ask panics if from or build is nil, an option is nil, or from is a Context outside its turn.

func (*Reply[R]) Await 

func (r *Reply[R]) Await(ctx context.Context) (R, error)

Await waits for the ask to resolve and returns its value or error. If ctx ends first it returns the zero R and ctx.Err(), and the ask stays pending. Called during a turn it returns ErrAwaitInActor at once. In simulation Await never waits and never drives the run: on a pending reply it fails the seed and returns an error wrapping ErrAwaitInActor, from a turn or from the test body (test bodies use sim.Ask). Await may be called any number of times from any goroutine; every call that sees the resolution returns the same result.

type ReplyTo 

type ReplyTo[R any] struct {
	// contains filtered or unexported fields
}

ReplyTo is the address an answer goes to. Ask creates it and passes it to build; the message carries it to whoever answers. It is a comparable value without pointers, so messages that hold it stay safe. The zero ReplyTo answers nothing.

func (ReplyTo[R]) Fail 

func (rt ReplyTo[R]) Fail(from Sender, err error)

Fail answers the ask with err. It panics if err is nil, and like Send otherwise.

func (ReplyTo[R]) Send 

func (rt ReplyTo[R]) Send(from Sender, v R)

Send answers the ask with v. The first reply attempt resolves the ask; later ones are dropped and traced. Send panics if from is nil or is a Context outside its turn.

type Sender 

type Sender interface {
	// contains filtered or unexported methods
}

Sender is the origin of a send, an ask, a reply or a stop. It is implemented only by *System and *Context[X] for any X.

type Signal 

type Signal interface {
	// contains filtered or unexported methods
}

Signal is a notification from lockstep itself, delivered through SignalReceiver[M].Signal and never as an M. It is implemented only by Terminated and ChildFailed (SUP), Overloaded (OVL) and, from Phase 3, NodeDown and NodeUp (CLU).

type SignalReceiver 

type SignalReceiver[M any] interface {
	Signal(ctx *Context[M], sig Signal)
}

SignalReceiver is implemented by actors that receive lockstep's notifications (MSG-810). An actor without it ignores signals; its supervisor strategy still handles its children's failures (SUP-150 to SUP-230).

type Sizer 

type Sizer interface{ Size() int }

Sizer is implemented by messages that report their size in bytes for memory estimates.

type SpawnOption 

type SpawnOption func(*spawnConfig)

SpawnOption configures one Spawn. WithMailbox (MBX), WithStrategy (SUP) and the throughput hint (SCH) are SpawnOptions.

func WithMailbox 

func WithMailbox(capacity int, policy MailboxPolicy) SpawnOption

WithMailbox sets the spawned actor's mailbox capacity and full-mailbox policy. It never panics: if capacity < 1 or policy is not one of the four constants, Spawn returns an error wrapping ErrInvalidOption (ACT).

func WithStrategy 

func WithStrategy(s Strategy) SpawnOption

WithStrategy sets the strategy for the children of the spawned actor. Without it, the strategy is OneForOne(nil). Spawn returns an error wrapping ErrInvalidOption if s is invalid (SUP-370).

type Spawner 

type Spawner interface {
	// contains filtered or unexported methods
}

Spawner is a parent for Spawn. It is implemented only by *System (the user guardian is the parent) and *Context[X] for any X (that actor is the parent).

type Starter 

type Starter[M any] interface {
	Started(ctx *Context[M])
}

Starter is implemented by actors that need a hook when an incarnation starts. Started runs after the factory and before the first message of the incarnation (SUP-030).

type StopKind 

type StopKind uint8

StopKind classifies a StopReason. 

const (
	StopNormal         StopKind = iota + 1 // Stop(from, addr) or ctx.Stop
	StopFailed                             // the supervisor's directive was to stop a failed actor
	StopParentStopping                     // the parent is stopping or restarting
	StopShutdown                           // System.Shutdown
	StopRestarting                         // the incarnation ends for a restart; only Stopped sees it
	StopNoActor                            // Terminated only: the watched address was not alive
)

func (StopKind) String 

func (k StopKind) String() string

String returns "normal", "failed", "parent-stopping", "shutdown", "restarting", "no-actor", or "StopKind(<n>)".

type StopReason 

type StopReason struct {
	Kind StopKind
	Err  error
}

StopReason tells why an incarnation ended, or, in Terminated, why a watched actor is gone. Err is non-nil exactly for StopFailed and StopRestarting.

func Failed 

func Failed(err error) StopReason

Failed returns a StopFailed reason carrying err: the reason of an actor stopped by its supervisor after failing with err, and of a Terminated whose watched actor was lost for a reason another package reports, such as cluster.ErrNodeDown or REM's ErrPeerRestarted. It panics if err is nil.

func Restarting 

func Restarting(err error) StopReason

Restarting returns the reason Stopped receives when the incarnation ends for a restart caused by err. It panics if err is nil.

func (StopReason) String 

func (r StopReason) String() string

String returns Kind.String(), followed by ": " and Err.Error() when Err is non-nil.

type Stopper 

type Stopper[M any] interface {
	Stopped(ctx *Context[M], reason StopReason)
}

Stopper is implemented by actors that release resources. Stopped runs exactly once per incarnation: when the actor stops, before every restart, and after a failure, but not when a simulated node crashes. Children have already stopped when it runs (SUP-100).

type Strategy 

type Strategy struct {

	// Decider maps a child's failure to a directive. nil means DirectiveRestart for every error.
	Decider func(err error) Directive
	// MaxRestarts is how many restarts are allowed within Window; one more becomes
	// DirectiveEscalate. 0 allows none.
	MaxRestarts int
	// Window is the sliding period over which restarts are counted, on the engine clock.
	Window time.Duration
	// MinBackoff is the delay before the first restart in the window; it doubles for each further
	// restart in the window, up to MaxBackoff. 0 restarts without delay.
	MinBackoff time.Duration
	MaxBackoff time.Duration
	// contains filtered or unexported fields
}

Strategy is a supervision strategy for the children of one actor. Build it with OneForOne or AllForOne and adjust the exported fields; a zero Strategy is invalid.

func AllForOne 

func AllForOne(decider func(err error) Directive) Strategy

AllForOne returns a strategy that restarts or stops every child of the supervisor when one fails, with the same limits as OneForOne. Restarts are counted for the group.

func OneForOne 

func OneForOne(decider func(err error) Directive) Strategy

OneForOne returns a strategy that applies a directive to the failed child only, with MaxRestarts 10, Window 1 minute, MinBackoff 10 ms and MaxBackoff 10 s.

type System 

type System struct {
	// contains filtered or unexported fields
}

System is one runtime instance. All of its exported functions and methods are safe for concurrent use when *System is the Spawner or Sender.

func New 

func New(opts ...Option) *System

New builds a system on the local engine (Phase 1) or the production engine (Phase 2, SCH), creates its two guardians and returns it. It panics with an error wrapping ErrInvalidOption if an option is nil or invalid.

func (*System) Estimates 

func (s *System) Estimates(n int) []ActorEstimate

Estimates returns up to n live actors with the largest estimates, largest first, ties in byte order of Addr.String(). It returns nil when n <= 0 or estimates are off.

func (*System) Incarnation 

func (s *System) Incarnation() uint32

Incarnation returns the epoch the system started with: the faultline node incarnation in simulation, a random value otherwise. It is never 0 and does not change for the system's life, even when uid allocation moves to a new epoch (ACT-040).

func (*System) Node 

func (s *System) Node() string

Node returns the system's node name: the node part of every address it allocates.

func (*System) Shutdown 

func (s *System) Shutdown(ctx context.Context) error

Shutdown stops every actor, user actors first, then system actors, and waits until every Stopped hook has returned. It returns nil when the system is stopped, or ctx.Err() if ctx ends first; shutdown then continues and a later call waits again. Called during a turn (an actor may shut down its own system) or on an engine that cannot block, such as the simulation engine, it starts the shutdown and returns nil at once without waiting (ACT-060).

type Terminated 

type Terminated struct {
	Addr   Address
	Reason StopReason
}

Terminated reports that a watched actor stopped for good, or that the watched address was not alive (Reason NoActor). Reason is the actor's final stop reason, NoActor, or a Failed reason set by REM or CLU for an actor lost with its node or connection.

type TimerID 

type TimerID uint64

TimerID identifies a timer of one actor. IDs are unique within the actor for its whole life, across restarts. 0 is never a valid timer.

Source Files

View all Source files

Directories

Path Synopsis
analysis module
internal
dettest
Package dettest is lockstep's determinism lint, import-boundary check, exported-API snapshot generator, run-twice switch and golden-hash file format.
 Package dettest is lockstep's determinism lint, import-boundary check, exported-API snapshot generator, run-twice switch and golden-hash file format.
engine
Package engine is the seam between the lockstep runtime and the engines that run it (architecture §12, RFC-0001).
 Package engine is the seam between the lockstep runtime and the engines that run it (architecture §12, RFC-0001).
engine/enginetest
Package enginetest is a deterministic manual engine and a recording tracer for unit tests of the runtime.
 Package enginetest is a deterministic manual engine and a recording tracer for unit tests of the runtime.
engine/local
Package local is the single-goroutine engine behind lockstep.New in Phase 1 (architecture §12.3): one loop goroutine runs every turn, timer function and effect completion.
 Package local is the single-goroutine engine behind lockstep.New in Phase 1 (architecture §12.3): one loop goroutine runs every turn, timer function and effect completion.
hooks
Package hooks connects packages that must not import each other.
 Package hooks connects packages that must not import each other.
mailbox
Package mailbox holds the two queues of an actor: the bounded user Queue with its full-mailbox policy, and the unbounded SystemQueue for lifecycle commands and signals (MBX-030 to MBX-080).
 Package mailbox holds the two queues of an actor: the bounded user Queue with its full-mailbox policy, and the unbounded SystemQueue for lifecycle commands and signals (MBX-030 to MBX-080).
overload
Package overload holds the parts of overload protection every engine shares: the actor cap, the admission gate and the hysteresis switch (OVL §4.2).
 Package overload holds the parts of overload protection every engine shares: the actor cap, the admission gate and the hysteresis switch (OVL §4.2).
registry
Package registry holds a system's actor names, child lists, generated-name counters and watch tables (ACT-500).
 Package registry holds a system's actor names, child lists, generated-name counters and watch tables (ACT-500).
supervise
Package supervise holds the pure logic of actor lifecycle and supervision: the cell state machine, directives, restart limits, sliding-window restart intensity and capped exponential backoff (SUP-010, SUP-180, SUP-270, SUP-280, SUP-370).
 Package supervise holds the pure logic of actor lifecycle and supervision: the cell state machine, directives, restart limits, sliding-window restart intensity and capped exponential backoff (SUP-010, SUP-180, SUP-270, SUP-280, SUP-370).
toys
Package toys holds the toy actor systems of the golden suite: small actors with fixed behavior whose simulation runs have the same trace on every machine (DET-200).
 Package toys holds the toy actor systems of the golden suite: small actors with fixed behavior whose simulation runs have the same trace on every machine (DET-200).
Package sim runs lockstep actor systems as seeded faultline simulations inside go test.
 Package sim runs lockstep actor systems as seeded faultline simulations inside go test.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.