devnet

package
v0.84.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 16, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

README

DevNet

A self-contained private Cardano network for end-to-end testing of Dingo as a block producer. The whole thing runs locally under Docker Compose: a configurator container generates fresh genesis files and pool keys on each start, then nodes come up and forge blocks against each other while a txpump sidecar continuously submits payment transactions into the mempool.

Two networks are available, selected by a Docker Compose profile:

  • dingo (default, profile dingo) — three Dingo producers plus one Dingo relay, all running the tree under test. This is what run-tests.sh, start.sh, and stop.sh bring up with no flags. It validates the generic consensus/liveness suite dingo-vs-dingo and hosts dingo-only feature tests (e.g. CIP-50 pledge leverage) that have no cardano-node reference to compare against.
  • conformance (opt-in, profile conformance, pass --conformance) — one Dingo producer alongside cardano-node (producer + relay), for compatibility/reference validation against the upstream implementation.

The Go test harness lives alongside this directory: internal/test/devnet/ (helpers and config loader at the top level, runnable scenarios under internal/test/devnet/scenarios/). Every Go file in the tree has a linux build constraint because the harness requires a native Linux Docker engine, Bash, Linux container networking, and Unix ownership semantics. Code that talks to a running network additionally requires the devnet build tag, and conformance-only tests also require devnet_conformance (see Test scenarios below).

The pure logic the harness is built on — the network-spec loader and its validation, the observed-chain state machine, and the scenario plan — carries only the linux constraint, so its unit tests run in the ordinary Linux go test ./... with no Docker involved. That is what keeps an invalid accelerated spec or a broken rollback rule from only surfacing as a mysterious DevNet stall.

Topology — dingo mode (default)

Service Role Container IP NtN host port NtC (LocalStateQuery) host port
configurator-dingo One-shot: generates keys + genesis for 3 pools — — —
dingo-1 Dingo, forging with pool 1 keys 172.20.0.13 3010 3020
dingo-2 Dingo, forging with pool 2 keys 172.20.0.14 3013 3021
dingo-3 Dingo, forging with pool 3 keys 172.20.0.15 3014 3022
dingo-relay Dingo relay (no forging) 172.20.0.16 3015 3023
txpump-dingo Submits payment txs into dingo-1's mempool 172.20.0.21 — —

The three producers and the relay are wired into a ring topology by configurator.sh (DINGO_POOL_IDS="1 2 3"), with the relay peering with all three producers so block diffusion is exercised in addition to direct producer-to-producer sync. Network spec: testnet-dingo.yaml (3 pools, poolPledge: 0 — required for the CIP-50 scenario below).

Topology — conformance mode (--conformance)

Service Role Container IP NtN host port NtC (LocalStateQuery) host port
configurator One-shot: generates keys + genesis — — —
dingo-producer Dingo, forging with pool 1 keys 172.20.0.10 3010 3030
cardano-producer cardano-node, forging with pool 2 172.20.0.11 3011 3031
cardano-relay cardano-node relay (no forging) 172.20.0.12 3012 —
txpump Submits payment txs into Dingo 172.20.0.20 — —

This is unchanged from the original two-pool network: pool 1 and pool 2 are wired into a ring topology, and the relay peers with both producers. Network spec: testnet.yaml.

dingo-producer and cardano-producer — the two nodes that actually forge blocks — both have an NtC host port mapped, so the host test harness can run LocalStateQuery against each and compare ledger state (see TestLedgerStateConsensus under Test scenarios below). cardano-relay has no NtC host port mapped; it isn't part of that comparison.

In both modes, txpump is a load generator: it talks Ouroboros NtC over the Dingo node's container-network TCP endpoint and submits one payment transaction per round with a 5–15s cooldown, funded from the genesis UTxO keys generated by the configurator. Limiting each round to one transaction prevents txpump from immediately building an unconfirmed dependency chain that can be invalidated by an early fork. txpump starts only after every node in the active profile is healthy. Outputs created by a submitted transaction remain unavailable to txpump for 600 slots on canonical runs. --accelerated selects 100 slots (50 seconds at the accelerated slot length), allowing further rounds within the scenario budget. This delay reduces immediate dependency reuse; txpump does not verify on-chain confirmation or recover outputs lost to a rollback, so expiry alone does not prove an output is spendable. It exists to keep the mempool exercised while the consensus tests run, so block bodies are non-empty and tx-submission / mempool paths are continuously hit. Compose pulls the digest-pinned ghcr.io/blinklabs-io/cardano-txpump:main image by default; set TXPUMP_IMAGE to select another image tag. Its source and container build live in the cardano-txpump repository.

The dedicated --leios runner uses the all-Dingo topology with testnet-dingo-leios.yaml: Dijkstra is active from genesis, test-only Leios keys are registered on the three pools, and a producer-to-peer scenario checks EB notification, relay fetch, ranking-header reference, and ledger application. It selects that feature scenario only; it does not change the canonical Conway specs or the --accelerated timeline.

The Leios spec uses 300-slot, 150-second epochs with f: 0.05 and k: 3. Its 240-slot nonce stability window contains 12 expected blocks, which keeps the finite era horizon advancing while leaving half of the ten-minute scenario after voting registrations become usable in epoch 2.

Network parameters

Dingo mode reads testnet-dingo.yaml, conformance mode reads testnet.yaml. Notable values shared by both:

  • networkMagic: 42
  • epochLength: 500 slots, slotLength: 1s (~8 min epochs)
  • activeSlotsCoeff: 0.4, securityParam (k): 40
  • All hard forks at epoch 0 — the network starts in Conway with protocol version 10.0.
  • systemStart is set to now + 30s by the configurator after key generation, so nodes have time to come up before slot 0.

Dingo mode differs in pool count and pledge: poolCount: 3, poolPledge: 0, delegatedSupply: 2100000000000 (divisible by 3, required by the generator). The zero pledge is deliberate — it's what makes the CIP-50 pledge-leverage scenario meaningful (see Test scenarios below).

Accelerated network parameters

--accelerated swaps the network spec for a compressed-timing variant — testnet-dingo-accelerated.yaml (dingo mode) or testnet-accelerated.yaml (conformance mode) — so a whole scenario fits the reference-runner budget:

  • epochLength: 120 slots, slotLength: 0.5s → 60s epochs
  • activeSlotsCoeff: 0.4, securityParam (k): 10 → a block every ~1.25s
  • Byron protocolConsts.k tracks securityParam
  • Everything else (network magic, hard forks at epoch 0, supply) matches the canonical specs.

Shortening an epoch is only safe if the stability windows still fit inside it, and both are derived from k and f rather than configured directly:

Window Formula Accelerated Canonical
nonce stability 4k/f 100 slots 400 slots
blockfetch stability 3k/f 75 slots 300 slots
epoch length — 120 slots 500 slots

DevNetConfig.Validate() enforces 4k/f < epochLength and 3k/f < epochLength, and TestCheckedInSpecsAreValid in internal/test/devnet/config_test.go runs it against every checked-in spec. That test carries only the linux constraint, so an edit that shrinks an epoch without shrinking k fails in the ordinary Linux go test ./... run instead of turning into a DevNet stall nobody can explain.

The canonical specs are deliberately not accelerated: they are what soak and canary runs use, and TestCanonicalSpecsKeepCanonicalTiming fails if someone quietly speeds them up.

Docker runtime prerequisites

  • Docker with the Compose plugin (docker compose version must work).
  • Go 1.26+ on the host (matching go.mod) to run the integration tests.
  • Outbound network access on first run to pull ghcr.io/blinklabs-io/cardano-node:11.0.1 (conformance mode only) and to clone cardano-foundation/testnet-generation-tool inside the configurator image.
  • Local build of Dingo via the repo root Dockerfile — Compose builds it automatically on up / run-tests.sh.
  • Pull of the digest-pinned ghcr.io/blinklabs-io/cardano-txpump:main image for the transaction generator; set TXPUMP_IMAGE to select a different tag.

Building Dingo

Apple Container on macOS

run-tests.sh optionally uses Apple Container on an Apple silicon Mac with the Apple Container CLI installed and its system running. A host Go installation and host Docker are not required. Docker remains the default runtime:

container system start
./internal/test/devnet/run-tests.sh --runtime container --accelerated
./internal/test/devnet/run-tests.sh --runtime container --accelerated --conformance
# Equivalent environment selection; an explicit --runtime flag takes precedence:
DEVNET_RUNTIME=container ./internal/test/devnet/run-tests.sh --accelerated

This runs Docker Engine and Compose inside an Apple Container Linux VM. Docker Desktop and a host Docker daemon are not required. The existing Linux harness, bridge networking, health checks, named volumes, and disruption tests run unchanged inside that VM. Apple Container 1.4.1 is the tested CLI version; the runner needs its capability and path-mask options for the nested engine. The Docker API listens only on a Unix socket inside the VM.

The VM receives 6 CPUs and 12 GiB of RAM by default. Set DEVNET_CONTAINER_CPUS and DEVNET_CONTAINER_MEMORY before the first run to change those allocations. A stopped VM retains its filesystem, including images, build caches, and the Go toolchain selected by this repository. It is reused for the same checkout and stopped after tests finish. Delete the stopped runner with container delete <runner-name> to reclaim its disk or change its resource allocation. The runner name is printed at startup. Concurrent runs against the same checkout are rejected by a lock directory.

The checkout is mounted at the same absolute path inside Linux. Failure artifacts default to .devnet/apple-container/run.XXXXXX on the Mac and survive VM shutdown. DEVNET_ARTIFACT_DIR must be inside the checkout. Other file-path overrides must also refer to files available inside the VM; arbitrary host directories are not mounted. Test configuration environment variables (DEVNET_*, COMPOSE_PROJECT_NAME, COMPOSE_PROFILES, and MODE) are forwarded.

--keep-up leaves both the network and VM running after success. Use container exec -e COMPOSE_PROJECT_NAME=<project-name> -w <checkout-path> <runner-name> bash internal/test/devnet/stop.sh (using the project name selected for the run and adding --conformance if selected) to tear the network down, then container stop <runner-name> to stop the VM. Run manual queries inside the VM with container exec; its DevNet ports are not published to macOS.

Docker

You do not need to make build Dingo locally — Compose builds the Dingo node images (dingo-1/dingo-2/dingo-3/dingo-relay in dingo mode, dingo-producer in conformance mode) directly from the repo by referencing the top-level Dockerfile (build.context: ../../.. in docker-compose.yml). The build runs make build inside the Go builder image pinned by that Dockerfile against your working tree, so any uncommitted local changes are included.

Force a rebuild after editing Dingo source:

# from this directory (internal/test/devnet), dingo mode
docker compose -f docker-compose.yml build dingo-1 dingo-2 dingo-3 dingo-relay

# conformance mode
docker compose -f docker-compose.yml build dingo-producer

run-tests.sh always rebuilds the active profile's images before starting (docker compose build, scoped to COMPOSE_PROFILES). start.sh does not — it only runs up -d, so add an explicit build (or pass --build to up) when you want fresh code in the running containers.

The configurator image (Dockerfile.configurator) is built from this directory and pinned to upstream testnet-generation-tool@v0.1.0; it rarely needs rebuilding once cached.

Where to run from

start.sh, stop.sh, and run-tests.sh all resolve their own location via SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)", so they work from any current working directory:

# from this directory
./start.sh

# from the repo root
./internal/test/devnet/start.sh

# from anywhere
/home/me/dingo/internal/test/devnet/run-tests.sh -run TestBasicBlockForging

Plain docker compose invocations are not location-independent — pass -f <path>/docker-compose.yml if you are not in this directory, since the compose file's build.context paths (. and ../../..) are resolved relative to the compose file itself, not your shell's CWD.

The block replay validation path is opt-in, matching production defaults. Set both passthroughs when a change needs that path in either DevNet profile:

DEVNET_BLOCK_PIPELINE_ENABLED=true \
DEVNET_BLOCK_PIPELINE_VALIDATE_ENABLED=true \
  ./run-tests.sh --accelerated

The scripts must, however, stay at this path inside the repo: the compose file references ../../.. to pick up the Dingo source tree for its build context, and the topology/*.json and testnet*.yaml files are mounted by relative path. Don't copy the directory elsewhere expecting it to work standalone.

Manual usage

./start.sh               # dingo mode (default): 3 producers + relay
./start.sh --conformance # conformance mode: dingo + cardano-node
./start.sh --accelerated # bring the network up on the accelerated spec
./stop.sh                 # tear down the default dingo network
./stop.sh --conformance   # tear down the conformance network
./stop.sh --accelerated   # accepted and ignored; teardown is spec-independent

Both scripts set COMPOSE_PROFILES for you (dingo or conformance), so docker compose up -d / down -v only touch the services in the selected profile. Tail logs while it runs:

docker compose -f docker-compose.yml logs -f
docker compose -f docker-compose.yml logs -f dingo-1

# conformance profile
COMPOSE_PROFILES=conformance docker compose -f docker-compose.yml logs -f
COMPOSE_PROFILES=conformance docker compose -f docker-compose.yml logs -f cardano-producer

Open a shell inside a node:

docker exec -it dingo-1 sh

# conformance mode
docker exec -it cardano-producer bash

Each Dingo node's socket is on its own named *-ipc volume at /ipc/dingo.socket (dingo-1-ipc, dingo-2-ipc, dingo-3-ipc, dingo-relay-ipc in dingo mode; dingo-producer-ipc in conformance mode); the cardano-node sockets live on each node's *-ipc volume at /ipc/node.socket. These are container-internal named volumes, not host bind mounts — see LocalStateQuery access below for how the host reaches a node's NtC endpoint.

Preview observability and Koios parity

The koios-parity Compose profile runs a separate Dingo node syncing Preview with Prometheus and Grafana. It shares this DevNet Compose file and project layout, while keeping its database and monitoring data in separate volumes. The profile can run alongside either generated DevNet profile. It uses the node's embedded Preview configuration and does not use the local DevNet genesis network.

From this directory, select only the Preview profile to start or stop it:

COMPOSE_PROFILES=koios-parity docker compose up -d
COMPOSE_PROFILES=koios-parity docker compose down -v

Set DEVNET_KOIOS_PARITY_ENABLED=true before startup to enable reward parity checks against Preview Koios. It defaults to false. Grafana is available on 127.0.0.1:13930, Prometheus on 127.0.0.1:13900, and the node's metrics endpoint on 127.0.0.1:13798. Override these ports with DEVNET_KOIOS_PARITY_GRAFANA_PORT, DEVNET_KOIOS_PARITY_PROMETHEUS_PORT, DEVNET_KOIOS_PARITY_METRICS_PORT, and DEVNET_KOIOS_PARITY_RELAY_PORT when they are already in use. The Grafana dashboards show Preview sync progress and Koios parity results. Grafana's default login is admin / admin; set DEVNET_GRAFANA_ADMIN_USER and DEVNET_GRAFANA_ADMIN_PASSWORD before startup, and change them before exposing Grafana beyond localhost.

Reward parity is checked only at closed-epoch boundaries. Preview epochs last about five days, so a short sync can show the observer enabled without reaching its first parity check.

Running the integration tests

run-tests.sh is the entry point for a complete native-Linux DevNet run:

./run-tests.sh                              # dingo mode (default): bring up, run devnet tests, tear down
./run-tests.sh --conformance                 # conformance mode: dingo + cardano-node
./run-tests.sh --accelerated                # fast timeline, including Conway governance scenarios
./run-tests.sh --accelerated --conformance  # the same timeline against the reference topology
./run-tests.sh --leios                      # Dijkstra/Leios producer-to-peer path
./run-tests.sh -run TestBasicBlockForging   # forward -run (and other flags) to `go test`
./run-tests.sh --keep-up                    # leave the network running on success (for poking around)
DEVNET_MEMPOOL_PROVIDER=dag ./run-tests.sh  # exercise the DAG mempool on every Dingo node

What it does:

  1. Builds the images for the selected profile (dingo by default, conformance with --conformance).
  2. docker compose up -d and waits up to 120s for all nodes in the profile to report healthy (four nodes in dingo mode: dingo-1, dingo-2, dingo-3, dingo-relay; three in conformance mode).
  3. Verifies the profile's txpump service (txpump-dingo or txpump) is running.
  4. Dingo mode only: copies the genesis stake keys out of the utxo-keys Docker volume to a host temp directory and exports DEVNET_STAKE_KEYS_DIR, so the CIP-50 scenario can load them. This is best-effort — if the volume or keys aren't found, the copy is skipped and the CIP-50 scenario just skips (see Test scenarios below).
  5. From the repo root, runs go test -tags "<mode tags>" -count=1 -v ./internal/test/devnet/... — devnet alone in dingo mode, devnet devnet_conformance in conformance mode.
  6. Tears the network down. On failure it dumps the last 100 lines of compose logs first, then preserves the full failure evidence (see Failure artifacts below). It also checks that txpump logged at least one accepted submission; zero accepted submissions fails the run even if the Go tests passed.

It also exports DEVNET_TESTNET_YAML pointing at the spec it actually brought the network up with, so devnet.LoadDevNetConfig() in the tests reads the same parameters the configurator generated genesis from rather than falling back to testnet.yaml.

Accelerated scenario timeline

--accelerated is the fast path used for scheduled and release integration evidence. It brings the network up on the accelerated spec and runs TestAcceleratedScenarioTimeline and, in all-Dingo mode, TestConwaySPORatificationUsesBoundaryMarkAndEnactsNextEpoch instead of the full suite.

What makes it fast is not just the shorter slots. The canonical suite queries each node's tip by opening a fresh Node-to-Node connection every couple of seconds, because a short-lived ChainSync client only ever reports the tip it captured when it intersected. The accelerated scenario instead opens one persistent ChainSync session per node and follows the chain, so RollForward and RollBackward arrive as the nodes produce them. Assertions are then conditions over those observed events bounded by a context deadline — there is no polling interval and no time.Sleep anywhere in the synchronisation path.

The second difference is that every assertion shares one timeline. Each phase's deadline is measured from the scenario's start rather than from the end of the previous phase, so a phase that finishes early hands its slack forward instead of each test paying for its own relative slot window:

Phase What it verifies
readiness Every node accepted a ChainSync session and sent a header
propagation A block forged after the baseline reaches every node, including the relay, with matching hashes; and one carrying transactions does too
agreement Every node agrees on the block hash at the deepest slot they have all observed
epoch-transition The chain crosses the next epoch boundary and the nodes agree on a header built above it (exercising the new epoch nonce)
peer-interruption A producer is stopped, the network visibly advances without it, and it rejoins and reconverges
relay-restart The same for the relay

Both disruption phases hold the node down until the rest of the network has advanced a derived number of blocks — k/2, so the outage stays inside what the security parameter can reconcile — rather than for a fixed wall-clock time. That keeps the disruption equally meaningful on a fast and a slow runner.

The whole schedule is derived from the network spec by devnet.NewScenarioPlan. For the accelerated spec it works out as readiness<=45s propagation<=1m10s agreement<=1m22.5s epoch-transition<=2m50s peer-interruption<=3m33.75s relay-restart<=4m17.5s — a 4m17.5s worst case against a 5-minute hard timeout. Those are ceilings, not expected times; the run ends as soon as the last condition is observed. Both bounds are asserted without Docker by TestAcceleratedPlanFitsReferenceBudgetAndCanonicalDoesNot, which also asserts that the canonical spec does not fit — so the fast scenario can never be quietly pointed at the soak configuration.

Reference runner: a 4-vCPU / 16 GB Linux runner (GitHub Actions ubuntu-latest class) with images already built. Image build time is excluded from the budget; run-tests.sh rebuilds before starting, and the scenario's clock only starts once the containers are healthy.

The scenario runs in both topologies: it derives the producer it interrupts and the relay it restarts from LoadEndpoints(), so --accelerated gives the all-Dingo network and --accelerated --conformance gives Dingo beside cardano-node. In both cases the node it interrupts is the last producer, which is never the node txpump submits to, so mempool traffic keeps flowing through the outage.

To drive it by hand against a network you brought up yourself:

./start.sh --accelerated

start.sh --accelerated prints the exact go test command to run next, including the DEVNET_COMPOSE_PROJECT and DEVNET_*_ADDR values for whichever project/ports this run actually landed on — devnet_ports may have derived non-default host ports, so copy that printed command rather than retyping one with the default localhost:3010-style addresses, which would connect to the wrong ports. Then:

./stop.sh --accelerated

TestAcceleratedScenarioTimeline skips unless DEVNET_ACCELERATED=1 is set, so it never runs against the canonical-timing network, where its budget could not be met.

Dijkstra/Leios producer-to-peer scenario

--leios selects testnet-dingo-leios.yaml, runs the all-Dingo profile, registers a valid test-only Leios key in each generated pool registration, starts each Dingo node in explicit leios run mode and Dijkstra start era, and activates Dijkstra at epoch 0. Its transaction pump submits signed Dijkstra payments so the endorser block contains ledger-valid transactions. The pump keeps their outputs unavailable for 1000 slots so it cannot spend them before the scenario queries the relay ledger; this wallet delay is not confirmation evidence. The test observes an endorser-block offer over LeiosNotify, fetches its manifest and every referenced body from the non-forging relay over LeiosFetch, matches the ranking header's announcement to that EB, and queries the relay's ledger for an output from the fetched transactions. This exercises the producer, network, and ledger path through Dingo's interfaces and protocol outputs.

Run the complete scenario with:

./run-tests.sh --leios

The devnet-leios workflow runs this scenario daily and on manual dispatch, and uploads the failure artifacts described below when it fails.

For a manually started network, ./start.sh --leios prints the environment and Go command for the matching producer-to-peer test. The test-only signing key and proof are paired fixtures under testdata/; a focused unit test checks that the registered proof and signing key agree. The configurator places the secret only in the generated local pool-key volume.

Failure artifacts

When a run fails, run-tests.sh keeps DEVNET_ARTIFACT_DIR (a temp directory it creates, or one you set) and prints its path instead of deleting it. On success the directory is removed.

Teardown is best-effort and never replaces the test exit status. In dingo mode, the temporary stake-key copy is written with the host uid/gid so the runner can remove its own temporary tree after either result.

Path Contents
network/container-status.txt docker compose ps --all for the profile
network/compose.log Full compose logs for every service
network/generated-configs/ The genesis and node configuration the configurator generated
network/testnet*.yaml The network spec the run used
<scenario>/observed-chains.json Every node's observed chain: tip, retained headers, roll-forward/roll-backward counts, deepest rollback, connect/disconnect churn
<scenario>/container-status.txt Container status as the scenario saw it
<scenario>/<service>.log Per-service logs captured by the scenario, capped at the last CapturedLogTailLines lines

<scenario> is accelerated-timeline for the accelerated timeline, which captures explicitly through NodeControl.CaptureFailureArtifacts, and the Go test name for every canonical scenario, which NewTestHarness wires up on its own. A subtest renders as parent/child, so the separator is percent-escaped to keep the evidence in one directory per test, as is anything a filesystem would reject or rewrite -- Windows refuses : * ? " < > | and strips a trailing dot or space, which would otherwise put two scenarios in one directory. A name built from Go identifiers is used as written. Case is left alone, so directories stay readable; on a filesystem that folds case, two scenario names differing only in case would share one.

Observation and writing happen at different times. Chain observers run for the whole of a scenario, so observed-chains.json is a continuous record of what every node's chain did while that scenario was failing. The files themselves are written from the scenario's t.Cleanup, which runs once that test finishes and before the network is torn down, so container status and the service logs describe the network as it stood at the end of the failing scenario rather than at the instant of the failure.

The network/ entries, by contrast, are collected once after the whole run, by which point a chain that stalled and then recovered looks healthy again. That is the difference that separates a stall nobody forged through from one nobody propagated.

The per-scenario service logs are capped, because a DevNet node logs at debug level and emits tens of megabytes a minute; network/compose.log is the complete, uncapped record for the whole run, so the per-scenario copy only carries the window around the failure.

Capture stays off unless DEVNET_ARTIFACT_DIR is set and the topology names containers, so a harness built over endpoints that are never dialled does not reach for Docker.

The harness reads endpoint addresses from mode-specific environment variables that run-tests.sh sets based on the host port mappings (dingo mode: DEVNET_DINGO1_ADDR, DEVNET_DINGO2_ADDR, DEVNET_DINGO3_ADDR, DEVNET_DINGO_RELAY_ADDR; conformance mode: DEVNET_DINGO_ADDR, DEVNET_CARDANO_ADDR, DEVNET_RELAY_ADDR). Tests also re-parse the mode's testnet*.yaml at runtime via devnet.LoadDevNetConfig(), so changes to network parameters flow through without code edits.

LocalStateQuery access

The host Go test harness reaches a Dingo node's LocalStateQuery interface over node-to-client (NtC) Ouroboros, dialed as plain TCP — not a UNIX socket. Each Dingo node sets DINGO_PRIVATE_BIND_ADDR=0.0.0.0 (private/NtC port 3002 inside the container), and docker-compose.yml maps that port to a host port per node: dingo-1 → 3020, dingo-2 → 3021, dingo-3 → 3022, dingo-relay → 3023 (override with DEVNET_DINGO1_NTC_ADDR etc., or the DEVNET_DINGO{1,2,3}_NTC_PORT / DEVNET_DINGO_RELAY_NTC_PORT host port vars). internal/test/devnet/endpoints_dingo.go's DingoNtcAddrs() returns these as a map[string]string keyed by node name, and internal/test/devnet/lsq.go's RewardAccountsByNtc(addr, magic, creds) dials that address with ouroboros.New(...).DialTimeout("tcp", addr, ...) and queries delegations + reward balances via GetFilteredDelegationsAndRewardAccounts.

This TCP-based approach replaced an earlier unix-socket host bind-mount design: the Dingo image runs as uid 1000 and cannot bind a socket inside a host-owned bind mount, so the in-container socket (used by txpump and the healthcheck) stays on a named Docker volume, and the host harness talks NtC over TCP instead. There is no host socket bind mount and no DEVNET_IPC_DIR environment variable.

LocalStateQuery in conformance mode

Conformance mode exposes NtC for both producers, so ledger state (not just chain tip/growth) can be compared against the reference implementation (TestLedgerStateConsensus, blinklabs-io/dingo#1900):

  • dingo-producer (in --conformance mode) exposes NtC the same way dingo mode's nodes do (DINGO_PRIVATE_BIND_ADDR=0.0.0.0 on private port 3002), mapped to host port 3030 unless overridden with DEVNET_DINGO_NTC_ADDR or DEVNET_DINGO_NTC_PORT.
  • cardano-producer (in --conformance mode) has no built-in TCP NtC support — cardano-node only serves LocalStateQuery over its unix socket. The ghcr.io/blinklabs-io/cardano-node image's run-node entrypoint bridges this: setting SOCAT_PORT spawns a background socat TCP-LISTEN:<port>,fork UNIX-CLIENT:<socket-path> inside the container. docker-compose.yml sets SOCAT_PORT: "3002" on cardano-producer (the same private-port number Dingo uses, purely by convention — it has no special meaning to cardano-node), mapped to host port 3031 unless overridden with DEVNET_CARDANO_NTC_ADDR or DEVNET_CARDANO_NTC_PORT.

internal/test/devnet/endpoints_conformance.go's DingoProducerNtcAddr() and CardanoProducerNtcAddr() return these host addresses, and internal/nodeparity's SnapshotAtTip(addr, magic) (shared with cmd/node-parity; this scenario is not the only caller) queries one node's current protocol parameters, stake distribution, and whole UTxO set (normalized into a comparable form) via a single acquired LocalStateQuery session; DiffSnapshots(a, b) reports every divergence between two such snapshots.

TestLedgerStateConsensus additionally queries treasury/reserves and absolute mark/set/go stake snapshots at the same acquired point. Its samples cover epochs 1 and 2: the first update must leave the d=0 genesis treasury empty, and the next must use epoch 0's block production. Both samples compare pots and stake distribution with cardano-node. Run it on a fresh network so the epoch-1 sample is still available:

./internal/test/devnet/run-tests.sh --conformance --accelerated -run TestLedgerStateConsensus

The runner compiles the host-side tests before starting the nodes so a cold build cannot consume the bootstrap sampling window.

GetStakeDistribution and GetUTxOWhole (the two queries SnapshotAtTip needs beyond the ones already used elsewhere in this harness) did not have server-side support in Dingo before this scenario — they were part of the // TODO (#394) block in ledger/queries.go's query dispatcher. They are implemented in ledger/queries_stakedistribution.go and ledger/queries_utxowhole.go, closing out #394 for those two query types specifically (the rest of that TODO block is unrelated to this scenario and remains open).

Dingo's LocalStateQuery server (ouroboros/localstatequery.go) answers every query in a session from a ledger snapshot opened at Acquire, and Acquire(point) reads at that block for every query this scenario issues (protocol parameters, stake distribution, whole UTxO set, account state and stake snapshots). Each TestLedgerStateConsensus sample therefore acquires the trailing node's tip as a specific point on both producers, so both answers describe the same block however far either node advances during the queries. If the two nodes have forked, the leading node's Acquire fails with point-not-on-chain and the sample retries with a new candidate; a candidate at or past the end of the intended epoch fails the test instead. The scenario samples rather than replaying every block because each sample walks the whole UTxO set on both nodes.

Running the CIP-50 pledge-leverage scenario

TestCIP50PledgeLeverageRewardEffect (dingo mode only) is long-running: it waits into epoch 4 so the delayed reward update (applied from the stake snapshot three epochs back, see ledger/chainsync.go) has had a chance to credit rewards, which takes roughly 35 minutes per pass at epochLength=500, slotLength=1s. It is gated behind DEVNET_CIP50_TEST=1 and never runs as part of the default suite, and it requires two independent passes against two separately launched networks to be meaningful:

# 1. baseline: bring up the network with leverage off (the default), then run
./start.sh
DEVNET_CIP50_TEST=1 DEVNET_STAKE_KEYS_DIR=/path/to/stake/keys \
  go test -tags devnet -run TestCIP50PledgeLeverageRewardEffect -timeout 50m \
  ./internal/test/devnet/scenarios/
./stop.sh

# 2. leveraged: bring up the network with pledge leverage enabled, then run
DEVNET_DINGO_PLEDGE_LEVERAGE_ENABLED=true ./start.sh
DEVNET_CIP50_TEST=1 DEVNET_DINGO_PLEDGE_LEVERAGE_ENABLED=true \
  DEVNET_STAKE_KEYS_DIR=/path/to/stake/keys \
  go test -tags devnet -run TestCIP50PledgeLeverageRewardEffect -timeout 50m \
  ./internal/test/devnet/scenarios/
./stop.sh

The baseline pass expects total member rewards greater than zero (rewards flow without a cap); the leveraged pass expects every delegated credential's reward to be exactly zero (the leverage cap with zero pledge zeroes reward-eligible stake). run-tests.sh does not run this scenario by default; setting DEVNET_CIP50_TEST=1 can trigger one pass when the stake-key copy succeeds. A meaningful baseline-versus-leveraged comparison still requires the two separately launched passes shown above. DEVNET_STAKE_KEYS_DIR must point at the genesis stake key directory exposed by the configurator (copy it out of the utxo-keys Docker volume, the same way run-tests.sh does for its own runs).

Test scenarios

All scenarios live in internal/test/devnet/scenarios/ and use the harness in internal/test/devnet/.

Generic scenarios (//go:build linux && devnet) run in both dingo and conformance mode:

Test What it verifies
TestBasicBlockForging Dingo forges at least one block and all nodes converge within securityParam
TestDingoChainAdvances Dingo's tip slot advances over a short observation window
TestChainGrowthRate Block production rate is within the expected range derived from activeSlotsCoeff and slotLength
TestRelayPropagation Blocks reach the non-forging relay
TestSustainedConsensus All nodes stay in agreement across multiple sampling intervals
TestEpochBoundaryConsensus All nodes remain in consensus across at least one epoch boundary (exercises candidate-nonce freeze, lab nonce roll, and new-epoch VRF verification)
TestAcceleratedScenarioTimeline The accelerated scenario timeline: readiness, block and transaction propagation, chain agreement, an epoch transition, a peer interruption with recovery, and a relay restart — all on one shared clock, driven by streamed ChainSync events. Skipped unless DEVNET_ACCELERATED=1; see Accelerated scenario timeline above.
TestConwaySPORatificationUsesBoundaryMarkAndEnactsNextEpoch Moves delegated stake across pools, ratifies and enacts an above-threshold no-confidence action, then checks that its below-threshold committee-update child remains active. Requires the accelerated all-Dingo run and its exposed test keys.

Reference-conformance scenario (//go:build linux && devnet && devnet_conformance) runs only with --conformance:

Test What it verifies
TestCardanoProducerChainAdvances cardano-producer's tip advances (sanity check on the reference node)
TestLedgerStateConsensus dingo-producer's and cardano-producer's ledger state (protocol parameters, stake distribution, whole UTxO set) match at a point acquired on both nodes in epochs 1 and 2; fails with a diagnostic naming every divergence found

Dingo-only feature scenario (//go:build linux && devnet && !devnet_conformance) runs only in the default dingo mode — no cardano-node reference exists for this feature:

Test What it verifies
TestCIP50PledgeLeverageRewardEffect With poolPledge: 0, compares a leverage-off baseline (member rewards greater than zero by epoch 4) against a leveraged pass (member rewards exactly zero for every delegated stake credential). Skipped unless DEVNET_CIP50_TEST=1 is set; requires two separately launched networks to run both passes (see above).
TestLeiosEndorserBlockProducerToPeer On the dedicated Dijkstra network, follows a producer EB offer through LeiosNotify, relay LeiosFetch, a Dijkstra ranking-header reference, and presence of a fetched transaction output in the relay ledger. Runs only via run-tests.sh --leios.

Port and address overrides

start.sh/run-tests.sh derive a worktree-specific block for all 13 variables below by default (see devnet_ports in Cleanup), so the defaults in this table only apply to a bare docker compose up or a single-worktree run with DEVNET_*_PORT set explicitly. Set any one of them to opt out of auto-derivation entirely and take full manual control (via environment or a local .env file next to docker-compose.yml; the checked-in .env sets COMPOSE_PROFILES=dingo as the default).

Dingo mode:

Variable Default Used by
DEVNET_DINGO1_PORT 3010 docker-compose host port for dingo-1 NtN
DEVNET_DINGO2_PORT 3013 docker-compose host port for dingo-2 NtN
DEVNET_DINGO3_PORT 3014 docker-compose host port for dingo-3 NtN
DEVNET_DINGO_RELAY_PORT 3015 docker-compose host port for dingo-relay NtN
DEVNET_DINGO1_NTC_PORT 3020 docker-compose host port for dingo-1 NtC
DEVNET_DINGO2_NTC_PORT 3021 docker-compose host port for dingo-2 NtC
DEVNET_DINGO3_NTC_PORT 3022 docker-compose host port for dingo-3 NtC
DEVNET_DINGO_RELAY_NTC_PORT 3023 docker-compose host port for dingo-relay NtC
DEVNET_DINGO{1,2,3}_ADDR, DEVNET_DINGO_RELAY_ADDR localhost:<port above> run-tests.sh-derived NtN addresses for the Go harness
DEVNET_DINGO{1,2,3}_NTC_ADDR, DEVNET_DINGO_RELAY_NTC_ADDR localhost:<port above> NtC addresses for DingoNtcAddrs()
TEST_TIMEOUT 20m go test -timeout in run-tests.sh

Conformance mode:

Variable Default Used by
DEVNET_DINGO_PORT 3010 docker-compose host port for Dingo NtN
DEVNET_CARDANO_PORT 3011 docker-compose host port for cardano NtN
DEVNET_RELAY_PORT 3012 docker-compose host port for relay NtN
DEVNET_DINGO_NTC_PORT 3030 docker-compose host port for dingo-producer NtC
DEVNET_CARDANO_NTC_PORT 3031 docker-compose host port for cardano-producer NtC (bridged by socat)
DEVNET_DINGO_NTC_ADDR localhost:<port above> DingoProducerNtcAddr() override
DEVNET_CARDANO_NTC_ADDR localhost:<port above> CardanoProducerNtcAddr() override
DINGO_PORT falls back to DEVNET_DINGO_PORT run-tests.sh only
CARDANO_PORT falls back to DEVNET_CARDANO_PORT run-tests.sh only
RELAY_PORT falls back to DEVNET_RELAY_PORT run-tests.sh only

run-tests.sh derives the DEVNET_*_ADDR variables from these so the test harness and the compose port mappings always agree.

Files

File Purpose
docker-compose.yml Service, volume, and network definitions for the dingo, conformance, and koios-parity profiles
Dockerfile.configurator Builds the genesis/key generator image (cardano-foundation/testnet-generation-tool v0.1.0)
configurator.sh Runs inside the configurator: drives genesis-cli.py, builds ring topology (mode-aware pool count via DINGO_POOL_IDS), sets systemStart, relaxes key permissions for non-root node containers and chowns each Dingo pool's keys to DINGO_UID/DINGO_GID (passed in by compose, defaulting to the 1000:1000 pinned in the repo root Dockerfile)
testnet-dingo.yaml Canonical network spec for dingo mode: 3 pools, poolPledge: 0
testnet.yaml Canonical network spec for conformance mode: 2 pools
testnet-dingo-accelerated.yaml Accelerated dingo-mode spec: 60s epochs, k: 10, 0.5s slots
testnet-accelerated.yaml Accelerated conformance-mode spec: same timing, 2 pools
testnet-dingo-leios.yaml Dijkstra/Leios all-Dingo integration spec: Dijkstra at genesis, 300-slot epochs, 0.5s slots
topology/dingo-1.json, dingo-2.json, dingo-3.json, dingo-relay.json Static peer lists for dingo mode
topology/dingo-producer.json, cardano-producer.json, relay.json Static peer lists for conformance mode
.env Sets the default COMPOSE_PROFILES=dingo
compose-project.sh Derives a stable, worktree-specific Compose project name, a collision-checked bridge subnet and host port block, and a rendered topology directory; wraps docker compose up with a retry on subnet collision
start.sh / stop.sh Convenience wrappers around docker compose up -d / down -v; accept --conformance
run-tests.sh Full native-Linux bring-up → test → tear-down runner; accepts --conformance, --keep-up, and forwards other flags to go test
TXPUMP_IMAGE (default is digest-pinned ghcr.io/blinklabs-io/cardano-txpump:main) Transaction generator container image
harness.go Go test harness: Ouroboros NtN client, tip queries, consensus checks, the WaitForChainStart genesis gate, and per-scenario failure capture (build tag devnet)
config.go testnet*.yaml loader, derived timings, and spec validation (linux)
chainstate.go Observed-chain state machine: applies RollForward/RollBackward, tracks tip and retained headers, and exposes cross-node agreement helpers and bounded-context conditions (linux)
timeline.go ScenarioPlan: derives the accelerated scenario's phases, deadlines, outage length, and hard timeout from the network spec (linux)
observer.go Persistent per-node ChainSync sessions feeding chainstate.go, with automatic reconnect across container restarts (build tag devnet)
nodectl.go Stops/starts compose services for the disruption phases and supplies the Docker side of failure capture (build tag devnet)
artifacts.go What a failed scenario preserves and where: capture planning and artifact writing (linux)
endpoints.go The NodeEndpoint description shared by the harness, observers, and failure capture (linux)
endpoints_dingo.go Dingo-mode node endpoints and NtC addresses (//go:build devnet && !devnet_conformance)
endpoints_conformance.go Conformance-mode node endpoints, plus DingoProducerNtcAddr() / CardanoProducerNtcAddr() (//go:build devnet && devnet_conformance)
lsq.go RewardAccountsByNtc / RewardAccountsByNtcForCreds: LocalStateQuery over NtC TCP (build tag devnet)
credentials.go Loads genesis stake credentials for the CIP-50 scenario (build tag devnet)
harness_test.go, credentials_test.go Tests for the harness/credential helpers themselves (build tag devnet)
scenarios/ Devnet test scenarios (one or more Test* per file, gated per the Test scenarios table above)

Cleanup

start.sh, stop.sh, and run-tests.sh derive COMPOSE_PROJECT_NAME from the worktree's absolute path. Compose therefore gives each worktree distinct containers, networks, volumes, and project state. Set COMPOSE_PROJECT_NAME explicitly to run more than one DevNet in the same worktree. NodeControl receives that project as DEVNET_COMPOSE_PROJECT, resolves service containers inside it, and still uses direct docker stop/docker start for disruption phases.

A distinct network name isn't enough on its own: the compose network's subnet is a separate axis, and Docker refuses to create two networks with an overlapping subnet even under different project names ("Pool overlaps with other one on this address space"). The same three scripts also derive DEVNET_NET_BASE, a 172.24-172.31.x /24, and use it for the compose network's subnet and each service's static IP — the topology still needs static IPs because the peer lists in topology/*.json are static, checked-in files. devnet_render_topology rewrites a worktree-local copy of those files with the run's DEVNET_NET_BASE into DEVNET_TOPOLOGY_DIR, which docker-compose.yml mounts instead of the checked-in ./topology (its fallback default, so a bare docker compose up without going through these scripts still works — for a single run at a time).

A worktree-path hash only picks a starting subnet: two worktrees can hash to the same one, and this range isn't reserved for DevNet, so an unrelated Docker network could already sit on part of it. devnet_net_base actually checks every subnet docker network ls/inspect currently reports and walks forward from the hash to a genuinely free /24 (falling back to the hash alone if Docker isn't reachable). A residual race remains between that check and the network actually being created — devnet_compose_up (used by start.sh/run-tests.sh in place of a bare docker compose up -d) covers it: on a "Pool overlaps" failure it recomputes DEVNET_NET_BASE (which now sees the winner's network and skips it), re-renders topology, and retries, up to 3 attempts. Set DEVNET_NET_BASE explicitly for the same reason you'd set COMPOSE_PROJECT_NAME.

Published host ports are a fourth axis Compose does not scope by project at all: two worktrees' default ports (3010/3013-3015, 3020-3023, and conformance's 3010-3012, 3030-3031) collide outright with "port is already allocated". devnet_ports derives a worktree-specific block of 13 ports (one hash-based starting point, actually checked against what's listening on 127.0.0.1 and shifted forward a whole block at a time until every port in it is free) and exports it as the DEVNET_*_PORT / DEVNET_*_NTC_PORT variables docker-compose.yml already reads. It's skipped entirely if the caller has already set any of those variables, so a manual port override stays in full manual control.

stop.sh and the run-tests.sh trap run docker compose down -v only for that project, removing its config and data volumes without touching another worktree's run, and remove that project's rendered topology directory. Genesis is regenerated on every start, so this is the desired default — there's no state worth preserving between runs. If a previous run left orphaned containers or volumes around, use the same project name:

COMPOSE_PROJECT_NAME=<project> docker compose -f docker-compose.yml down -v --remove-orphans
docker volume ls | grep devnet                       # inspect leftovers
docker network ls | grep dingo-devnet                # inspect leftover networks

Troubleshooting

  • Nodes don't reach healthy within 120s. run-tests.sh dumps docker compose ps and the last 100 log lines on timeout. Most often this is a build issue with the Dingo image or genesis-time skew (the configurator schedules systemStart 30s in the future; if your machine's clock is far off, regenerate by tearing down and starting again).
  • port is already allocated. Set the DEVNET_*_PORT overrides above.
  • Configurator fails on first run. It clones cardano-foundation/testnet-generation-tool at build time; check outbound network access and re-run docker compose build configurator (or configurator-dingo in dingo mode).
  • Dingo image is stale. run-tests.sh rebuilds the active profile's Dingo images on every invocation; for start.sh runs, force a rebuild with docker compose -f docker-compose.yml build (scoped by COMPOSE_PROFILES).
  • A single scenario fails on its own but the whole suite passes. Every scenario now calls WaitForChainStart before taking a baseline, so this should no longer happen. The cause was that the configurator schedules systemStart 30s after it exits while the compose health checks pass as soon as a node opens its socket: a node answers tip queries reporting slot 0 / block 0 for that whole window. Timeouts derived from slot counts only measure chain time, so charging one against the pre-genesis wait expired it before the chain produced anything — and a test only passed because an earlier one in the package had already absorbed the wait. If you add a scenario, gate it on WaitForChainStart rather than relying on ordering.
  • TestAcceleratedScenarioTimeline skips. It requires DEVNET_ACCELERATED=1, which run-tests.sh --accelerated sets. Running it against the canonical-timing network is deliberately prevented: its phase budgets are derived from the spec and cannot be met at epochLength: 500, slotLength: 1s.
  • The accelerated scenario fails at NewNodeControl. It needs to reach Docker Compose to stop and start nodes. Run it through run-tests.sh, which exports DEVNET_COMPOSE_FILE and DEVNET_COMPOSE_PROJECT, or set both variables yourself. It fails rather than skipping the disruption phases on purpose — a pass that quietly omitted them would not be release evidence.
  • TestCIP50PledgeLeverageRewardEffect skips. It requires DEVNET_CIP50_TEST=1 and a non-empty DEVNET_STAKE_KEYS_DIR to run at all, and needs two separately launched networks (leverage off, then DEVNET_DINGO_PLEDGE_LEVERAGE_ENABLED=true) to be meaningful; see Running the CIP-50 pledge-leverage scenario above.

Documentation

Overview

Package devnet provides a test harness for running integration tests against a private Cardano DevNet consisting of Dingo and cardano-node instances connected via Docker Compose.

The chain-observation types in this file carry only the package-wide Linux constraint, so the state machine driving the DevNet scenarios is covered by an ordinary Linux `go test ./...` run without Docker. The pieces that dial real nodes additionally require the `devnet` tag.

Index

Constants

View Source 
const (
	PhaseReadiness        = "readiness"
	PhasePropagation      = "propagation"
	PhaseAgreement        = "agreement"
	PhaseEpochTransition  = "epoch-transition"
	PhasePeerInterruption = "peer-interruption"
	PhaseRelayRestart     = "relay-restart"
)

Phase names on the shared scenario timeline.

View Source 
const CapturedLogTailLines = 2000

CapturedLogTailLines bounds the per-service logs a failed scenario copies out. A DevNet node logs at debug level and emits tens of megabytes a minute, so an unbounded copy per failed scenario is a multi-gigabyte artifact directory on a long canonical run. run-tests.sh separately preserves the complete compose log for the whole run under network/, so this copy only has to carry the window around the failure.

View Source 
const ReferenceRunnerBudget = 5 * time.Minute

ReferenceRunnerBudget is the wall-clock ceiling for one accelerated scenario, excluding image build, on the reference runner documented in internal/test/devnet/README.md. It is a hard timeout, not a target: the scenario is event-driven and normally finishes well inside it.

Variables

This section is empty.

Functions

func ArtifactDir added in v0.70.1 

func ArtifactDir() (string, bool)

ArtifactDir returns the directory failure evidence is written to, and whether one is configured. run-tests.sh creates it and preserves it on failure.

func ArtifactName added in v0.70.1 

func ArtifactName(testName string) string

ArtifactName encodes a Go test name as one path segment. t.Name() renders a subtest as parent/child, which would otherwise scatter a scenario's evidence across nested directories.

The separator is escaped rather than replaced, because replacing it is lossy: with the separator rewritten to an ordinary character, TestX/a-b and TestX/a/b both become TestX-a-b and two failing subtests write into one directory. Escaping keeps the mapping one-to-one, so distinct test names always get distinct directories -- a property of the encoding rather than a probability, which is what a digest of the name would give instead.

String-level injectivity only delivers that guarantee if the filesystem stores the name it is given. Windows does not: it rejects : * ? " < > | outright, and it strips a trailing dot or space, which would quietly land "." and ".." -- encoded here as %2E and %2E. -- in one directory. Those are escaped for the same reason "." and ".." are: not because they escape the segment, but because the filesystem reads them as something other than the name asked for.

Case is the one difference deliberately left unescaped, so the guarantee above is a guarantee on any filesystem that distinguishes case, and not on one that folds it: on macOS or Windows, TestX/a and TestX/A encode differently but land in one directory. Closing that would cost the readable name, which is the thing the encoding exists to protect -- escaping case renders TestSustainedConsensus as %54est%53ustained%43onsensus, and a digest suffix trades the exact name for a probability. DevNet captures on Linux, where case is significant, and two Go test names differing only in case is a naming problem of its own. TestArtifactNameKeepsCase pins this boundary.

Only what a filesystem would reject or rewrite is escaped, so a name built from Go identifiers comes back exactly as written. That is every scenario in the canonical suite, and it is what makes the directory findable from the name in the failure output.

func MaxBlockNumber 

func MaxBlockNumber(snaps []ChainSnapshot) uint64

MaxBlockNumber returns the highest observed block height across snapshots. The scenario uses it to require forward progress between disruption phases, so that each outage starts from a network that is demonstrably still producing rather than merely converged.

func MaxServerTipSlot 

func MaxServerTipSlot(snaps []ChainSnapshot) uint64

MaxServerTipSlot returns the highest slot any node reported as its own tip.

This is not the same as MaxTipSlot: an observer intersects at origin and replays history before it reaches the tip, so its observed tip lags during catch-up. Assertions about what the chain is doing *now* — a baseline for "forged after this point", or the next epoch boundary to cross — must use the peer's reported tip, or replayed history could satisfy them without the network having done anything.

func MaxTipSlot 

func MaxTipSlot(snaps []ChainSnapshot) uint64

MaxTipSlot returns the highest tip slot across snapshots.

func MinTipSlot 

func MinTipSlot(snaps []ChainSnapshot) uint64

MinTipSlot returns the lowest tip slot across snapshots, i.e. how far the slowest node has got.

func SlotsDuration 

func SlotsDuration(slots uint64, slotDuration time.Duration) time.Duration

SlotsDuration returns the wall-clock time n slots take. DevNet slot counts come from a checked-in spec and are small, but the conversion is clamped rather than trusted so a nonsensical spec cannot wrap a duration into the past.

func WriteFailureArtifacts added in v0.70.1 

func WriteFailureArtifacts(
	ctx context.Context,
	src ArtifactSource,
	plan FailureCapturePlan,
	snapshots []ChainSnapshot,
	logf func(format string, args ...any),
)

WriteFailureArtifacts writes the evidence a failed scenario needs to be diagnosable after the network is gone: what every node's chain actually did, which containers were up, and each service's logs.

It is best-effort by design — a capture error must not mask the test failure that triggered it, and one unreadable service must not cost the rest of the evidence — so problems are logged rather than returned. A nil src means the Docker side is unavailable; the observed chains are recorded in-process and are still written.

Types

type AgreementResult 

type AgreementResult struct {
	Slot   uint64
	Agree  bool
	Hashes map[string]string
}

AgreementResult reports whether a set of nodes agree on the chain at the deepest slot they have all observed.

func AgreementAtDeepestCommonSlot 

func AgreementAtDeepestCommonSlot(
	snaps []ChainSnapshot,
) (AgreementResult, bool)

AgreementAtDeepestCommonSlot picks the highest slot every snapshot observed a header for and compares the hashes there. Using a slot all nodes actually reached makes the check a deterministic expected point rather than a tolerance window: nodes at different tips still get compared on the chain they share, and a fork shows up as a hash mismatch instead of being masked by a slot-distance allowance.

The second return is false when the snapshots share no observed slot, which means the comparison could not be made at all.

func (AgreementResult) String 

func (r AgreementResult) String() string

String renders the per-node hashes for failure messages.

type ArtifactSource added in v0.70.1 

type ArtifactSource interface {
	ContainerStatus(ctx context.Context) (string, error)
	// Logs returns a service's container logs, limited to the last
	// tailLines lines when tailLines is positive.
	Logs(
		ctx context.Context,
		service string,
		tailLines int,
	) (string, error)
}

ArtifactSource supplies the Docker-side evidence a capture writes out. NodeControl implements it.

type ChainGroup 

type ChainGroup struct {
	// contains filtered or unexported fields
}

ChainGroup observes several nodes at once and lets a scenario wait on conditions that span them. Every member shares one change broadcaster, so a group wait costs a single channel receive regardless of how many nodes the topology has.

func NewChainGroup 

func NewChainGroup(nodes ...string) *ChainGroup

NewChainGroup returns a group observing the named nodes.

func (*ChainGroup) Await 

func (g *ChainGroup) Await(
	ctx context.Context,
	desc string,
	cond func([]ChainSnapshot) bool,
) error

Await blocks until cond holds across every node's observed chain or ctx expires. On timeout the error reports each node's state.

func (*ChainGroup) Chain 

func (g *ChainGroup) Chain(node string) *ObservedChain

Chain returns the observer for a node, or nil if it is not in the group.

func (*ChainGroup) Chains 

func (g *ChainGroup) Chains() []*ObservedChain

Chains returns every observer in the order the group was built.

func (*ChainGroup) Snapshots 

func (g *ChainGroup) Snapshots() []ChainSnapshot

Snapshots returns one snapshot per node, in group order.

type ChainPoint 

type ChainPoint struct {
	Slot uint64 `json:"slot"`
	Hash []byte `json:"hash"`
}

ChainPoint is a slot/hash pair identifying a position on a chain. The zero value is the origin.

func (ChainPoint) IsOrigin 

func (p ChainPoint) IsOrigin() bool

IsOrigin reports whether the point refers to the chain origin, which ChainSync encodes as an empty point rather than a slot/hash pair.

type ChainSnapshot 

type ChainSnapshot struct {
	Node             string           `json:"node"`
	Tip              ChainTip         `json:"tip"`
	ServerTip        ChainTip         `json:"serverTip"`
	Headers          []ObservedHeader `json:"headers"`
	RollForwards     int              `json:"rollForwards"`
	RollBackwards    int              `json:"rollBackwards"`
	MaxRollbackDepth uint64           `json:"maxRollbackDepth"`
	Connects         int              `json:"connects"`
	Disconnects      int              `json:"disconnects"`
	Connected        bool             `json:"connected"`
	LastError        string           `json:"lastError,omitempty"`
}

ChainSnapshot is a consistent point-in-time copy of one node's observed chain, safe to read without holding the observer's lock. The json tags matter: a snapshot is what NodeControl.CaptureFailureArtifacts writes to observed-chains.json, and that file is often the only surviving record of what a node's chain did once the DevNet has been torn down.

func (ChainSnapshot) HashAt 

func (s ChainSnapshot) HashAt(slot uint64) ([]byte, bool)

HashAt returns the observed header hash at slot, if the slot is still inside the retained window.

func (ChainSnapshot) String 

func (s ChainSnapshot) String() string

String renders the snapshot as a single diagnostic line. Await failures embed it so a timeout says what the node was actually doing.

type ChainTip 

type ChainTip struct {
	SlotNumber  uint64 `json:"slot"`
	BlockNumber uint64 `json:"block"`
	Hash        []byte `json:"hash"`
}

ChainTip holds the chain tip information retrieved from a node.

type DevNetConfig 

type DevNetConfig struct {
	PoolCount               int
	NetworkMagic            uint32
	EpochLength             uint64
	SlotLength              float64
	ActiveSlotsCoeff        float64
	SecurityParam           uint64
	DijkstraHardForkAtEpoch *uint64
}

DevNetConfig holds the parsed configuration values from testnet.yaml.

func LoadDevNetConfig 

func LoadDevNetConfig() (*DevNetConfig, error)

LoadDevNetConfig reads the active network spec and returns the parsed DevNetConfig. The path is taken from the DEVNET_TESTNET_YAML environment variable; if unset, it defaults to defaultTestnetYAMLPath (relative to the test package).

func LoadDevNetConfigFrom 

func LoadDevNetConfigFrom(path string) (*DevNetConfig, error)

LoadDevNetConfigFrom parses the network spec at an explicit path. It is the form used by tests that check the checked-in specs directly, without depending on which mode is currently exported to the environment.

func (*DevNetConfig) BlockFetchStabilityWindowSlots 

func (c *DevNetConfig) BlockFetchStabilityWindowSlots() uint64

BlockFetchStabilityWindowSlots returns 3k/f, the shorter window used for blockfetch stability.

func (*DevNetConfig) EpochDuration 

func (c *DevNetConfig) EpochDuration() time.Duration

EpochDuration returns the wall-clock length of one epoch.

func (*DevNetConfig) ExpectedBlockTime 

func (c *DevNetConfig) ExpectedBlockTime() time.Duration

ExpectedBlockTime returns the average wall-clock time between blocks. This is slotDuration / activeSlotsCoeff (e.g. 2.5s with 1s slots and f=0.4). Panics if ActiveSlotsCoeff is zero or negative (invalid configuration).

func (*DevNetConfig) ExpectedBlocksPerSlot 

func (c *DevNetConfig) ExpectedBlocksPerSlot() float64

ExpectedBlocksPerSlot returns the approximate probability that any given slot produces a block (i.e. activeSlotsCoeff, ignoring per-pool stake fraction).

func (*DevNetConfig) NextEpochBoundary 

func (c *DevNetConfig) NextEpochBoundary(slot uint64) uint64

NextEpochBoundary returns the first epoch-start slot strictly after slot. Deriving the target from the observed tip rather than assuming slot == epochLength lets a scenario cross a real transition even when it attaches to a network that has been running for a while.

func (*DevNetConfig) NonceStabilityWindowSlots 

func (c *DevNetConfig) NonceStabilityWindowSlots() uint64

NonceStabilityWindowSlots returns 4k/f, the window cardano-node uses for the candidate-nonce freeze. The candidate nonce for the next epoch stops evolving this many slots before the epoch ends, so an epoch shorter than the window can never freeze it.

func (*DevNetConfig) SlotDuration 

func (c *DevNetConfig) SlotDuration() time.Duration

SlotDuration returns the wall-clock duration of a single slot.

func (*DevNetConfig) Validate 

func (c *DevNetConfig) Validate() error

Validate checks that the spec describes an internally consistent network. Every violation is reported, not just the first, so shrinking a network's timing tells you everything that has to move with it.

type FailureCapturePlan added in v0.70.1 

type FailureCapturePlan struct {
	Root     string
	Name     string
	Services []string
}

FailureCapturePlan is what a scenario preserves when it fails: the directory to write under, the name that separates one scenario's evidence from another's, and the compose services whose logs to keep.

func PlanFailureCapture added in v0.70.1 

func PlanFailureCapture(
	root string,
	testName string,
	endpoints []NodeEndpoint,
) (FailureCapturePlan, bool)

PlanFailureCapture decides whether a scenario can preserve evidence and what it would preserve. Capture needs somewhere to write and at least one container to read, so an unset DEVNET_ARTIFACT_DIR or a topology of endpoints that name no container disables it rather than producing an empty directory or asking Docker for services that do not exist.

type NodeEndpoint added in v0.70.1 

type NodeEndpoint struct {
	Name        string
	Address     string // host:port
	Role        string // "producer" or "relay"
	IsDingo     bool   // node runs Dingo
	IsReference bool   // node runs the cardano-node reference impl
	// Container is the compose service name, used by the scenario to
	// interrupt and restart the node, and by failure capture to name the
	// service whose logs to preserve. A disruption step against an
	// endpoint with no container fails rather than being skipped: a run
	// that quietly omitted its interruption phases would not be the
	// release evidence it claims to be.
	Container string
}

NodeEndpoint describes a node that the test harness can connect to using the Ouroboros Node-to-Node mini-protocol over TCP.

This is plain data and carries only the package-wide Linux constraint, so the failure-capture planning in artifacts.go stays testable without a running DevNet. The code that dials an endpoint additionally requires the devnet tag.

type ObservedChain 

type ObservedChain struct {
	// contains filtered or unexported fields
}

ObservedChain accumulates the ChainSync messages one node sends and exposes them as timeout-bound conditions. It replaces polling a node's tip over repeated short-lived connections: the chain view is pushed by the node, so a condition becomes true as soon as the protocol event that satisfies it arrives.

All methods are safe for concurrent use; the observer goroutine writes while scenario assertions read.

func NewObservedChain 

func NewObservedChain(node string) *ObservedChain

NewObservedChain returns an observer for a single named node.

func (*ObservedChain) Await 

func (c *ObservedChain) Await(
	ctx context.Context,
	desc string,
	cond func(ChainSnapshot) bool,
) error

Await blocks until cond holds for this node's observed chain or ctx expires. cond is evaluated once before waiting, then again after every observed protocol event, so no polling interval is involved.

func (*ObservedChain) Connected 

func (c *ObservedChain) Connected()

Connected records that the observer established (or re-established) a ChainSync session with the node.

func (*ObservedChain) Disconnected 

func (c *ObservedChain) Disconnected(err error)

Disconnected records that the ChainSync session dropped. The observed chain is deliberately preserved so a reconnect resumes from it.

func (*ObservedChain) RollBackward 

func (c *ObservedChain) RollBackward(point ChainPoint, serverTip ChainTip)

RollBackward records a rollback to point, dropping every observed header above it. A rollback to the origin clears the observed chain.

The recorded depth counts dropped headers still inside the retained window, so a rollback deeper than that window under-reports rather than over-reports.

func (*ObservedChain) RollForward 

func (c *ObservedChain) RollForward(h ObservedHeader, serverTip ChainTip)

RollForward records a header the node sent, extending the observed chain. serverTip is the node's own reported tip from the same message.

func (*ObservedChain) Snapshot 

func (c *ObservedChain) Snapshot() ChainSnapshot

Snapshot returns a deep-enough copy for assertions to read safely.

type ObservedHeader 

type ObservedHeader struct {
	Slot        uint64 `json:"slot"`
	BlockNumber uint64 `json:"block"`
	Hash        []byte `json:"hash"`
	PrevHash    []byte `json:"prevHash"`
	// BodySize is the header's declared block body size. It is what
	// lets a scenario tell a block carrying transactions from an empty
	// one without a node-to-client connection, which the mixed
	// cardano-node topology does not expose.
	BodySize uint64 `json:"bodySize"`
	// LeiosAnnouncementHash is set when a Dijkstra header extension names
	// an endorser block. Scenarios can match it to the separately fetched EB.
	LeiosAnnouncementHash []byte `json:"leiosAnnouncementHash,omitempty"`
	LeiosAnnouncementSize uint64 `json:"leiosAnnouncementSize,omitempty"`
	LeiosCertified        bool   `json:"leiosCertified,omitempty"`
}

ObservedHeader is one block header seen in a ChainSync RollForward.

func AgreedHeaderAbove 

func AgreedHeaderAbove(
	snaps []ChainSnapshot,
	minSlot uint64,
) (ObservedHeader, bool)

AgreedHeaderAbove returns the lowest header above minSlot that every snapshot observed with an identical hash, and whether one exists.

This is the propagation check: a header only becomes "agreed" once it has reached every node in the topology, including the non-forging relay, so a block that never diffused is never mistaken for one that did. Taking the lowest such header rather than the deepest makes the answer the first block produced after the caller's baseline, which is what a propagation assertion is actually about.

type ScenarioPlan 

type ScenarioPlan struct {
	Config *DevNetConfig
	Phases []TimelinePhase

	// HardTimeout bounds the whole scenario.
	HardTimeout time.Duration

	// InterruptionHold is how long a node stays down during the
	// interruption and restart phases.
	InterruptionHold time.Duration

	// RecoveryBudget is the allowance for a restarted node to rejoin and
	// catch back up to the network's chain.
	RecoveryBudget time.Duration

	// OutageBlocks is how far the rest of the network must advance while
	// a node is stopped. Holding the outage to observed chain progress
	// rather than wall clock keeps the disruption meaningful on a fast
	// or slow runner alike, and keeps it inside what k can reconcile.
	OutageBlocks uint64

	// EpochMargin is how far past an epoch boundary the scenario runs
	// before asserting agreement, so the assertion covers headers built
	// on the new epoch nonce rather than the boundary block alone.
	EpochMargin uint64
}

ScenarioPlan is the deterministic schedule an accelerated DevNet run follows. Every budget is derived from the network's own timing parameters, so changing the spec moves the schedule with it instead of silently invalidating hardcoded waits.

func NewScenarioPlan 

func NewScenarioPlan(cfg *DevNetConfig) (*ScenarioPlan, error)

NewScenarioPlan derives the schedule for a network spec. It rejects a spec that is not internally consistent, since a plan built on one would describe a network that cannot run.

func (*ScenarioPlan) FitsReferenceBudget 

func (p *ScenarioPlan) FitsReferenceBudget() bool

FitsReferenceBudget reports whether the plan fits the documented reference-runner budget.

func (*ScenarioPlan) Phase 

func (p *ScenarioPlan) Phase(name string) (TimelinePhase, bool)

Phase returns the named phase.

func (*ScenarioPlan) String 

func (p *ScenarioPlan) String() string

String renders the schedule for the scenario log, so a run records the timeline it was held to.

func (*ScenarioPlan) Total 

func (p *ScenarioPlan) Total() time.Duration

Total returns the scenario length: the last phase's deadline on the shared clock.

type TimelinePhase 

type TimelinePhase struct {
	Name     string
	Deadline time.Duration
}

TimelinePhase is one stage of the shared scenario timeline.

Deadline is measured from the scenario's start, not from the end of the previous phase. That is what makes this one timeline rather than a series of independent waits: a phase that completes early hands its slack to everything after it, and no assertion adds its own relative slot window on top of the phases before it.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.