Skip to content

Add layered integration test framework with simulated and real-node tiers - #314

Open
carlaKC wants to merge 4 commits into
bitcoin-dev-project:mainfrom
carlaKC:integration-tests
Open

Add layered integration test framework with simulated and real-node tiers#314
carlaKC wants to merge 4 commits into
bitcoin-dev-project:mainfrom
carlaKC:integration-tests

Conversation

@carlaKC

@carlaKC carlaKC commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Adds an integration test framework covering sim-ln's node backends, payment modalities and configuration surface, structured as independent layers:

  • Environment layer (integration-tests/src/env/): provisions a network and emits its part of the sim.json config. Two providers: a deterministic in-process simulated graph, and a dockerized heterogeneous regtest network (bitcoind + one node each of LND, CLN, Eclair and ldk-server in a ring of announced channels, via testcontainers). Knows nothing about payments.
  • Scenario layer (src/scenario.rs): describes activity (defined/random) and config style (alias vs pubkey references, scalar vs [min, max] values). Knows nothing about provisioning.
  • Runner + assertions (src/runner.rs, src/asserts.rs): assembles real config files, drives them through the same public entry points the sim-cli binary uses, and asserts over parsing results, dispatched payments and the results CSV.

Test tiers

Simulated (tests/sim_matrix.rs, runs in the existing test CI job, ~0.2s on virtual time): the full cross-product of node reference styles and value shapes for defined activity, random activity with/without exclusions, seeded determinism, count- and time-bounded runs, untagged connector inference, and negative validation cases asserting specific errors.

Real nodes (tests/real_nodes.rs, #[ignore], new CI job / make integration-real, ~8 min): one shared network, scenarios run sequentially — defined keysends across every directed ring edge (each connector proves it sends and receives cross-implementation, with receipts checked against each destination node's own books), a multi-hop route, alias-referenced activities, and random activity.

Error resistance

Every real-node startup step polls with capped exponential backoff and names the node/step in its failure. Container logs are dumped on startup or scenario failure. Readiness keeps mining while waiting for channels because some implementations (ldk-node) broadcast funding transactions asynchronously after the open call returns.

Notes

  • ldk-server has no published image: the harness (and CI, with a docker layer cache) builds one from the upstream repo at the rev the ldk-server-client dependency already pins; SIMLN_LDK_SERVER_IMAGE overrides.
  • ACINQ ships no versioned eclair tags past 0.8.0, so eclair is pinned by digest; the image is built from the post-release dev commit and needs -Declair.allow-unsafe-startup=true (a static dev-build guard, nothing dynamic).
  • testcontainers is pinned to =0.25.0, the last release whose MSRV fits the repo's Rust 1.85 toolchain; serde_with/time are pinned in Cargo.lock for the same reason.
  • Count-bounded assertions tolerate the loss of the final payment record per activity: meeting a payment count triggers shutdown in the same instant as the last dispatch, and the results consumer's select! prefers the shutdown listener over draining pending results. Possibly worth an upstream fix (drain the channel before exiting) — happy to file separately.

Both tiers run green locally on macOS (Apple Silicon, Docker Desktop; eclair runs under Rosetta since its image is amd64-only).

🤖 Generated with Claude Code

carlaKC and others added 4 commits August 4, 2026 10:27
…ario layers

Adds a workspace crate housing an integration test framework built from
independent layers: an environment layer that provisions a network and
emits its partial sim.json config, a scenario layer that describes
payment activity and config style, a runner that assembles config files
and drives them through the same public entry points the sim-cli binary
uses, and shared assertions over the observable output. Simulated
networks run on virtual time so time-bounded scenarios complete
instantly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers defined activity across the cross-product of node reference
styles (alias/pubkey) and value shapes (scalar/range), random activity
with and without exclusions, seeded determinism, count- and time-bounded
runs, connector implementation inference from untagged config, and
negative validation cases.

Note that count-bounded assertions tolerate the loss of the final
payment record: meeting a payment count shuts the simulation down in the
same instant as the last dispatch, and the results consumer prefers the
shutdown signal over draining pending results.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a container-based environment provider that brings up a
heterogeneous regtest network: bitcoind plus one node each of LND, CLN,
Eclair and ldk-server (built from the upstream repository at the same
rev the client dependency pins), connected in a ring of announced
channels. All startup steps poll with capped exponential backoff, and
container logs are dumped when a node fails to come up. Because some
implementations broadcast funding transactions asynchronously after the
open call returns, readiness polling keeps mining blocks until every
channel is active rather than confirming once.

The real-node test runs scenarios sequentially against one shared
network: defined keysends across every directed ring edge with receipts
verified against each destination's own books, a multi-hop route, alias
references resolved from real node configuration, and random activity.
It is marked ignored so it only runs when requested explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The simulated tier already runs under the existing build job's cargo
test invocation; the new CI job covers the real-node tier, building the
ldk-server image from the rev pinned in Cargo.lock with a docker layer
cache so repeat runs reuse it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant