Add layered integration test framework with simulated and real-node tiers - #314
Open
carlaKC wants to merge 4 commits into
Open
Add layered integration test framework with simulated and real-node tiers#314carlaKC wants to merge 4 commits into
carlaKC wants to merge 4 commits into
Conversation
…ario layers Adds a workspace crate housing an integration test framework built from independent layers: an environment layer that provisions a network and emits its partial sim.json config, a scenario layer that describes payment activity and config style, a runner that assembles config files and drives them through the same public entry points the sim-cli binary uses, and shared assertions over the observable output. Simulated networks run on virtual time so time-bounded scenarios complete instantly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers defined activity across the cross-product of node reference styles (alias/pubkey) and value shapes (scalar/range), random activity with and without exclusions, seeded determinism, count- and time-bounded runs, connector implementation inference from untagged config, and negative validation cases. Note that count-bounded assertions tolerate the loss of the final payment record: meeting a payment count shuts the simulation down in the same instant as the last dispatch, and the results consumer prefers the shutdown signal over draining pending results. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a container-based environment provider that brings up a heterogeneous regtest network: bitcoind plus one node each of LND, CLN, Eclair and ldk-server (built from the upstream repository at the same rev the client dependency pins), connected in a ring of announced channels. All startup steps poll with capped exponential backoff, and container logs are dumped when a node fails to come up. Because some implementations broadcast funding transactions asynchronously after the open call returns, readiness polling keeps mining blocks until every channel is active rather than confirming once. The real-node test runs scenarios sequentially against one shared network: defined keysends across every directed ring edge with receipts verified against each destination's own books, a multi-hop route, alias references resolved from real node configuration, and random activity. It is marked ignored so it only runs when requested explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The simulated tier already runs under the existing build job's cargo test invocation; the new CI job covers the real-node tier, building the ldk-server image from the rev pinned in Cargo.lock with a docker layer cache so repeat runs reuse it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an integration test framework covering sim-ln's node backends, payment modalities and configuration surface, structured as independent layers:
integration-tests/src/env/): provisions a network and emits its part of the sim.json config. Two providers: a deterministic in-process simulated graph, and a dockerized heterogeneous regtest network (bitcoind + one node each of LND, CLN, Eclair and ldk-server in a ring of announced channels, via testcontainers). Knows nothing about payments.src/scenario.rs): describes activity (defined/random) and config style (alias vs pubkey references, scalar vs[min, max]values). Knows nothing about provisioning.src/runner.rs,src/asserts.rs): assembles real config files, drives them through the same public entry points the sim-cli binary uses, and asserts over parsing results, dispatched payments and the results CSV.Test tiers
Simulated (
tests/sim_matrix.rs, runs in the existing test CI job, ~0.2s on virtual time): the full cross-product of node reference styles and value shapes for defined activity, random activity with/without exclusions, seeded determinism, count- and time-bounded runs, untagged connector inference, and negative validation cases asserting specific errors.Real nodes (
tests/real_nodes.rs,#[ignore], new CI job /make integration-real, ~8 min): one shared network, scenarios run sequentially — defined keysends across every directed ring edge (each connector proves it sends and receives cross-implementation, with receipts checked against each destination node's own books), a multi-hop route, alias-referenced activities, and random activity.Error resistance
Every real-node startup step polls with capped exponential backoff and names the node/step in its failure. Container logs are dumped on startup or scenario failure. Readiness keeps mining while waiting for channels because some implementations (ldk-node) broadcast funding transactions asynchronously after the open call returns.
Notes
ldk-server-clientdependency already pins;SIMLN_LDK_SERVER_IMAGEoverrides.-Declair.allow-unsafe-startup=true(a static dev-build guard, nothing dynamic).testcontainersis pinned to=0.25.0, the last release whose MSRV fits the repo's Rust 1.85 toolchain;serde_with/timeare pinned in Cargo.lock for the same reason.select!prefers the shutdown listener over draining pending results. Possibly worth an upstream fix (drain the channel before exiting) — happy to file separately.Both tiers run green locally on macOS (Apple Silicon, Docker Desktop; eclair runs under Rosetta since its image is amd64-only).
🤖 Generated with Claude Code