Rust insta based tests - #210
Conversation
This patch series introduces the rust insta crate based testing upgrade. The insta crate allows for snapshot comparison which is more powerful than the current bash script based testing framework. For documentation https://docs.rs/insta/latest/insta/ This patch introduces a new crate named qemu-tests. * The qemu-tests has a mod named common, which lists the bins available in each of the examples * Once bins are populated, we use rust's command execution from std, and call the `cargo run` with specific flags and --bin * Before validating the test suite needs to be bootstrapped for the insta to load the snapshot for comparsion please refer to justfile for notes on how to initialize the test suite. Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
… with insta crate Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
…rkflow to use updated test-qemu Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
jonathanpallant
left a comment
jonathanpallant
left a comment
There was a problem hiding this comment.
Thank you for these changes. I think the proposed solution is going to work well and I left you some thoughts.
| #[test] | ||
| #[ignore = "requires QEMU_TARGET; run via the justfile"] | ||
| fn test_examples() { | ||
| let target = std::env::var("QEMU_TARGET") |
There was a problem hiding this comment.
Would it make sense to have one test per target, instead of passing in the target as an environment variable? Then bare cargo test would work, and you'd see:
armv4t-none-eabi PASS
armv5te-none-eabi PASS
armv6-none-eabi PASS
etc etc
There was a problem hiding this comment.
I'd still like to see this, but I think I'm going to merge this as-is and work on that as a follow-up. Basically the existing #[test] will stay, but as a non-test function, and we can add a bunch of #[test] functions that call it, setting the parameters for the test as appropriate.
There was a problem hiding this comment.
@jonathanpallant Could you check the last push, i've reworked the infra a bit by using libtest_mimic crate. This now satisfies the comment and enables running of target specific tests
Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
* Move back to older style target based invocation of tests. * Allow user to run target specific insta tests. * Adapt test framework accordingly. Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
|
@jonathanpallant Please have a look when you've time. Hopefully I've address most of the comments in one way or the other. |
tests. And control the execution with libtest_mimic addressing the review comments Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
f299118 to
988e18e
Compare
2 participants
This patch series introduces rust insta based testing framework. The insta framework documentation is available (https://docs.rs/insta/latest/insta/).
qemu-testsaarch32-testsThe PR address #138