Add GitHub Action step to verify random_test_runner execution #1627
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Checklist for reviewers ☑️All references to "You" in the following text refer to the code reviewer.
|
- Added `test/ci_test_inputs/` directory with predefined input data for CI tests - Modified test executor to support the `spawn_ego_as_npc` parameter (default: false) - Updated CI runner script to execute `random_test_runner` with input_dir from `ci_test_inputs` - Ensured test results are validated via `result_checker` and proper exit status is returned
SzymonParapura
added
the
bump patch
… condition handling
… without Autoware
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR ensures that
random_test_runneris correctly executed and verified in CI by running it with predefined test input and validating the results using result_checker.py. The tests confirm that the runner launches successfully and help detect whether recent changes to its implementation introduce any issues with execution.Abstract
Enable CI verification for random_test_runner.
Background
An issue was reported on GitHub indicating that
random_test_runnerwas not functioning as expected.However, the CI system did not catch this failure, because the previous workflow lacked any mechanism to verify its results. As a result, test failures from
random_test_runnercould silently pass through the pipeline unnoticed.See: #1441
Details
workflow.shscript to include execution ofrandom_test_runner[link]input_dir, a predefined test input file is used. It contains only seeds from previously successful local runs.spawn_ego_as_npcparameterspawn_ego_as_npcparameter was added to therandom_test_runnerlaunch file. It allows users to specify whether the EGO vehicle should be spawned as an npc (i.e., not controlled by Autoware).traffic_simulator::VehicleBehavior::autoware(), which requires an Autoware. When spawn_ego_as_npc is set to true, the EGO is spawned as a regular NPC entity link.In short, this enables running
random_test_runnertests in CI without requiringAutoware.launch_rvizparameter (default: false) link.spawn_ego_as_npc:=true), RViz can be useful for local visualization and debugging.References
Jira ticket
Destructive Changes
N/A
Known Limitations
N/A