[https://nvbugs/6435112][test] Unwaive Wan 2.2 I2V perf sanity test#16409
[https://nvbugs/6435112][test] Unwaive Wan 2.2 I2V perf sanity test#16409taianz-nv wants to merge 1 commit into
Conversation
Signed-off-by: Taian Zhang <taianz@nvidia.com>
|
/bot run --extra-stage "DGX_B200-8_GPUs-PyTorch-VisualGen-PerfSanity-Post-Merge-1" |
📝 WalkthroughWalkthroughRemoves one SKIP waiver for a specific visual generation end-to-end performance test parameterization from the integration waiver list. ChangesVisual generation waiver
Estimated code review effort: 1 (Trivial) | ~2 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
PR_Github #59383 [ run ] triggered by Bot. Commit: |
|
PR_Github #59383 [ run ] completed with state
|
|
/bot run --extra-stage "DGX_B200-8_GPUs-PyTorch-VisualGen-PerfSanity-Post-Merge-1" |
|
PR_Github #59446 [ run ] triggered by Bot. Commit: |
|
PR_Github #59446 [ run ] completed with state
|
|
Quick question to confirm the bug attribution before this merges. The line removed in this PR is That waiver is tagged nvbugs/6422339, but the PR title/description says this un-waives the case tracked by nvbugs/6435112 (NCCL ALLTOALL_BASE watchdog timeout). Could you confirm which bug actually caused this specific case to be skipped? If the case was originally waived under 6422339 (which covers several perf-sanity cases) and only the 6435112 NCCL timeout was fixed, removing this waiver may re-expose the 6422339 failure. If 6422339 is in fact resolved for this case, it would help to update the description (or the waiver's bug reference) so the attribution is consistent. Thanks! |
|
/bot run --extra-stage "DGX_B200-8_GPUs-PyTorch-VisualGen-PerfSanity-Post-Merge-1" |
|
As discussed in DM, 6422339 is the Post-Merge aggregation bug that originally carried this waiver, while 6435112 specifically tracks the Wan 2.2 NCCL timeout. I have also updated the PR description to clarify this. Thanks! |
BowenFu
left a comment
There was a problem hiding this comment.
LGTM — single waiver removal (nvbugs/6435112, Wan 2.2 I2V perf sanity).
Summary
Background
NVBug 6435112 tracks an NCCL ALLTOALL_BASE watchdog timeout in this exact test. The repair bot reran the test multiple times on B200 x8 against current main and all attempts passed, so this PR re-enables the test for full CI validation.
NVBug context
This test was initially waived under NVBug 6422339, which groups multiple failures from Post-Merge #2824. NVBug 6435112 tracks this specific Wan 2.2 I2V NCCL timeout, so this PR is titled under 6435112 while removing the existing waiver linked to 6422339. Other 6422339 waivers are unchanged.
Validation
pre-commit run --files tests/integration/test_lists/waives.txtCI
Please run
DGX_B200-8_GPUs-PyTorch-VisualGen-PerfSanity-Post-Merge-1to validate the previously flaky distributed test before closing NVBug 6435112.Summary by CodeRabbit