[https://nvbugs/6525059][fix] Drop `8` from `FP8BlockScaleMoeRunner::mSupportedTileN`, covering both the… by trtllm-agent · Pull Request #17135 · NVIDIA/TensorRT-LLM

trtllm-agent · 2026-07-31T19:01:36Z

Summary

Root cause: On sm100f/sm103 every DeepSeek-FP8 GEMM2 cubin at tile_tokens_dim=8 faults with cudaErrorIllegalAddress in its TMA-OOB epilogue store, and tileN=8 is selected for warmup/tiny batches.
Fix: Drop 8 from FP8BlockScaleMoeRunner::mSupportedTileN, covering both the autotuner tactic list and the tileN == -1 fallback without constructing a config-less tileN=8 runner; unwaive the GB200/GB300 entries.
Automated fix generated by repair-bot

Test plan

Verify fix on the same GPU type as the original failure
Check for regressions in related tests

Links

Bug: https://nvbugs/6525059

Dev Engineer Review

Removed tileN=8 from FP8BlockScaleMoeRunner::mSupportedTileN.
The change applies to autotuner tactics and the tileN == -1 fallback.
The runner no longer constructs a config-less tileN=8 runner.
The change addresses cudaErrorIllegalAddress faults on sm100f/sm103.
Removed the GB200, GB300, and B200 waiver entries for TestQwen3_30B_A3B::test_dummy_load_format.
No public API declarations changed.
No configuration errors, duplicate entries, or unintended scope changes were identified.

QA Engineer Review

No test functions changed.
Removed three SKIP entries from tests/integration/test_lists/waives.txt.
TensorRT-LLM CI completed successfully at commit 03f178b.
QA verification completed successfully for NVBug 6525059.
The customized QA test list was empty.

Verdict: sufficient

coderabbitai · 2026-07-31T19:07:22Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c8e6cd85-5ecb-42c0-aaaa-e3a980b2b0bc

📥 Commits

Reviewing files that changed from the base of the PR and between 409bccf and 7cf564a.

📒 Files selected for processing (2)

cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp
tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

tests/integration/test_lists/waives.txt

🚧 Files skipped from review as they are similar to previous changes (1)

cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp

Walkthrough

The FP8 MoE runner no longer supports tileN=8 because its GEMM2 cubins fault. Supported tile sizes are 16, 32, 64, and 128. Three Qwen3 integration-test waivers are removed.

Changes

FP8 MoE support and test waivers

Layer / File(s)	Summary
Update FP8 MoE tile support `cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp`	Runner construction and fallback selection exclude tileN=8 because of a GEMM2 illegal-address failure.
Remove integration-test waivers `tests/integration/test_lists/waives.txt`	The B200, GB200, and GB300 `TestQwen3_30B_A3B::test_dummy_load_format` skip entries are removed.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#17120: Modifies FP8 MoE fallback tactic support.
NVIDIA/TensorRT-LLM#17328: Modifies FP8BlockScaleMoE fallback and tactic validation.

Suggested reviewers: mzweilz, qijune

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the NVBug, fix type, and removal of tileN=8 from the FP8BlockScaleMoeRunner supported tile sizes.
Description check	✅ Passed	The description explains the root cause, fix, affected entries, test plan, and bug link; it omits the repository checklist but is otherwise complete.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp (1)
367-373: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Name the supported tile sizes.

Line 373 uses non-exempt numeric literals directly in mSupportedTileN. Replace them with k-prefixed constexpr constants.
Proposed change
-        : mSupportedTileN{16, 32, 64, 128}
+        : mSupportedTileN{kSupportedTileN16, kSupportedTileN32, kSupportedTileN64, kSupportedTileN128}

 private:
+    static constexpr int32_t kSupportedTileN16{16};
+    static constexpr int32_t kSupportedTileN32{32};
+    static constexpr int32_t kSupportedTileN64{64};
+    static constexpr int32_t kSupportedTileN128{128};
As per coding guidelines, avoid magic literals except 0, nullptr, true, and false, and use k-prefixed camelCase constants.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp` around lines 367 - 373, Define
k-prefixed constexpr constants for the supported tile sizes near the relevant
configuration declarations, then initialize mSupportedTileN with those named
constants instead of the numeric literals 16, 32, 64, and 128. Preserve the
existing supported-size set and ordering.
Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp`:
- Around line 367-373: Validate cached and requested tile values against
mSupportedTileN before runner lookup in the relevant run() path, so unsupported
tileN=8 configurations are rejected or invalidated instead of reaching
mRunners.at(). Preserve valid cached configurations and the existing supported
tile set.

---

Nitpick comments:
In `@cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp`:
- Around line 367-373: Define k-prefixed constexpr constants for the supported
tile sizes near the relevant configuration declarations, then initialize
mSupportedTileN with those named constants instead of the numeric literals 16,
32, 64, and 128. Preserve the existing supported-size set and ordering.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3cdeff4c-99e0-47d4-91f8-d7821162500c

📥 Commits

Reviewing files that changed from the base of the PR and between 7443b7f and 03f178b.

📒 Files selected for processing (2)

cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp
tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

tests/integration/test_lists/waives.txt

crazydemo · 2026-08-03T06:16:18Z

/bot run --only-qa-verify

tensorrt-cicd · 2026-08-03T06:23:04Z

PR_Github #63374 [ run ] triggered by Bot. Commit: 03f178b Link to invocation

tensorrt-cicd · 2026-08-03T07:24:41Z

PR_Github #63374 [ run ] completed with state SUCCESS. Commit: 03f178b
LLM_FUNCTION_AUTO_V2C #345 completed with status: 'SUCCESS'
QA verify (empty customized_test_list) (NVBug 6525059, branch repair-bot-bug6525059, fork tensorrt-cicd, dry_run_close=true)

Link to invocation

brnguyen2

Change is right and minimal — mSupportedTileN gates runner construction, the tactic list, and the -1 clamp floor, so dropping the entry covers every path. One thing to confirm before merge: the fault is described as sm100f/sm103, but the removal is unconditional, so sm100a (B200/GB200) also loses tileN=8 for tiny batches and now clamps to 16. If that's an accepted small-batch perf cost, fine; if not, this needs an arch guard. Worth a sentence in the commit either way.

… MoE The TRTLLM-Gen FP8 block-scale MoE picks tile_tokens_dim=8 whenever num_tokens*top_k/local_num_experts <= 8, i.e. for warmup and tiny decode batches. On sm100f/sm103 every DeepSeek-FP8 GEMM2 cubin at that tile (bmm_Bfloat16_E4m3E4m3_Fp32_t128x8x128{,u2}_..._dsFp8_{schedS,schPd4x2x2x3}_bN_...) faults with cudaErrorIllegalAddress in its TMA-OOB epilogue store, so the warmup forward of Qwen3-30B-A3B-FP8 dies. Both the Static and the Persistent variant fault, so the tile size is the discriminator rather than a single bad cubin. Remove 8 from the runner's supported tile list. This covers the autotuner tactic list and the tileN == -1 fallback (which clamps on front()) in one place, and avoids ever constructing the tileN=8 runner -- filtering the cubins in TrtllmGenBatchedGemmRunner::skipQuirks instead would leave that runner with no passing config and throw during construction. FP8 block-scale MoE stays enabled; small batches now run at tileN=16. Unwaive the GB200/GB300 TestQwen3_30B_A3B::test_dummy_load_format entries. Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

coderabbitai · 2026-08-06T05:47:14Z

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

trtllm-agent requested review from a team as code owners July 31, 2026 19:01

trtllm-agent requested review from QiJune, crazydemo, mlefeb01, mzweilz, schetlur-nv, sunnyqgg and xxi-nv July 31, 2026 19:01

github-actions Bot assigned trtllm-agent Jul 31, 2026

coderabbitai Bot reviewed Jul 31, 2026

View reviewed changes

Comment thread cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp

trtllm-agent force-pushed the repair-bot-bug6525059 branch from 03f178b to b8db274 Compare August 3, 2026 06:37

trtllm-agent force-pushed the repair-bot-bug6525059 branch from b8db274 to 05b8927 Compare August 3, 2026 14:45

coderabbitai Bot mentioned this pull request Aug 4, 2026

[https://nvbugs/6550749][fix] Size the DeepGEMM warmup bucket stride to the 16-token config quantum #17242

Closed

brnguyen2 approved these changes Aug 5, 2026

View reviewed changes

xxi-nv removed their request for review August 6, 2026 05:35

trtllm-agent added 2 commits August 5, 2026 22:45

[nvbugs/6525059][chore] Remove stale waiver after fix

7cf564a

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

trtllm-agent force-pushed the repair-bot-bug6525059 branch from 05b8927 to 7cf564a Compare August 6, 2026 05:47

coderabbitai Bot mentioned this pull request Aug 6, 2026

[https://nvbugs/6432948][fix] Add per-engine flag _trtllm_gen_jit_warmup_done; skip… #16205

Open

2 tasks

Navigation Menu

Uh oh!

[https://nvbugs/6525059][fix] Drop `8` from `FP8BlockScaleMoeRunner::mSupportedTileN`, covering both the… - #17135

[https://nvbugs/6525059][fix] Drop `8` from `FP8BlockScaleMoeRunner::mSupportedTileN`, covering both the…#17135
trtllm-agent wants to merge 2 commits into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6525059

trtllm-agent commented Jul 31, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

crazydemo commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

brnguyen2 left a comment

Uh oh!

coderabbitai Bot commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

trtllm-agent commented Jul 31, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Links

Dev Engineer Review

QA Engineer Review

Uh oh!

coderabbitai Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

crazydemo commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

brnguyen2 left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

trtllm-agent commented Jul 31, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading