Fix IndexError during Qwen3.5/3.6 quantization by disabling in_proj_qkv/in_proj_z by default

[heyparth1]

This PR fixes the IndexError: tuple index out of range encountered during NVPF4 quantization of Qwen3.5/3.6 models.

Root Cause:
TRT-LLM's Qwen3.5 weight mapper repacks in_proj_qkv and in_proj_z into the fused in_proj_qkvz by splitting the checkpoint tensor along dim 0. The scalar per-tensor scales of quantized modules are 0-dim, which crashes the split operation.

Changes:

• Disable *linear_attn.in_proj_qkv* and *linear_attn.in_proj_z* by default in default_disabled_quantizers.yaml.
• Update the curated Qwen3.5 FP8 recipes to explicitly re-enable these quantizers after importing the default disabled quantizers, preserving their intended quantization behavior while fixing the default path.

Closes #1933

Summary by CodeRabbit

• Bug Fixes
  • Improved PTQ quantization rules for Qwen3.5/3.6 so packed linear-attention projection weights (in_proj_qkv / in_proj_z) are kept in the intended precision to prevent repacking-related runtime crashes.
  • Adjusted FP8 config/preset ordering so these packed projections follow the correct enable/disable outcome.
• Tests
  • Added unit tests to verify Qwen3.5 recipes and generic presets keep (or disable) packed linear-attention quantizers as expected.
• Documentation
  • Refined example script logging to better explain when quantization is skipped and how to re-quantize with updated configs.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6a97c257-c88f-4cae-8742-5a1cfdd2d4b9

📥 Commits

Reviewing files that changed from the base of the PR and between c75cc0c and 81a4d89.

📒 Files selected for processing (2)

• examples/hf_ptq/scripts/huggingface_example.sh
• tests/unit/torch/quantization/test_quantize_cpu.py

✅ Files skipped from review due to trivial changes (1)

• examples/hf_ptq/scripts/huggingface_example.sh

---

📝 Walkthrough

Walkthrough

PTQ configs now default-disable packed linear-attention projections, Qwen3.5 quant configs re-enable FP8 for those projections after the default import, and tests validate the resulting enable state for recipes, presets, and a CPU quantization model. The example HF PTQ script also updates its skip-quantization logging.

Changes

Packed linear-attn FP8 quantizer config and tests

Layer / File(s)	Summary
Default disable rule for packed projections modelopt_recipes/configs/ptq/units/default_disabled_quantizers.yaml	Adds enable: false entries and explanatory comments for *linear_attn.in_proj_qkv* and *linear_attn.in_proj_z*.
Re-enable FP8 after default disable (nvfp4-fp8) modelopt_recipes/huggingface/qwen3_5/ptq/w4a16_nvfp4-fp8_attn-kv_fp8_cast.quant_cfg.yaml	Removes earlier FP8 enable rules for in_proj_qkv/in_proj_z and re-adds them after default_disabled_quantizers so they override the new default disabling, while in_proj_a/in_proj_b and conv1d remain disabled.
Re-enable FP8 after default disable (nvfp4_mse-fp8) modelopt_recipes/huggingface/qwen3_5/ptq/w4a16_nvfp4_mse-fp8_attn-kv_fp8_cast.quant_cfg.yaml	Relocates FP8 enable rules for in_proj_qkv/in_proj_z weight/input quantizers to after default_disabled_quantizers for the same override effect.
Effective enable-state tests tests/unit/recipe/test_loader.py, tests/unit/recipe/test_presets.py	Adds an _effective_enable helper resolving final quantizer state from ordered quant_cfg entries, a loader test confirming in_proj_qkv/in_proj_z stay enabled while in_proj_a/in_proj_b/conv1d stay disabled for two Qwen3.5 recipes, and a presets smoke test confirming nvfp4/fp8 presets disable packed projections by default.
CPU quantization coverage for packed projections tests/unit/torch/quantization/test_quantize_cpu.py	Adds a hybrid linear-attention test model and asserts default NVFP4/FP8 presets leave packed linear-attention projections unquantized while other projections remain enabled.
Skip-quantization message update examples/hf_ptq/scripts/huggingface_example.sh	Splits the skip-quantization log line into two messages and adds guidance that the saved checkpoint is reused as-is until the output directory is deleted.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant QuantConfigYAML
  participant DefaultDisabledQuantizers
  participant RecipeLoader
  QuantConfigYAML->>DefaultDisabledQuantizers: import default_disabled_quantizers
  DefaultDisabledQuantizers->>RecipeLoader: disable in_proj_qkv/in_proj_z rules
  QuantConfigYAML->>RecipeLoader: add FP8 enable rules after import
  RecipeLoader->>RecipeLoader: apply quant_cfg entries in order
  RecipeLoader-->>QuantConfigYAML: final state - in_proj_qkv/z enabled, in_proj_a/b and conv1d disabled

Loading

Suggested reviewers: kevalmorabia97, shengliangxu, cjluo-nv

🚥 Pre-merge checks | ✅ 6

✅ Passed checks (6 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title matches the main fix: preventing Qwen3.5/3.6 quantization crashes by disabling packed projections by default.
Linked Issues check	✅ Passed	The config changes, recipe overrides, and regression tests align with #1933’s goal of fixing Qwen3.6 NVFP4 quantization failures.
Out of Scope Changes check	✅ Passed	The script message update and added tests are support changes for the quantization fix, with no clear unrelated scope creep.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Security Anti-Patterns	✅ Passed	Touched files only add tests/YAML/logging; no new torch.load(weights_only=False), numpy.load(allow_pickle=True), trust_remote_code=True, eval/exec, or # nosec.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[kristianpaul]

This is not fixing the error, IndexError: tuple index out of range still there. I'm happy to test it again

https://gist.github.com/kristianpaul/34b3421fb84be82339c1f5e3efa1b118

[heyparth1]

Thanks for re-testing! Your log shows the run reused the checkpoint exported before the fix — note the line "Quantized model config /workspace/output_models/saved_models_Qwen3_6-35B-A3B_nvfp4/config.json exists, skipping the quantization stage" — so the corrected quantization config was never applied. Please delete output_models/saved_models_Qwen3_6-35B-A3B_nvfp4 (or the whole mounted output_models) and rerun; the linear_attn.in_proj_qkv/in_proj_z projections will then be exported unquantized, which removes the 0-dim scale tensors that crash TRT-LLM's QKV split. I've also made the script print an explicit warning when it reuses an existing checkpoint, and added an end-to-end mtq.quantize test confirming the exclusions apply to the real Qwen3.5/3.6 module names.