Skip to content

[PyTorch] Fix strict load_state_dict for te.ops.Linear without bias - #3915

Open
bhargavchintam wants to merge 1 commit into
NVIDIA:mainfrom
bhargavchintam:fix/ops-linear-nobias-state-dict
Open

bhargavchintam wants to merge 1 commit into
NVIDIA:mainfrom
bhargavchintam:fix/ops-linear-nobias-state-dict

Conversation

@bhargavchintam

@bhargavchintam bhargavchintam commented Nov 21, 2026 •

Copy link
Copy Markdown

Description

A bias-free te.ops.Linear can't load the state dict it just saved with a strict load_state_dict. It fails with Unexpected key(s) in state_dict: "0.bias". Linear._load_from_state_dict added state_dict["bias"] = None when the op has no bias. torch leaves None parameters out of the keys it expects, so the added key was reported as unexpected. This PR adds the bias key only when the op has a bias.

Loading a bias=True checkpoint into a bias-free op still fails a strict load, as it should. With this change the unexpected keys are 0.basic_ops.1.bias and 0.basic_ops.1._extra_state; main also listed the inserted 0.bias.

Fixes #3914

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

  • transformer_engine/pytorch/ops/linear.py: Linear._load_from_state_dict adds {prefix}bias only when the op has a bias. Two lines replace five.
  • tests/pytorch/test_fusible_ops.py: TestCheckpointing.test_linear is parametrized over bias in (False, True) and passes it to both te_ops.Linear constructions.

Testing

On an NVIDIA RTX PRO 6000 Blackwell (compute capability 12.0) in the NGC PyTorch 26.10 container (torch 2.15.0a0, CUDA 13.4), I built TE from main e6f9e86 with NVTE_FRAMEWORK=pytorch NVTE_CUDA_ARCHS=120 python -m pip install -e . -v --no-build-isolation. I applied the test change first and the fix second, so both runs use the same build.

python -m pytest -v tests/pytorch/test_fusible_ops.py -k "TestCheckpointing and test_linear" with main's ops/linear.py and the new test cases:

FAILED tests/pytorch/test_fusible_ops.py::TestCheckpointing::test_linear[False-False-None]
FAILED tests/pytorch/test_fusible_ops.py::TestCheckpointing::test_linear[False-False-fp8_delayed_scaling]
FAILED tests/pytorch/test_fusible_ops.py::TestCheckpointing::test_linear[False-False-fp8_current_scaling]
FAILED tests/pytorch/test_fusible_ops.py::TestCheckpointing::test_linear[False-True-None]
FAILED tests/pytorch/test_fusible_ops.py::TestCheckpointing::test_linear[False-True-fp8_delayed_scaling]
FAILED tests/pytorch/test_fusible_ops.py::TestCheckpointing::test_linear[False-True-fp8_current_scaling]
===== 6 failed, 6 passed, 12 skipped, 5830 deselected, 8 warnings in 4.89s =====

The first case again with --tb=short:

tests/pytorch/test_fusible_ops.py:4655: in test_linear
    model_load.load_state_dict(state_dict["model"])
/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py:2647: in load_state_dict
    raise RuntimeError(
E   RuntimeError: Error(s) in loading state_dict for Sequential:
E   	Unexpected key(s) in state_dict: "0.bias".

The same command on this branch (source tree 22b31bc8, the tree of commit 71968de):

========= 12 passed, 12 skipped, 5830 deselected, 11 warnings in 4.55s =========

The 12 skipped cases are the nvfp4, nvfp4_4over6 and fp8_block_scaling recipes, which skip on this GPU.

The whole file on this branch, as qa/L0_pytorch_unittest/test.sh runs it (python -m pytest tests/pytorch/test_fusible_ops.py):

84 failed, 2292 passed, 3478 skipped, 873 warnings in 49.01s

The same 84 tests fail with main's ops/linear.py (84 failed in 5.75s), so none of them comes from this change. They are fp8_delayed_scaling cases of test_activation (36), test_layernorm_mlp (24), test_clamped_swiglu (16) and test_swiglu (8). All of them stop in cast_gated_tma with "Only rowwise cast supported in this function" on this GPU.

TE_PATH=$PWD PYTHON_ONLY=1 bash qa/L0_pytorch_lint/test.sh on this branch:

------------------------------------
Your code has been rated at 10.00/10

pre-commit run --from-ref origin/main --to-ref HEAD on this branch:

check for merge conflicts................................................Passed
check for added large files..............................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
Format python code.......................................................Passed
clang-format.........................................(no files to check)Skipped
Check minimum Python version.............................................Passed

TE_PATH="$PWD" bash qa/L0_license/test.sh exits 0. The MXFP8 recipe did not run: the test file adds it only when mxfp8_available is true, and it is false on this GPU.

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Linear._load_from_state_dict added state_dict["bias"] = None when the op
had no bias. torch leaves None parameters out of the keys it expects, so
a strict load reported the added key as unexpected, and a bias-free
te.ops.Linear could not load the state dict it had just saved. Add the
bias key only when the op has a bias.

Run TestCheckpointing.test_linear with bias=False as well as bias=True.

Signed-off-by: Bindu Bhargava R Chintam <bindubhargavareddy@gmail.com>
@github-actions github-actions Bot added the community-contribution PRs from external contributor outside the core maintainers, representing community-driven work. label Nov 21, 2026
@bhargavchintam
bhargavchintam marked this pull request as ready for review November 21, 2026 00:08
@greptile-apps

greptile-apps Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

Fixes strict checkpoint loading for te.ops.Linear without bias.

  • Bias-free te.ops.Linear checkpoints now load with strict load_state_dict.

No actionable issues found. Tests were not run during this review.

Reviews (1) · Last reviewed commit: "[PyTorch] Fix strict load_state_dict for..." · Reviewed by Greptile

@lxcxjxhx lxcxjxhx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks right, and I checked the two things that usually make a fix of this shape incomplete.

Is the pattern repeated in the sibling ops? No. state_dict[...] = None inside _load_from_state_dict exists only in ops/linear.py:214 across transformer_engine/pytorch/ops/*.py, so fixing Linear alone is the whole fix rather than the first of several.

if self._bias_idx is not None and f"{prefix}bias" not in state_dict:
    state_dict[f"{prefix}bias"] = state_dict[f"{prefix}basic_ops.{self._bias_idx}.bias"]

Is dropping the None insertion safe for the reverse direction? A model with bias=False now leaves a checkpoint's bias key alone, so a bias-carrying checkpoint loaded into a bias-free op fails strict load as an unexpected key — which is the correct strict-load answer, and it is the same failure class the old code produced (the inserted None never matched either). Worth one sentence in the description so the next reader does not read it as a regression.

The parametrisation covers what the issue reports: bias × quantization × quantized_weight turns the single passing case into six, and the bias-free six are exactly the ones that failed on main before this change.

One thing I could not check here: the test needs a GPU and the compiled extension, so I did not run it — the reasoning above is from the diff plus a grep for the pattern.

@bhargavchintam

Copy link
Copy Markdown
Author

Thanks for checking the sibling ops. I added the reverse case to the description. A bias=True checkpoint loaded strictly into a bias-free op fails on main and on this branch alike; the only change is that main also listed the inserted 0.bias among the unexpected keys. I ran that on a Mac with TE's Python code and the extension stubbed out.

@bhargavchintam

Copy link
Copy Markdown
Author

@timmoon10, this is a two-line fix in Linear._load_from_state_dict, the code that came in with #2027. Lint, License, Documentation and Minimum supported versions need a maintainer to approve the runs, since I'm an outside contributor. TE's own TestCheckpointing.test_linear with the new bias=False cases passes on an RTX PRO 6000, and the output is in the description. Could you approve the runs?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PRs from external contributor outside the core maintainers, representing community-driven work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[PyTorch] te.ops.Linear(bias=False) can't load its own state_dict

2 participants