Finegrained kernels: unified quantization kernels api (separate quantizers) - #49015

Draft

IlyasMoutawwakil wants to merge 64 commits into

mainfrom

finegrained-per-scheme-demo

Commits on Aug 13, 2026

finegrained kernels
IlyasMoutawwakil
committed
64708a2

Commits on Aug 15, 2026

deepgemm and tp/ep fixes
IlyasMoutawwakil
committed
7b21bdb

Commits on Aug 16, 2026

finegrained: adopt the interleaved gate|up layout
IlyasMoutawwakil
committed
05bd0f3

Commits on Aug 18, 2026

finegrained: keep DeepGEMM Mega MoE on the stacked gate|up layout
IlyasMoutawwakil
committed
686238e
fix
IlyasMoutawwakil
committed
641fcb2
Merge remote-tracking branch 'origin/main' into finegrained-unified
IlyasMoutawwakil
committed
e6b20c4

Commits on Aug 21, 2026

finegrained: build the per-call kernel objects instead of caching them

IlyasMoutawwakil
and
claude
committed
cb04cbe
finegrained: fuse both MoE biases into the GEMMs

IlyasMoutawwakil
and
claude
committed
e1c7a92

Commits on Sep 9, 2026

Merge origin/main into finegrained-unified

IlyasMoutawwakil
and
claude
committed
0a1b49b
Merge remote-tracking branch 'origin/main' into finegrained-unified
IlyasMoutawwakil
committed
9041320

Commits on Sep 11, 2026

Commits on Sep 13, 2026

Revert "finegrained: pass input_recipe to the dispatchers, drop the Quantization/Epilogue bundle fields"

IlyasMoutawwakil
and
claude
committed
0b6ca4c

Commits on Sep 14, 2026

finegrained: pass activation_format straight through; let deep-gemm skip the kernels arch check

IlyasMoutawwakil
and
claude
committed
513e231
hub_kernels: finegrained-kernels ships as version 0

IlyasMoutawwakil
and
claude
committed
240abc9
hub_kernels: load finegrained-kernels from the kernels-staging build (pr-1018)

IlyasMoutawwakil
and
claude
committed
30ab2c2

Commits on Sep 15, 2026

finegrained: consume modelopt's calibrated globals, and fuse a model's per-expert output norm

IlyasMoutawwakil
and
claude
committed
a562fb8
finegrained: address the integration review

IlyasMoutawwakil
and
claude
committed
89267bf
finegrained: freeze the mxfp4 and nvfp4 integrations, and serve what they served

IlyasMoutawwakil
and
claude
committed
09ea21c
Merge branch 'main' into finegrained-unified
IlyasMoutawwakil
committed
d59f903
fix style
IlyasMoutawwakil
committed
e139d2a
FineGrainedFP8Config is its own class again, not an alias of the new one

IlyasMoutawwakil
and
claude
committed
c665644
FineGrainedFP8Config: drop the frozen-notice docstring line

IlyasMoutawwakil
and
claude
committed
29cca04

Commits on Sep 16, 2026

Tolerate an experts module without the norm, and a machine without CUDA

IlyasMoutawwakil
and
claude
committed
f63ad97
Pin that every modelopt converter target is a slot the module holds

IlyasMoutawwakil
and
claude
committed
1265f0c
is_sm100: answer False without CUDA, and fake both halves in the tests

IlyasMoutawwakil
and
claude
committed
97dfa10
fix
IlyasMoutawwakil
committed
788d02d
finegrained: drop the DTensor unwrap, and pass the kernel operands by name

IlyasMoutawwakil
and
claude
committed
53a8970
deepgemm: only Mega MoE cannot apply a post-expert norm

IlyasMoutawwakil
and
claude
committed
e8e6b87

Commits on Sep 17, 2026

Shard quantized expert companions under EP and TP

IlyasMoutawwakil
and
claude
committed
43b9a74
fixes for the new model
IlyasMoutawwakil
committed
fe0b718
fix multimodal and add some testing
IlyasMoutawwakil
committed
22b1001

Commits on Sep 18, 2026

static
IlyasMoutawwakil
committed
f9302cc
Load a calibrated checkpoint's input_scale, and fix what per-tensor weights reach

IlyasMoutawwakil
and
claude
committed
daa2420
fix
IlyasMoutawwakil
committed
b4dcb7b

Commits on Sep 21, 2026

Finish the finegrained integration: reload, TP sharding, dsv4 norms

IlyasMoutawwakil
and
claude
committed
f7081ec
Fix a destructive Qwen3 TP-plan override; split out the conversions

IlyasMoutawwakil
and
claude
committed
9ddb214
Collapse the conversion-op boilerplate; shorten the longest units

IlyasMoutawwakil
and
claude
committed
e83eadf

Commits on Sep 22, 2026

Key on capability not name; drop the last op boilerplate

IlyasMoutawwakil
and
claude
committed
60f904e
Quantize per module group; split the arms by producer, not by format

IlyasMoutawwakil
and
claude
committed
4cb5d9c
Carry the quantized-load fixes onto the per-scheme arms, and the same simplification

IlyasMoutawwakil
and
claude
committed
a67e7ac

Commits on Sep 23, 2026

Address the per-scheme review

IlyasMoutawwakil
and
claude
committed
9fc683d
trim
IlyasMoutawwakil
committed
95b3e44
fixes
IlyasMoutawwakil
committed
2392e6a
last
IlyasMoutawwakil
committed
f7a8e54
Cover the groups feature, and export what it asks callers to build

IlyasMoutawwakil
and
claude
committed
4cbbe7f
Point everything at the fine-grained integration; deprecate the frozen files

IlyasMoutawwakil
and
claude
committed
f687860
Bring two comment blocks under the noisy-comment limits

IlyasMoutawwakil
and
claude
committed
fab7f74
Bring two comment blocks under the noisy-comment limits

IlyasMoutawwakil
and
claude
committed
21c0a07
Do not let the tied-head plan raise in post_init

IlyasMoutawwakil
and
claude
committed
12f5aa8
Do not let the tied-head plan raise in post_init

IlyasMoutawwakil
and
claude
committed
7abcedf

Commits on Sep 24, 2026

Address the second review round

IlyasMoutawwakil
and
claude
committed
db85a44
Merge finegrained-replace-frozen-fp8 into finegrained-per-scheme-demo

IlyasMoutawwakil
and
claude
committed
d5c361c
Carry the review's comment and lint fixes to the sites it did not name

IlyasMoutawwakil
and
claude
committed
9f7326c
Make tests/kernels a package so the moved fp8 test can import its fixtures

IlyasMoutawwakil
and
claude
committed
f1b593d
Declare GLM-4v-MoE's head plan instead of deriving it in the mixin

IlyasMoutawwakil
and
claude
committed
0067189
cleanup guards and post expert norm in deepgemm
IlyasMoutawwakil
committed
bb24085
not now
IlyasMoutawwakil
committed
39eb1d1
minimal peft integration preparation
IlyasMoutawwakil
committed
bf0e7ec

Commits on Sep 25, 2026

addressing review comments
IlyasMoutawwakil
committed
5b2b7f0
Tiny checkpoints for every shipped layout, and the save/load fixes they found

IlyasMoutawwakil
and
claude
committed
f3c7cc8
Keep pre-quantized scale dtypes through the dtype plan

IlyasMoutawwakil
and
claude
committed
6a7c92e