Chenghao/triton moe dense mlp opt#249

Open

nvchenghaoz wants to merge 25 commits into

chenghao/triton_moe_dense_mlpfrom

chenghao/triton_moe_dense_mlp_opt

Commits on Mar 20, 2026

[None][perf] triton_moe_dense_mlp opt iter 0: baseline measurements
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 1: num_warps x num_stages sweep
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 2: apply sweep best configs
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 3: adaptive 2D/1D grid for kernel 2
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 4: deinterleave K1 for coalesced loads
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 4-5: K1 coalesced loads + K2 adaptive BLOCK_H
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 6: K2 remove dtype_probe load
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 6-7: K2 cleanup + failed weight rearrange
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 8: K2 skip zero routing weights
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 9: K1 2D grid with adaptive BLOCK_I
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt: update running doc through iter 9
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 10: K1 unified num_warps=16
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 11: remove deinterleave, use stride-2 kernel
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 12: remove redundant .contiguous()
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 13: K1 native dtype compute
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 14: bandwidth analysis + doc update
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 15: K2 gather kernel for sparse routing
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 16: optimize sparse detect + BLOCK_I tuning
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 17: gather threshold + sweep findings
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 18: K1 num_warps=8 for large T
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt iter 19-22: launcher profiling + cleanup
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt: update running doc through iter 22
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt: final summary after 22 iterations
nvchenghaoz
committed
[None][perf] triton_moe_dense_mlp opt: document K1 regression tradeoff
nvchenghaoz
committed
[None][fix] Remove guardword from running doc
nvchenghaoz
committed