Skip to content

[bugfix] LoRA + TP>1 + sequence_parallel: sum grads of the replicated LoRA factor over the TP group (exported adapter had last-layer linear_proj.lora_B == 0) - #173

Open
hojuna wants to merge 1 commit into
modelscope:mainfrom
hojuna:fix/lora-sp-tp-grad-allreduce
Open

[bugfix] LoRA + TP>1 + sequence_parallel: sum grads of the replicated LoRA factor over the TP group (exported adapter had last-layer linear_proj.lora_B == 0)#173
hojuna wants to merge 1 commit into
modelscope:mainfrom
hojuna:fix/lora-sp-tp-grad-allreduce

Commits

Commits on Aug 26, 2026