Qwen edit 2511 training speed

[preethamp0197]

When training with zero2offloading, I have 95% of GPU memory utilization but but only 20% of GPU utilization. And with no offloading it gives me OOM error. I am training it on 8 H100s.
HOw can I increase my GPU utilization
Also would like to know if I have to train it on 5 nodes (each with 8 h100s) Whats the best configuration? Can I use deepspeed 3? Or something like Deepspeed zero++?

[preethamp0197]

I tried data process and then sft. but it takes around 15-25 sec/iteration. And in this 10-15 secs taken by backward pass. Any tips on how to make it faster?