A Training Bug

[zjykzj]

There is a training bug in this project. That is I only set teacher model's require_grad_=False, but still put it's parameters to optimizer. So teacher model will update in training while it doesn't compute grad.

I don't have plan to fix it, because the training result shows it also works well.

[zjykzj]

May be that's fine if I had set requires_grad=False. See What is the behavior of passing model parameters with requires_grad == False to an optimizer?

Optimizer has done the filter func implicitly, for example sgd. To be on the safe side, filter the parameters entered into the optimizer

[thancaocuong]

it's not a bug 👍