The learning rate warms up to the preset value during training and quickly becomes 0

[si520]

Hello author, thank you for your work. I have a question to ask you: The learning rate warms up to the preset value during training and quickly becomes 0，but the training is still ongoing

[si520]

@jaehyunnn

[guanyonglai]

I also encountered this problem

[guanyonglai]

you should set layer_decay_rate=1.0, rather than 0.75