フリー問題

問題文

In the documented mixed precision loop, in what order are the scaler's methods called relative to the backward pass and the optimizer?

選択肢

1. Scale the loss and call backward on the scaled value, then step through the scaler, then update the scaler's factor.
2. The scaler replaces the optimizer entirely.
3. Call backward on the unscaled loss first, then let the scaler multiply the gradients afterwards, because scaling the loss itself would change the value that is reported for monitoring purposes.
4. Update the scaler's factor before the backward pass, so that the gradients are produced and scaled with the factor already in place.