Learn the hidden signal inside teacher probabilities (“dark knowledge”) and use cross-entropy + KL to transfer it to a smaller model.

Continue reading on Towards AI »