BERT vs GPT-2 Performance (opens in new tab)

There are quite a few BERT vs GPT-2 breakdowns online, mostly focusing on the architectural differences between the two models. However, I am more interested in the performance differences between the two models, specifically their predictive capabilities. This blog post outlines the results of my experiments. The code used in this experiment can be found on my Github BERT The Devlin et al. model was released in November 2018. It is a transformer-based language model pretrained on masked inpu...