92. BERT: The Model That Reads in Both Directions (opens in new tab)

GPT generates text by predicting the next word. It reads left to right. BERT does something different. It masks random words in a sentence and tries to predict what they are. To do that well, it has to understand every word in relation to every other word simultaneously. Left and right context both matter. That bidirectional understanding is why BERT dominated NLP benchmarks when it came out in 2018, and why encoder-only transformers are still the go-to for understanding tasks. What You'll Le...