Next Token Prediction is a Misleading Term (opens in new tab)

I’m fed up of hearing about how LLMs are next token predictors, and therefore they .There’s lots of philosophical objections, but fundamentally, framing AI as next token predictors in the first places is just misleading and inaccurate. Here’s why LLMs aren’t naive next token predictors.What is Next Token PredictionLet’s first briefly cover what “Next Token Prediction” even means. It is referring to base training (also ca...