How LLMs Keep on Getting Better (opens in new tab)

If you look at the source code of a modern open source LLM, it looks very similar to the transformer described in the “Attention is all you need” paper from 2017. It’s just a stac…