Implementing a GPT model from scratch (opens in new tab)

Implementing a GPT model from scratch 27 May 2026 by karlosmid This post implements a GPT-style model from scratch in Elixir/Nx, building on the attention mechanism post. We start with the GPT-2 configuration and parameter counts, then add layer normalization, GELU feed-forward layers, shortcut connections, transformer blocks, the full GPTModel, and a simple greedy text generation loop.