How Transformer Decoders Generate Text — From Causal Masking to Decoding (opens in new tab)

A Transformer Decoder does not generate a sentence all at once. It predicts one token. Then it feeds that token back and predicts the next one. That simple loop is the core of modern LLM generation. Core Idea A Transformer Decoder is built for autoregressive generation. That means: previous tokens → next token prediction → repeat The Decoder creates hidden representations. The LM Head converts those representations into vocabulary scores. A decoding strategy chooses the actual next token. Thi...