An analysis on why LLMs perform bad on long loop tasks (opens in new tab)

Why does protocol compliance degrade over rounds? From softmax attention dilution and RoPE distance decay to the EOS bias and transformer statelessness, layer by layer to the algorithmic root causes of LLM loop failure.