Why Longer System Prompts Usually Make LLMs Worse (opens in new tab)

There’s a pattern that shows up constantly in LLM deployments: something isn’t working quite right, so someone adds more instructions to the system prompt. The model ignores a constraint, so you restate it more forcefully. It produces the wrong tone, so you add a tone guide. Repeat until the prompt is 2,000 words long and the model is somehow worse than when you started. This isn’t a fringe experience. It’s close to a law of LLM prompt engineering. Here’s why it keeps happening. 1. LLMs Don’t...