Temperature in LLMs Changes Much More Than Randomness (opens in new tab)

Most developers treat the temperature parameter like a volume knob: turn it up for creative writing, turn it down for factual summaries. This mental model is wrong in ways that matter in production. Temperature doesn’t add randomness on top of the model’s output. It transforms the probability distribution that the model samples from, and that transformation is nonlinear, meaning small changes at the extremes have outsized effects. Understanding what’s actually happening explains a lot of othe...