Temperature and Sampling in Transformers: How LLMs Decide the Next Word (opens in new tab)

When working with Large Language Models (LLMs) like ChatGPT, many people assume that the model simply predicts the next word with the…