Comparing LLM Token Distributions: An Interactive Zipf–KL Explorer (opens in new tab)

An interactive visualisation of KL divergence between two Zipf-shaped token distributions — the kind a language model produces. Set each distribution’s entropy, shuffle the rank ordering by a chosen Spearman correlation, and watch forward and reverse KL respond in real time.