Beyond Mutual Information: Extension Profiles and Shape Functions of Random Variable Pairs (opens in new tab)

We study the extension profile of a pair of jointly distributed finite-valued random variables $(X,Y)$, defined as the set of all triples of numbers $ (H(X|W), H(Y|W), I(X:Y|W)) $ obtained by extending the pair with an auxiliary random variable $W$. This object captures structural properties of joint distributions that are not determined solely by the entropies of $X$ and $Y$ and their mutual information. To describe the boundary of the extensio...