Published on January 23, 2026 10:21 PM GMT
(This post elaborates on a few ideas from my review of Sam Eisenstat’s Condensation: a theory of concepts. It should be somewhat readable on its own but doesn’t fully explain what condensation is on its own; for that, see my review or Sam’s paper. The post came out of conversations with Sam.)
As I mentioned in my Condensation review, the difference between compression and condensation fits the physical analogy suggested by their names: compression mashes all the information together, while condensation (still compresses size, but) sorts information into discrete droplets.
Thus, condensation has a property we might call local relevance: typical questions can be answered at a glance, ie, retrieving small subsets of the information. This type of representation is sometimes called “symbolic”:
| Symbolic | Not Symbolic |
|---|---|
| A number can be quickly determined positive or negative by checking whether there is a “-” symbol in front. | “reading the room” at a social gathering requires integrating diverse cues. |
| The topic of a paper can be determined by reading the title and abstract. | The semantic content in a vector representation inside an artificial neural network is often represented redundantly, spread across the whole vector. |
| A person’s age can be determined by looking at their birthdate on a government-issued ID. | The quality of a work of art is spread throughout the whole piece. |
| The subject of a sentence can be found before the verb. | Determining the subject of a photograph requires understanding the whole image. |
| A target library book can be quickly retrieved from the shelves. | Finding gold nuggets requires sifting huge amounts of sand. |
This notion of “symbolic” seems related to interpretability (and the theory of condensation seeks to clarify this relationship).
The notion of “relevance” in condensation is what Sam calls the contribution relation.