Dense Retrievers Know More Than They Can Express (opens in new tab)

Demonstrating that dense retrieval models learn much more information than they can express through their usual scoring mechanism: they also contain an indexable, natural-language-like sparse vocabulary, which is plug-and-play with BM25.