Paper Summary: Dual-Encoders in Ranking (opens in new tab)

In Defense of Dual-Encoders for Neural Ranking by Menon et. al. discusses the question of why dual-encoder (DE) models, also called Bi-Encoders elsewhere, don’t match the performance of cross-attention (CA) models. The authors investigate what is actually going on, and demonstrate some improved performance over baseline DE models with a new model distillation method. Background Search requires an automatic way to find the most relevant documents to a query. There are bag-of-word approaches to...