Adji Bousso Dieng

Author directory

2026

Retrieval-augmented generation (RAG) for multi-hop question answering is limited by retrieving semantically redundant documents, when reasoning instead needs diverse, complementary sources. We introduce Vendi-RAG, which treats set-level semantic diversity as a first-class retrieval objective. It combines an eigenvalue-based diversity score (the Vendi Score) with query relevance into a single Vendi Retrieval Score, embedded in an iterative loop that uses an LLM-based answer-quality signal to adaptively rebalance diversity and relevance. On HotpotQA, MuSiQue, and 2WikiMultiHopQA, Vendi-RAG improves accuracy by +2.4, +1.6, and +2.4 points over the strongest diversity-aware iterative baseline (Adaptive Retrieval + MMR). A fixed-s variant captures most of the gain at no extra inference cost, while the LLM-judge loop adds a smaller but significant gain on hard queries. Results hold across five instruction-tuned backbones and two dense retrievers, identifying set-level retrieval diversity as a primary, controllable axis for improving multi-hop RAG.
Embedding-based topic models such as BERTopic routinely produce hundreds of fine-grained, overlapping topics that must be reduced before downstream use. Existing approaches couple reduction with outlier rejection, rely on local geometric linkage, or use costly and stochastic LLM prompting. We introduce Vendi Clustering (VC), a topic reduction method that reframes the problem as global diversity-preserving compression: at each step, VC selects the merge that induces the smallest loss in the Vendi Score, a spectral measure of diversity. A naive implementation of this global criterion is prohibitive at O(n6); we derive algebraic update rules at order q = 2 that evaluate each candidate merge in O(1) via cached row-square sums and a lookahead Gram matrix, reducing total complexity to a tractable O(n3). On 20 Newsgroups and AG News, VC outperforms standard reduction baselines on coherence and diversity, surpasses LLM-prompted reduction on the majority of metrics against both an open-weight model and a frontier API model, with topic diversity gains of up to 35% while running over 40 × faster.