Emily Black
Author directory2026
Do AI Reviewers Converge? Diversity Collapse in Model-Generated Peer Review
Xuzhe Hou | Shengxin Zhang | Riley Stacy | Emily Black | Varun Chandrasekaran
Findings of the Association for Computational Linguistics: EMNLP 2026
Xuzhe Hou | Shengxin Zhang | Riley Stacy | Emily Black | Varun Chandrasekaran
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) are increasingly used to generate or assist reviews. This raises the risk of generative monoculture, where models converge on homogeneous judgments and language patterns, limiting the novelty of scientific progress. We study this phenomenon over 1,943 human reviews from 509 ICLR papers with 10,172 reviews generated by five state-of-the-art LLMs under multiple prompting strategies. We evaluate human and LLM review diversity across several dimensions, including praise-to-critique balance and semantic similarity. Across metrics, LLM-generated reviews exhibit systematic convergence: for example, they are substantially more praise-heavy, and more semantically similar across unrelated papers, suggesting a lack of paper-specific feedback. Specialized review agents reduce—but do not eliminate—these effects. Our work demonstrates the risk of introducing epistemic monoculture into peer review and develops methods to measure it. If left unchecked, this monoculture could narrow the range of perspectives that shape scientific evaluation.
2025
Multilingual Prompting for Improving LLM Generation Diversity
Qihan Wang | Shidong Pan | Tal Linzen | Emily Black
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Qihan Wang | Shidong Pan | Tal Linzen | Emily Black
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Large Language Models (LLMs) are known to lack cultural representation and overall diversity in their generations, from expressing opinions to answering factual questions. To mitigate this problem, we propose multilingual prompting: a prompting method which generates several variations of a base prompt with added cultural and linguistic cues from several cultures, generates responses, and then combines the results. Building on evidence that LLMs have language-specific knowledge, multilingual prompting seeks to increase diversity by activating a broader range of cultural knowledge embedded in model training data. Through experiments across multiple models (GPT-4o, GPT-4o-mini, LLaMA 70B, and LLaMA 8B), we show that multilingual prompting consistently outperforms existing diversity-enhancing techniques such as high-temperature sampling, step-by-step recall, and persona prompting. Further analyses show that the benefits of multilingual prompting vary between high and low resource languages and across model sizes, and that aligning the prompting language with cultural cues reduces hallucination about culturally-specific information.