Authority Bias in Conversational Search Engines for Academic Paper Recommendation.

Uthman Jinadu, Parsa Ghazvinian, Anjila Budathoki, Benjamin M. Ampel, Rajshekhar Sunderraman, Yi Ding


Abstract
Large Language Models (LLMs) are increasingly used as conversational search engines for academic literature, yet whether they judge papers on content or on authority signals has not been tested causally. We investigate authority bias: systematic preference for papers based on author prestige, venue, and citations rather than content. Holding title and abstract constant, we vary authority metadata across three counterfactual conditions (original, flipped, boosted) over eight LLMs (five open-weight and three frontier closed-weight) in an in-context, single-turn, top-1 recommendation setting. Our experiments show that authority bias is substantial and directional, varies markedly across models, and is only partially addressable through prompt-level debiasing. We further document a say-do gap: debiasing instructions suppress authority mentions far faster than authority-driven flips, so surface auditing systematically underestimates behavioral bias.

Code: https://github.com/jinaduuthman/Authority-Bias-In-Conversational-Search-Engine

Data: https://huggingface.co/datasets/uthmanjinadu/authority-bias-paper-recommendation
Anthology ID:
2026.emnlp-main.1069
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
24548–24568
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1069/
DOI:
Bibkey:
Cite (ACL):
Uthman Jinadu, Parsa Ghazvinian, Anjila Budathoki, Benjamin M. Ampel, Rajshekhar Sunderraman, and Yi Ding. 2026. Authority Bias in Conversational Search Engines for Academic Paper Recommendation.. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 24548–24568, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Authority Bias in Conversational Search Engines for Academic Paper Recommendation. (Jinadu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1069.pdf