Persona-Aware Evaluation of Cognitive Bias in LLMs: From Benchmark to Applied Decision-Making

Katsumasa Yoshikawa, Junya Takayama, Takato Yamazaki


Abstract
We present a persona-aware evaluation suite that couples a 12-category cognitive-bias benchmark with 100 applied financial framing tasks to assess how large language models (LLMs) respond under systematically varied persona conditions. Using a factorized set of 162 personas spanning gender, age, political orientation, income, and education, we analyze how persona conditioning modulates bias-consistent responding across ten instruction-tuned models. On applied tasks, persona conditioning reduces framing reversals on average and slightly increases decision confidence, with substantial variation across model families and scales. Correlation analyses further reveal that benchmark bias tendencies—particularly availability, social proof, and framing—predict applied framing sensitivity, suggesting that standardized bias scores can serve as indicators of real-world decision variability. This work provides a unified framework for linking cognitive-bias evaluation with persona-conditioned decision behavior in LLMs. (All data and prompts will be released after acceptance to preserve anonymity.)
Anthology ID:
2026.lrec-1.332
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
4213–4225
Language:
External URL:
https://lrec.elra.info/lrec2026-main-332
DOI:
10.63317/2dvjjaywrket
Bibkey:
Cite (ACL):
Katsumasa Yoshikawa, Junya Takayama, and Takato Yamazaki. 2026. Persona-Aware Evaluation of Cognitive Bias in LLMs: From Benchmark to Applied Decision-Making. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 4213–4225, Palma de Mallorca, Spain. ELRA Language Resource Association.
Cite (Informal):
Persona-Aware Evaluation of Cognitive Bias in LLMs: From Benchmark to Applied Decision-Making (Yoshikawa et al., LREC 2026)
Copy Citation: