Daniel Renken
Author directory2026
Exploring Confirmation Bias: LLMs Perceive Democratic Claims as More Truthful Independently of their Content
Charlott Jakob | Jing Yang | Daniel Renken | Veronika Solopova | Max Upravitelev | Sebastian Möller | Vera Schmitt
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Charlott Jakob | Jing Yang | Daniel Renken | Veronika Solopova | Max Upravitelev | Sebastian Möller | Vera Schmitt
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large Language Models (LLMs) are increasingly used to detect misinformation and support human fact-checkers. However, their truthfulness assessments may be affected by systematic political biases. Motivated by confirmation bias in humans, i.e., interpreting evidence in ways that are partial to existing beliefs or expectations, we investigate whether LLMs assess truthfulness in line with their political attitudes. We compare LLMs’ political attitudinal bias (PB-A) with political bias in truthfulness perception (PB-T). We assess PB-T by prompting LLMs to evaluate Democratic and Republican claims, including conditions where the claim-making party is explicitly stated. Across 20 instruction-tuned LLMs, all models exhibit a left-leaning tendency in PB-T, statistically significant for roughly half of them. Moreover, 16 out of 20 models significantly factor in the claim maker’s party affiliation when predicting truthfulness. PB-T and PB-A correlate weakly to moderately, and larger models tend to exhibit more left-leaning PB-T. The code for reproducibility is available on GitHub.