Hidden Persuaders: LLMs’ Political Leaning and Their Influence on Voters

Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, Dawn Song


Abstract
Do LLMs have political leanings and are LLMs able to shift our political views? This paper explores these questions in the context of the 2024 U.S. presidential election. Through a voting simulation, we demonstrate 18 open-weight and closed-source LLMs’ political preference for Biden over Trump. We show how Biden-leaning becomes more pronounced in instruction-tuned and reinforced models compared to their base versions by analyzing their responses to political questions related to the two nominees. We further explore the potential impact of LLMs on voter choice by recruiting 935 U.S. registered voters. Participants interacted with LLMs (Claude-3, Llama-3, and GPT-4) over five exchanges. Intriguingly, although LLMs were not asked to persuade users to support Biden, about 20% of Trump supporters reduced their support for Trump after LLM interaction. This result is noteworthy given that many studies on the persuasiveness of political campaigns have shown minimal effects in presidential elections. Many users also expressed a desire for further interaction with LLMs on political subjects. Further research on how LLMs affect users’ political views is required, as their use becomes more widespread.
Anthology ID:
2024.emnlp-main.244
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4244–4275
Language:
URL:
https://aclanthology.org/2024.emnlp-main.244
DOI:
10.18653/v1/2024.emnlp-main.244
Bibkey:
Cite (ACL):
Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, and Dawn Song. 2024. Hidden Persuaders: LLMs’ Political Leaning and Their Influence on Voters. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4244–4275, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Hidden Persuaders: LLMs’ Political Leaning and Their Influence on Voters (Potter et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-main.244.pdf