What Sounds “Right” to Me? Experiential Factors in the Perception of Political Ideology

Qinlan Shen, Carolyn Rose


Abstract
In this paper, we challenge the assumption that political ideology is inherently built into text by presenting an investigation into the impact of experiential factors on annotator perceptions of political ideology. We construct an annotated corpus of U.S. political discussion, where in addition to ideology labels for texts, annotators provide information about their political affiliation, exposure to political news, and familiarity with the source domain of discussion, Reddit. We investigate the variability in ideology judgments across annotators, finding evidence that these experiential factors may influence the consistency of how political ideologies are perceived. Finally, we present evidence that understanding how humans perceive and interpret ideology from texts remains a challenging task for state-of-the-art language models, pointing towards potential issues when modeling user experiences that may require more contextual knowledge.
Anthology ID:
2021.eacl-main.152
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Month:
April
Year:
2021
Address:
Online
Editors:
Paola Merlo, Jorg Tiedemann, Reut Tsarfaty
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1762–1771
Language:
URL:
https://aclanthology.org/2021.eacl-main.152
DOI:
10.18653/v1/2021.eacl-main.152
Bibkey:
Cite (ACL):
Qinlan Shen and Carolyn Rose. 2021. What Sounds “Right” to Me? Experiential Factors in the Perception of Political Ideology. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1762–1771, Online. Association for Computational Linguistics.
Cite (Informal):
What Sounds “Right” to Me? Experiential Factors in the Perception of Political Ideology (Shen & Rose, EACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.eacl-main.152.pdf