How “Loco” Is the LOCO Corpus? Annotating the Language of Conspiracy Theories

Ludovic Mompelat, Zuoyu Tian, Amanda Kessler, Matthew Luettgen, Aaryana Rajanala, Sandra Kübler, Michelle Seelig


Abstract
Conspiracy theories have found a new channel on the internet and spread by bringing together like-minded people, thus functioning as an echo chamber. The new 88-million word corpus Language of Conspiracy (LOCO) was created with the intention to provide a text collection to study how the language of conspiracy differs from mainstream language. We use this corpus to develop a robust annotation scheme that will allow us to distinguish between documents containing conspiracy language and documents that do not contain any conspiracy content or that propagate conspiracy theories via misinformation (which we explicitly disregard in our work). We find that focusing on indicators of a belief in a conspiracy combined with textual cues of conspiracy language allows us to reach a substantial agreement (based on Fleiss’ kappa and Krippendorff’s alpha). We also find that the automatic retrieval methods used to collect the corpus work well in finding mainstream documents, but include some documents in the conspiracy category that would not belong there based on our definition.
Anthology ID:
2022.law-1.14
Volume:
Proceedings of the 16th Linguistic Annotation Workshop (LAW-XVI) within LREC2022
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Sameer Pradhan, Sandra Kuebler
Venue:
LAW
SIG:
SIGANN
Publisher:
European Language Resources Association
Note:
Pages:
111–119
Language:
URL:
https://aclanthology.org/2022.law-1.14
DOI:
Bibkey:
Cite (ACL):
Ludovic Mompelat, Zuoyu Tian, Amanda Kessler, Matthew Luettgen, Aaryana Rajanala, Sandra Kübler, and Michelle Seelig. 2022. How “Loco” Is the LOCO Corpus? Annotating the Language of Conspiracy Theories. In Proceedings of the 16th Linguistic Annotation Workshop (LAW-XVI) within LREC2022, pages 111–119, Marseille, France. European Language Resources Association.
Cite (Informal):
How “Loco” Is the LOCO Corpus? Annotating the Language of Conspiracy Theories (Mompelat et al., LAW 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.law-1.14.pdf