Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media

Shubham Mittal, Megha Sundriyal, Preslav Nakov


Abstract
Claim span identification (CSI) is an important step in fact-checking pipelines, aiming to identify text segments that contain a check-worthy claim or assertion in a social media post. Despite its importance to journalists and human fact-checkers, it remains a severely understudied problem, and the scarce research on this topic so far has only focused on English. Here we aim to bridge this gap by creating a novel dataset, X-CLAIM, consisting of 7K real-world claims collected from numerous social media platforms in five Indian languages and English. We report strong baselines with state-of-the-art encoder-only language models (e.g., XLM-R) and we demonstrate the benefits of training on multiple languages over alternative cross-lingual transfer methods such as zero-shot transfer, or training on translated data, from a high-resource language such as English. We evaluate generative large language models from the GPT series using prompting methods on the X-CLAIM dataset and we find that they underperform the smaller encoder-only language models for low-resource languages.
Anthology ID:
2023.emnlp-main.236
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3887–3902
Language:
URL:
https://aclanthology.org/2023.emnlp-main.236
DOI:
10.18653/v1/2023.emnlp-main.236
Bibkey:
Cite (ACL):
Shubham Mittal, Megha Sundriyal, and Preslav Nakov. 2023. Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3887–3902, Singapore. Association for Computational Linguistics.
Cite (Informal):
Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media (Mittal et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.236.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.236.mp4