Are People Located in the Places They Mention in Their Tweets? A Multimodal Approach

Zhaomin Xiao, Eduardo Blanco


Abstract
This paper introduces the problem of determining whether people are located in the places they mention in their tweets. In particular, we investigate the role of text and images to solve this challenging problem. We present a new corpus of tweets that contain both text and images. Our analyses show that this problem is multimodal at its core: human judgments depend on whether annotators have access to the text, the image, or both. Experimental results show that a neural architecture that combines both modalities yields better results. We also conduct an error analysis to provide insights into why and when each modality is beneficial.
Anthology ID:
2022.coling-1.226
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Editors:
Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, Seung-Hoon Na
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
2561–2571
Language:
URL:
https://aclanthology.org/2022.coling-1.226
DOI:
Bibkey:
Cite (ACL):
Zhaomin Xiao and Eduardo Blanco. 2022. Are People Located in the Places They Mention in Their Tweets? A Multimodal Approach. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2561–2571, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Are People Located in the Places They Mention in Their Tweets? A Multimodal Approach (Xiao & Blanco, COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.226.pdf
Code
 zhaomin1995/coling2022_repo