Multimodal Sarcasm Target Identification in Tweets

Jiquan Wang, Lin Sun, Yi Liu, Meizhi Shao, Zengwei Zheng


Abstract
Sarcasm is important to sentiment analysis on social media. Sarcasm Target Identification (STI) deserves further study to understand sarcasm in depth. However, text lacking context or missing sarcasm target makes target identification very difficult. In this paper, we introduce multimodality to STI and present Multimodal Sarcasm Target Identification (MSTI) task. We propose a novel multi-scale cross-modality model that can simultaneously perform textual target labeling and visual target detection. In the model, we extract multi-scale visual features to enrich spatial information for different sized visual sarcasm targets. We design a set of convolution networks to unify multi-scale visual features with textual features for cross-modal attention learning, and correspondingly a set of transposed convolution networks to restore multi-scale visual information. The results show that visual clues can improve the performance of TSTI by a large margin, and VSTI achieves good accuracy.
Anthology ID:
2022.acl-long.562
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8164–8175
Language:
URL:
https://aclanthology.org/2022.acl-long.562
DOI:
10.18653/v1/2022.acl-long.562
Bibkey:
Cite (ACL):
Jiquan Wang, Lin Sun, Yi Liu, Meizhi Shao, and Zengwei Zheng. 2022. Multimodal Sarcasm Target Identification in Tweets. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8164–8175, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Multimodal Sarcasm Target Identification in Tweets (Wang et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.562.pdf
Software:
 2022.acl-long.562.software.zip
Code
 wjq-learning/msti