Descriptive Prompt Paraphrasing for Target-Oriented Multimodal Sentiment Classification

Dan Liu, Lin Li, Xiaohui Tao, Jian Cui, Qing Xie


Abstract
Target-Oriented Multimodal Sentiment Classification (TMSC) aims to perform sentiment polarity on a target jointly considering its corresponding multiple modalities including text, image, and others. Current researches mainly work on either of two types of targets in a decentralized manner. One type is entity, such as a person name, a location name, etc. and the other is aspect, such as ‘food’, ‘service’, etc. We believe that this target type based division in task modelling is not necessary because the sentiment polarity of the specific target is not governed by its type but its context. For this reason, we propose a unified model for target-oriented multimodal sentiment classification, so called UnifiedTMSC. It is prompt-based language modelling and performs well on four datasets spanning the above two target types. Specifically, we design descriptive prompt paraphrasing to reformulate TMSC task via (1) task paraphrasing, which obtains paraphrased prompts based on the task description through a paraphrasing rule, and (2) image prefix tuning, which optimizes a small continuous image vector throughout the multimodal representation space of text and images. Conducted on two entity-level multimodal datasets: Twitter-2015 and Twitter-2017, and two aspect-level multimodal datasets: Multi-ZOL and MASAD, the experimental results show the effectiveness of our UnifiedTMSC.
Anthology ID:
2023.findings-emnlp.275
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4174–4186
Language:
URL:
https://aclanthology.org/2023.findings-emnlp.275
DOI:
10.18653/v1/2023.findings-emnlp.275
Bibkey:
Cite (ACL):
Dan Liu, Lin Li, Xiaohui Tao, Jian Cui, and Qing Xie. 2023. Descriptive Prompt Paraphrasing for Target-Oriented Multimodal Sentiment Classification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4174–4186, Singapore. Association for Computational Linguistics.
Cite (Informal):
Descriptive Prompt Paraphrasing for Target-Oriented Multimodal Sentiment Classification (Liu et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-emnlp.275.pdf