Beyond Emotion: A Multi-Modal Dataset for Human Desire Understanding

Ao Jia, Yu He, Yazhou Zhang, Sagar Uprety, Dawei Song, Christina Lioma


Abstract
Desire is a strong wish to do or have something, which involves not only a linguistic expression, but also underlying cognitive phenomena driving human feelings. As the most primitive and basic human instinct, conscious desire is often accompanied by a range of emotional responses. As a strikingly understudied task, it is difficult for machines to model and understand desire due to the unavailability of benchmarking datasets with desire and emotion labels. To bridge this gap, we present MSED, the first multi-modal and multi-task sentiment, emotion and desire dataset, which contains 9,190 text-image pairs, with English text. Each multi-modal sample is annotated with six desires, three sentiments and six emotions. We also propose the state-of-the-art baselines to evaluate the potential of MSED and show the importance of multi-task and multi-modal clues for desire understanding. We hope this study provides a benchmark for human desire analysis. MSED will be publicly available for research.
Anthology ID:
2022.naacl-main.108
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1512–1522
Language:
URL:
https://aclanthology.org/2022.naacl-main.108
DOI:
10.18653/v1/2022.naacl-main.108
Bibkey:
Cite (ACL):
Ao Jia, Yu He, Yazhou Zhang, Sagar Uprety, Dawei Song, and Christina Lioma. 2022. Beyond Emotion: A Multi-Modal Dataset for Human Desire Understanding. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1512–1522, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Beyond Emotion: A Multi-Modal Dataset for Human Desire Understanding (Jia et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.108.pdf
Video:
 https://aclanthology.org/2022.naacl-main.108.mp4
Data
IEMOCAPMELDMultimodal Opinionlevel Sentiment Intensity