Multilogue-Net: A Context-Aware RNN for Multi-modal Emotion Detection and Sentiment Analysis in Conversation

Aman Shenoy, Ashish Sardana


Abstract
Sentiment Analysis and Emotion Detection in conversation is key in several real-world applications, with an increase in modalities available aiding a better understanding of the underlying emotions. Multi-modal Emotion Detection and Sentiment Analysis can be particularly useful, as applications will be able to use specific subsets of available modalities, as per the available data. Current systems dealing with Multi-modal functionality fail to leverage and capture - the context of the conversation through all modalities, the dependency between the listener(s) and speaker emotional states, and the relevance and relationship between the available modalities. In this paper, we propose an end to end RNN architecture that attempts to take into account all the mentioned drawbacks. Our proposed model, at the time of writing, out-performs the state of the art on a benchmark dataset on a variety of accuracy and regression metrics.
Anthology ID:
2020.challengehml-1.3
Volume:
Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML)
Month:
July
Year:
2020
Address:
Seattle, USA
Editors:
Amir Zadeh, Louis-Philippe Morency, Paul Pu Liang, Soujanya Poria
Venue:
Challenge-HML
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
19–28
Language:
URL:
https://aclanthology.org/2020.challengehml-1.3
DOI:
10.18653/v1/2020.challengehml-1.3
Bibkey:
Cite (ACL):
Aman Shenoy and Ashish Sardana. 2020. Multilogue-Net: A Context-Aware RNN for Multi-modal Emotion Detection and Sentiment Analysis in Conversation. In Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML), pages 19–28, Seattle, USA. Association for Computational Linguistics.
Cite (Informal):
Multilogue-Net: A Context-Aware RNN for Multi-modal Emotion Detection and Sentiment Analysis in Conversation (Shenoy & Sardana, Challenge-HML 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.challengehml-1.3.pdf
Video:
 http://slideslive.com/38931258
Data
Multimodal Opinionlevel Sentiment Intensity