Zara: A Virtual Interactive Dialogue System Incorporating Emotion, Sentiment and Personality Recognition

Pascale Fung, Anik Dey, Farhad Bin Siddique, Ruixi Lin, Yang Yang, Dario Bertero, Yan Wan, Ricky Ho Yin Chan, Chien-Sheng Wu


Abstract
Zara, or ‘Zara the Supergirl’ is a virtual robot, that can exhibit empathy while interacting with an user, with the aid of its built in facial and emotion recognition, sentiment analysis, and speech module. At the end of the 5-10 minute conversation, Zara can give a personality analysis of the user based on all the user utterances. We have also implemented a real-time emotion recognition, using a CNN model that detects emotion from raw audio without feature extraction, and have achieved an average of 65.7% accuracy on six different emotion classes, which is an impressive 4.5% improvement from the conventional feature based SVM classification. Also, we have described a CNN based sentiment analysis module trained using out-of-domain data, that recognizes sentiment from the speech recognition transcript, which has a 74.8 F-measure when tested on human-machine dialogues.
Anthology ID:
C16-2058
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations
Month:
December
Year:
2016
Address:
Osaka, Japan
Editor:
Hideo Watanabe
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
278–281
Language:
URL:
https://aclanthology.org/C16-2058
DOI:
Bibkey:
Cite (ACL):
Pascale Fung, Anik Dey, Farhad Bin Siddique, Ruixi Lin, Yang Yang, Dario Bertero, Yan Wan, Ricky Ho Yin Chan, and Chien-Sheng Wu. 2016. Zara: A Virtual Interactive Dialogue System Incorporating Emotion, Sentiment and Personality Recognition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pages 278–281, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Zara: A Virtual Interactive Dialogue System Incorporating Emotion, Sentiment and Personality Recognition (Fung et al., COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-2058.pdf