Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset

Hannah Rashkin, Eric Michael Smith, Margaret Li, Y-Lan Boureau


Abstract
One challenge for dialogue agents is recognizing feelings in the conversation partner and replying accordingly, a key communicative skill. While it is straightforward for humans to recognize and acknowledge others’ feelings in a conversation, this is a significant challenge for AI systems due to the paucity of suitable publicly-available datasets for training and evaluation. This work proposes a new benchmark for empathetic dialogue generation and EmpatheticDialogues, a novel dataset of 25k conversations grounded in emotional situations. Our experiments indicate that dialogue models that use our dataset are perceived to be more empathetic by human evaluators, compared to models merely trained on large-scale Internet conversation data. We also present empirical comparisons of dialogue model adaptations for empathetic responding, leveraging existing models or datasets without requiring lengthy re-training of the full model.
Anthology ID:
P19-1534
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5370–5381
Language:
URL:
https://aclanthology.org/P19-1534
DOI:
10.18653/v1/P19-1534
Bibkey:
Cite (ACL):
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset (Rashkin et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1534.pdf
Supplementary:
 P19-1534.Supplementary.pdf
Code
 facebookresearch/EmpatheticDialogues +  additional community code
Data
EmpatheticDialogues