Evaluating Dialogs based on Grice’s Maxims

Prathyusha Jwalapuram


Abstract
There is no agreed upon standard for the evaluation of conversational dialog systems, which are well-known to be hard to evaluate due to the difficulty in pinning down metrics that will correspond to human judgements and the subjective nature of human judgment itself. We explored the possibility of using Grice’s Maxims to evaluate effective communication in conversation. We collected some system generated dialogs from popular conversational chatbots across the spectrum and conducted a survey to see how the human judgements based on Gricean maxims correlate, and if such human judgments can be used as an effective evaluation metric for conversational dialog.
Anthology ID:
R17-2003
Volume:
Proceedings of the Student Research Workshop Associated with RANLP 2017
Month:
September
Year:
2017
Address:
Varna
Editors:
Venelin Kovatchev, Irina Temnikova, Pepa Gencheva, Yasen Kiprov, Ivelina Nikolova
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
17–24
Language:
URL:
https://doi.org/10.26615/issn.1314-9156.2017_003
DOI:
10.26615/issn.1314-9156.2017_003
Bibkey:
Cite (ACL):
Prathyusha Jwalapuram. 2017. Evaluating Dialogs based on Grice’s Maxims. In Proceedings of the Student Research Workshop Associated with RANLP 2017, pages 17–24, Varna. INCOMA Ltd..
Cite (Informal):
Evaluating Dialogs based on Grice’s Maxims (Jwalapuram, RANLP 2017)
Copy Citation:
PDF:
https://doi.org/10.26615/issn.1314-9156.2017_003