Is It Dish Washer Safe? Automatically Answering “Yes/No” Questions Using Customer Reviews

Daria Dzendzik, Carl Vogel, Jennifer Foster


Abstract
It has become commonplace for people to share their opinions about all kinds of products by posting reviews online. It has also become commonplace for potential customers to do research about the quality and limitations of these products by posting questions online. We test the extent to which reviews are useful in question-answering by combining two Amazon datasets and focusing our attention on yes/no questions. A manual analysis of 400 cases reveals that the reviews directly contain the answer to the question just over a third of the time. Preliminary reading comprehension experiments with this dataset prove inconclusive, with accuracy in the range 50-66%.
Anthology ID:
N19-3001
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Sudipta Kar, Farah Nadeem, Laura Burdick, Greg Durrett, Na-Rae Han
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–6
Language:
URL:
https://aclanthology.org/N19-3001
DOI:
10.18653/v1/N19-3001
Bibkey:
Cite (ACL):
Daria Dzendzik, Carl Vogel, and Jennifer Foster. 2019. Is It Dish Washer Safe? Automatically Answering “Yes/No” Questions Using Customer Reviews. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 1–6, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Is It Dish Washer Safe? Automatically Answering “Yes/No” Questions Using Customer Reviews (Dzendzik et al., NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/N19-3001.pdf
Data
MovieQA