ClarQ: A large-scale and diverse dataset for Clarification Question Generation

Vaibhav Kumar, Alan W Black


Abstract
Question answering and conversational systems are often baffled and need help clarifying certain ambiguities. However, limitations of existing datasets hinder the development of large-scale models capable of generating and utilising clarification questions. In order to overcome these limitations, we devise a novel bootstrapping framework (based on self-supervision) that assists in the creation of a diverse, large-scale dataset of clarification questions based on post-comment tuples extracted from stackexchange. The framework utilises a neural network based architecture for classifying clarification questions. It is a two-step method where the first aims to increase the precision of the classifier and second aims to increase its recall. We quantitatively demonstrate the utility of the newly created dataset by applying it to the downstream task of question-answering. The final dataset, ClarQ, consists of ~2M examples distributed across 173 domains of stackexchange. We release this dataset in order to foster research into the field of clarification question generation with the larger goal of enhancing dialog and question answering systems.
Anthology ID:
2020.acl-main.651
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7296–7301
Language:
URL:
https://aclanthology.org/2020.acl-main.651
DOI:
10.18653/v1/2020.acl-main.651
Bibkey:
Cite (ACL):
Vaibhav Kumar and Alan W Black. 2020. ClarQ: A large-scale and diverse dataset for Clarification Question Generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7296–7301, Online. Association for Computational Linguistics.
Cite (Informal):
ClarQ: A large-scale and diverse dataset for Clarification Question Generation (Kumar & Black, ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.651.pdf
Video:
 http://slideslive.com/38928968
Code
 vaibhav4595/ClarQ
Data
ClarQ