Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sources

Yerin Hwang, Yongil Kim, Hyunkyung Bae, Hwanhee Lee, Jeesoo Bang, Kyomin Jung


Abstract
To address the data scarcity issue in Conversational question answering (ConvQA), a dialog inpainting method, which utilizes documents to generate ConvQA datasets, has been proposed. However, the original dialog inpainting model is trained solely on the dialog reconstruction task, resulting in the generation of questions with low contextual relevance due to insufficient learning of question-answer alignment. To overcome this limitation, we propose a novel framework called Dialogizer, which has the capability to automatically generate ConvQA datasets with high contextual relevance from textual sources. The framework incorporates two training tasks: question-answer matching (QAM) and topic-aware dialog generation (TDG). Moreover, re-ranking is conducted during the inference phase based on the contextual relevance of the generated questions. Using our framework, we produce four ConvQA datasets by utilizing documents from multiple domains as the primary source. Through automatic evaluation using diverse metrics, as well as human evaluation, we validate that our proposed framework exhibits the ability to generate datasets of higher quality compared to the baseline dialog inpainting model.
Anthology ID:
2023.emnlp-main.545
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8806–8828
Language:
URL:
https://aclanthology.org/2023.emnlp-main.545
DOI:
10.18653/v1/2023.emnlp-main.545
Bibkey:
Cite (ACL):
Yerin Hwang, Yongil Kim, Hyunkyung Bae, Hwanhee Lee, Jeesoo Bang, and Kyomin Jung. 2023. Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sources. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8806–8828, Singapore. Association for Computational Linguistics.
Cite (Informal):
Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sources (Hwang et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.545.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.545.mp4