DiffQG: Generating Questions to Summarize Factual Changes

Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos, Michael J.Q. Zhang, Eunsol Choi, Bhuwan Dhingra


Abstract
Identifying the difference between two versions of the same article is useful to update knowledge bases and to understand how articles evolve. Paired texts occur naturally in diverse situations: reporters write similar news stories and maintainers of authoritative websites must keep their information up to date. We propose representing factual changes between paired documents as question-answer pairs, where the answer to the same question differs between two versions. We find that question-answer pairs can flexibly and concisely capture the updated contents. Provided with paired documents, annotators identify questions that are answered by one passage but answered differently or cannot be answered by the other. We release DiffQG which consists of 759 QA pairs and 1153 examples of paired passages with no factual change. These questions are intended to be both unambiguous and information-seeking and involve complex edits, pushing beyond the capabilities of current question generation and factual change detection systems. Our dataset summarizes the changes between two versions of the document as questions and answers, studying automatic update summarization in a novel way.
Anthology ID:
2023.eacl-main.225
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3088–3101
Language:
URL:
https://aclanthology.org/2023.eacl-main.225
DOI:
10.18653/v1/2023.eacl-main.225
Bibkey:
Cite (ACL):
Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos, Michael J.Q. Zhang, Eunsol Choi, and Bhuwan Dhingra. 2023. DiffQG: Generating Questions to Summarize Factual Changes. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3088–3101, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
DiffQG: Generating Questions to Summarize Factual Changes (Cole et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.225.pdf
Video:
 https://aclanthology.org/2023.eacl-main.225.mp4