Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality

Tanay Dixit, Fei Wang, Muhao Chen


Abstract
Improving factual consistency of abstractive summarization has been a widely studied topic. However, most of the prior works on training factuality-aware models have ignored the negative effect it has on summary quality. We propose {pasted macro ‘MODEL’}name (i.e. Effective Factual Summarization), a candidate summary generation and ranking technique to improve summary factuality without sacrificing quality. We show that using a contrastive learning framework with our refined candidate summaries leads to significant gains on both factuality and similarity-based metrics. Specifically, we propose a ranking strategy in which we effectively combine two metrics, thereby preventing any conflict during training. Models trained using our approach show up to 6 points of absolute improvement over the base model with respect to FactCC on XSUM and 11 points on CNN/DM, without negatively affecting either similarity-based metrics or absractiveness.
Anthology ID:
2023.acl-short.78
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
902–913
Language:
URL:
https://aclanthology.org/2023.acl-short.78
DOI:
10.18653/v1/2023.acl-short.78
Bibkey:
Cite (ACL):
Tanay Dixit, Fei Wang, and Muhao Chen. 2023. Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 902–913, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality (Dixit et al., ACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.acl-short.78.pdf
Video:
 https://aclanthology.org/2023.acl-short.78.mp4