Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization

Zhiyuan Zeng, Jiaze Chen, Weiran Xu, Lei Li


Abstract
Neural abstractive summarization systems have gained significant progress in recent years. However, abstractive summarization often produce inconsisitent statements or false facts. How to automatically generate highly abstract yet factually correct summaries? In this paper, we proposed an efficient weak-supervised adversarial data augmentation approach to form the factual consistency dataset. Based on the artificial dataset, we train an evaluation model that can not only make accurate and robust factual consistency discrimination but is also capable of making interpretable factual errors tracing by backpropagated gradient distribution on token embeddings. Experiments and analysis conduct on public annotated summarization and factual consistency datasets demonstrate our approach effective and reasonable.
Anthology ID:
2021.emnlp-main.337
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4102–4108
Language:
URL:
https://aclanthology.org/2021.emnlp-main.337
DOI:
10.18653/v1/2021.emnlp-main.337
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.337.pdf
Data
CNN/Daily Mail