On Reducing Repetition in Abstractive Summarization

Pranav Nair, Anil Kumar Singh


Abstract
Repetition in natural language generation reduces the informativeness of text and makes it less appealing. Various techniques have been proposed to alleviate it. In this work, we explore and propose techniques to reduce repetition in abstractive summarization. First, we explore the application of unlikelihood training and embedding matrix regularizers from previous work on language modeling to abstractive summarization. Next, we extend the coverage and temporal attention mechanisms to the token level to reduce repetition. In our experiments on the CNN/Daily Mail dataset, we observe that these techniques reduce the amount of repetition and increase the informativeness of the summaries, which we confirm via human evaluation.
Anthology ID:
2021.ranlp-srw.18
Volume:
Proceedings of the Student Research Workshop Associated with RANLP 2021
Month:
September
Year:
2021
Address:
Online
Editors:
Souhila Djabri, Dinara Gimadi, Tsvetomila Mihaylova, Ivelina Nikolova-Koleva
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
126–134
Language:
URL:
https://aclanthology.org/2021.ranlp-srw.18
DOI:
Bibkey:
Cite (ACL):
Pranav Nair and Anil Kumar Singh. 2021. On Reducing Repetition in Abstractive Summarization. In Proceedings of the Student Research Workshop Associated with RANLP 2021, pages 126–134, Online. INCOMA Ltd..
Cite (Informal):
On Reducing Repetition in Abstractive Summarization (Nair & Singh, RANLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.ranlp-srw.18.pdf
Data
CNN/Daily Mail