Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization

Puyuan Liu, Chenyang Huang, Lili Mou


Abstract
Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS first performs edit-based search towards a heuristically defined score, and generates a summary as pseudo-groundtruth. Then, we train an encoder-only non-autoregressive Transformer based on the search result. We also propose a dynamic programming approach for length-control decoding, which is important for the summarization task. Experiments on two datasets show that NAUS achieves state-of-the-art performance for unsupervised summarization, yet largely improving inference efficiency. Further, our algorithm is able to perform explicit length-transfer summary generation.
Anthology ID:
2022.acl-long.545
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7916–7929
Language:
URL:
https://aclanthology.org/2022.acl-long.545
DOI:
10.18653/v1/2022.acl-long.545
Bibkey:
Cite (ACL):
Puyuan Liu, Chenyang Huang, and Lili Mou. 2022. Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7916–7929, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization (Liu et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.545.pdf
Code
 manga-uofa/naus