Inverse Reinforcement Learning for Text Summarization

Yu Fu, Deyi Xiong, Yue Dong


Abstract
We introduce inverse reinforcement learning (IRL) as an effective paradigm for training abstractive summarization models, imitating human summarization behaviors. Our IRL model estimates the reward function using a suite of important sub-rewards for summarization and concurrently optimizes the policy network. Experimental results across datasets in different domains (CNN/DailyMail and WikiHow) and various model sizes (BART-base and BART-large) demonstrate the superiority of our proposed IRL model for summarization over MLE and RL baselines. The resulting summaries exhibit greater similarity to human-crafted gold references, outperforming MLE and RL baselines on metrics such as ROUGE, coverage, novelty, compression ratio, factuality, and human evaluations.
Anthology ID:
2023.findings-emnlp.436
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6559–6570
Language:
URL:
https://aclanthology.org/2023.findings-emnlp.436
DOI:
10.18653/v1/2023.findings-emnlp.436
Bibkey:
Cite (ACL):
Yu Fu, Deyi Xiong, and Yue Dong. 2023. Inverse Reinforcement Learning for Text Summarization. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6559–6570, Singapore. Association for Computational Linguistics.
Cite (Informal):
Inverse Reinforcement Learning for Text Summarization (Fu et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-emnlp.436.pdf