Unsupervised Extractive Summarization using Pointwise Mutual Information

Vishakh Padmakumar, He He


Abstract
Unsupervised approaches to extractive summarization usually rely on a notion of sentence importance defined by the semantic similarity between a sentence and the document. We propose new metrics of relevance and redundancy using pointwise mutual information (PMI) between sentences, which can be easily computed by a pre-trained language model. Intuitively, a relevant sentence allows readers to infer the document content (high PMI with the document), and a redundant sentence can be inferred from the summary (high PMI with the summary). We then develop a greedy sentence selection algorithm to maximize relevance and minimize redundancy of extracted sentences. We show that our method outperforms similarity-based methods on datasets in a range of domains including news, medical journal articles, and personal anecdotes.
Anthology ID:
2021.eacl-main.213
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Month:
April
Year:
2021
Address:
Online
Editors:
Paola Merlo, Jorg Tiedemann, Reut Tsarfaty
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2505–2512
Language:
URL:
https://aclanthology.org/2021.eacl-main.213
DOI:
10.18653/v1/2021.eacl-main.213
Bibkey:
Cite (ACL):
Vishakh Padmakumar and He He. 2021. Unsupervised Extractive Summarization using Pointwise Mutual Information. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2505–2512, Online. Association for Computational Linguistics.
Cite (Informal):
Unsupervised Extractive Summarization using Pointwise Mutual Information (Padmakumar & He, EACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.eacl-main.213.pdf
Code
 vishakhpk/mi-unsup-summ
Data
Reddit TIFU