The patient is more dead than alive: exploring the current state of the multi-document summarisation of the biomedical literature

Yulia Otmakhova, Karin Verspoor, Timothy Baldwin, Jey Han Lau


Abstract
Although multi-document summarisation (MDS) of the biomedical literature is a highly valuable task that has recently attracted substantial interest, evaluation of the quality of biomedical summaries lacks consistency and transparency. In this paper, we examine the summaries generated by two current models in order to understand the deficiencies of existing evaluation approaches in the context of the challenges that arise in the MDS task. Based on this analysis, we propose a new approach to human evaluation and identify several challenges that must be overcome to develop effective biomedical MDS systems.
Anthology ID:
2022.acl-long.350
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5098–5111
Language:
URL:
https://aclanthology.org/2022.acl-long.350
DOI:
10.18653/v1/2022.acl-long.350
Bibkey:
Cite (ACL):
Yulia Otmakhova, Karin Verspoor, Timothy Baldwin, and Jey Han Lau. 2022. The patient is more dead than alive: exploring the current state of the multi-document summarisation of the biomedical literature. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5098–5111, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
The patient is more dead than alive: exploring the current state of the multi-document summarisation of the biomedical literature (Otmakhova et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.350.pdf