What Makes a Good and Useful Summary? Incorporating Users in Automatic Summarization Research

Maartje Ter Hoeve, Julia Kiseleva, Maarten de Rijke


Abstract
Automatic text summarization has enjoyed great progress over the years and is used in numerous applications, impacting the lives of many. Despite this development, there is little research that meaningfully investigates how the current research focus in automatic summarization aligns with users’ needs. To bridge this gap, we propose a survey methodology that can be used to investigate the needs of users of automatically generated summaries. Importantly, these needs are dependent on the target group. Hence, we design our survey in such a way that it can be easily adjusted to investigate different user groups. In this work we focus on university students, who make extensive use of summaries during their studies. We find that the current research directions of the automatic summarization community do not fully align with students’ needs. Motivated by our findings, we present ways to mitigate this mismatch in future research on automatic summarization: we propose research directions that impact the design, the development and the evaluation of automatically generated summaries.
Anthology ID:
2022.naacl-main.4
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
46–75
Language:
URL:
https://aclanthology.org/2022.naacl-main.4
DOI:
10.18653/v1/2022.naacl-main.4
Bibkey:
Cite (ACL):
Maartje Ter Hoeve, Julia Kiseleva, and Maarten de Rijke. 2022. What Makes a Good and Useful Summary? Incorporating Users in Automatic Summarization Research. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 46–75, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
What Makes a Good and Useful Summary? Incorporating Users in Automatic Summarization Research (Ter Hoeve et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.4.pdf
Video:
 https://aclanthology.org/2022.naacl-main.4.mp4