An Empirical Analysis of Diversity in Argument Summarization

Michiel Van Der Meer, Piek Vossen, Catholijn Jonker, Pradeep Murukannaiah


Abstract
Presenting high-level arguments is a crucial task for fostering participation in online societal discussions. Current argument summarization approaches miss an important facet of this task—capturing diversity—which is important for accommodating multiple perspectives. We introduce three aspects of diversity: those of opinions, annotators, and sources. We evaluate approaches to a popular argument summarization task called Key Point Analysis, which shows how these approaches struggle to (1) represent arguments shared by few people, (2) deal with data from various sources, and (3) align with subjectivity in human-provided annotations. We find that both general-purpose LLMs and dedicated KPA models exhibit this behavior, but have complementary strengths. Further, we observe that diversification of training data may ameliorate generalization in zero-shot cases. Addressing diversity in argument summarization requires a mix of strategies to deal with subjectivity.
Anthology ID:
2024.eacl-long.123
Volume:
Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2028–2045
Language:
URL:
https://aclanthology.org/2024.eacl-long.123
DOI:
Bibkey:
Cite (ACL):
Michiel Van Der Meer, Piek Vossen, Catholijn Jonker, and Pradeep Murukannaiah. 2024. An Empirical Analysis of Diversity in Argument Summarization. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2028–2045, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
An Empirical Analysis of Diversity in Argument Summarization (Van Der Meer et al., EACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eacl-long.123.pdf
Software:
 2024.eacl-long.123.software.zip