Examining Bias in Opinion Summarisation through the Perspective of Opinion Diversity

Nannan Huang, Lin Tian, Haytham Fayek, Xiuzhen Zhang


Abstract
Opinion summarisation is a task that aims to condense the information presented in the source documents while retaining the core message and opinions. A summary that only represents the majority opinions will leave the minority opinions unrepresented in the summary. In this paper, we use the stance towards a certain target as an opinion. We study bias in opinion summarisation from the perspective of opinion diversity, which measures whether the model generated summary can cover a diverse set of opinions. In addition, we examine opinion similarity, a measure of how closely related two opinions are in terms of their stance on a given topic, and its relationship with opinion diversity. Through the lense of stances towards a topic, we examine opinion diversity and similarity using three debatable topics under COVID-19. Experimental results on these topics revealed that a higher degree of similarity of opinions did not indicate good diversity or fairly cover the various opinions originally presented in the source documents. We found that BART and ChatGPT can better capture diverse opinions presented in the source documents.
Anthology ID:
2023.wassa-1.14
Volume:
Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Jeremy Barnes, Orphée De Clercq, Roman Klinger
Venue:
WASSA
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
149–161
Language:
URL:
https://aclanthology.org/2023.wassa-1.14
DOI:
10.18653/v1/2023.wassa-1.14
Bibkey:
Cite (ACL):
Nannan Huang, Lin Tian, Haytham Fayek, and Xiuzhen Zhang. 2023. Examining Bias in Opinion Summarisation through the Perspective of Opinion Diversity. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 149–161, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Examining Bias in Opinion Summarisation through the Perspective of Opinion Diversity (Huang et al., WASSA 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.wassa-1.14.pdf
Video:
 https://aclanthology.org/2023.wassa-1.14.mp4