Is Stance Detection Topic-Independent and Cross-topic Generalizable? - A Reproduction Study

Myrthe Reuver, Suzan Verberne, Roser Morante, Antske Fokkens


Abstract
Cross-topic stance detection is the task to automatically detect stances (pro, against, or neutral) on unseen topics. We successfully reproduce state-of-the-art cross-topic stance detection work (Reimers et. al, 2019), and systematically analyze its reproducibility. Our attention then turns to the cross-topic aspect of this work, and the specificity of topics in terms of vocabulary and socio-cultural context. We ask: To what extent is stance detection topic-independent and generalizable across topics? We compare the model’s performance on various unseen topics, and find topic (e.g. abortion, cloning), class (e.g. pro, con), and their interaction affecting the model’s performance. We conclude that investigating performance on different topics, and addressing topic-specific vocabulary and context, is a future avenue for cross-topic stance detection. References Nils Reimers, Benjamin Schiller, Tilman Beck, Johannes Daxenberger, Christian Stab, and Iryna Gurevych. 2019. Classification and Clustering of Arguments with Contextualized Word Embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 567–578, Florence, Italy. Association for Computational Linguistics.
Anthology ID:
2021.argmining-1.5
Volume:
Proceedings of the 8th Workshop on Argument Mining
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Venues:
ArgMining | EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
46–56
Language:
URL:
https://aclanthology.org/2021.argmining-1.5
DOI:
10.18653/v1/2021.argmining-1.5
Bibkey:
Cite (ACL):
Myrthe Reuver, Suzan Verberne, Roser Morante, and Antske Fokkens. 2021. Is Stance Detection Topic-Independent and Cross-topic Generalizable? - A Reproduction Study. In Proceedings of the 8th Workshop on Argument Mining, pages 46–56, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Is Stance Detection Topic-Independent and Cross-topic Generalizable? - A Reproduction Study (Reuver et al., ArgMining 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.argmining-1.5.pdf