Can Large Language Models perform Relation-based Argument Mining?

Deniz Gorur, Antonio Rago, Francesca Toni


Abstract
Relation-based Argument Mining (RbAM) is the process of automatically determining agreement (support) and disagreement (attack) relations amongst textual arguments (in the binary prediction setting), or neither relation (in the ternary prediction setting). As the number of platforms supporting online debate increases, the need for RbAM becomes ever more urgent, especially in support of downstream tasks. RbAM is a challenging classification task, with existing state-of-the-art methods, based on Language Models (LMs), failing to perform satisfactorily across different datasets. In this paper, we show that general-purpose Large LMs (LLMs), appropriately primed and prompted, can significantly outperform the best performing (RoBERTa-based) baseline. Specifically, we experiment with two open-source LLMs (Llama-2 and Mistral) and with GPT-3.5-turbo on several datasets for (binary and ternary) RbAM, as well as with GPT-4o-mini on samples (to limit costs) from the datasets.
Anthology ID:
2025.coling-main.569
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8518–8534
Language:
URL:
https://aclanthology.org/2025.coling-main.569/
DOI:
Bibkey:
Cite (ACL):
Deniz Gorur, Antonio Rago, and Francesca Toni. 2025. Can Large Language Models perform Relation-based Argument Mining?. In Proceedings of the 31st International Conference on Computational Linguistics, pages 8518–8534, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
Can Large Language Models perform Relation-based Argument Mining? (Gorur et al., COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.569.pdf