Revisiting Pre-trained Language Models for Conversation Disentanglement

Tung-Thien Lam, Cheng-Zen Yang


Abstract
Multi-party conversation is a popular form in online group chatting. However, the interweaving of utterance threads complicates the understanding of the dialogues for participants. Many conversation disentanglement models have been proposed using transformer-based pre-trained language models (PrLMs). However, advanced transformer-based PrLMs have not been extensively studied. This paper investigates the effectiveness of five advanced PrLMs: BERT, XLNet, ELECTRA, RoBERTa, and ModernBERT. The experimental results show that ELECTRA and RoBERTa are two PrLMs with outstanding performance than other PrLMs for the conversation disentanglement task.
Anthology ID:
2025.rocling-main.31
Volume:
Proceedings of the 37th Conference on Computational Linguistics and Speech Processing (ROCLING 2025)
Month:
November
Year:
2025
Address:
National Taiwan University, Taipei City, Taiwan
Editors:
Kai-Wei Chang, Ke-Han Lu, Chih-Kai Yang, Zhi-Rui Tam, Wen-Yu Chang, Chung-Che Wang
Venue:
ROCLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
296–302
Language:
URL:
https://aclanthology.org/2025.rocling-main.31/
DOI:
Bibkey:
Cite (ACL):
Tung-Thien Lam and Cheng-Zen Yang. 2025. Revisiting Pre-trained Language Models for Conversation Disentanglement. In Proceedings of the 37th Conference on Computational Linguistics and Speech Processing (ROCLING 2025), pages 296–302, National Taiwan University, Taipei City, Taiwan. Association for Computational Linguistics.
Cite (Informal):
Revisiting Pre-trained Language Models for Conversation Disentanglement (Lam & Yang, ROCLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.rocling-main.31.pdf