Comparison of methods for explicit discourse connective identification across various domains

Merel Scholman, Tianai Dong, Frances Yung, Vera Demberg


Abstract
Existing parse methods use varying approaches to identify explicit discourse connectives, but their performance has not been consistently evaluated in comparison to each other, nor have they been evaluated consistently on text other than newspaper articles. We here assess the performance on explicit connective identification of three parse methods (PDTB e2e, Lin et al., 2014; the winner of CONLL2015, Wang et al., 2015; and DisSent, Nie et al., 2019), along with a simple heuristic. We also examine how well these systems generalize to different datasets, namely written newspaper text (PDTB), written scientific text (BioDRB), prepared spoken text (TED-MDB) and spontaneous spoken text (Disco-SPICE). The results show that the e2e parser outperforms the other parse methods in all datasets. However, performance drops significantly from the PDTB to all other datasets. We provide a more fine-grained analysis of domain differences and connectives that prove difficult to parse, in order to highlight the areas where gains can be made.
Anthology ID:
2021.codi-main.9
Volume:
Proceedings of the 2nd Workshop on Computational Approaches to Discourse
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic and Online
Editors:
Chloé Braud, Christian Hardmeier, Junyi Jessy Li, Annie Louis, Michael Strube, Amir Zeldes
Venue:
CODI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
95–106
Language:
URL:
https://aclanthology.org/2021.codi-main.9
DOI:
10.18653/v1/2021.codi-main.9
Bibkey:
Cite (ACL):
Merel Scholman, Tianai Dong, Frances Yung, and Vera Demberg. 2021. Comparison of methods for explicit discourse connective identification across various domains. In Proceedings of the 2nd Workshop on Computational Approaches to Discourse, pages 95–106, Punta Cana, Dominican Republic and Online. Association for Computational Linguistics.
Cite (Informal):
Comparison of methods for explicit discourse connective identification across various domains (Scholman et al., CODI 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.codi-main.9.pdf
Video:
 https://aclanthology.org/2021.codi-main.9.mp4