Analyzing Neural Discourse Coherence Models

Youmna Farag, Josef Valvoda, Helen Yannakoudakis, Ted Briscoe


Abstract
In this work, we systematically investigate how well current models of coherence can capture aspects of text implicated in discourse organisation. We devise two datasets of various linguistic alterations that undermine coherence and test model sensitivity to changes in syntax and semantics. We furthermore probe discourse embedding space and examine the knowledge that is encoded in representations of coherence. We hope this study shall provide further insight into how to frame the task and improve models of coherence assessment further. Finally, we make our datasets publicly available as a resource for researchers to use to test discourse coherence models.
Anthology ID:
2020.codi-1.11
Volume:
Proceedings of the First Workshop on Computational Approaches to Discourse
Month:
November
Year:
2020
Address:
Online
Editors:
Chloé Braud, Christian Hardmeier, Junyi Jessy Li, Annie Louis, Michael Strube
Venue:
CODI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
102–112
Language:
URL:
https://aclanthology.org/2020.codi-1.11
DOI:
10.18653/v1/2020.codi-1.11
Bibkey:
Cite (ACL):
Youmna Farag, Josef Valvoda, Helen Yannakoudakis, and Ted Briscoe. 2020. Analyzing Neural Discourse Coherence Models. In Proceedings of the First Workshop on Computational Approaches to Discourse, pages 102–112, Online. Association for Computational Linguistics.
Cite (Informal):
Analyzing Neural Discourse Coherence Models (Farag et al., CODI 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.codi-1.11.pdf
Video:
 https://slideslive.com/38939699
Code
 Youmna-H/coherence-analysis