Discourse Self-Attention for Discourse Element Identification in Argumentative Student Essays

Wei Song, Ziyao Song, Ruiji Fu, Lizhen Liu, Miaomiao Cheng, Ting Liu


Abstract
This paper proposes to adapt self-attention to discourse level for modeling discourse elements in argumentative student essays. Specifically, we focus on two issues. First, we propose structural sentence positional encodings to explicitly represent sentence positions. Second, we propose to use inter-sentence attentions to capture sentence interactions and enhance sentence representation. We conduct experiments on two datasets: a Chinese dataset and an English dataset. We find that (i) sentence positional encoding can lead to a large improvement for identifying discourse elements; (ii) a structural relative positional encoding of sentences shows to be most effective; (iii) inter-sentence attention vectors are useful as a kind of sentence representations for identifying discourse elements.
Anthology ID:
2020.emnlp-main.225
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2820–2830
Language:
URL:
https://aclanthology.org/2020.emnlp-main.225
DOI:
10.18653/v1/2020.emnlp-main.225
Bibkey:
Cite (ACL):
Wei Song, Ziyao Song, Ruiji Fu, Lizhen Liu, Miaomiao Cheng, and Ting Liu. 2020. Discourse Self-Attention for Discourse Element Identification in Argumentative Student Essays. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2820–2830, Online. Association for Computational Linguistics.
Cite (Informal):
Discourse Self-Attention for Discourse Element Identification in Argumentative Student Essays (Song et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.225.pdf
Optional supplementary material:
 2020.emnlp-main.225.OptionalSupplementaryMaterial.zip
Video:
 https://slideslive.com/38938811