Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT

Rik van Noord, Antonio Toral, Johan Bos


Abstract
We combine character-level and contextual language model representations to improve performance on Discourse Representation Structure parsing. Character representations can easily be added in a sequence-to-sequence model in either one encoder or as a fully separate encoder, with improvements that are robust to different language models, languages and data sets. For English, these improvements are larger than adding individual sources of linguistic information or adding non-contextual embeddings. A new method of analysis based on semantic tags demonstrates that the character-level representations improve performance across a subset of selected semantic phenomena.
Anthology ID:
2020.emnlp-main.371
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4587–4603
Language:
URL:
https://aclanthology.org/2020.emnlp-main.371
DOI:
10.18653/v1/2020.emnlp-main.371
Bibkey:
Cite (ACL):
Rik van Noord, Antonio Toral, and Johan Bos. 2020. Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4587–4603, Online. Association for Computational Linguistics.
Cite (Informal):
Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT (van Noord et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.371.pdf
Video:
 https://slideslive.com/38939012
Code
 RikVN/Neural_DRS +  additional community code