Does Character-level Information Always Improve DRS-based Semantic Parsing?

Tomoya Kurosawa, Hitomi Yanaka


Abstract
Even in the era of massive language models, it has been suggested that character-level representations improve the performance of neural models. The state-of-the-art neural semantic parser for Discourse Representation Structures uses character-level representations, improving performance in the four languages (i.e., English, German, Dutch, and Italian) in the Parallel Meaning Bank dataset. However, how and why character-level information improves the parser’s performance remains unclear. This study provides an in-depth analysis of performance changes by order of character sequences. In the experiments, we compare F1-scores by shuffling the order and randomizing character sequences after testing the performance of character-level information. Our results indicate that incorporating character-level information does not improve the performance in English and German. In addition, we find that the parser is not sensitive to correct character order in Dutch. Nevertheless, performance improvements are observed when using character-level information.
Anthology ID:
2023.starsem-1.23
Volume:
Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Alexis Palmer, Jose Camacho-collados
Venue:
*SEM
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
249–258
Language:
URL:
https://aclanthology.org/2023.starsem-1.23
DOI:
10.18653/v1/2023.starsem-1.23
Bibkey:
Cite (ACL):
Tomoya Kurosawa and Hitomi Yanaka. 2023. Does Character-level Information Always Improve DRS-based Semantic Parsing?. In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 249–258, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Does Character-level Information Always Improve DRS-based Semantic Parsing? (Kurosawa & Yanaka, *SEM 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.starsem-1.23.pdf