BERT-Based Simplification of Japanese Sentence-Ending Predicates in Descriptive Text

Taichi Kato, Rei Miyata, Satoshi Sato


Abstract
Japanese sentence-ending predicates intricately combine content words and functional elements, such as aspect, modality, and honorifics; this can often hinder the understanding of language learners and children. Conventional lexical simplification methods, which replace difficult target words with simpler synonyms acquired from lexical resources in a word-by-word manner, are not always suitable for the simplification of such Japanese predicates. Given this situation, we propose a BERT-based simplification method, the core feature of which is the high ability to substitute the whole predicates with simple ones while maintaining their core meanings in the context by utilizing pre-trained masked language models. Experimental results showed that our proposed methods consistently outperformed the conventional thesaurus-based method by a wide margin. Furthermore, we investigated in detail the effectiveness of the average token embedding and dropout, and the remaining errors of our BERT-based methods.
Anthology ID:
2020.inlg-1.31
Volume:
Proceedings of the 13th International Conference on Natural Language Generation
Month:
December
Year:
2020
Address:
Dublin, Ireland
Editors:
Brian Davis, Yvette Graham, John Kelleher, Yaji Sripada
Venue:
INLG
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
242–251
Language:
URL:
https://aclanthology.org/2020.inlg-1.31
DOI:
10.18653/v1/2020.inlg-1.31
Bibkey:
Cite (ACL):
Taichi Kato, Rei Miyata, and Satoshi Sato. 2020. BERT-Based Simplification of Japanese Sentence-Ending Predicates in Descriptive Text. In Proceedings of the 13th International Conference on Natural Language Generation, pages 242–251, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
BERT-Based Simplification of Japanese Sentence-Ending Predicates in Descriptive Text (Kato et al., INLG 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.inlg-1.31.pdf