Juman++: A Morphological Analysis Toolkit for Scriptio Continua

Arseny Tolmachev, Daisuke Kawahara, Sadao Kurohashi


Abstract
We present a three-part toolkit for developing morphological analyzers for languages without natural word boundaries. The first part is a C++11/14 lattice-based morphological analysis library that uses a combination of linear and recurrent neural net language models for analysis. The other parts are a tool for exposing problems in the trained model and a partial annotation tool. Our morphological analyzer of Japanese achieves new SOTA on Jumandic-based corpora while being 250 times faster than the previous one. We also perform a small experiment and quantitive analysis and experience of using development tools. All components of the toolkit is open source and available under a permissive Apache 2 License.
Anthology ID:
D18-2010
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
Eduardo Blanco, Wei Lu
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
54–59
Language:
URL:
https://aclanthology.org/D18-2010
DOI:
10.18653/v1/D18-2010
Bibkey:
Cite (ACL):
Arseny Tolmachev, Daisuke Kawahara, and Sadao Kurohashi. 2018. Juman++: A Morphological Analysis Toolkit for Scriptio Continua. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 54–59, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Juman++: A Morphological Analysis Toolkit for Scriptio Continua (Tolmachev et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-2010.pdf
Code
 ku-nlp/jumanpp