KWJA: A Unified Japanese Analyzer Based on Foundation Models

Nobuhiro Ueda, Kazumasa Omura, Takashi Kodama, Hirokazu Kiyomaru, Yugo Murawaki, Daisuke Kawahara, Sadao Kurohashi


Abstract
We present KWJA, a high-performance unified Japanese text analyzer based on foundation models.KWJA supports a wide range of tasks, including typo correction, word segmentation, word normalization, morphological analysis, named entity recognition, linguistic feature tagging, dependency parsing, PAS analysis, bridging reference resolution, coreference resolution, and discourse relation analysis, making it the most versatile among existing Japanese text analyzers.KWJA solves these tasks in a multi-task manner but still achieves competitive or better performance compared to existing analyzers specialized for each task.KWJA is publicly available under the MIT license at https://github.com/ku-nlp/kwja.
Anthology ID:
2023.acl-demo.52
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Danushka Bollegala, Ruihong Huang, Alan Ritter
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
538–548
Language:
URL:
https://aclanthology.org/2023.acl-demo.52
DOI:
10.18653/v1/2023.acl-demo.52
Bibkey:
Cite (ACL):
Nobuhiro Ueda, Kazumasa Omura, Takashi Kodama, Hirokazu Kiyomaru, Yugo Murawaki, Daisuke Kawahara, and Sadao Kurohashi. 2023. KWJA: A Unified Japanese Analyzer Based on Foundation Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 538–548, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
KWJA: A Unified Japanese Analyzer Based on Foundation Models (Ueda et al., ACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.acl-demo.52.pdf
Video:
 https://aclanthology.org/2023.acl-demo.52.mp4