Construction of a Quality Estimation Dataset for Automatic Evaluation of Japanese Grammatical Error Correction

Daisuke Suzuki, Yujin Takahashi, Ikumi Yamashita, Taichi Aida, Tosho Hirasawa, Michitaka Nakatsuji, Masato Mita, Mamoru Komachi


Abstract
In grammatical error correction (GEC), automatic evaluation is considered as an important factor for research and development of GEC systems. Previous studies on automatic evaluation have shown that quality estimation models built from datasets with manual evaluation can achieve high performance in automatic evaluation of English GEC. However, quality estimation models have not yet been studied in Japanese, because there are no datasets for constructing quality estimation models. In this study, therefore, we created a quality estimation dataset with manual evaluation to build an automatic evaluation model for Japanese GEC. By building a quality estimation model using this dataset and conducting a meta-evaluation, we verified the usefulness of the quality estimation model for Japanese GEC.
Anthology ID:
2022.lrec-1.596
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
5565–5572
Language:
URL:
https://aclanthology.org/2022.lrec-1.596
DOI:
Bibkey:
Cite (ACL):
Daisuke Suzuki, Yujin Takahashi, Ikumi Yamashita, Taichi Aida, Tosho Hirasawa, Michitaka Nakatsuji, Masato Mita, and Mamoru Komachi. 2022. Construction of a Quality Estimation Dataset for Automatic Evaluation of Japanese Grammatical Error Correction. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5565–5572, Marseille, France. European Language Resources Association.
Cite (Informal):
Construction of a Quality Estimation Dataset for Automatic Evaluation of Japanese Grammatical Error Correction (Suzuki et al., LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.596.pdf
Data
GUG