Automatic Error Type Annotation for Arabic

Riadh Belkebir, Nizar Habash


Abstract
We present ARETA, an automatic error type annotation system for Modern Standard Arabic. We design ARETA to address Arabic’s morphological richness and orthographic ambiguity. We base our error taxonomy on the Arabic Learner Corpus (ALC) Error Tagset with some modifications. ARETA achieves a performance of 85.8% (micro average F1 score) on a manually annotated blind test portion of ALC. We also demonstrate ARETA’s usability by applying it to a number of submissions from the QALB 2014 shared task for Arabic grammatical error correction. The resulting analyses give helpful insights on the strengths and weaknesses of different submissions, which is more useful than the opaque M2 scoring metrics used in the shared task. ARETA employs a large Arabic morphological analyzer, but is completely unsupervised otherwise. We make ARETA publicly available.
Anthology ID:
2021.conll-1.47
Volume:
Proceedings of the 25th Conference on Computational Natural Language Learning
Month:
November
Year:
2021
Address:
Online
Editors:
Arianna Bisazza, Omri Abend
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
596–606
Language:
URL:
https://aclanthology.org/2021.conll-1.47
DOI:
10.18653/v1/2021.conll-1.47
Bibkey:
Cite (ACL):
Riadh Belkebir and Nizar Habash. 2021. Automatic Error Type Annotation for Arabic. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 596–606, Online. Association for Computational Linguistics.
Cite (Informal):
Automatic Error Type Annotation for Arabic (Belkebir & Habash, CoNLL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.conll-1.47.pdf
Video:
 https://aclanthology.org/2021.conll-1.47.mp4
Code
 camel-lab/arabic_error_type_annotation +  additional community code