Ahmad Alahmari
Author directory2026
Quanta: A Fine-Grained Dataset and Benchmark for Arabic Machine-Translation Error Analysis
Afraa Alshammari | Ahmad Alahmari | Amal Alaboud | Jaber Ahmed Harthi | Karima Ali Shahwan Alzahrani | Ahnaf Adib | Patrick Brandt | Javier Osorio | Vito D’Orazio | Latifur Khan
Proceedings of The Fourth Arabic Natural Language Processing Conference
Afraa Alshammari | Ahmad Alahmari | Amal Alaboud | Jaber Ahmed Harthi | Karima Ali Shahwan Alzahrani | Ahnaf Adib | Patrick Brandt | Javier Osorio | Vito D’Orazio | Latifur Khan
Proceedings of The Fourth Arabic Natural Language Processing Conference
Automatic machine-translation (MT) metrics collapse heterogeneous errors into a single score. For Arabic, no fine-grained resource exists for diagnosing error types and severity. We introduce QUANTA, a 9,032-instance En to Ar MT-error dataset (738 real UNPC + 7,856 synthetic training instances, both structurally validated, plus a held-out 438-instance real Gold test set) annotated with Mizan (ميزان, "scale"), a new 23-class MQM-compatible taxonomy organised in a four-level hierarchy: 23 leaf classes rolling up to 14 subtypes, 5 parent groups, and 4 TQA dimensions. To make the resource usable, we release the Raqeeb (رقيب, "observer") classifier, an AraBERT-v2 model for predicting Mizan 23-class labels (5-fold Gold macro-F1 = 0.614 ± 0.017). Data quality is supported by ETCA (Error Taxonomy Cross-vendor Audit), an LLM-as-judge check in which a different-vendor model audits QUANTA instances for anchor validity and label fidelity, curbing same-family self-preference bias, and by dual-annotator validation (Gwet’s AC1 = 0.97 synthetic; 0.83 real) on core taxonomy criteria. Ablations show that anchor-guided framing is decisive (without it, macro-F1 collapses to 0.114); that 738 structurally validated real samples are ≈12× more sample-efficient than synthetic data; and that, restricted to the classes with real-tier training support, combining both tiers outperforms real (0.656) or synthetic (0.312) alone. Code, data, prompts, and checkpoints are publicly available.