Atahan Uz
Author directory2026
TurCOMET: A Morphologically-Aware Evaluation System for Document-Level Turkish-English Machine Translation
Atahan Uz | Hüseyin Emir Akdağ | Susan Üsküdarlı
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Atahan Uz | Hüseyin Emir Akdağ | Susan Üsküdarlı
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
We introduce TurCOMET, the first Turkish-specific augmentation of COMET for evaluating translations between Turkish and English. By extending xCOMET-XL with Turkish morphological features and training it on additional Turkish-English data via an additive bottleneck layer, TurCOMET resolves neural evaluation failures caused by sub-word over-segmentation. To validate our metric, we introduce TurkDoc-MT, a meticulously curated Turkish-English document-level aligned dataset of 1,322 instances spanning seven domains. Meta-evaluation demonstrates that TurCOMET significantly outperforms the off-the-shelf baseline, yielding absolute Kendall’s τ correlation gains of +0.113 (system-level) and +0.024 (segment-level), with the largest gains in the TR→EN direction. All models, datasets, code, and annotations are publicly released for the benefit of Turkish natural language processing and benchmarking any translation system.