Stan Kharevich
Author directory2026
Findings of the WMT26 General Machine Translation Shared Task: Contrastive Dynamic Human Evaluation at Scale
Tom Kocmi | Ekaterina Artemova | Eleftherios Avramidis | Rachel Bawden | Ondřej Bojar | Sergey Dukanov | Anton Dvorkovich | Mark Fishel | Markus Freitag | Samuel Frontull | Thamme Gowda | Roman Grundkiewicz | Barry Haddow | Stan Kharevich | Philipp Koehn | Zheng Li | Jean Maillard | Christof Monz | Alexander Murauski | Kenton Murray | Masaaki Nagata | Stefano Perrella | Martin Popel | Maja Popović | Lorenzo Proietti | Sara Rajaee | Parker Riley | Mariya Shmatova | Steinthór Steingrímsson | Lisa Yankovskaya | Vilem Zouhar
Proceedings of the Eleventh Conference on Machine Translation
Tom Kocmi | Ekaterina Artemova | Eleftherios Avramidis | Rachel Bawden | Ondřej Bojar | Sergey Dukanov | Anton Dvorkovich | Mark Fishel | Markus Freitag | Samuel Frontull | Thamme Gowda | Roman Grundkiewicz | Barry Haddow | Stan Kharevich | Philipp Koehn | Zheng Li | Jean Maillard | Christof Monz | Alexander Murauski | Kenton Murray | Masaaki Nagata | Stefano Perrella | Martin Popel | Maja Popović | Lorenzo Proietti | Sara Rajaee | Parker Riley | Mariya Shmatova | Steinthór Steingrímsson | Lisa Yankovskaya | Vilem Zouhar
Proceedings of the Eleventh Conference on Machine Translation
This paper presents the results of the General Machine Translation Task organized under the 2026 Conference on Machine Translation (WMT). Participants build systems for any of the 23 language pairs spanning four to five domains. This year we brought major changes to the human evaluation: (1) new annotation platform Pearmut for better reproducibility, (2) new human protocol "contrastive Error Span Annotation" for higher quality and side-by-side evaluation, and (3) Dynamic human evaluation that allows assessing all submitted models while evaluating higher-performing models more frequently. Beyond human evaluation, we (4) extended difficulty sampling with a human-driven stage, (5) added two new domains, (6) made the test sets fully document-level without requiring segment-level alignment and relying on HTML or JSON structure, (7) prepared some human references by post-editing open-weight model outputs, and (8) introduced contextual instructions governing formality and structural style. We evaluated 46 systems in total: 30 submitted by participants and 16 consisting of translations from large language models (LLMs) and industry translation providers.
Search
Fix author
Co-authors
- Ekaterina Artemova 1
- Eleftherios Avramidis 1
- Rachel Bawden 1
- Ondřej Bojar 1
- Sergey Dukanov 1
- Anton Dvorkovich 1
- Mark Fishel 1
- Markus Freitag 1
- Samuel Frontull 1
- Thamme Gowda 1
- Roman Grundkiewicz 1
- Barry Haddow 1
- Tom Kocmi 1
- Philipp Koehn 1
- Zheng Li 1
- Jean Maillard 1
- Christof Monz 1
- Alexander Murauski 1
- Kenton Murray 1
- Masaaki Nagata 1
- Stefano Perrella 1
- Martin Popel 1
- Maja Popović 1
- Lorenzo Proietti 1
- Sara Rajaee 1
- Parker Riley 1
- Mariya Shmatova 1
- Steinþór Steingrímsson 1
- Lisa Yankovskaya 1
- Vilém Zouhar 1