Comparing Machine and Human Simultaneous Interpreting

Dongpeng Pan, Kilian Seeber


Abstract
We evaluate four machine simultaneous interpreting (SI) systems, an open-weight and a commercial instance of both the cascaded and the end-to-end speech-native architecture, on simulated conference discourse interpreted into five languages. All outputs are compared with professional interpretations. Reference-free quality metrics are first calibrated against blind expert ratings of the human renditions: their association with expert ratings is strongest for content, weaker for presentation, and weakest for style. Machine systems attain content-similarity scores within or above the professional range and higher median segment coverage at the reported alignment threshold. Coverage can count professional compression as omission and is sensitive to transcription and alignment error, so the surplus does not establish better interpreting. For latency, only the commercial speech-native system approaches professional timing. Estimated content lag thus ranges from under twice to more than ten times the professional median, and each system’s delay is associated with an observable operating property: commit granularity, synthesis queuing, or compute limits. Together, the rating-calibrated metrics and professional distribution provide a reference-free procedure for evaluating machine interpreting on unsegmented, full-length conference discourse.
Anthology ID:
2026.wmt-main.30
Volume:
Proceedings of the Eleventh Conference on Machine Translation
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Barry Haddow, Tom Kocmi, Philipp Koehn, Christof Monz
Venues:
WMT | WS
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
541–553
Language:
URL:
https://aclanthology.org/2026.wmt-main.30/
DOI:
Bibkey:
Cite (ACL):
Dongpeng Pan and Kilian Seeber. 2026. Comparing Machine and Human Simultaneous Interpreting. In Proceedings of the Eleventh Conference on Machine Translation, pages 541–553, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Comparing Machine and Human Simultaneous Interpreting (Pan & Seeber, WMT 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wmt-main.30.pdf