IsoChronoMeter: A Simple and Effective Isochronic Translation Evaluation Metric

Nikolai Rozanov, Vikentiy Pankov, Dmitrii Mukhutdinov, Dima Vypirailenko


Abstract
Machine translation (MT) has come a long way and is readily employed in production systems to serve millions of users daily. With the recent advances in generative AI, a new form of translation is becoming possible - video dubbing. This work motivates the importance of isochronic translation, especially in the context of automatic dubbing, and introduces ‘IsoChronoMeter’ (ICM). ICM is a simple yet effective metric to measure isochrony of translations in a scalable and resource-efficient way without the need for gold data, based on state-of-the-art text-to-speech (TTS) duration predictors. We motivate IsoChronoMeter and demonstrate its effectiveness. Using ICM we demonstrate the shortcomings of state-of-the-art translation systems and show the need for new methods. We release the code at this URL: https://github.com/braskai/isochronometer.
Anthology ID:
2024.wmt-1.29
Volume:
Proceedings of the Ninth Conference on Machine Translation
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Barry Haddow, Tom Kocmi, Philipp Koehn, Christof Monz
Venue:
WMT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
372–379
Language:
URL:
https://aclanthology.org/2024.wmt-1.29
DOI:
Bibkey:
Cite (ACL):
Nikolai Rozanov, Vikentiy Pankov, Dmitrii Mukhutdinov, and Dima Vypirailenko. 2024. IsoChronoMeter: A Simple and Effective Isochronic Translation Evaluation Metric. In Proceedings of the Ninth Conference on Machine Translation, pages 372–379, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
IsoChronoMeter: A Simple and Effective Isochronic Translation Evaluation Metric (Rozanov et al., WMT 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.wmt-1.29.pdf