On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translation

Kelly Marchisio, Markus Freitag, David Grangier


Abstract
Modern unsupervised machine translation (MT) systems reach reasonable translation quality under clean and controlled data conditions. As the performance gap between supervised and unsupervised MT narrows, it is interesting to ask whether the different training methods result in systematically different output beyond what is visible via quality metrics like adequacy or BLEU. We compare translations from supervised and unsupervised MT systems of similar quality, finding that unsupervised output is more fluent and more structurally different in comparison to human translation than is supervised MT. We then demonstrate a way to combine the benefits of both methods into a single system which results in improved adequacy and fluency as rated by human evaluators. Our results open the door to interesting discussions about how supervised and unsupervised MT might be different yet mutually-beneficial.
Anthology ID:
2022.naacl-main.161
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2214–2225
Language:
URL:
https://aclanthology.org/2022.naacl-main.161
DOI:
10.18653/v1/2022.naacl-main.161
Bibkey:
Cite (ACL):
Kelly Marchisio, Markus Freitag, and David Grangier. 2022. On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2214–2225, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translation (Marchisio et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.161.pdf
Video:
 https://aclanthology.org/2022.naacl-main.161.mp4