Tower v2: Unbabel-IST 2024 Submission for the General MT Shared Task

Ricardo Rei, Jose Pombal, Nuno M. Guerreiro, João Alves, Pedro Henrique Martins, Patrick Fernandes, Helena Wu, Tania Vaz, Duarte Alves, Amin Farajian, Sweta Agrawal, Antonio Farinhas, José G. C. De Souza, André Martins


Abstract
In this work, we present Tower v2, an improved iteration of the state-of-the-art open-weight Tower models, and the backbone of our submission to the WMT24 General Translation shared task. Tower v2 introduces key improvements including expanded language coverage, enhanced data quality, and increased model capacity up to 70B parameters. Our final submission combines these advancements with quality-aware decoding strategies, selecting translations based on multiple translation quality signals. The resulting system demonstrates significant improvement over previous versions, outperforming closed commercial systems like GPT-4o, Claude 3.5, and DeepL even at a smaller 7B scale.
Anthology ID:
2024.wmt-1.12
Volume:
Proceedings of the Ninth Conference on Machine Translation
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Barry Haddow, Tom Kocmi, Philipp Koehn, Christof Monz
Venue:
WMT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
185–204
Language:
URL:
https://aclanthology.org/2024.wmt-1.12
DOI:
Bibkey:
Cite (ACL):
Ricardo Rei, Jose Pombal, Nuno M. Guerreiro, João Alves, Pedro Henrique Martins, Patrick Fernandes, Helena Wu, Tania Vaz, Duarte Alves, Amin Farajian, Sweta Agrawal, Antonio Farinhas, José G. C. De Souza, and André Martins. 2024. Tower v2: Unbabel-IST 2024 Submission for the General MT Shared Task. In Proceedings of the Ninth Conference on Machine Translation, pages 185–204, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Tower v2: Unbabel-IST 2024 Submission for the General MT Shared Task (Rei et al., WMT 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.wmt-1.12.pdf