Occiglot at WMT24: European Open-source Large Language Models Evaluated on Translation

Eleftherios Avramidis, Annika Grützner-Zahn, Manuel Brack, Patrick Schramowski, Pedro Ortiz Suarez, Malte Ostendorff, Fabio Barth, Shushen Manakhimova, Vivien Macketanz, Georg Rehm, Kristian Kersting


Abstract
This document describes the submission of the very first version of the Occiglot open-source large language model to the General MT Shared Task of the 9th Conference of Machine Translation (WMT24). Occiglot is an open-source, community-based LLM based on Mistral-7B, which went through language-specific continual pre-training and subsequent instruction tuning, including instructions relevant to machine translation.We examine the automatic metric scores for translating the WMT24 test set and provide a detailed linguistically-motivated analysis.Despite Occiglot performing worse than many of the other system submissions, we observe that it performs better than Mistral7B, which has been based upon, which indicates the positive effect of the language specific continual-pretraining and instruction tuning. We see the submission of this very early version of the model as a motivation to unite community forces and pursue future LLM research on the translation task.
Anthology ID:
2024.wmt-1.23
Volume:
Proceedings of the Ninth Conference on Machine Translation
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Barry Haddow, Tom Kocmi, Philipp Koehn, Christof Monz
Venue:
WMT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
292–298
Language:
URL:
https://aclanthology.org/2024.wmt-1.23
DOI:
10.18653/v1/2024.wmt-1.23
Bibkey:
Cite (ACL):
Eleftherios Avramidis, Annika Grützner-Zahn, Manuel Brack, Patrick Schramowski, Pedro Ortiz Suarez, Malte Ostendorff, Fabio Barth, Shushen Manakhimova, Vivien Macketanz, Georg Rehm, and Kristian Kersting. 2024. Occiglot at WMT24: European Open-source Large Language Models Evaluated on Translation. In Proceedings of the Ninth Conference on Machine Translation, pages 292–298, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Occiglot at WMT24: European Open-source Large Language Models Evaluated on Translation (Avramidis et al., WMT 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.wmt-1.23.pdf