Machine Translation in the AI Era: Comparing previous methods of machine translation with large language models

William Jock Boyd, Ruslan Mitkov


Abstract
The aim of this paper is to compare the efficacy of multiple different methods of machine translation in the French-English language pair. There is a particular focus on Large Language Models given they are an emerging technology that could have a profound effect on the field of machine translation. This study used the European Parliament’s parallel French-English corpus, testing each method on the same section of data, with multiple different Neural Translation, Large Language Model and Rule-Based solutions being used. The translations were then evaluated using BLEU and METEOR scores to gain an accurate understanding of both precision and semantic accuracy of translation. Statistical analysis was then performed to ensure the results validity and statistical significance. This study found that Neural Translation was the best translation technology overall, with Large Language Models coming second and Rule-Based translation coming last by a significant margin. It was also discovered that within Large Language Model implementations that specifically trained translation capabilities outperformed emergent translation capabilities.
Anthology ID:
2025.case-1.5
Volume:
Proceedings of the 8th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Texts
Month:
September
Year:
2025
Address:
Varna, Bulgaria
Editors:
Ali Hürriyetoğlu, Hristo Tanev, Surendrabikram Thapa
Venues:
CASE | WS
SIG:
Publisher:
INCOMA Ltd., Shoumen, Bulgaria
Note:
Pages:
38–51
Language:
URL:
https://aclanthology.org/2025.case-1.5/
DOI:
Bibkey:
Cite (ACL):
William Jock Boyd and Ruslan Mitkov. 2025. Machine Translation in the AI Era: Comparing previous methods of machine translation with large language models. In Proceedings of the 8th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Texts, pages 38–51, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
Cite (Informal):
Machine Translation in the AI Era: Comparing previous methods of machine translation with large language models (Boyd & Mitkov, CASE 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.case-1.5.pdf