Dan He


2023

pdf bib
Gradient-based Gradual Pruning for Language-Specific Multilingual Neural Machine Translation
Dan He | Minh-Quang Pham | Thanh-Le Ha | Marco Turchi
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

Multilingual neural machine translation (MNMT) offers the convenience of translating between multiple languages with a single model. However, MNMT often suffers from performance degradation in high-resource languages compared to bilingual counterparts. This degradation is commonly attributed to parameter interference, which occurs when parameters are fully shared across all language pairs. In this work, to tackle this issue we propose a gradient-based gradual pruning technique for MNMT. Our approach aims to identify an optimal sub-network for each language pair within the multilingual model by leveraging gradient-based information as pruning criterion and gradually increasing the pruning ratio as schedule. Our approach allows for partial parameter sharing across language pairs to alleviate interference, and each pair preserves its unique parameters to capture language-specific information. Comprehensive experiments on IWSLT and WMT datasets show that our approach yields a notable performance gain on both datasets.

2021

pdf bib
Multilingual Speech Translation KIT @ IWSLT2021
Ngoc-Quan Pham | Tuan Nam Nguyen | Thanh-Le Ha | Sebastian Stüker | Alexander Waibel | Dan He
Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021)

This paper contains the description for the submission of Karlsruhe Institute of Technology (KIT) for the multilingual TEDx translation task in the IWSLT 2021 evaluation campaign. Our main approach is to develop both cascade and end-to-end systems and eventually combine them together to achieve the best possible results for this extremely low-resource setting. The report also confirms certain consistent architectural improvement added to the Transformer architecture, for all tasks: translation, transcription and speech translation.