Language Model Augmented Monotonic Attention for Simultaneous Translation

Sathish Reddy Indurthi, Mohd Abbas Zaidi, Beomseok Lee, Nikhil Kumar Lakumarapu, Sangha Kim


Abstract
The state-of-the-art adaptive policies for Simultaneous Neural Machine Translation (SNMT) use monotonic attention to perform read/write decisions based on the partial source and target sequences. The lack of sufficient information might cause the monotonic attention to take poor read/write decisions, which in turn negatively affects the performance of the SNMT model. On the other hand, human translators make better read/write decisions since they can anticipate the immediate future words using linguistic information and domain knowledge. In this work, we propose a framework to aid monotonic attention with an external language model to improve its decisions. Experiments on MuST-C English-German and English-French speech-to-text translation tasks show the future information from the language model improves the state-of-the-art monotonic multi-head attention model further.
Anthology ID:
2022.naacl-main.3
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
38–45
Language:
URL:
https://aclanthology.org/2022.naacl-main.3
DOI:
10.18653/v1/2022.naacl-main.3
Bibkey:
Cite (ACL):
Sathish Reddy Indurthi, Mohd Abbas Zaidi, Beomseok Lee, Nikhil Kumar Lakumarapu, and Sangha Kim. 2022. Language Model Augmented Monotonic Attention for Simultaneous Translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 38–45, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Language Model Augmented Monotonic Attention for Simultaneous Translation (Indurthi et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.3.pdf
Data
MuST-C