Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework

Shaolei Zhang, Yang Feng


Abstract
Simultaneous machine translation (SiMT) starts translating while receiving the streaming source inputs, and hence the source sentence is always incomplete during translating. Different from the full-sentence MT using the conventional seq-to-seq architecture, SiMT often applies prefix-to-prefix architecture, which forces each target word to only align with a partial source prefix to adapt to the incomplete source in streaming inputs. However, the source words in the front positions are always illusoryly considered more important since they appear in more prefixes, resulting in position bias, which makes the model pay more attention on the front source positions in testing. In this paper, we first analyze the phenomenon of position bias in SiMT, and develop a Length-Aware Framework to reduce the position bias by bridging the structural gap between SiMT and full-sentence MT. Specifically, given the streaming inputs, we first predict the full-sentence length and then fill the future source position with positional encoding, thereby turning the streaming inputs into a pseudo full-sentence. The proposed framework can be integrated into most existing SiMT methods to further improve performance. Experiments on two representative SiMT methods, including the state-of-the-art adaptive policy, show that our method successfully reduces the position bias and thereby achieves better SiMT performance.
Anthology ID:
2022.acl-long.467
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6775–6788
Language:
URL:
https://aclanthology.org/2022.acl-long.467
DOI:
10.18653/v1/2022.acl-long.467
Bibkey:
Cite (ACL):
Shaolei Zhang and Yang Feng. 2022. Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6775–6788, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework (Zhang & Feng, ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.467.pdf