What Have We Achieved on Non-autoregressive Translation?

Yafu Li, Huajian Zhang, Jianhao Yan, Yongjing Yin, Yue Zhang


Abstract
Recent advances have made non-autoregressive (NAT) translation comparable to autoregressive methods (AT). However, their evaluation using BLEU has been shown to weakly correlate with human annotations. Limited research compares non-autoregressive translation and autoregressive translation comprehensively, leaving uncertainty about the true proximity of NAT to AT. To address this gap, we systematically evaluate four representative NAT methods across various dimensions, including human evaluation. Our empirical results demonstrate that despite narrowing the performance gap, state-of-the-art NAT still underperforms AT under more reliable evaluation metrics. Furthermore, we discover that explicitly modeling dependencies is crucial for generating natural language and generalizing to out-of-distribution sequences.
Anthology ID:
2024.findings-acl.452
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7585–7606
Language:
URL:
https://aclanthology.org/2024.findings-acl.452
DOI:
Bibkey:
Cite (ACL):
Yafu Li, Huajian Zhang, Jianhao Yan, Yongjing Yin, and Yue Zhang. 2024. What Have We Achieved on Non-autoregressive Translation?. In Findings of the Association for Computational Linguistics ACL 2024, pages 7585–7606, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
What Have We Achieved on Non-autoregressive Translation? (Li et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.452.pdf