Discriminative Reranking for Neural Machine Translation

Ann Lee, Michael Auli, Marc’Aurelio Ranzato


Abstract
Reranking models enable the integration of rich features to select a better output hypothesis within an n-best list or lattice. These models have a long history in NLP, and we revisit discriminative reranking for modern neural machine translation models by training a large transformer architecture. This takes as input both the source sentence as well as a list of hypotheses to output a ranked list. The reranker is trained to predict the observed distribution of a desired metric, e.g. BLEU, over the n-best list. Since such a discriminator contains hundreds of millions of parameters, we improve its generalization using pre-training and data augmentation techniques. Experiments on four WMT directions show that our discriminative reranking approach is effective and complementary to existing generative reranking approaches, yielding improvements of up to 4 BLEU over the beam search output.
Anthology ID:
2021.acl-long.563
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7250–7264
Language:
URL:
https://aclanthology.org/2021.acl-long.563
DOI:
10.18653/v1/2021.acl-long.563
Bibkey:
Cite (ACL):
Ann Lee, Michael Auli, and Marc’Aurelio Ranzato. 2021. Discriminative Reranking for Neural Machine Translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7250–7264, Online. Association for Computational Linguistics.
Cite (Informal):
Discriminative Reranking for Neural Machine Translation (Lee et al., ACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.563.pdf
Video:
 https://aclanthology.org/2021.acl-long.563.mp4