%0 Conference Proceedings %T Generalizing Back-Translation in Neural Machine Translation %A Graça, Miguel %A Kim, Yunsu %A Schamper, Julian %A Khadivi, Shahram %A Ney, Hermann %Y Bojar, Ondřej %Y Chatterjee, Rajen %Y Federmann, Christian %Y Fishel, Mark %Y Graham, Yvette %Y Haddow, Barry %Y Huck, Matthias %Y Yepes, Antonio Jimeno %Y Koehn, Philipp %Y Martins, André %Y Monz, Christof %Y Negri, Matteo %Y Névéol, Aurélie %Y Neves, Mariana %Y Post, Matt %Y Turchi, Marco %Y Verspoor, Karin %S Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers) %D 2019 %8 August %I Association for Computational Linguistics %C Florence, Italy %F graca-etal-2019-generalizing %X Back-translation — data augmentation by translating target monolingual data — is a crucial component in modern neural machine translation (NMT). In this work, we reformulate back-translation in the scope of cross-entropy optimization of an NMT model, clarifying its underlying mathematical assumptions and approximations beyond its heuristic usage. Our formulation covers broader synthetic data generation schemes, including sampling from a target-to-source NMT model. With this formulation, we point out fundamental problems of the sampling-based approaches and propose to remedy them by (i) disabling label smoothing for the target-to-source model and (ii) sampling from a restricted search space. Our statements are investigated on the WMT 2018 German \textless-\textgreater English news translation task. %R 10.18653/v1/W19-5205 %U https://aclanthology.org/W19-5205 %U https://doi.org/10.18653/v1/W19-5205 %P 45-52