%0 Conference Proceedings %T When and Why is Document-level Context Useful in Neural Machine Translation? %A Kim, Yunsu %A Tran, Duc Thanh %A Ney, Hermann %Y Popescu-Belis, Andrei %Y Loáiciga, Sharid %Y Hardmeier, Christian %Y Xiong, Deyi %S Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019) %D 2019 %8 November %I Association for Computational Linguistics %C Hong Kong, China %F kim-etal-2019-document %X Document-level context has received lots of attention for compensating neural machine translation (NMT) of isolated sentences. However, recent advances in document-level NMT focus on sophisticated integration of the context, explaining its improvement with only a few selected examples or targeted test sets. We extensively quantify the causes of improvements by a document-level model in general test sets, clarifying the limit of the usefulness of document-level context in NMT. We show that most of the improvements are not interpretable as utilizing the context. We also show that a minimal encoding is sufficient for the context modeling and very long context is not helpful for NMT. %R 10.18653/v1/D19-6503 %U https://aclanthology.org/D19-6503 %U https://doi.org/10.18653/v1/D19-6503 %P 24-34