Interrogating the Explanatory Power of Attention in Neural Machine Translation

Pooya Moradi, Nishant Kambhatla, Anoop Sarkar


Abstract
Attention models have become a crucial component in neural machine translation (NMT). They are often implicitly or explicitly used to justify the model’s decision in generating a specific token but it has not yet been rigorously established to what extent attention is a reliable source of information in NMT. To evaluate the explanatory power of attention for NMT, we examine the possibility of yielding the same prediction but with counterfactual attention models that modify crucial aspects of the trained attention model. Using these counterfactual attention mechanisms we assess the extent to which they still preserve the generation of function and content words in the translation process. Compared to a state of the art attention model, our counterfactual attention models produce 68% of function words and 21% of content words in our German-English dataset. Our experiments demonstrate that attention models by themselves cannot reliably explain the decisions made by a NMT model.
Anthology ID:
D19-5624
Volume:
Proceedings of the 3rd Workshop on Neural Generation and Translation
Month:
November
Year:
2019
Address:
Hong Kong
Editors:
Alexandra Birch, Andrew Finch, Hiroaki Hayashi, Ioannis Konstas, Thang Luong, Graham Neubig, Yusuke Oda, Katsuhito Sudoh
Venue:
NGT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
221–230
Language:
URL:
https://aclanthology.org/D19-5624
DOI:
10.18653/v1/D19-5624
Bibkey:
Cite (ACL):
Pooya Moradi, Nishant Kambhatla, and Anoop Sarkar. 2019. Interrogating the Explanatory Power of Attention in Neural Machine Translation. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 221–230, Hong Kong. Association for Computational Linguistics.
Cite (Informal):
Interrogating the Explanatory Power of Attention in Neural Machine Translation (Moradi et al., NGT 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-5624.pdf
Code
 sfu-natlang/attention_explanation