Measuring Semantic Abstraction of Multilingual NMT with Paraphrase Recognition and Generation Tasks

Jörg Tiedemann, Yves Scherrer


Abstract
In this paper, we investigate whether multilingual neural translation models learn stronger semantic abstractions of sentences than bilingual ones. We test this hypotheses by measuring the perplexity of such models when applied to paraphrases of the source language. The intuition is that an encoder produces better representations if a decoder is capable of recognizing synonymous sentences in the same language even though the model is never trained for that task. In our setup, we add 16 different auxiliary languages to a bidirectional bilingual baseline model (English-French) and test it with in-domain and out-of-domain paraphrases in English. The results show that the perplexity is significantly reduced in each of the cases, indicating that meaning can be grounded in translation. This is further supported by a study on paraphrase generation that we also include at the end of the paper.
Anthology ID:
W19-2005
Volume:
Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP
Month:
June
Year:
2019
Address:
Minneapolis, USA
Editors:
Anna Rogers, Aleksandr Drozd, Anna Rumshisky, Yoav Goldberg
Venue:
RepEval
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
35–42
Language:
URL:
https://aclanthology.org/W19-2005
DOI:
10.18653/v1/W19-2005
Bibkey:
Cite (ACL):
Jörg Tiedemann and Yves Scherrer. 2019. Measuring Semantic Abstraction of Multilingual NMT with Paraphrase Recognition and Generation Tasks. In Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP, pages 35–42, Minneapolis, USA. Association for Computational Linguistics.
Cite (Informal):
Measuring Semantic Abstraction of Multilingual NMT with Paraphrase Recognition and Generation Tasks (Tiedemann & Scherrer, RepEval 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-2005.pdf
Supplementary:
 W19-2005.Supplementary.pdf
Data
Tatoeba