Incorporate Semantic Structures into Machine Translation Evaluation via UCCA

Jin Xu, Yinuo Guo, Junfeng Hu


Abstract
Copying mechanism has been commonly used in neural paraphrasing networks and other text generation tasks, in which some important words in the input sequence are preserved in the output sequence. Similarly, in machine translation, we notice that there are certain words or phrases appearing in all good translations of one source text, and these words tend to convey important semantic information. Therefore, in this work, we define words carrying important semantic meanings in sentences as semantic core words. Moreover, we propose an MT evaluation approach named Semantically Weighted Sentence Similarity (SWSS). It leverages the power of UCCA to identify semantic core words, and then calculates sentence similarity scores on the overlap of semantic core words. Experimental results show that SWSS can consistently improve the performance of popular MT evaluation metrics which are based on lexical similarity.
Anthology ID:
2020.wmt-1.104
Volume:
Proceedings of the Fifth Conference on Machine Translation
Month:
November
Year:
2020
Address:
Online
Editors:
Loïc Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Yvette Graham, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
934–939
Language:
URL:
https://aclanthology.org/2020.wmt-1.104
DOI:
Bibkey:
Cite (ACL):
Jin Xu, Yinuo Guo, and Junfeng Hu. 2020. Incorporate Semantic Structures into Machine Translation Evaluation via UCCA. In Proceedings of the Fifth Conference on Machine Translation, pages 934–939, Online. Association for Computational Linguistics.
Cite (Informal):
Incorporate Semantic Structures into Machine Translation Evaluation via UCCA (Xu et al., WMT 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.wmt-1.104.pdf
Video:
 https://slideslive.com/38939565