Generating Diverse Translations via Weighted Fine-tuning and Hypotheses Filtering for the Duolingo STAPLE Task

Sweta Agrawal, Marine Carpuat


Abstract
This paper describes the University of Maryland’s submission to the Duolingo Shared Task on Simultaneous Translation And Paraphrase for Language Education (STAPLE). Unlike the standard machine translation task, STAPLE requires generating a set of outputs for a given input sequence, aiming to cover the space of translations produced by language learners. We adapt neural machine translation models to this requirement by (a) generating n-best translation hypotheses from a model fine-tuned on learner translations, oversampled to reflect the distribution of learner responses, and (b) filtering hypotheses using a feature-rich binary classifier that directly optimizes a close approximation of the official evaluation metric. Combination of systems that use these two strategies achieves F1 scores of 53.9% and 52.5% on Vietnamese and Portuguese, respectively ranking 2nd and 4th on the leaderboard.
Anthology ID:
2020.ngt-1.21
Volume:
Proceedings of the Fourth Workshop on Neural Generation and Translation
Month:
July
Year:
2020
Address:
Online
Venues:
ACL | NGT | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
178–187
Language:
URL:
https://aclanthology.org/2020.ngt-1.21
DOI:
10.18653/v1/2020.ngt-1.21
Bibkey:
Cite (ACL):
Sweta Agrawal and Marine Carpuat. 2020. Generating Diverse Translations via Weighted Fine-tuning and Hypotheses Filtering for the Duolingo STAPLE Task. In Proceedings of the Fourth Workshop on Neural Generation and Translation, pages 178–187, Online. Association for Computational Linguistics.
Cite (Informal):
Generating Diverse Translations via Weighted Fine-tuning and Hypotheses Filtering for the Duolingo STAPLE Task (Agrawal & Carpuat, NGT 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.ngt-1.21.pdf
Video:
 http://slideslive.com/38929835
Data
Duolingo STAPLE Shared Task