Multi-task Learning for Paraphrase Generation With Keyword and Part-of-Speech Reconstruction

Xuhang Xie, Xuesong Lu, Bei Chen


Abstract
Paraphrase generation using deep learning has been a research hotspot of natural language processing in the past few years. While previous studies tackle the problem from different aspects, the essence of paraphrase generation is to retain the key semantics of the source sentence and rewrite the rest of the content. Inspired by this observation, we propose a novel two-stage model, PGKPR, for paraphrase generation with keyword and part-of-speech reconstruction. The rationale is to capture simultaneously the possible keywords of a source sentence and the relations between them to facilitate the rewriting. In the first stage, we identify the possible keywords using a prediction attribution technique, where the words obtaining higher attribution scores are more likely to be the keywords. In the second stage, we train a transformer-based model via multi-task learning for paraphrase generation. The novel learning task is the reconstruction of the keywords and part-of-speech tags, respectively, from a perturbed sequence of the source sentence. The learned encodings are then decoded to generate the paraphrase. We conduct the experiments on two commonly-used datasets, and demonstrate the superior performance of PGKPR over comparative models on multiple evaluation metrics.
Anthology ID:
2022.findings-acl.97
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1234–1243
Language:
URL:
https://aclanthology.org/2022.findings-acl.97
DOI:
10.18653/v1/2022.findings-acl.97
Bibkey:
Cite (ACL):
Xuhang Xie, Xuesong Lu, and Bei Chen. 2022. Multi-task Learning for Paraphrase Generation With Keyword and Part-of-Speech Reconstruction. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1234–1243, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Multi-task Learning for Paraphrase Generation With Keyword and Part-of-Speech Reconstruction (Xie et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.97.pdf
Software:
 2022.findings-acl.97.software.zip
Data
MS COCO