Adversarial Example Generation with Syntactically Controlled Paraphrase Networks

Mohit Iyyer, John Wieting, Kevin Gimpel, Luke Zettlemoyer


Abstract
We propose syntactically controlled paraphrase networks (SCPNs) and use them to generate adversarial examples. Given a sentence and a target syntactic form (e.g., a constituency parse), SCPNs are trained to produce a paraphrase of the sentence with the desired syntax. We show it is possible to create training data for this task by first doing backtranslation at a very large scale, and then using a parser to label the syntactic transformations that naturally occur during this process. Such data allows us to train a neural encoder-decoder model with extra inputs to specify the target syntax. A combination of automated and human evaluations show that SCPNs generate paraphrases that follow their target specifications without decreasing paraphrase quality when compared to baseline (uncontrolled) paraphrase systems. Furthermore, they are more capable of generating syntactically adversarial examples that both (1) “fool” pretrained models and (2) improve the robustness of these models to syntactic variation when used to augment their training data.
Anthology ID:
N18-1170
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marilyn Walker, Heng Ji, Amanda Stent
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1875–1885
Language:
URL:
https://aclanthology.org/N18-1170
DOI:
10.18653/v1/N18-1170
Bibkey:
Cite (ACL):
Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial Example Generation with Syntactically Controlled Paraphrase Networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1875–1885, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Adversarial Example Generation with Syntactically Controlled Paraphrase Networks (Iyyer et al., NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-1170.pdf
Video:
 https://aclanthology.org/N18-1170.mp4
Code
 miyyer/scpn +  additional community code
Data
SST