ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model

Chuyun Deng, Mingxuan Liu, Yue Qin, Jia Zhang, Hai-Xin Duan, Donghong Sun


Abstract
Adversarial texts help explore vulnerabilities in language models, improve model robustness, and explain their working mechanisms. However, existing word-level attack methods trap in a one-to-one attack pattern, i.e., only a single word can be modified in one transformation round, and they ignore the interactions between several consecutive words. In this paper, we propose ValCAT, a black-box attack framework that misleads the language model by applying variable-length contextualized transformations to the original text. Compared to word-level methods, ValCAT expands the basic units of perturbation from single words to spans composed of multiple consecutive words, enhancing the perturbation capability. Experiments show that our method outperforms state-of-the-art methods in terms of attack success rate, perplexity, and semantic similarity on several classification tasks and inference tasks. The comprehensive human evaluation demonstrates that ValCAT has a significant advantage in ensuring the fluency of the adversarial examples and achieves better semantic consistency. We release the code at https://github.com/linerxliner/ValCAT.
Anthology ID:
2022.naacl-main.125
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1735–1746
Language:
URL:
https://aclanthology.org/2022.naacl-main.125
DOI:
10.18653/v1/2022.naacl-main.125
Bibkey:
Cite (ACL):
Chuyun Deng, Mingxuan Liu, Yue Qin, Jia Zhang, Hai-Xin Duan, and Donghong Sun. 2022. ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1735–1746, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model (Deng et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.125.pdf
Code
 linerxliner/valcat
Data
AG NewsGLUEIMDb Movie ReviewsMultiNLIQNLISNLI