Adversarial Semantic Collisions

Congzheng Song, Alexander Rush, Vitaly Shmatikov


Abstract
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and demonstrate that state-of-the-art models for many tasks which rely on analyzing the meaning and similarity of texts—including paraphrase identification, document retrieval, response suggestion, and extractive summarization—are vulnerable to semantic collisions. For example, given a target query, inserting a crafted collision into an irrelevant document can shift its retrieval rank from 1000 to top 3. We show how to generate semantic collisions that evade perplexity-based filtering and discuss other potential mitigations. Our code is available at https://github.com/csong27/collision-bert.
Anthology ID:
2020.emnlp-main.344
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4198–4210
Language:
URL:
https://aclanthology.org/2020.emnlp-main.344
DOI:
10.18653/v1/2020.emnlp-main.344
Bibkey:
Cite (ACL):
Congzheng Song, Alexander Rush, and Vitaly Shmatikov. 2020. Adversarial Semantic Collisions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4198–4210, Online. Association for Computational Linguistics.
Cite (Informal):
Adversarial Semantic Collisions (Song et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.344.pdf
Video:
 https://slideslive.com/38939027
Code
 csong27/collision-bert
Data
MRPC