How effective is BERT without word ordering? Implications for language understanding and data privacy

Jack Hessel, Alexandra Schofield


Abstract
Ordered word sequences contain the rich structures that define language. However, it’s often not clear if or how modern pretrained language models utilize these structures. We show that the token representations and self-attention activations within BERT are surprisingly resilient to shuffling the order of input tokens, and that for several GLUE language understanding tasks, shuffling only minimally degrades performance, e.g., by 4% for QNLI. While bleak from the perspective of language understanding, our results have positive implications for cases where copyright or ethics necessitates the consideration of bag-of-words data (vs. full documents). We simulate such a scenario for three sensitive classification tasks, demonstrating minimal performance degradation vs. releasing full language sequences.
Anthology ID:
2021.acl-short.27
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
204–211
Language:
URL:
https://aclanthology.org/2021.acl-short.27
DOI:
10.18653/v1/2021.acl-short.27
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-short.27.pdf
Optional supplementary material:
 2021.acl-short.27.OptionalSupplementaryMaterial.zip