UnNatural Language Inference

Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, Adina Williams


Abstract
Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to understand human-like syntax, at least to some extent. We provide novel evidence that complicates this claim: we find that state-of-the-art Natural Language Inference (NLI) models assign the same labels to permuted examples as they do to the original, i.e. they are invariant to random word-order permutations. This behavior notably differs from that of humans; we struggle to understand the meaning of ungrammatical sentences. To measure the severity of this issue, we propose a suite of metrics and investigate which properties of particular permutations lead models to be word order invariant. For example, in MNLI dataset we find almost all (98.7%) examples contain at least one permutation which elicits the gold label. Models are even able to assign gold labels to permutations that they originally failed to predict correctly. We provide a comprehensive empirical evaluation of this phenomenon, and further show that this issue exists in pre-Transformer RNN / ConvNet based encoders, as well as across multiple languages (English and Chinese). Our code and data are available at https://github.com/facebookresearch/unlu.
Anthology ID:
2021.acl-long.569
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7329–7346
Language:
URL:
https://aclanthology.org/2021.acl-long.569
DOI:
10.18653/v1/2021.acl-long.569
Award:
 Outstanding Paper
Bibkey:
Cite (ACL):
Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, and Adina Williams. 2021. UnNatural Language Inference. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7329–7346, Online. Association for Computational Linguistics.
Cite (Informal):
UnNatural Language Inference (Sinha et al., ACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.569.pdf
Video:
 https://aclanthology.org/2021.acl-long.569.mp4
Code
 facebookresearch/UNLU
Data
ANLIGLUEMultiNLIOCNLISNLI