Counterfactual Adversarial Learning with Representation Interpolation

Wei Wang, Boxin Wang, Ning Shi, Jinfeng Li, Bingyu Zhu, Xiangyu Liu, Rong Zhang


Abstract
Deep learning models exhibit a preference for statistical fitting over logical reasoning. Spurious correlations might be memorized when there exists statistical bias in training data, which severely limits the model performance especially in small data scenarios. In this work, we introduce Counterfactual Adversarial Training framework (CAT) to tackle the problem from a causality perspective. Particularly, for a specific sample, CAT first generates a counterfactual representation through latent space interpolation in an adversarial manner, and then performs Counterfactual Risk Minimization (CRM) on each original-counterfactual pair to adjust sample-wise loss weight dynamically, which encourages the model to explore the true causal effect. Extensive experiments demonstrate that CAT achieves substantial performance improvement over SOTA across different downstream tasks, including sentence classification, natural language inference and question answering.
Anthology ID:
2021.findings-emnlp.413
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
4809–4820
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.413
DOI:
10.18653/v1/2021.findings-emnlp.413
Bibkey:
Cite (ACL):
Wei Wang, Boxin Wang, Ning Shi, Jinfeng Li, Bingyu Zhu, Xiangyu Liu, and Rong Zhang. 2021. Counterfactual Adversarial Learning with Representation Interpolation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4809–4820, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Counterfactual Adversarial Learning with Representation Interpolation (Wang et al., Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.413.pdf
Video:
 https://aclanthology.org/2021.findings-emnlp.413.mp4
Code
 shininglab/cat
Data
SNLISQuADYahoo! Answers