iFlip: Iterative Feedback-driven Counterfactual Example

Yilong Wang, Qianli Wang, Nils Feldhus


Abstract
Counterfactual examples are minimal edits to an input that alter a model’s prediction. They are widely employed in explainable AI to probe model behavior and in natural language processing (NLP) to augment training data. However, generating valid counterfactuals with large language models (LLMs) remains challenging, as existing single-pass methods often fail to induce reliable label changes, neglecting LLMs’ self-correction capabilities. To explore this untapped potential, we propose iFlip, an iterative refinement approach that leverages three types of feedback, including model confidence, feature attribution, and natural language. Our results show that iFlip achieves an average 57.8% higher validity than the three state-of-the-art baselines, as measured by the label flipping rate. The user study further corroborates that iFlip outperforms baselines in completeness, overall satisfaction, and feasibility. In addition, ablation studies demonstrate that three components are paramount for iFlip to generate valid counterfactuals: leveraging an appropriate number of iterations, pointing to highly attributed words, and early stopping. Finally, counterfactuals generated by iFlip enable effective counterfactual data augmentation, substantially improving model performance and robustness.
Anthology ID:
2026.findings-emnlp.38
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
749–779
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.38/
DOI:
Bibkey:
Cite (ACL):
Yilong Wang, Qianli Wang, and Nils Feldhus. 2026. iFlip: Iterative Feedback-driven Counterfactual Example. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 749–779, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
iFlip: Iterative Feedback-driven Counterfactual Example (Wang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.38.pdf