ICCU: In-Context Continual Unlearning via Pattern-Induced Refusal Rules

Ruihao Pan, Suhang Wang


Abstract
Machine unlearning aims to remove the influence of specific data from trained language models. In real-world deployments, unlearning requests often arrive sequentially, which challenges existing fine-tuning-based methods: fine-tuning each request is costly, accumulates utility loss, and may cause cross-request interference. To address these issues, we propose ICCU (In-Context Continual Unlearning), an in-context continual unlearning framework that induces readable refusal rules from unlearning datasets and applies them at inference time either as a filter or via the system prompt, without modifying model parameters. Because rules are accumulated as an order-independent union, ICCU is compositional and free of cross-request interference, and the original forget-set data can be discarded after rule induction. Extensive experiments show that ICCU effectively suppresses target knowledge while preserving utility, scales across long sequences of requests, and remains robust to paraphrased, cross-lingual, and adversarially framed queries. Our code is available at https://github.com/rhpan/ICCU.
Anthology ID:
2026.aacl-long.269
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5963–5989
Language:
URL:
https://aclanthology.org/2026.aacl-long.269/
DOI:
Bibkey:
Cite (ACL):
Ruihao Pan and Suhang Wang. 2026. ICCU: In-Context Continual Unlearning via Pattern-Induced Refusal Rules. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5963–5989, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
ICCU: In-Context Continual Unlearning via Pattern-Induced Refusal Rules (Pan & Wang, AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-long.269.pdf