FLOUR: Mixing the Right Ingredients for LLM Unlearning in Latent Space

Thuy Dung Nguyen, Giang Nguyen, Taylor T Johnson, Kevin Leach


Abstract
As large language models are deployed at scale, the ability to selectively remove knowledge has become critical for privacy, safety, and legal compliance. Existing post-training unlearning methods suffer from a persistent tension: unbounded divergence signals over-unlearn, degrading utility and producing incoherent outputs, while fixed sample-agnostic targets and reference-relative penalties under-unlearn, leaving traces recoverable under paraphrase or membership-inference attacks. We attribute both failures to a common structural cause: the forgetting objective is decoupled from any sample- or parameter-level notion of what retention requires. We present FLOUR (Forgetting via Learned On-manifold Unlearning Representations), a latent unlearning framework in which a lightweight encoder produces a per-sample suppression direction. This direction is anchored to the null region of the reference model’s activations, keeping it on-manifold and disentangled from retention-critical directions, which enables forgetting while preserving coherent generation and enhancing privacy. Results on the Open-Unlearning benchmark suite show that FLOUR most closely matches the retrain oracle across the unlearning trilemma of forgetting, retention, and privacy, substantially outperforming prior baselines. Our code is publicly available at https://github.com/ judydnguyen/flour-unlearning.
Anthology ID:
2026.findings-emnlp.2376
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
51033–51051
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2376/
DOI:
10.18653/2026.findings-emnlp.2376
Bibkey:
Cite (ACL):
Thuy Dung Nguyen, Giang Nguyen, Taylor T Johnson, and Kevin Leach. 2026. FLOUR: Mixing the Right Ingredients for LLM Unlearning in Latent Space. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 51033–51051, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
FLOUR: Mixing the Right Ingredients for LLM Unlearning in Latent Space (Nguyen et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2376.pdf