Mohamed Waseem
Author directory2026
A False Positive in Cross-Lingual Unlearning Evaluation
Mohamed Waseem | Mohammed Fawaz
Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
Mohamed Waseem | Mohammed Fawaz
Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
Cross-lingual audits of LLM unlearning ask whether a fact forgotten in one language remains recoverable in another, and read a small loss increase in the second language as residual recall. We show that this inference is unsafe without a control that is routinely omitted. Auditing GradDiff unlearning of TOFU fictional-persona facts on Qwen2.5-0.5B across English, Modern Standard Arabic, Egyptian Arabic and Hindi, we find the expected signature of leakage: the unlearned model’s forget-set loss rises by +2.922 nats in English but only +0.774 in MSA, 26% as much. Measuring the base model as well shows that signature to be an artefact. English-only fine-tuning left no measurable trace of the facts in the other three varieties — it raised forget-set loss there, and raised retain-set loss by an indistinguishable amount — so there is no evidence of cross-lingual memory for unlearning to have left behind. Once the never-learned and the uniform-degradation components are removed, the MSA effect is +0.054 nats, 2.5% of English rather than 26%. We argue that a base-model transfer control and a retain-set control are jointly necessary for any cross-lingual unlearning claim, and report the audit design that yields them at no additional training cost.