Large language models (LLMs) often memorize personally identifiable information (PII) during training, raising serious privacy concerns. While machine unlearning has emerged as a promising mitigation, its effectiveness against privacy attacks remains unclear. We introduce PrivUn, an evaluation framework for privacy unlearning under recovery attacks, paired with three measurement tools: forgetting scores, association metrics, and layer-wise analysis of forgetting depth. This framework studies not only whether a model forgets, but also how forgetting propagates and where it fails.

Using PrivUn, we make two findings. (i) The PII ripple effect is gradient-driven, not semantic. Graph-based proximity is uncorrelated with PII forgetting, while gradient-based association is more strongly correlated. (ii) Label-based unlearning is typically shallow, while PII is stored across deeper layers. Layer-wise evidence suggests that label-based methods mainly change the final layers, explaining why they reduce direct retrieval but remain vulnerable to stronger recovery attacks.