Sheng-Jun Huang
Author directory2026
Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMs
Xianya Fang | Feiyang Ren | Xiang Chen | Yu Tian | Zhen Bi | Haiyang Yu | Sheng-Jun Huang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xianya Fang | Feiyang Ren | Xiang Chen | Yu Tian | Zhen Bi | Haiyang Yu | Sheng-Jun Huang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Multimodal LLMs are powerful but prone to object hallucinations, which describe non-existent entities and harm reliability. While recent unlearning methods attempt to mitigate this, we identify a critical flaw: structural fragility. We empirically demonstrate that standard erasure achieves only superficial suppression, trapping the model in sharp minima where hallucinations catastrophically resurge after lightweight relearning. To ensure geometric stability, we propose Sharpness-Aware Robust Erasure (SARE), which casts unlearning as a targeted min-max optimization problem and leverages a targeted Sharpness-Aware Minimization (Targeted-SAM) mechanism to explicitly flatten the loss landscape around hallucinated concepts. By suppressing hallucinations under simulated worst-case parameter perturbations, our framework ensures robust removal stable against weight shifts. Extensive experiments demonstrate that SARE outperforms baselines in erasure efficacy while preserving general generation quality. Crucially, it maintains persistent hallucination suppression against relearning and parameter updates, validating the effectiveness of geometric stabilization.
2025
Graph-guided Cross-composition Feature Disentanglement for Compositional Zero-shot Learning
Yuxia Geng | Runkai Zhu | Jiaoyan Chen | Jintai Chen | Xiang Chen | Zhuo Chen | Shuofei Qiao | Yuxiang Wang | Xiaoliang Xu | Sheng-Jun Huang
Findings of the Association for Computational Linguistics: ACL 2025
Yuxia Geng | Runkai Zhu | Jiaoyan Chen | Jintai Chen | Xiang Chen | Zhuo Chen | Shuofei Qiao | Yuxiang Wang | Xiaoliang Xu | Sheng-Jun Huang
Findings of the Association for Computational Linguistics: ACL 2025
Disentanglement of visual features of primitives (i.e., attributes and objects) has shown exceptional results in Compositional Zero-shot Learning (CZSL). However, due to the feature divergence of an attribute (resp. object) when combined with different objects (resp. attributes), it is challenging to learn disentangled primitive features that are general across different compositions. To this end, we propose the solution of cross-composition feature disentanglement, which takes multiple primitive-sharing compositions as inputs and constrains the disentangled primitive features to be general across these compositions. More specifically, we leverage a compositional graph to define the overall primitive-sharing relationships between compositions, and build a task-specific architecture upon the recently successful large pre-trained vision-language model (VLM) CLIP, with dual cross-composition disentangling adapters (called L-Adapter and V-Adapter) inserted into CLIP’s frozen text and image encoders, respectively. Evaluation on three popular CZSL benchmarks shows that our proposed solution significantly improves the performance of CZSL, and its components have been verified by solid ablation studies. Our code and data are available at: https://github.com/zhurunkai/DCDA.