Dongsheng Xu
Author directory2026
Mitigating Object Hallucination in MLLMs with Global-to-Local Visual Attention Correction
Xingxing Huang | Zhiguo Yang | Jiacheng Pi | Dongsheng Xu | Ruizhi Zhong | Peipei Xu | Wenjie Ruan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xingxing Huang | Zhiguo Yang | Jiacheng Pi | Dongsheng Xu | Ruizhi Zhong | Peipei Xu | Wenjie Ruan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Multimodal Large Language Models (MLLMs) have achieved impressive progress in vision-language understanding, yet they remain susceptible to object hallucination, which refers to generating descriptions of objects that do not exist in the input image. Through comprehensive analysis, we uncover a fundamental trade-off in MLLMs: strengthening visual attention enhances grounding completeness but also increases hallucinations, especially additive hallucinations that appear in the trailing part of the generated sentence. To alleviate this trade-off, we propose **GLVAC**, **G**lobal-to-**L**ocal **V**isual **A**ttention **C**orrection, a training-free dual-path inference framework that adaptively combines global conservatism and local enhancement in visual attention. The global path maintains stable and grounded predictions under moderate visual attention, while the local correction branch is dynamically activated for high-uncertainty tokens identified via entropy-based estimation. A confidence-guided gating mechanism then determines whether to adopt the correction output, ensuring safe and reliable refinement. GLVAC can be seamlessly integrated into existing MLLMs without retraining or external modules, effectively reducing object hallucinations while maintaining perceptual completeness across diverse benchmarks and model families. Our code is available on https://github.com/Trusted-LLM/GLVAC.
2023
Multi-modal Sarcasm Generation: Dataset and Solution
Wenye Zhao | Qingbao Huang | Dongsheng Xu | Peizhi Zhao
Findings of the Association for Computational Linguistics: ACL 2023
Wenye Zhao | Qingbao Huang | Dongsheng Xu | Peizhi Zhao
Findings of the Association for Computational Linguistics: ACL 2023
As an interesting and challenging task, sarcasm generation has attracted widespread attention. Although very recent studies have made promising progress, none of them considers generating a sarcastic description for a given image - as what people are doing on Twitter. In this paper, we present a Multi-modal Sarcasm Generation (MSG) task: Given an image with hashtags that provide the sarcastic target, MSG aims to generate sarcastic descriptions like humans. Different from textual sarcasm generation, MSG is more challenging as it is difficult to accurately capture the key information from images, hashtags, and OCR tokens and exploit multi-modal incongruity to generate sarcastic descriptions. To support the research on MSG, we develop MuSG, a new dataset with 5000 images and related Twitter text. We also propose a multi-modal Transformer-based method as a solution to this MSG task. The input features are embedded in the common space and passed through the multi-modal Transformer layers to generate the sarcastic descriptions by the auto-regressive paradigm. Both automatic and manual evaluations demonstrate the superiority of our method. The dataset and code will be available soon.