Rui Gao
Author directory2026
Bridging Modality Disconnect in Self-Reflection via Closed-Loop Visually Grounded Verification
Haoyu Zhang | Zhi Gao | Pengxiang Li | Xintong Zhang | Yuwei Wu | Rui Gao | Mingyang Gao | Xuanyan Chen | Anqian Cen | Che Sun | Yunde Jia
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Haoyu Zhang | Zhi Gao | Pengxiang Li | Xintong Zhang | Yuwei Wu | Rui Gao | Mingyang Gao | Xuanyan Chen | Anqian Cen | Che Sun | Yunde Jia
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Self-reflection has become a key mechanism for improving reasoning in Vision-Language Models (VLMs), yet this corrective mechanism often fails when resolving complex fine-grained visual ambiguities. This performance degradation stems from the issue of modality disconnect in self-reflection: most existing models execute self-reflection either within textual or latent space, lacking an explicit mechanism to align textual reasoning with visual evidence. In this paper, we propose MIRROR, a closed-loop visual reflection framework comprising four steps: response generation, error identification, region-based visual verification, and revision. In this cycle, the model first generates an initial response, identifies uncertain logical statements in the response that require visual verification, then grounds them in relevant image regions, and finally revises based on the visual evidence. We construct a multi-turn visual reflection dataset ReflectV, which empowers the model with such a reflective capability. Extensive experiments across diverse multimodal benchmarks show that MIRROR improves performance and reduces visual hallucinations by an average of about 7.2%, demonstrating the advantage of transforming self-reflection from open-loop textual revision into closed-loop, visually grounded verification.
2024
High-Order Semantic Alignment for Unsupervised Fine-Grained Image-Text Retrieval
Rui Gao | Miaomiao Cheng | Xu Han | Wei Song
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Rui Gao | Miaomiao Cheng | Xu Han | Wei Song
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Cross-modal retrieval is an important yet challenging task due to the semantic discrepancy between visual content and language. To measure the correlation between images and text, most existing research mainly focuses on learning global or local correspondence, failing to explore fine-grained local-global alignment. To infer more accurate similarity scores, we introduce a novel High Order Semantic Alignment (HOSA) model that can provide complementary and comprehensive semantic clues. Specifically, to jointly learn global and local alignment and emphasize local-global interaction, we employ tensor-product (t-product) operation to reconstruct one modal’s representation based on another modal’s information in a common semantic space. Such a cross-modal reconstruction strategy would significantly enhance inter-modal correlation learning in a fine-grained manner. Extensive experiments on two benchmark datasets validate that our model significantly outperforms several state-of-the-art baselines, especially in retrieving the most relevant results.