Kaixiang Zhao
Author directory2026
Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation
Kaixiang Zhao | Tianrun Yu | Shawn Huang | Porter Jenkins | Yushun Dong | Amanda Hughes
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Kaixiang Zhao | Tianrun Yu | Shawn Huang | Porter Jenkins | Yushun Dong | Amanda Hughes
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
We study fact-level repair for cross-modal factual generation, where a fluent textual output may contain specific facts that are not supported by the input. Existing inference-time repair methods often generate feedback by jointly conditioning on the input and the current output. This design can let hallucinated claims in the output bias the model’s interpretation of the input, and its free-form feedback is difficult to rank or schedule at the fact level. We present TIGER, an inference-time framework that redesigns feedback for localized repair. TIGER independently extracts an observation graph from the input and a claim graph from the current output, then assigns each claim a graph-conditioned risk score based on support and conflict. The model repairs selected high-risk claims while keeping the backbone frozen. We provide a convergence analysis showing that the expected total risk decreases geometrically to an explicit asymptotic bound under mild assumptions. Experiments on four text-output cross-modal paths, including image-to-text, image+text-to-text, audio-to-text, and video-to-text, show that TIGER reduces unsupported content while preserving task quality. The gains hold across multiple backbones, and a CrisisFACTS case study suggests that the same repair mechanism can improve grounding in noisy multi-source settings.