We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for visual language models (VLMs). Existing state-of-the-art UQ methods rely on consistency among multiple responses, but often report higher confidence when the model is consistently incorrect, leading to poorly calibrated confidence. To address this, we leverage cross-modal consistency by grounding back the textual responses to visual inputs. The confidence from the grounding model (GM) is used to calibrate the overall confidence. Given that using a GM adds its own uncertainty, we apply temperature scaling to calibrate the GM’s confidence in the accuracy of the generated responses. We comprehensively evaluate the proposed approach across diverse range of settings, including datasets with varying visuo-linguistic complexity, Q&A and caption generation tasks, general visual and specialized medical contexts, VLMs differing in scale, openness, and different categories of GMs. Experimental results show that our approach consistently improves confidence calibration, with expected calibration error (ECE) improvements ranging from 32.5% to 96.7% over both self and non-self consistency-based baselines. Code is available here https://github.com/trilokpadhi/Multimodal-Uncertainty-Quantification.

We propose a novel inference-time out-of-domain (OOD) detection algorithm for specialized large language models (LLMs). Despite achieving state-of-the-art performance on in-domain tasks through fine-tuning, specialized LLMs remain vulnerable to incorrect or unreliable outputs when presented with OOD inputs, posing risks in critical applications. Our method leverages the Inductive Conformal Anomaly Detection (ICAD) framework, using a new non-conformity measure based on the model’s dropout tolerance. Motivated by recent findings on polysemanticity and redundancy in LLMs, we hypothesize that in-domain inputs exhibit higher dropout tolerance than OOD inputs. We aggregate dropout tolerance across multiple layers via a valid ensemble approach, improving detection while maintaining theoretical false alarm bounds from ICAD. Experiments with medical-specialized LLMs show that our approach detects OOD inputs better than baseline methods, with AUROC improvements of 2% to 37% when treating OOD datapoints as positives and in-domain test datapoints as negatives.