Anuridhi Gupta

Author directory

2026

In this paper, we evaluate three state-of-the-art (SOTA) MM-LLMs on consistency of their outputs across text and audio modalities using real emergency alerts. Building on prior semantic-similarity and factual-overlap evaluation, we introduce a LIWC aligned analysis of modality dependent differences in linguistic structure, interpersonal framing, urgency, risk, and action-oriented language. Our findings indicate that despite being unified models, capable of taking in input from different modalities, no single model was able to achieve reliable cross-modal consistency. Persona-level results further indicate that cross-modal disparities vary across user groups. This introduces modality dependent inequity that unfairly affects individuals with access and functional needs. We release prompts, audio recordings, model outputs and evaluation material to support the crisis language technology community in building equitable solution for crisis management.