Syed Faizan
Author directory2026
Auditing and Mitigating Paraphrase Fragility in Medical Vision-Language Models
Syed Faizan | Anuradha Chandramohan | Parameswari Krishnamurthy
Findings of the Association for Computational Linguistics: EMNLP 2026
Syed Faizan | Anuradha Chandramohan | Parameswari Krishnamurthy
Findings of the Association for Computational Linguistics: EMNLP 2026
Paraphrases express the same intent in different words, so a robust vision-language model (VLM) should preserve its clinical conclusion when a question is rephrased. Medical VLMs often violate this expectation. We audit this failure on VQA-RAD using nine clinically grounded linguistic operations and 9,723 answer-preserving paraphrases. Across eight checkpoints from five architecture stacks (2.2B–27.4B), Negation Reframing is consistently the most fragile category, with clinical flip rates from 30.7% to 95.3%. A locked, category-conditioned LoRA reduces this failure by 59.0–75.8% across all eight checkpoints; on MedGemma-27B, paraphrase accuracy rises from 68.4% to 75.5%, although base accuracy falls from 79.6% to 73.9%. A pooled adapter performs nearly as well, so we use category-conditioned adapters primarily as diagnostic instruments. Two clinicians agree on 97.8% of a 500-pair calibration sample (κ=0.901), whereas agreement with an LLM equivalence judge is only κ≈0.14. Sparse-autoencoder clamping yields only a 5.8% reduction, and activation patching provides suggestive rather than conclusive localization evidence. Together, these results provide a reproducible robustness audit for radiology VLMs and make its dataset, generator, evaluation, and deployment limits explicit.