Eunkyu Park
Author directory2026
Jury Duty: Calibration and Orientation Failures in MLLM-as-a-Judge Under Cultural Ambiguity
Daniel Lee | Harshit Sharma | Eunkyu Park | Pranav Narayanan Venkit | Jeonghwan Kim | Kah Mun Chia | Andreas Vlachos | Shafiq Joty
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Daniel Lee | Harshit Sharma | Eunkyu Park | Pranav Narayanan Venkit | Jeonghwan Kim | Kah Mun Chia | Andreas Vlachos | Shafiq Joty
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
MLLM-as-a-Judge is conventionally validated by agreement with human annotations, but agreement against a culturally heterogeneous pool is pool-dependent: it silently conditions on whose judgments the aggregate represents. We introduce VOIR DIRE, a multimodal benchmark of 626 culturally paired imageāprompt items spanning U.S. and mainland Chinese contexts across food, fashion, and architecture, with annotator pools that are within-pool reliable (š¼ = 0.86 / 0.74) but cross-pool divergent on evaluation (Q1 r = -0.12). Across six MLLM judges, the bias decomposes into two failures: a positivity-floor calibration failure (compressed scale use) and an orientation failure (default to one cultural norm). On this corpus, contested items are sampled so the two pools split, and because models default to high ratings, the floor mechanically sides with the more-permissive Chinese reading. Persona prompting partially recovers calibration, but an orientation residual survivesāevidence the tilt is not reducible to scale compression. Reference-pool in-context demonstrations deepen the orientation residual and inflate the high end rather than restoring use of the low end. Model origin adds a small additive tilt (ā¼ 0.10 MAE) that is approximately invariant under demonstration. We recommend reporting alignment against each reference pool separately and treating cross-pool divergence as a judge property.
Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations
Eunkyu Park | Wesley Hanwen Deng | Vasudha Varadarajan | Mingxi Yan | Gunhee Kim | Maarten Sap | Motahhare Eslami
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Eunkyu Park | Wesley Hanwen Deng | Vasudha Varadarajan | Mingxi Yan | Gunhee Kim | Maarten Sap | Motahhare Eslami
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Explanations are often promoted as tools for transparency, but they can also foster blind trust; users may assume reasoning is correct whenever model outputs appear acceptable. We study this double-edged role of Chain-of-Thought (CoT) explanations in multimodal moral scenarios by systematically perturbing reasoning chains and manipulating delivery tones. Specifically, we perturb reasoning chains generated by vision-language models (VLMs) with three error types (omissions, contradictions, hallucinations) and measure their effects on user trust and error detection through a study with 800 participants, alongside a parallel detection task with models. Our findings reveal two key effects: (1) users often equate trust with outcome agreement regardless of flaws in the modelās reasoning, and (2) confident tones suppress scrutiny while leaving agreement and self-reported trust largely unchanged. These results highlight how CoT explanations can simultaneously clarify and mislead in tandem, and point to user-side scrutiny, not just model-side faithfulness, as a critical dimension for calibrated trust in AI reasoning. All code will be released publicly upon acceptance.