Eunkyu Park

Author directory

2026

MLLM-as-a-Judge is conventionally validated by agreement with human annotations, but agreement against a culturally heterogeneous pool is pool-dependent: it silently conditions on whose judgments the aggregate represents. We introduce VOIR DIRE, a multimodal benchmark of 626 culturally paired image–prompt items spanning U.S. and mainland Chinese contexts across food, fashion, and architecture, with annotator pools that are within-pool reliable (š›¼ = 0.86 / 0.74) but cross-pool divergent on evaluation (Q1 r = -0.12). Across six MLLM judges, the bias decomposes into two failures: a positivity-floor calibration failure (compressed scale use) and an orientation failure (default to one cultural norm). On this corpus, contested items are sampled so the two pools split, and because models default to high ratings, the floor mechanically sides with the more-permissive Chinese reading. Persona prompting partially recovers calibration, but an orientation residual survives—evidence the tilt is not reducible to scale compression. Reference-pool in-context demonstrations deepen the orientation residual and inflate the high end rather than restoring use of the low end. Model origin adds a small additive tilt (∼ 0.10 MAE) that is approximately invariant under demonstration. We recommend reporting alignment against each reference pool separately and treating cross-pool divergence as a judge property.
Explanations are often promoted as tools for transparency, but they can also foster blind trust; users may assume reasoning is correct whenever model outputs appear acceptable. We study this double-edged role of Chain-of-Thought (CoT) explanations in multimodal moral scenarios by systematically perturbing reasoning chains and manipulating delivery tones. Specifically, we perturb reasoning chains generated by vision-language models (VLMs) with three error types (omissions, contradictions, hallucinations) and measure their effects on user trust and error detection through a study with 800 participants, alongside a parallel detection task with models. Our findings reveal two key effects: (1) users often equate trust with outcome agreement regardless of flaws in the model’s reasoning, and (2) confident tones suppress scrutiny while leaving agreement and self-reported trust largely unchanged. These results highlight how CoT explanations can simultaneously clarify and mislead in tandem, and point to user-side scrutiny, not just model-side faithfulness, as a critical dimension for calibrated trust in AI reasoning. All code will be released publicly upon acceptance.