Juan Yeo
Author directory2026
Compliance, Capability, and Conflict: Benchmarking Multimodal LLMs under System Messages
Juan Yeo | Geewook Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
Juan Yeo | Geewook Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
Production deployments of Multimodal Large Language Models (MLLMs) increasingly rely on system messages to govern model behavior. Yet existing MLLM benchmarks either evaluate constraints in text only or embed them into the user turn, leaving system-message adherence in multimodal contexts largely unmeasured; they also leave open whether compliance comes at the cost of foundational vision-language capabilities. We introduce **VSysBench**, a simple yet effective MLLM benchmark that organizes system-message constraints into 5 main categories and 22 sub-categories, ranging from textual directives in visual contexts to fully vision-grounded ones, each paired with a misaligned counterpart that stress-tests the instructional hierarchy. **VSysBench** scores each response jointly along two axes, constraint compliance and answer correctness, via the Joint Satisfaction Rate (JSR) and Cross-Constraint Sensitivity (CCS). Across 16 MLLMs, we find that imposing system messages substantially erodes base task accuracy, that compliance collapses under user conflict for open-weight models while remaining stable for top proprietary ones, and that vision-grounded constraints are the hardest category for nearly every model.
Stable On-Policy Distillation through Adaptive Target Reformulation
Ijun Jang | Jewon Yeom | Juan Yeo | Hyunggyu Lim | Taesup Kim
Findings of the Association for Computational Linguistics: ACL 2026
Ijun Jang | Jewon Yeom | Juan Yeo | Hyunggyu Lim | Taesup Kim
Findings of the Association for Computational Linguistics: ACL 2026
Knowledge distillation (KD) is a widely adopted technique for transferring capabilities from large language models to smaller student models. However, conventional supervised KD often suffers from a distribution mismatch between training and inference. While on-policy KD approaches attempt to mitigate this issue by learning directly from student-generated outputs, they frequently encounter training instabilities and noisy teacher feedback during early optimization stages. These challenges manifest as pathological gradients in forward KL objectives when students encounter unfamiliar tokens, or as a collapse in distributional diversity within reverse KL regimes. To address these limitations, we propose Veto, an objective-level reformulation that constructs a geometric target distribution in logit space to emphasize agreement between the teacher and the student. By introducing a tunable parameter 𝛽, Veto serves as an Adaptive Gradient Veto that stabilizes optimization by suppressing harmful gradients on low-confidence tokens, while simultaneously acting as a Decisiveness Knob to balance reward-driven performance with output diversity. Extensive experiments across various reasoning and generation tasks demonstrate that Veto consistently outperforms supervised fine-tuning and existing on-policy baselines.