Harry Yang
Author directory2026
Hallucination as Exploit: Evidence-Carrying Multimodal Agents
Guijia Zhang | Hao Zheng | Harry Yang
Findings of the Association for Computational Linguistics: EMNLP 2026
Guijia Zhang | Hao Zheng | Harry Yang
Findings of the Association for Computational Linguistics: EMNLP 2026
Multimodal agents increasingly choose tool calls from screenshots, documents, and webpages, where a false perceptual claim can turn hallucination from an answer-quality error into an authorization failure. We formalize this failure as hallucination-to-action conversion (H2AC) and propose evidence-carrying multimodal agents (ECA), in which model text may propose actions but cannot authorize them: constrained DOM/OCR/AX verifiers issue typed certificates for action-critical predicates, and a deterministic gate executes only what the certificates support. Across verifier red-teaming, 7,488 replayed authorization traces, and end-to-end and live-browser evaluations, ECA executes no unsafe action, while GPT-5.4 judge baselines admit 79–99% of unsafe actions under the same threat model; benign actions are occasionally deferred to user confirmation (6.4% on live pages) but never hard-blocked. The guarantee is conditional, extending exactly as far as the verifiers, schemas, and gate implementation that form ECA’s trusted base.
LoopViT: Scaling Visual ARC with Looped Transformers
Wen-Jie Shu | Xuerui Qiu | Rui-Jie Zhu | Harold Haodong Chen | Yexin Liu | Harry Yang
Findings of the Association for Computational Linguistics: EMNLP 2026
Wen-Jie Shu | Xuerui Qiu | Rui-Jie Zhu | Harold Haodong Chen | Yexin Liu | Harry Yang
Findings of the Association for Computational Linguistics: EMNLP 2026
Recent ARC-AGI systems have shown that pure vision can solve abstract grid reasoning without language backbones, but most strong models still rely on fixed-depth feed-forward computation. We study whether recurrent computation is a better scaling axis for this setting. We propose Loop-ViT, a vision-native recurrent model that reuses a weight-tied Hybrid Block across multiple steps, with local convolutions for image-token refinement and attention for global rule induction. We also use an entropy-based exit rule to reduce unnecessary iterations at inference time. On ARC-AGI-1, Loop-ViT improves the parameter-efficiency trade-off over fixed-depth vision baselines and reaches 65.8% accuracy with an 18M model. Overall, the results suggest that repeated computation is a useful inductive bias for ARC-style visual reasoning. The code is available at https://github.com/WenjieShu/LoopViT.