Yukai Huang
Author directory2026
MSQA: A Natively Sourced Multilingual and Multicultural SimpleQA Benchmark
Yukai Huang | Xianru Chen | Mingxiang Chen | Xinping Lei | Fangbing Deng | Jin Chen | Jiaheng Liu | Ge Zhang | Wenhao Huang
Findings of the Association for Computational Linguistics: EMNLP 2026
Yukai Huang | Xianru Chen | Mingxiang Chen | Xinping Lei | Fangbing Deng | Jin Chen | Jiaheng Liu | Ge Zhang | Wenhao Huang
Findings of the Association for Computational Linguistics: EMNLP 2026
Multilingual fluency often invites a stronger assumption: a model that can speak a user’s language must also understand the culture encoded by that language. We call this the Illusion of Cultural Alignment. To test this assumption directly, we introduce MMC-QA, a benchmark of 1,086 natively sourced questions across 11 language groups, five cultural dimensions, and three difficulty tiers. Unlike translated benchmarks, MMC-QA targets locally grounded knowledge and reduces shortcuts from English-centric cross-lingual transfer. Evaluating 15 LLMs, we find substantial cultural degradation and a pronounced Locality Effect: cultural competence tracks pre-training exposure more closely than general reasoning ability. We further show that common inference-time remedies do not dissolve the illusion. Models remain overconfident on unfamiliar cultural questions, repeated sampling yields unstable rather than reliable correctness, and retrieval augmentation helps unevenly on long-tail facts. These findings indicate that cultural alignment cannot be inferred from multilingual ability alone and requires deeper intervention than calibration, sampling, or retrieval at inference time.
Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding
Weitong Cai | Hang Zhang | Yukai Huang | Yiqiao Xie | Shan Gao | Jiankang Deng | Songcen Xu | Jifei Song | Zhensong Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Weitong Cai | Hang Zhang | Yukai Huang | Yiqiao Xie | Shan Gao | Jiankang Deng | Songcen Xu | Jifei Song | Zhensong Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Long-video understanding on edge devices must reason over hours of content under tight compute and bandwidth budgets. Subsampling visual tokens loses temporal structure, while text-only video memories lose fine-grained visual attributes. We observe a visual-textual duality: language memories carry long-range temporal structure better than dense frames, while pixels remain decisive for attribute-level perception. Building on this insight, we propose Caption-once, Frames-onDemand (CFD), a budget-aware edge-cloud agentic framework. The edge runs a single offline captioning pass that builds a dual-track narrative index, an event-level story skeleton plus a clip-level micro-log, cached and reused across queries without re-captioning. At query time, a cloud-side MLLM reasons over the index in a story-first loop centered on a lightweight Visual-Need Router: a per-query gating module that triggers bounded keyframe retrieval only for perceptual questions (appearance, on-screen text, attribute disambiguation) and keeps temporal-structural questions in language space. The router turns visual access into a first-class, query-conditioned cost, capping per-query frame consumption regardless of video length. Experiments on long-video benchmarks demonstrate strong accuracy-efficiency trade-offs while substantially reducing online visual processing.
WebCompass: Evaluating Multimodal Code Agents in Web Development from Vision, Instruction, and Interaction to Implementation
Xinping Lei | Xinyu Che | Junqi Xiong | Chenchen Zhang | Yukai Huang | Chenyu Zhou | Haoyang Huang | Minghao Liu | Letian Zhu | Hongyi Ye | Ken Deng | Zizheng Zhan | Han Li | Yifan Yao | Ming Sun | Zhaoxiang Zhang | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xinping Lei | Xinyu Che | Junqi Xiong | Chenchen Zhang | Yukai Huang | Chenyu Zhou | Haoyang Huang | Minghao Liu | Letian Zhu | Hongyi Ye | Ken Deng | Zizheng Zhan | Han Li | Yifan Yao | Ming Sun | Zhaoxiang Zhang | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models are increasingly deployed as interactive coding agents for end-to-end web development, yet existing benchmarks evaluate narrow slices of this workflow—typically text-conditioned generation with static correctness metrics—overlooking visual fidelity, interactivity, and codebase-level reasoning. We introduce WebCompass, a benchmark that covers the full web development lifecycle by spanning three input modalities (text, image, video) and three task types (generation, editing, repair). To match the diverse nature of these tasks, we pair checklist-guided LLM-as-a-Judge for patch-based evaluation with a novel Agent-as-a-Judge protocol that executes generated websites in a real browser and synthesizes targeted test cases. Experiments on ten models reveal a 25-point gap between the best closed- and open-source models, persistent weaknesses in visual aesthetics even among frontier systems, and divergent performance profiles across task types that challenge the notion of a single "best" web coding agent.
CapMem: A Benchmark for Caption-Based Episodic Memory in Egocentric Video
Dingli Liang | Yiqiao Xie | Yukai Huang | Zhaokai Wang | Weitong Cai | Guangwen Feng | Jifei Song | Zhensong Zhang | Hang Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Dingli Liang | Yiqiao Xie | Yukai Huang | Zhaokai Wang | Weitong Cai | Guangwen Feng | Jifei Song | Zhensong Zhang | Hang Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Wearable assistants require episodic memory over egocentric video, yet current vision-language models face bounded frame budgets, growing visual-token costs, and long-context retrieval failures. Under these practical constraints, we study whether textual captions can serve as reusable episodic memory. We define the Episodic Memory Video Caption QA task and introduce CapMem, a human-annotated benchmark with 75 videos totaling 33.7 hours, and 1,000 multiple-choice questions across 16 scenarios. On long videos (>20 min), full-coverage CaptionQA with 30s and 60s caption windows outperforms direct VideoQA for 10/12 and 8/12 models, respectively. On the same video subset, a matched-frame control across six Qwen models retains mean accuracy gains of 3.22 and 2.55 points, respectively. Our caption-guided retrieve-and-verify harness further improves accuracy by up to 5.3 points. These results support the effectiveness of caption memory for episodic reasoning over long egocentric video.
Search
Fix author
Co-authors
- Weitong Cai 2
- Xinping Lei 2
- Jiaheng Liu 2
- Jifei Song 2
- Yiqiao Xie 2
- Hang Zhang 2
- Zhensong Zhang 2
- Xinyu Che 1
- Jin Chen (陈瑾) 1
- Mingxiang Chen 1
- Xianru Chen 1
- Fangbing Deng 1
- Jiankang Deng 1
- Ken Deng 1
- Guangwen Feng 1
- Shan Gao 1
- Haoyang Huang 1
- Wenhao Huang 1
- Han Li 1
- Dingli Liang 1
- Minghao Liu 1
- Ming Sun 1
- Zhaokai Wang 1
- Junqi Xiong 1
- Songcen Xu 1
- Yifan Yao 1
- Hongyi Ye 1
- Zizheng Zhan 1
- Chenchen Zhang 1
- Ge Zhang 1
- Zhaoxiang Zhang 1
- Chenyu Zhou 1
- Letian Zhu 1