Changyu Zeng
Author directory2026
MEUR: A Benchmark for Evaluating Vision-Language Models on Multimodal Event Understanding and Reasoning
Zimu Wang | Yuqi Wang | Tong Chen | Changyu Zeng | Hongbin Na | Nijia Han | Fuyu Xing | Qi Chen | Qiufeng Wang | Anh Nguyen | Shuihua Wang | Ling Chen | Jionglong Su | Haiyang Zhang | Wei Wang
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Zimu Wang | Yuqi Wang | Tong Chen | Changyu Zeng | Hongbin Na | Nijia Han | Fuyu Xing | Qi Chen | Qiufeng Wang | Anh Nguyen | Shuihua Wang | Ling Chen | Jionglong Su | Haiyang Zhang | Wei Wang
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Event understanding and reasoning play critical roles in thoroughly evaluating the capabilities of Vision-Language Models (VLMs); however, existing Visual Question Answering (VQA) datasets predominantly focus on entity-centric questions, while event- or action-related questions are limited in scale and suffer from significant shortcut issues. We introduce MEUR, the first Multimodal Event Understanding and Reasoning dataset consisting of 1,200 images and 4,217 questions, necessitating VLMs with a diverse range of multimodal understanding and reasoning capabilities to answer, ranging from basic event recognition to more complex tasks such as counting and comparison. To streamline the annotation process, we propose a novel semi-automated pipeline that combines advanced VLMs with human annotators, achieving high quality and efficiency. We conduct extensive experiments on state-of-the-art non-thinking and thinking VLMs to demonstrate their capabilities and limitations in multimodal event understanding and reasoning. Furthermore, we provide a detailed error analysis that points out promising directions for future research.
SinoGlyphBench: A Diagnostic Benchmark for Chinese Glyph-Level Obfuscation in Language-Model Moderation
Yifan Wang | Zimu Wang | Suliu Qin | Changyu Zeng | Tong Chen | Siqi Chen | Yijie Lin | Lingyu Jiang | Jionglong Su | Yushan Pan | Haiyang Zhang | Wei Wang | Qiaoyu Tan
Findings of the Association for Computational Linguistics: EMNLP 2026
Yifan Wang | Zimu Wang | Suliu Qin | Changyu Zeng | Tong Chen | Siqi Chen | Yijie Lin | Lingyu Jiang | Jionglong Su | Yushan Pan | Haiyang Zhang | Wei Wang | Qiaoyu Tan
Findings of the Association for Computational Linguistics: EMNLP 2026
Glyph-level obfuscation can leave harmful Chinese content readable to humans while degrading automated moderation. We introduce SinoGlyphBench, a diagnostic benchmark that identifies label-critical semantic anchors and creates matched original and glyph-obfuscated inputs in text and image modalities. By perturbing anchors, background context, or both, this design distinguishes corruption of moderation-relevant evidence from general surface variation. Across 176,916 paired evaluations of 12 LLMs and MLLMs, obfuscation increases harmful false-negative and false-positive rates by 6.1 and 4.7 percentage points, respectively, and reduces four-way accuracy by 5.0 points. Models retain 75.7% of the decisions that were correct on the matched original inputs. Full-scope perturbations cause the largest degradation, anchor-only perturbations are more damaging than background-only perturbations, and cross-script substitution is particularly difficult in the text modality. Analysis of structured outputs identifies observable mismatches in visible-form reading, intended-message recovery, and final safety judgment. The evaluated models, therefore, remain brittle to Chinese content written with non-canonical glyphs. Resources are available at https://github.com/fengshun124/SinoGlyphBench.
2025
FinDebate: Multi-Agent Collaborative Intelligence for Financial Analysis
Tianshi Cai | Guanxu Li | Nijia Han | Ce Huang | Zimu Wang | Changyu Zeng | Yuqi Wang | Jingshi Zhou | Haiyang Zhang | Qi Chen | Yushan Pan | Shuihua Wang | Wei Wang
Proceedings of The 10th Workshop on Financial Technology and Natural Language Processing
Tianshi Cai | Guanxu Li | Nijia Han | Ce Huang | Zimu Wang | Changyu Zeng | Yuqi Wang | Jingshi Zhou | Haiyang Zhang | Qi Chen | Yushan Pan | Shuihua Wang | Wei Wang
Proceedings of The 10th Workshop on Financial Technology and Natural Language Processing
NUMINA: A Natural Understanding Benchmark for Multi-dimensional Intelligence and Numerical Reasoning Abilities
Changyu Zeng | Yifan Wang | Zimu Wang | Wei Wang | Zhengni Yang | Muyi Bao | Jimin Xiao | Anh Nguyen | Yutao Yue
Findings of the Association for Computational Linguistics: EMNLP 2025
Changyu Zeng | Yifan Wang | Zimu Wang | Wei Wang | Zhengni Yang | Muyi Bao | Jimin Xiao | Anh Nguyen | Yutao Yue
Findings of the Association for Computational Linguistics: EMNLP 2025
Recent advancements in 2D multimodal large language models (MLLMs) have significantly improved performance in vision-language tasks. However, extending these capabilities to 3D environments remains a distinct challenge due to the complexity of spatial reasoning. Nevertheless, existing 3D benchmarks often lack fine-grained numerical reasoning task annotations, limiting MLLMs’ ability to perform precise spatial measurements and complex numerical reasoning. To address this gap, we introduce NUMINA, the first Natural Understanding benchmark for Multi-dimensional Intelligence and Numerical reasoning Abilities to enhance multimodal indoor perceptual understanding. NUMINA features multi-scale annotations and various question-answer pairs, generated using NUMINA-Flow, an automated annotation pipeline that integrates LLM rewriting and rule-based self-verification. We evaluate the performance of various state-of-the-art LLMs on NUMINA following the Chat-Scene framework, demonstrating that current LLMs struggle with multimodal numerical reasoning, particularly in performing precise computations such as distance and volume estimation, highlighting the need for further advancements in 3D models. The dataset and source codes can be obtained from https://github.com/fengshun124/NUMINA.
Search
Fix author
Co-authors
- Zimu Wang 4
- Qi Chen 2
- Nijia Han 2
- Anh Nguyen 2
- Yushan Pan 2
- Jionglong Su 2
- Shuihua Wang 2
- Wei Wang 2
- Wei Wang 2
- Yifan Wang 2
- Yuqi Wang 2
- Haiyang Zhang 2
- Muyi Bao 1
- Tianshi Cai 1
- Ling Chen 1
- Siqi Chen 1
- Tong Chen 1
- Tong Chen 1
- Ce Huang 1
- Lingyu Jiang 1
- Guanxu Li 1
- Yijie Lin 1
- Hongbin Na 1
- Suliu Qin 1
- Qiaoyu Tan 1
- Qiufeng Wang 1
- Jimin Xiao 1
- Fuyu Xing 1
- Zhengni Yang 1
- Yutao Yue 1
- Haiyang Zhang 1
- Jingshi Zhou 1