Ziyu Han
Author directory2026
ChartREG++: Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referring
Tianhao Niu | Ziyu Han | Xuan Dong | Qingfu Zhu | Wanxiang Che
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tianhao Niu | Ziyu Han | Xuan Dong | Qingfu Zhu | Wanxiang Che
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Referring expression grounding is a core problem in visual grounding and is widely used as a diagnostic of spatial grounding and reasoning in vision–language models, yet most prior work focuses on natural images. In contrast, existing chart referring expression grounding-related benchmarks remain limited: (1) they largely adopt bounding boxes, constraining localization precision for fine chart elements; (2) they mostly assume one and two referred target instances, failing to handle multi-instance target references; (3) the language expressions over-rely on textual cues or data-ranking clues in the chart; (4) they cover only a narrow range of chart types. To address these issues, we introduce a chart referring expression grounding benchmark that first systematically supports multiple localization formats, multiple referred targets, diverse grounding cues and diverse chart types. Results across representative multi-modal large models reveal a significant performance gap. We further introduce an LLM-free synthesis pipeline that exploits the inherent alignment between matplotlib internal artist hierarchy and rendered chart primitives to derive pixel-accurate, multi-granularity instance segmentation masks. We train an instance segmentation model with the synthesized masks and integrate it into a general-purpose multi-modal grounding framework. The resulting system consistently outperforms baselines on our benchmark and generalizes well to a ChartQA-derived real-chart grounding benchmark.
Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards
Tianhao Niu | Ziyu Han | Qiguang Chen | Shiqi Zhou | Baocai Shan | Hengjie Fang | Qingfu Zhu | Wanxiang Che
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Tianhao Niu | Ziyu Han | Qiguang Chen | Shiqi Zhou | Baocai Shan | Hengjie Fang | Qingfu Zhu | Wanxiang Che
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Automatic data visualization generation have advanced rapidly with multi-modal large language models, yet existing efforts largely focus on static charts and overlook the interactive dashboards commonly used for real-world data exploration. We introduce Dashboard2Code, a novel task that requires a model to proactively explore an interactive dashboard, acquire and integrate feedback from its own interactions (e.g., clicking and filtering), and generate code that reproduces the target dashboard. To support comprehensive evaluation, we present DashboardMimic, the first Plotly+Dash benchmark for Dashboard2Code, comprising 180 carefully designed and manually verified dashboard–code pairs spanning three difficulty levels and covering eight common real-world interaction patterns. We further propose an automated evaluation framework tailored to dashboards that combines code semantic analysis with dynamic interaction-based testing to assess visual and interaction consistency, showing strong agreement with human judgments. Experiments across a range of open- and closed-source multi-modal models reveal that even the strongest systems struggle on high-complexity dashboards and that a substantial performance gap remains between open-source and closed-source models on the Dashboard2Code task.