Kangjia Zhao
Author directory2026
Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling
Kangjia Zhao | Jiajun Li | Haozhan Shen | Wei Chow | Linfeng Li | Hang Song | Lingdong Kong | Chen Zhi | Tiancheng Zhao | Songhua Liu | Jianwei Yin
Findings of the Association for Computational Linguistics: EMNLP 2026
Kangjia Zhao | Jiajun Li | Haozhan Shen | Wei Chow | Linfeng Li | Hang Song | Lingdong Kong | Chen Zhi | Tiancheng Zhao | Songhua Liu | Jianwei Yin
Findings of the Association for Computational Linguistics: EMNLP 2026
Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now approach or even surpass closed-source frontier models in aggregate accuracy. However, aggregate accuracy averages over many different multi-turn situations and obscures whether progress is balanced across them, masking failure redistribution. We propose an action-class-oriented diagnostic framework that decomposes failures into two orthogonal dimensions: capability and calibration. The framework operates over a four-class action space (tool_call/ask/refuse/confirm) and introduces a self-revealing upper bound, Acc ≤ GAR (Gold Action Recall), whose two violation directions separately localize action-class miscalibration and sub-action failures. We validate it on an extensive panel of tool-calling models that includes both open-weight families and closed-source frontier systems, across multiple multi-turn tool-calling benchmarks for cross-benchmark robustness. Across our panel the bottleneck is calibration, not capability: every model can produce all four classes, but which one it fires in which situation is highly model-specific. On no-tool-call scenarios, a top-ranked family’s GAR is roughly one-quarter that of size-matched peers, a 64.5 percentage point (pp) gap, indicating its ranking reflects benchmark-distribution fitting rather than genuine in-context calibration. We further show the calibration profile can be reshaped through context manipulation alone, with the reshaping direction strongly model-specific. We recommend that multi-turn tool-calling evaluations report per-category (GAR, tool-call-emit rate, Acc) triples, exposing the calibration structure aggregate accuracy hides. Our framework provides a falsifiable, reproducible diagnostic foundation for evaluation protocol design, training objective alignment, and inference-time calibration research.
2025
ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
Haozhan Shen | Kangjia Zhao | Tiancheng Zhao | Ruochen Xu | Zilun Zhang | Mingwei Zhu | Jianwei Yin
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Haozhan Shen | Kangjia Zhao | Tiancheng Zhao | Ruochen Xu | Zilun Zhang | Mingwei Zhu | Jianwei Yin
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in vision-language understanding. Recently, with the integration of test-time scaling techniques, these models have also shown strong potential in visual reasoning. However, most existing reasoning approaches remain text-level in nature: MLLMs are prompted to explore various combinations of textual tokens via their underlying language model, while the visual input remains fixed throughout the reasoning process. This paradigm limits the model’s ability to fully exploit rich visual information, particularly when dealing with images containing numerous fine-grained elements. In such cases, vision-level reasoning becomes crucial—where models dynamically zoom into specific regions of the image to gather detailed visual cues necessary for accurate decision-making. In this paper, we propose Zoom Eye, a training-free, model-agnostic tree search algorithm tailored for vision-level reasoning. Zoom Eye treats an image as a hierarchical tree structure, where each child node represents a zoomed-in sub-region of its parent, and the root corresponds to the full image. The algorithm enables MLLMs to simulate human-like zooming behavior by navigating from root to leaf nodes in search of task-relevant visual evidence. We experiment on a series of elaborate high-resolution benchmarks and the results demonstrate that Zoom Eye not only consistently improves the performance of a series of MLLMs with large margin (e.g., InternVL2.5-8B increases by 15.71% and 17.69% on HR-Bench) but also enables small 3-8B MLLMs to outperform strong large models such as GPT-4o.