Tiancheng Zhao

Author directory

Other people with similar names: Tiancheng Zhao

Unverified author pages with similar names: Tiancheng Zhao


2026

Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now approach or even surpass closed-source frontier models in aggregate accuracy. However, aggregate accuracy averages over many different multi-turn situations and obscures whether progress is balanced across them, masking failure redistribution. We propose an action-class-oriented diagnostic framework that decomposes failures into two orthogonal dimensions: capability and calibration. The framework operates over a four-class action space (tool_call/ask/refuse/confirm) and introduces a self-revealing upper bound, Acc ≤ GAR (Gold Action Recall), whose two violation directions separately localize action-class miscalibration and sub-action failures. We validate it on an extensive panel of tool-calling models that includes both open-weight families and closed-source frontier systems, across multiple multi-turn tool-calling benchmarks for cross-benchmark robustness. Across our panel the bottleneck is calibration, not capability: every model can produce all four classes, but which one it fires in which situation is highly model-specific. On no-tool-call scenarios, a top-ranked family’s GAR is roughly one-quarter that of size-matched peers, a 64.5 percentage point (pp) gap, indicating its ranking reflects benchmark-distribution fitting rather than genuine in-context calibration. We further show the calibration profile can be reshaped through context manipulation alone, with the reshaping direction strongly model-specific. We recommend that multi-turn tool-calling evaluations report per-category (GAR, tool-call-emit rate, Acc) triples, exposing the calibration structure aggregate accuracy hides. Our framework provides a falsifiable, reproducible diagnostic foundation for evaluation protocol design, training objective alignment, and inference-time calibration research.
Improving vision-language models (VLMs) on visual reasoning typically requires retraining or hand-designed prompts and tools. We present Dynamo, a training-free framework that adapts a frozen VLM without any weight updates. On a small labeled training subset, the agent inspects its own correct and incorrect attempts and evolves two complementary capabilities: reusable reasoning skills for cognitive bottlenecks, and executable visual tools for perceptual ones. Each generated tool is paired with a skill that specifies when to invoke it, and both capability types accumulate in a persistent library. Across four visual reasoning benchmarks and five VLM backbones, Dynamo improves direct inference on all 20 model–benchmark settings (avg. +5.6 acc). When the tool set is given in advance, the framework learns when to call each tool, and per-step tool choice improves on every tested backbone. Against task-specific RL (VTool-R1, DeepEyes), Dynamo closes 65–99% of the RL gap at a fraction of the compute, and combines additively with RL when available.
When retrieved evidence contradicts parametric memory, language models frequently ignore the context and revert to memorized priors—a failure that undermines the purpose of retrieval augmentation. Contrastive decoding amplifies context-conditioned output to suppress parametric bias, but fixed coefficients cannot track changing disagreement, while output-only adaptive coefficients observe only the final prediction distributions. We propose FIDES (Faithful Inference via Deep Evidence Signals), a training-free decoder that combines signals from output distributions, hidden representations, and prediction trajectories to recompute contrastive intervention strength at every step. A matched, formula-verified audit on 128 NQ-Swap pairs shows that conflict raises Opposition by 0.0969 and the resulting coefficient α by 0.0727 at the same answer-prediction position (both p < 10⁻⁴). The audit also finds only a small Shift effect and saturated clipped Noise, clarifying the limits of the mechanism interpretation. Across three benchmarks and six backbones—four primary 7B/8B models and two scaling backbones up to 70B—FIDES achieves the best context fidelity in all 18 settings, outperforming the strongest training-free baseline by +3 to +13 points. At 70B, fidelity reaches 92–94% while F1 reaches 62–63%, showing that multi-depth stepwise contrast can preserve generation utility while improving evidence following.

2025

Self-improving large language models (LLMs) – i.e., to improve the performance of an LLM by fine-tuning it with synthetic data generated by itself – is a promising way to advance the capabilities of LLMs while avoiding extensive supervision. Existing approaches to self-improvement often rely on external supervision signals in the form of seed data and/or assistance from third-party models. This paper presents Crescent – a simple yet effective framework for generating high-quality synthetic question-answer data in a fully autonomous manner. Crescent first elicits the LLM to generate raw questions via a bait prompt, then diversifies these questions leveraging a rejection sampling-based self-deduplication, and finally feeds the questions to the LLM and collects the corresponding answers by means of majority voting. We show that Crescent sheds light on the potential of true self-improvement with zero external supervision signals for math reasoning; in particular, Crescent-generated question-answer pairs suffice to (i) improve the reasoning capabilities of an LLM while preserving its general performance (especially in the 0-shot setting); and (ii) distill LLM knowledge to weaker models more effectively than existing methods based on seed-dataset augmentation.
Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in vision-language understanding. Recently, with the integration of test-time scaling techniques, these models have also shown strong potential in visual reasoning. However, most existing reasoning approaches remain text-level in nature: MLLMs are prompted to explore various combinations of textual tokens via their underlying language model, while the visual input remains fixed throughout the reasoning process. This paradigm limits the model’s ability to fully exploit rich visual information, particularly when dealing with images containing numerous fine-grained elements. In such cases, vision-level reasoning becomes crucial—where models dynamically zoom into specific regions of the image to gather detailed visual cues necessary for accurate decision-making. In this paper, we propose Zoom Eye, a training-free, model-agnostic tree search algorithm tailored for vision-level reasoning. Zoom Eye treats an image as a hierarchical tree structure, where each child node represents a zoomed-in sub-region of its parent, and the root corresponds to the full image. The algorithm enables MLLMs to simulate human-like zooming behavior by navigating from root to leaf nodes in search of task-relevant visual evidence. We experiment on a series of elaborate high-resolution benchmarks and the results demonstrate that Zoom Eye not only consistently improves the performance of a series of MLLMs with large margin (e.g., InternVL2.5-8B increases by 15.71% and 17.69% on HR-Bench) but also enables small 3-8B MLLMs to outperform strong large models such as GPT-4o.
Language agents powered by large language models (LLMs) have demonstrated remarkable capabilities in understanding, reasoning, and executing complex tasks. However, developing robust agents presents significant challenges: substantial engineering overhead, lack of standardized components, and insufficient evaluation frameworks for fair comparison. We introduce Agent Graph-based Orchestration for Reasoning and Assessment (AGORA), a flexible and extensible framework that addresses these challenges through three key contributions: (1) a modular architecture with a graph-based workflow engine, efficient memory management, and clean component abstraction; (2) a comprehensive suite of reusable agent algorithms implementing state-of-the-art reasoning approaches; and (3) a rigorous evaluation framework enabling systematic comparison across multiple dimensions. Through extensive experiments on mathematical reasoning and multimodal tasks, we evaluate various agent algorithms across different LLMs, revealing important insights about their relative strengths and applicability. Our results demonstrate that while sophisticated reasoning approaches can enhance agent capabilities, simpler methods like Chain-of-Thought often exhibit robust performance with significantly lower computational overhead. AGORA not only simplifies language agent development but also establishes a foundation for reproducible agent research through standardized evaluation protocols.We made a demo video at: https://www.youtube.com/watch?v=WRH-F1zegKI. The comparison agent of algorithms is also available at https://huggingface.co/spaces/omlab/open-agent-leaderboard. Source code of AGORA can be found at https://github.com/om-ai-lab/OmAgent.

2024

Recent advancements in Large Language Models (LLMs) have expanded their capabilities to multimodal contexts, including comprehensive video understanding. However, processing extensive videos such as 24-hour CCTV footage or full-length films presents significant challenges due to the vast data and processing demands. Traditional methods, like extracting key frames or converting frames to text, often result in substantial information loss. To address these shortcomings, we develop OmAgent, efficiently stores and retrieves relevant video frames for specific queries, preserving the detailed content of videos. Additionally, it features an Divide-and-Conquer Loop capable of autonomous reasoning, dynamically invoking APIs and tools to enhance query processing and accuracy. This approach ensures robust video understanding, significantly reducing information loss. Experimental results affirm OmAgent’s efficacy in handling various types of videos and complex tasks. Moreover, we have endowed it with greater autonomy and a robust tool-calling system, enabling it to accomplish even more intricate tasks.