Linfeng Li

Author directory

Other people with similar names: Linfeng Li


2026

Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now approach or even surpass closed-source frontier models in aggregate accuracy. However, aggregate accuracy averages over many different multi-turn situations and obscures whether progress is balanced across them, masking failure redistribution. We propose an action-class-oriented diagnostic framework that decomposes failures into two orthogonal dimensions: capability and calibration. The framework operates over a four-class action space (tool_call/ask/refuse/confirm) and introduces a self-revealing upper bound, Acc ≤ GAR (Gold Action Recall), whose two violation directions separately localize action-class miscalibration and sub-action failures. We validate it on an extensive panel of tool-calling models that includes both open-weight families and closed-source frontier systems, across multiple multi-turn tool-calling benchmarks for cross-benchmark robustness. Across our panel the bottleneck is calibration, not capability: every model can produce all four classes, but which one it fires in which situation is highly model-specific. On no-tool-call scenarios, a top-ranked family’s GAR is roughly one-quarter that of size-matched peers, a 64.5 percentage point (pp) gap, indicating its ranking reflects benchmark-distribution fitting rather than genuine in-context calibration. We further show the calibration profile can be reshaped through context manipulation alone, with the reshaping direction strongly model-specific. We recommend that multi-turn tool-calling evaluations report per-category (GAR, tool-call-emit rate, Acc) triples, exposing the calibration structure aggregate accuracy hides. Our framework provides a falsifiable, reproducible diagnostic foundation for evaluation protocol design, training objective alignment, and inference-time calibration research.

2025

Multi-modal large language models (MLLMs) have achieved remarkable success in fine-grained visual understanding across a range of tasks. However, they often encounter significant challenges due to inadequate alignment for fine-grained knowledge, which restricts their ability to accurately capture local details and attain a comprehensive global perception. While recent advancements have focused on aligning object expressions with grounding information, they typically lack explicit integration of object images, which contain affluent information beyond mere texts or coordinates. To bridge this gap, we introduce a novel fine-grained visual knowledge alignment method that effectively aligns and integrates multi-scale knowledge of objects, including texts, coordinates, and images. This innovative method is underpinned by our multi-scale fine-grained enhancement data synthesis pipeline, which provides over 300K essential training data to enhance alignment and improve overall performance. Furthermore, we present TinyGroundingGPT, a series of compact models optimized for high-level alignments. With a scale of approximately 3B parameters, TinyGroundingGPT achieves outstanding results in grounding tasks while delivering performance comparable to larger MLLMs in complex visual scenarios.