While Multimodal Large Language Models (MLLMs) have achieved remarkable success in difficult purely textual mathematical reasoning tasks, even advanced closed-source models such as GPT-o3 still struggle with geometric problems. This discrepancy motivates us to investigate the root cause: is the bottleneck of multimodal geometric reasoning rooted in reasoning itself, or in the perception of geometric information from diagrams? To answer this question, we conduct an exploratory experiment and find that providing high-quality captions consistently and substantially boosts performance of MLLMs, empirically validating the visual perception bottleneck in geometric reasoning. However, MLLMs’ capabilities in visual geometric perception remain insufficiently evaluated. To fill the gap, we propose CapGeo-Bench, a benchmark of 4,641 high-quality figure–caption pairs equipped with a fine-grained keypoint-based evaluation metric. Specifically, CapGeo-Bench encompasses three geometric classes and is devided into four difficulty levels according to the complexity of geometric primitives. Captions are scored at a fine-grained level by extracting keypoints along three dimensions-geometric elements, spatial relationships, and numerical attributes. CapGeo-Bench provides a reliable instrument and guideline for evaluating and advancing MLLMs’ geometric perception capabilities. Our code and data are publicly available at: https://github.com/YuYingLi0/CapGeo.

Recent advancements in Large Language Models (LLMs) have revolutionized Text-to-SQL parsing, achieving remarkable success in static, single-turn query generation. However, a significant disparity remains between these academic benchmarks and real-world utility. In practical applications, such as financial auditing or business analytics, user intents are rarely static; they evolve dynamically through iterative refinement, necessitating not just information retrieval (SELECT) but continuous state manipulation (INSERT, UPDATE, DELETE). To bridge this gap, we introduce DySQL-Bench, a novel benchmark designed to rigorously evaluate LLMs within a dynamic interaction framework. Unlike varying manual curation efforts, DySQL-Bench employs a two-stage automated synthesis pipeline: transforming raw relational schemas into hierarchical logic trees to generate user-database interactions, followed by a rigorous verify-and-refine protocol that ensures 100% distinct correctness via human expert validation. We further propose an interactive evaluation environment simulating a triadic workflow involving an LLM-simulated user, the agent under test, and an executable database system. Spanning 13 diverse domains with 1,072 complex tasks, our experiments reveal that current powerful models struggle in this realistic setting. Notably, GPT-4o achieves only 58.34% overall accuracy and a meager 23.81% on the strict Pass^5 metric, highlighting the substantial challenges DySQL-Bench poses for the future of database agents.

Synthetic math data is widely used to scale LLM training, but its correctness is not guaranteed. We introduce MathDebugger, the first text-only, type-aware benchmark for jointly detecting *and* classifying errors in synthetic mathematical questions and answers. MathDebugger contains 2,000 correct questions, 2,000 erroneous questions (balanced across four fine-grained error types), and 2,000 annotated answers (610 erroneous, across three error types), each annotated with both a correctness label and an error-type label, with per-error-type Fleiss’ 𝜅 ranging from 0.69 to 0.88. On 14 SOTA LLMs (6 closed-source, 8 open-source, 1.5B–72B) and 3 process reward models, even advanced systems such as GPT-o3 and DeepSeek-R1 remain far from saturating the benchmark, particularly on fine-grained error-type detection; we further surface a *solving-vs-verifying* gap that causes math-tuned and long-chain-reasoning models to trail their general-purpose siblings. Error-type annotations also provide actionable supervision for correction, with gains that are significant under paired bootstrap (p<0.01). Code and data: [https://anonymous.4open.science/r/MathDebugger-7C4A/README.md](https://anonymous.4open.science/r/MathDebugger-7C4A/README.md)

The application of autonomous agents to scientific literature review is often constrained by the reliability of online search infrastructures and linear text generation. To address these limitations, we introduce SciLENS (Scientific Localized Evidence Navigation and Synthesis), an autonomous agent framework operating entirely upon a localized dual-tier data infrastructure for deterministic access to massive academic corpora. To mitigate online dependency, SciLENS utilizes a comprehensive offline toolbox featuring topological graph traversal and structural visualization. This empowers the agent to navigate deep citation networks and organize macro-level trends into structured visual representations, mitigating context exhaustion. To train this system without human annotation, we develop an automated multi-hop data synthesis pipeline validated by cross-model consensus, followed by rubric-based reinforcement learning to align the logical planning of the agent. Evaluations across standard tasks and newly constructed complex synthesis benchmarks demonstrate that SciLENS achieves robust factual precision, outperforming existing open-source baselines and achieving performance comparable to frontier proprietary models. Our source code and data are released at https://anonymous.4open.science/r/SciLENS-48DB .

Vision-Language Models (VLMs) have demonstrated impressive capabilities in code generation across various domains. However, their ability to replicate complex, multi-panel visualizations from real-world data remains largely unassessed. To address this gap, we introduce RealChart2Code, a new large-scale benchmark with over 2,800 instances grounded in authentic datasets and featuring tasks with clear analytical intent. Crucially, it is the first benchmark to systematically evaluate chart generation from large-scale raw data and assess iterative code refinement in a multi-turn conversational setting. Our comprehensive evaluation of 14 leading VLMs on RealChart2Code reveals significant performance degradation compared to simpler benchmarks, highlighting their struggles with complex plot structures and authentic data. Our analysis uncovers a substantial performance gap between proprietary and open-weight models and confirms that even state-of-the-art VLMs often fail to accurately replicate intricate, multi-panel charts. These findings provide valuable insights into the current limitations of VLMs and guide future research directions.