Kaiwen Shi

Author directory

Other people with similar names: Kaiwen Shi


2026

The application of physics formulas is a fundamental human capability in numerical reasoning. While existing datasets often rely on implicit mathematical knowledge, they rarely explicitate the underlying formulas. To address this, we introduce FormulaReasoning, a new benchmark for formula-based numerical reasoning comprising 5,324 questions requiring calculations grounded in external physics principles. We provide high-quality, fine-grained annotations in English and Chinese—including formula structures, parameter names, symbols, values, and units—curated through manual effort and LLM-assisted validation. Additionally, we provide a consolidated formula database as an external knowledge source. To further challenge model performance, we develop an extended version of the dataset by coupling multiple questions. We evaluate various architectural and methodological frameworks, including retrieval-augmented methods, modular reasoning (formula generation, parameter extraction, and calculation), and preference-based optimization. Our analysis identifies critical challenges in formula-based reasoning, highlighting significant opportunities for future methodological advancement.
Flowcharts are widely used in business management and software development to represent process logic and user interactions. Understanding the semantics they encode is essential for downstream applications such as visual question answering. Existing methods either detect components and reconstruct topology, which suffers from cascaded errors and rigid shape assumptions, or feed the whole image to a vision language model (VLM), which dilutes attention on dense diagrams and often reverses edge directions. Inspired by cognitive science findings that humans read flowcharts by tracing arrows, we propose TRACE (Triplet Recovery via Arrowhead-Centric Extraction), a three-stage framework. An offline stage synthesizes aligned training data from text-format flowchart sources (e.g., Mermaid, BPMN XML) and fine-tunes the two models used online: a detector that locates every arrowhead, and then a VLM that extracts one triplet per arrowhead. Each output triplet is grounded to a detected arrowhead, enabling traceable diagnosis. Experiments show that TRACE substantially improves flowchart understanding accuracy and improves downstream QA accuracy.