Qiming Shi
Author directory2026
Modality Fault Lines: Structural Corruptions Reveal Fragile Omni-Modal Reasoning
Zhaolu Kang | Meixin Wu | Yu Xue | Yingjie He | Qiming Shi | Lei Wei | Yidi Wang | Richeng Xuan | Zhichao Hu
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhaolu Kang | Meixin Wu | Yu Xue | Yingjie He | Qiming Shi | Lei Wei | Yidi Wang | Richeng Xuan | Zhichao Hu
Findings of the Association for Computational Linguistics: EMNLP 2026
Omni-modal large language models are increasingly evaluated on clean text–vision–audio inputs, where every channel is present, synchronized, and readily interpretable. Such scores are often taken as evidence of robust cross-modal fusion, but clean evaluation cannot tell whether success depends on stable cross-modal structure or on cues sufficient only in intact inputs. To address this gap, we define a modality fault line: a boundary at which model behavior becomes unstable when a modality remains present and human-interpretable, but its internal evidence structure is perturbed. We introduce SCEval (Structure-Corruption Evaluation) a diagnostic evaluation protocol that keeps the question, answer space, and modality channels fixed while applying controlled structural corruptions to text, vision, and audio individually and jointly. Built from 273 human-verified tri-modal examples from Social-IQ, OmniBench, and VALOR, SCEval evaluates 15 proprietary and open-source omni-modal systems. The results show that structural corruption lowers clean accuracy, text–vision damage forms the most stable shared fault line, and multi-modal degradation is non-additive rather than a simple function of the number of corrupted modalities. Clean omni-modal accuracy therefore does not establish that a model will remain reliable when cross-modal evidence becomes structurally unreliable.
BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations
Yunfan Zhou | Qiming Shi | Yizhou Yang | Di Weng | Yingcai Wu
Findings of the Association for Computational Linguistics: EMNLP 2026
Yunfan Zhou | Qiming Shi | Yizhou Yang | Di Weng | Yingcai Wu
Findings of the Association for Computational Linguistics: EMNLP 2026
While recent Large Language Model (LLM)-based text-to-SQL systems achieve impressive performance on standard benchmarks, they struggle when user queries implicitly rely on domain-specific knowledge, such as business logic, data conventions, and analytical practices, that is neither captured by the schema nor explicitly stated in the natural language question. Historical SQL query logs offer a valuable source of such knowledge, yet existing benchmarks do not adequately support evaluation of history-driven approaches. To address this gap, we introduce BIRD-History, a benchmark consisting of 1,393 tasks across 11 databases, designed to evaluate text-to-SQL systems’ ability to ground underspecified natural language questions using historical SQL scripts. Each task is annotated with ground-truth labels specifying which historical queries contain relevant knowledge and which SQL clauses encode it, enabling systematic evaluation of both retrieval effectiveness and knowledge utilization. Alongside the benchmark, we propose a plug-in retriever that extracts five types of external knowledge from historical SQL scripts, then retrieves and reranks relevant fragments for query generation. The retriever integrates seamlessly into existing few-shot text-to-SQL pipelines without requiring prompt modifications. Experiments demonstrate consistent improvements across four text-to-SQL systems, highlighting the value of leveraging historical query logs for handling underspecified queries. Dataset and code are open-sourced on https://github.com/zjuidg/BIRD-History.
SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answering
Qiming Shi | Zhaolu Kang | Yunfan Zhou | Di Weng | Yingcai Wu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Qiming Shi | Zhaolu Kang | Yunfan Zhou | Di Weng | Yingcai Wu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models are increasingly deployed as tool-augmented agents to acquire information beyond parametric knowledge. While recent work has improved long-horizon tool-use reasoning, most approaches focus on tasks with a single correct answer. In contrast, many real-world queries require discovering a comprehensive set of valid answers, a setting known as Multi-Answer QA. This setting raises two challenges: fine-grained credit assignment over long search trajectories and reward alignment for sustained exploration beyond easy high-frequency entities.We propose SPADER, a reinforcement learning framework for long-horizon tool use in Multi-Answer QA. SPADER includes Step-wise Peer Advantage (SPA), a critic-free step-level credit assignment mechanism that aligns parallel trajectories by decision step and estimates advantages from peer returns. It also includes a diversity-aware exploration reward that promotes long-tail entity discovery by upweighting rare findings and downweighting redundant ones.Experiments on QAMPARI, Mintaka, WebQSP, and QUEST show that SPADER generally improves recall and overall F1 over prompting-based agents, outcome-supervised RL methods, and recent step-level supervision approaches.Our code and model weights are available at https://github.com/KhanCold/spader.