Ziqing Qiao
Author directory2026
StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?
Yinghao Chen | Zixi Chen | Bingxiang He | Ziqing Qiao | Huan-ang Gao | Yinuo Xu | Yuxin Zuo | Zeyuan Liu | Yuhao Zhan | Chaojun Xiao
Findings of the Association for Computational Linguistics: EMNLP 2026
Yinghao Chen | Zixi Chen | Bingxiang He | Ziqing Qiao | Huan-ang Gao | Yinuo Xu | Yuxin Zuo | Zeyuan Liu | Yuhao Zhan | Chaojun Xiao
Findings of the Association for Computational Linguistics: EMNLP 2026
Humans need to study only a handful of well-written textbooks to master a discipline and attempt its hardest problems. We argue that an ideal self-evolution method should share the same property, that is autonomously learning from raw training material for transferable problem-solving capability. However, we still lack a direct measurement for it. We introduce StudyBench, a controlled physics benchmark that directly measures how efficiently a self-evolution method converts training material into capability. We organise the test set into an Application Set, consisting of difficult textbook problems and evaluating absorption ability, and a Transfer Set, consisting of olympiad-level problems and evaluating transfer ability. Benchmarking representative self-evolution methods across three base models, we find that improvements on the Application Set rarely translate to the harder Transfer Set. A guidance ablation exposes a Guidance Gap: even the strongest method closes only a small fraction of what the same material unlocks when supplied as in-context guidance. Besides, every method hits a Compute Plateau, saturating well before exhausting its compute budget. The remaining gap is therefore a method problem rather than a data or compute problem. By offering a clean and controlled benchmark, StudyBench turns self-evolution progress from an open-ended pursuit into a measurable target for future research. Our code is released at https://github.com/thunlp/StudyBench.
CoTrust: Privacy-Preserving Collaboration Between Large and Small Language Models in Trusted Execution Environments
Zhenya Ma | Tingyi Wang | Yongheng Deng | Ziqing Qiao | Yinggui Wang | Tao Wei | Lei Wang | Ju Ren
Findings of the Association for Computational Linguistics: ACL 2026
Zhenya Ma | Tingyi Wang | Yongheng Deng | Ziqing Qiao | Yinggui Wang | Tao Wei | Lei Wang | Ju Ren
Findings of the Association for Computational Linguistics: ACL 2026
Services powered by large language models (LLMs) provide powerful text generation capabilities, but accessing sensitive user inputs raises serious privacy concerns. Trusted Execution Environments (TEEs) provide a secure computation environment, enabling sensitive inputs to be safely processed. However, directly deploying high-capacity LLMs in TEEs is often prohibitively expensive due to computation and memory constraints. To reconcile privacy, efficiency, and generation quality, we propose CoTrust, a privacy-preserving collaborative inference framework that combines LLMs with small language models (SLMs) inside TEE. CoTrust uses multiple de-identified views to let the LLM produce a consensus scaffold capturing answer reasoning without exposing private information, which the SLM then grounds in the full input to generate the final response. Experiments on multiple question answering and summarization benchmarks show that CoTrust approaches the performance of unconstrained LLMs, outperforms existing privacy-preserving baselines, and maintains strong privacy protection, while remaining efficient in a TDX-based TEE implementation.
Rethinking the Role of Efficient Attention in Hybrid Architectures
Ziqing Qiao | Yinuo Xu | Chaojun Xiao | Zhou su | Zihan Zhou | Yingfa Chen | Xiaoyue Xu | Xu Han | Zhiyuan Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ziqing Qiao | Yinuo Xu | Chaojun Xiao | Zhou su | Zihan Zhou | Yingfa Chen | Xiaoyue Xu | Xu Han | Zhiyuan Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Modern language models increasingly adopt hybrid architectures that combine full attention with efficient attention modules, such as sliding-window attention (SWA) and recurrent sequence mixers. However, how these efficient modules shape model capabilities remains poorly understood. To address this gap, we conduct a systematic analysis across hybrid architectures from three perspectives: scaling behavior, mechanism analysis, and architecture design. First, from a scaling perspective, we find that efficient-attention design primarily affects how fast long-context capability emerges, while different hybrids eventually converge to comparable long-context performance under sufficient training. Second, mechanistically, we show that long-range retrieval is mainly carried by full attention, whereas efficient attention shapes its optimization trajectory. This explains a counter-intuitive phenomenon we call Large-Window Laziness: larger SWA windows can delay the formation of retrieval heads in full-attention layers. Third, guided by this mechanism, we show that applying No Positional Encoding (NoPE) to only the full-attention layers of a small-window SWA hybrid substantially improves long-context performance with negligible impact on short-context performance.
2025
ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning
Ziqing Qiao | Yongheng Deng | Jiali Zeng | Dong Wang | Lai Wei | Guanbo Wang | Fandong Meng | Jie Zhou | Ju Ren | Yaoxue Zhang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Ziqing Qiao | Yongheng Deng | Jiali Zeng | Dong Wang | Lai Wei | Guanbo Wang | Fandong Meng | Jie Zhou | Ju Ren | Yaoxue Zhang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Large Reasoning Models (LRMs) perform strongly in complex reasoning tasks via Chain-of-Thought (CoT) prompting, but often suffer from verbose outputs, increasing computational overhead. Existing fine-tuning-based compression methods either operate post-hoc pruning, risking disruption to reasoning coherence, or rely on sampling-based selection, which fails to remove redundant content thoroughly. To address these limitations, this work begins by framing two key patterns of redundant reflection in LRMs—Confidence Deficit, wherein the model reflects on correct intermediate steps, and Termination Delay, where reflection continues after a verified, confident answer—through a confidence-guided perspective. Based on this, we introduce ConCISE (Confidence-guided Compression In Step-by-step Efficient Reasoning), a framework designed to generate concise reasoning chains, integrating Confidence Injection to boost reasoning confidence, and Early Stopping to terminate reasoning when confidence is sufficient. Extensive experiments demonstrate that compared to baseline methods, fine-tuning LRMs on ConCISE-generated data yields a better balance between compression and task performance, reducing length by up to ~50% under SimPO, while maintaining high task accuracy.