Pengjie Wang
Author directory2026
LoRE: Enhancing Search Relevance with Progressive Chain-of-Thought and Preference Alignment
Chenji Lu | Zhuo Chen | Hui Zhao | Zhiyuan Zeng | Gang Zhao | Junjie Ren | Lihaoran | Songyan Liu | Pengjie Wang | Chuan Yu | Jian Xu | Bo Zheng
Findings of the Association for Computational Linguistics: ACL 2026
Chenji Lu | Zhuo Chen | Hui Zhao | Zhiyuan Zeng | Gang Zhao | Junjie Ren | Lihaoran | Songyan Liu | Pengjie Wang | Chuan Yu | Jian Xu | Bo Zheng
Findings of the Association for Computational Linguistics: ACL 2026
E-commerce search relevance is a critical component of retrieval systems. While Large Language Models (LLMs)-driven Chain-of-Thought (CoT) modeling has become the dominant paradigm and yielded significant gains, a critical gap remains: the absence of a systematic definition for comprehensive relevance reasoning, which leads to significant blind spots in current approaches. In this paper, we deconstruct the task into three core competencies: reasoning knowledge, multi-modal understanding, and rule awareness. Accordingly, we propose LoRE(Large Generative Model for Search Relevance), a novel two-stage training framework. We first employ an SFT phase to instill these capabilities via a progressive CoT synthesis pipeline, followed by a Reinforcement Learning(RL) phase, which serves as a regularizer, pruning redundant logic to achieve precise and robust adjudication. Extensive experiments validate LoRE, outperforming GPT-5 by 29.1% in Macro-F1 and achieving a 27% online gain, offering a vital reference for industrial domain-specific post-training.
NOSA: Native and Offloadable Sparse Attention
Yuxiang Huang | Pengjie Wang | Jicheng Han | Weilin Zhao | Zhou su | Ao Sun | Hongya Lyu | Hengyu Zhao | Yudong Wang | Chaojun Xiao | Xu Han | Zhiyuan Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yuxiang Huang | Pengjie Wang | Jicheng Han | Weilin Zhao | Zhou su | Ao Sun | Hongya Lyu | Hengyu Zhao | Yudong Wang | Chaojun Xiao | Xu Han | Zhiyuan Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Decoding throughput gains from larger inference batches are often limited by GPU memory, mainly due to the large size of KV cache. Training-free KV cache offloading mitigates this issue by storing redundant context on the CPU and fetching sparse KV subsets, but often harms long-generation quality due to training-inference mismatch of sparse patterns. Trainable sparse attention can improve quality, yet its unconstrained KV accesses are inefficient for offloading and may incur excessive CPU–GPU transfers. To this end, we propose NOSA, a trainable sparse attention mechanism designed for KV cache offloading. NOSA explicitly bounds CPU–GPU KV transfers, enabling low communication overhead and high decoding throughput. We further develop NOSI, an offloading inference system that realizes these efficiency gains. Experiments on 1,3,8B LLMs show that NOSA outperforms KV cache offloading baselines on general, long-input, and long-generation tasks, while improving decoding throughput by up to 5.04×, 1.92×, and 1.83× over FullAttn, InfLLMv2, and ShadowKV, respectively.