Guoliang Zhao
Author directory2026
Enhancing Long-Context Reasoning via Verifiable Atomic Skills
Yanling Xiao | Huaibing Xie | Guoliang Zhao | Shihan Dou | Nantao Zheng | Shaolei Wang | Yiting Liu | Cheng Zhang | Pluto Zhou | Zhisong Zhang | Lemao Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yanling Xiao | Huaibing Xie | Guoliang Zhao | Shihan Dou | Nantao Zheng | Shaolei Wang | Yiting Liu | Cheng Zhang | Pluto Zhou | Zhisong Zhang | Lemao Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Long-context reasoning remains a critical challenge for Large Language Models (LLMs). Current research typically focuses on curating specialized training data by treating long-context reasoning as a monolithic task, yet it is challenging to verify its quality due to its internal cognitive complexity. In this paper, we propose a paradigm shift: viewing long-context reasoning as a hierarchical spectrum of core atomic skills. To operationalize this, we present an Anchor-based Reasoning (AbR) pipeline that automatically synthesizes strictly verifiable training data for each skill, thereby enabling reliable quality. Diagnostic analyses confirm that proficiency in these atomic skills strongly predicts general long-context reasoning performance. By training exclusively on a highly compact set of merely four thousand synthesized samples with Reinforcement Learning (RL), we explicitly sharpen these foundational capabilities. Extensive experiments across six challenging benchmarks demonstrate that this targeted RL causally delivers robust and generalizable performance gains, outperforming DeepSeek-R1-distill-32B by an average margin of 7.7% (improving from 46.3% to 54.0%). Furthermore, in-depth ablations reveal that these skills operate synergistically, establishing a scalable, data-efficient, and verifiable path to advancing long-context reasoning.
Towards a Comprehensive Scaling Law of Mixture-of-Experts
Guoliang Zhao | Yuhan Fu | Shuaipeng Li | Samm Sun | Ruobing Xie | An Wang | Weidong Han | Zhen Yang | Weixuan Sun | Yudong Zhang | Cheng-zhong Xu | Di Wang | Jie Jiang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Guoliang Zhao | Yuhan Fu | Shuaipeng Li | Samm Sun | Ruobing Xie | An Wang | Weidong Han | Zhen Yang | Weixuan Sun | Yudong Zhang | Cheng-zhong Xu | Di Wang | Jie Jiang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Mixture-of-Experts (MoE) has emerged as the dominant paradigm for parameter-efficient scaling and cost-effective deployment of Large Language Models (LLMs). However, scaling laws developed for dense models are fundamentally inapplicable to MoE architectures, owing to three critical challenges: the multiplicity of governing factors, their intricate interdependencies, and the non-monotonic nature of their effects on model performance. These challenges collectively demand a rigorous and fine-grained investigation into MoE-specific scaling laws. In this work, we present a systematic decomposition of MoE configurations, identifying five key factors that influence model performance from both size and structural perspectives: data size (D), total model size (N), activated model size (Na), number of active experts (G), and ratio of shared experts (S). We conduct 450 controlled experiments to characterize the marginal effects of each factor, culminating in a comprehensive and precise joint MoE scaling law. Building on this foundation, we derive the theoretically optimal and practically efficiency-aware optimal configurations for G, S and Na/N with detailed analyses. Our findings reveal that the optimal settings for G and S are invariant to model and data size, while the optimal activation ratio Na/N grows sparser as N increases. Our proposed MoE scaling law provides accurate and insightful guidance for future MoE model design and training.
A Novel Matching Paradigm: Unified Generative and Discriminative LLM with Prompt Compression for Relevance Learning
Guoliang Zhao | Zixin Cui | Chao Ye | Dengwu He | Fei Huang | Yubo Liu | Shuanglong Li | Tzungren Kuo | Bin Ding | Shuang Zhang | KunhongZhu | Zhi Guo | Liu Lin
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)
Guoliang Zhao | Zixin Cui | Chao Ye | Dengwu He | Fei Huang | Yubo Liu | Shuanglong Li | Tzungren Kuo | Bin Ding | Shuang Zhang | KunhongZhu | Zhi Guo | Liu Lin
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)
The matching paradigm is fundamental to large-scale information retrieval and is widely used in industrial search and advertising systems. Existing approaches employ Large Language Models (LLMs) primarily as feature extractors, underutilizing their full modeling capabilities. To address this limitation, we propose a novel matching paradigm, termed the Unified Generative and Discriminative LLM (UGD). It integrates two-tower, single-tower, and generative tasks within a unified LLM framework via attention-mask partitioning, enabling generative tasks to serve as auxiliary supervision for discriminative learning and facilitating distillation from single-tower to two-tower architectures through a multi-task fine-tuning mechanism. To satisfy online latency constraints, we further introduce a self-distillation variant of UGD with a KMeans-enhanced linearized RQVAE for prompt compression and quantization. This design compresses and quantizes landing-page documents during inference, improving serving efficiency and reducing storage overhead. Extensive experiments show that UGD achieves superior performance and strong practical value. The framework has been deployed in an industrial search engine serving hundreds of millions of users and hundreds of thousands of advertisers, significantly enhancing search experience. Open access upon publication.
Search
Fix author
Co-authors
- Zixin Cui 1
- Bin Ding 1
- Shihan Dou 1
- Yuhan Fu 1
- Zhi Guo 1
- Weidong Han 1
- Dengwu He 1
- Fei Huang 1
- Jie Jiang 1
- KunhongZhu 1
- Tzungren Kuo 1
- Shuaipeng Li 1
- Shuanglong Li 1
- Liu Lin 1
- Lemao Liu 1
- Yiting Liu 1
- Yubo Liu 1
- Weixuan Sun 1
- Xingwu Sun 1
- An Wang 1
- Di Wang 1
- Shaolei Wang 1
- Yanling Xiao 1
- Huaibing Xie 1
- Ruobing Xie 1
- Cheng-zhong Xu 1
- Zhen Yang 1
- Chao Ye 1
- Cheng Zhang 1
- Shuang Zhang 1
- Yudong Zhang 1
- Zhisong Zhang 1
- Nantao Zheng 1
- Pluto Zhou 1