Quality Over Clicks: Iterative Reinforcement Learning for Early-Stage E-Commerce Query Suggestion

Qi Sun, Kejun Xiao, Huaipeng Zhao, Tao Luo, Xiaoyi Zeng


Abstract
Existing dialogue systems rely on query suggestion to enhance user engagement. Recent approaches mainly optimize generative models using click-through rate (CTR) models to align with user preferences. However, these methods are less effective in early-stage deployment scenarios, where click feedback is sparse and insufficient for training a reliable CTR model. To bridge this gap, we propose QualEQS, a quality-first iterative reinforcement learning framework for e-commerce query suggestion. We formalize actionable suggestion quality along three dimensions that directly affect downstream usability: answerability, factuality, and information gain. To continuously improve from online traffic without click supervision, we further propose group-level disagreement among candidate suggestions to identify ambiguous query contexts and mine hard training cases for iterative refinement. We also introduce EQS-Benchmark, a dataset of 16,949 real-world e-commerce queries for offline training and evaluation. Experiments show that our quality-based offline metrics correlate strongly with online performance, providing a practical evaluation recipe for sparse-feedback deployment. In both offline and online settings, QualEQS consistently outperforms strong baselines, yielding a 6.81% improvement in online ChatPV in a real-world enterprise-level conversational shopping assistant system.
Anthology ID:
2026.emnlp-industry.45
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
751–762
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.45/
DOI:
Bibkey:
Cite (ACL):
Qi Sun, Kejun Xiao, Huaipeng Zhao, Tao Luo, and Xiaoyi Zeng. 2026. Quality Over Clicks: Iterative Reinforcement Learning for Early-Stage E-Commerce Query Suggestion. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 751–762, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Quality Over Clicks: Iterative Reinforcement Learning for Early-Stage E-Commerce Query Suggestion (Sun et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.45.pdf