CompCQR: Compositional Query Generation for Training-Free Conversational Search

Yunah Jang, Kang-il Lee, Joongbo Shin, Kyomin Jung


Abstract
Multi-turn interactions with LLMs are becoming increasingly common in information-seeking scenarios. However, user queries are often ambiguous and context-dependent, making them ill-suited for direct use as retriever queries. Conversational query reformulation (CQR) addresses this issue by rewriting the current utterance into a stand-alone query grounded in the dialogue history. Recent LLM-based CQR approaches achieve strong performance; however, their repeated LLM invocations and misalignment with downstream retrievers remain challenges. In this work, we begin from the observation that retrievers are highly sensitive to content ordering: simply reordering the same content can lead to changes in retrieval coverage and performance. Based on this, we propose a novel training-free method that generates a very large number of queries with minimal LLM usage by compositionally combining a small set of atomic components. We further apply LLM reasoning to construct a high-quality document set that balances precision and recall while capturing the user’s core intent. Our framework generalizes across both open- and closed-source LLMs as well as dense and sparse retrievers. It achieves strong performance on four widely used conversational benchmarks, with up to 22.5% relative MRR improvement over the previous state-of-the-art baseline with far fewer LLM calls.
Anthology ID:
2026.findings-emnlp.1225
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
25925–25943
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1225/
DOI:
Bibkey:
Cite (ACL):
Yunah Jang, Kang-il Lee, Joongbo Shin, and Kyomin Jung. 2026. CompCQR: Compositional Query Generation for Training-Free Conversational Search. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 25925–25943, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
CompCQR: Compositional Query Generation for Training-Free Conversational Search (Jang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1225.pdf