Useful Diversity or Hallucinated Variety? Persona-Conditioned Synthetic Query Generation for E-Commerce Search

Filippos Ventirozos, Tom Armitage, Mahmoud Oshagh, Peter A. Appleby, Matthew Shardlow


Abstract
Synthetic supervision can bootstrap semantic parsers in new search domains, but prompt-induced diversity is useful only when generations are faithful to the structured intent. We explore this for cold-start e-commerce search: 118,098 synthetic pairs produced by 6 LLMs with 9 persona-conditioned prompt families on a 47-field UK automotive schema, against 1,214 human-annotated and 15,304 production queries. We show that the IPIP personality bundle improves parser F1 in all four matched contrasts (Holm-corrected p < .001), and all three bundles have a positive main effect over the complete 2^3 grid. Syntax conditioning is the axis that moves generations towards real traffic. It also induces lexical diversity. Alignment sKL, the divergence between von Mises-Fisher distributions fitted to synthetic and human-annotated queries, is an unlabelled proxy for parser F1 (rho = -0.917 on intent-elicited human queries, lower is closer). An LLM committee, checked against a blinded author pass, triages the 927 queries missed by every parser and labels 37% as hallucinated. Dropped constraints dominate. The four syntax-bearing families have the largest hallucinated shares, 38 to 42%, while IPIP alone has the lowest at 28%.
Anthology ID:
2026.findings-aacl.20
Volume:
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
350–368
Language:
URL:
https://aclanthology.org/2026.findings-aacl.20/
DOI:
Bibkey:
Cite (ACL):
Filippos Ventirozos, Tom Armitage, Mahmoud Oshagh, Peter A. Appleby, and Matthew Shardlow. 2026. Useful Diversity or Hallucinated Variety? Persona-Conditioned Synthetic Query Generation for E-Commerce Search. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 350–368, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
Useful Diversity or Hallucinated Variety? Persona-Conditioned Synthetic Query Generation for E-Commerce Search (Ventirozos et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-aacl.20.pdf