Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems

Lidiya Murakhovs’ka, Philippe Laban, Tian Xie, Caiming Xiong, Chien-Sheng Wu


Abstract
Making big purchases requires consumers to research or consult a salesperson to gain domain expertise. However, existing conversational recommender systems (CRS) often overlook users’ lack of background knowledge, focusing solely on gathering preferences. In this work, we define a new problem space for conversational agents that aim to provide both product recommendations and educational value through mixed-type mixed-initiative dialog. We introduce SalesOps, a framework that facilitates the simulation and evaluation of such systems by leveraging recent advancements in large language models (LLMs). We build SalesBot and ShopperBot, a pair of LLM-powered agents that can simulate either side of the framework. A comprehensive human study compares SalesBot against professional salespeople, revealing that although SalesBot approaches professional performance in terms of fluency and informativeness, it lags behind in recommendation quality. We emphasize the distinct limitations both face in providing truthful information, highlighting the challenges of ensuring faithfulness in the CRS context. We release our code and make all data available.
Anthology ID:
2023.findings-emnlp.657
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9823–9838
Language:
URL:
https://aclanthology.org/2023.findings-emnlp.657
DOI:
10.18653/v1/2023.findings-emnlp.657
Bibkey:
Cite (ACL):
Lidiya Murakhovs’ka, Philippe Laban, Tian Xie, Caiming Xiong, and Chien-Sheng Wu. 2023. Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9823–9838, Singapore. Association for Computational Linguistics.
Cite (Informal):
Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems (Murakhovs’ka et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-emnlp.657.pdf