Fast and Light-Weight Answer Text Retrieval in Dialogue Systems

Hui Wan, Siva Sankalp Patel, J William Murdock, Saloni Potdar, Sachindra Joshi


Abstract
Dialogue systems can benefit from being able to search through a corpus of text to find information relevant to user requests, especially when encountering a request for which no manually curated response is available. The state-of-the-art technology for neural dense retrieval or re-ranking involves deep learning models with hundreds of millions of parameters. However, it is difficult and expensive to get such models to operate at an industrial scale, especially for cloud services that often need to support a big number of individually customized dialogue systems, each with its own text corpus. We report our work on enabling advanced neural dense retrieval systems to operate effectively at scale on relatively inexpensive hardware. We compare with leading alternative industrial solutions and show that we can provide a solution that is effective, fast, and cost-efficient.
Anthology ID:
2022.naacl-industry.37
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track
Month:
July
Year:
2022
Address:
Hybrid: Seattle, Washington + Online
Editors:
Anastassia Loukina, Rashmi Gangadharaiah, Bonan Min
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
334–343
Language:
URL:
https://aclanthology.org/2022.naacl-industry.37
DOI:
10.18653/v1/2022.naacl-industry.37
Bibkey:
Cite (ACL):
Hui Wan, Siva Sankalp Patel, J William Murdock, Saloni Potdar, and Sachindra Joshi. 2022. Fast and Light-Weight Answer Text Retrieval in Dialogue Systems. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track, pages 334–343, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
Cite (Informal):
Fast and Light-Weight Answer Text Retrieval in Dialogue Systems (Wan et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-industry.37.pdf
Video:
 https://aclanthology.org/2022.naacl-industry.37.mp4
Code
 IBM/ColBERT-practical
Data
Natural Questions