Efficient Answer Retrieval System (EARS): Combining Local DB Search and Web Search for Generative QA

Nikita Krayko, Ivan Sidorov, Fedor Laputin, Daria Galimzianova, Vasily Konovalov


Abstract
In this work, we propose an efficient answer retrieval system **EARS**: a production-ready, factual question answering (QA) system that combines local knowledge base search with generative, context-based QA. To assess the quality of the generated content, we devise comprehensive metrics for both manual and automatic evaluation of the answers to questions. A distinctive feature of our system is the Ranker component, which ranks answer candidates based on their relevance. This feature enhances the effectiveness of local knowledge base retrieval by 23%. Another crucial aspect of our system is the LLM, which utilizes contextual information from a web search API to generate responses. This results in substantial 92.8% boost in the usefulness of voice-based responses. **EARS** is language-agnostic and can be applied to any data domain.
Anthology ID:
2024.emnlp-industry.116
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
November
Year:
2024
Address:
Miami, Florida, US
Editors:
Franck Dernoncourt, Daniel Preoţiuc-Pietro, Anastasia Shimorina
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1584–1594
Language:
URL:
https://aclanthology.org/2024.emnlp-industry.116
DOI:
Bibkey:
Cite (ACL):
Nikita Krayko, Ivan Sidorov, Fedor Laputin, Daria Galimzianova, and Vasily Konovalov. 2024. Efficient Answer Retrieval System (EARS): Combining Local DB Search and Web Search for Generative QA. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1584–1594, Miami, Florida, US. Association for Computational Linguistics.
Cite (Informal):
Efficient Answer Retrieval System (EARS): Combining Local DB Search and Web Search for Generative QA (Krayko et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-industry.116.pdf
Poster:
 2024.emnlp-industry.116.poster.pdf