Reimagining Retrieval Augmented Language Models for Answering Queries

Wang-Chiew Tan, Yuliang Li, Pedro Rodriguez, Richard James, Xi Victoria Lin, Alon Halevy, Wen-tau Yih


Abstract
We present a reality check on large language models and inspect the promise of retrieval-augmented language models in comparison. Such language models are semi-parametric, where models integrate model parameters and knowledge from external data sources to make their predictions, as opposed to the parametric nature of vanilla large language models. We give initial experimental findings that semi-parametric architectures can be enhanced with views, a query analyzer/planner, and provenance to make a significantly more powerful system for question answering in terms of accuracy and efficiency, and potentially for other NLP tasks.
Anthology ID:
2023.findings-acl.382
Original:
2023.findings-acl.382v1
Version 2:
2023.findings-acl.382v2
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6131–6146
Language:
URL:
https://aclanthology.org/2023.findings-acl.382
DOI:
10.18653/v1/2023.findings-acl.382
Bibkey:
Cite (ACL):
Wang-Chiew Tan, Yuliang Li, Pedro Rodriguez, Richard James, Xi Victoria Lin, Alon Halevy, and Wen-tau Yih. 2023. Reimagining Retrieval Augmented Language Models for Answering Queries. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6131–6146, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Reimagining Retrieval Augmented Language Models for Answering Queries (Tan et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-acl.382.pdf