Retrieval-Augmented Chain-of-Thought in Semi-structured Domains

Vaibhav Mavi, Abulhair Saparov, Chen Zhao


Abstract
Applying existing question answering (QA) systems to specialized domains like law and finance presents challenges that necessitate domain expertise. Although large language models (LLMs) have shown impressive language comprehension and in-context learning capabilities, their inability to handle very long inputs/contexts is well known. Tasks specific to these domains need significant background knowledge, leading to contexts that can often exceed the maximum length that existing LLMs can process. This study explores leveraging the semi-structured nature of legal and financial data to efficiently retrieve relevant context, enabling the use of LLMs for domain-specialized QA. The resulting system outperforms contemporary models and also provides useful explanations for the answers, encouraging the integration of LLMs into legal and financial NLP systems for future research.
Anthology ID:
2023.nllp-1.18
Volume:
Proceedings of the Natural Legal Language Processing Workshop 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Daniel Preoțiuc-Pietro, Catalina Goanta, Ilias Chalkidis, Leslie Barrett, Gerasimos (Jerry) Spanakis, Nikolaos Aletras
Venues:
NLLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
178–191
Language:
URL:
https://aclanthology.org/2023.nllp-1.18
DOI:
10.18653/v1/2023.nllp-1.18
Bibkey:
Cite (ACL):
Vaibhav Mavi, Abulhair Saparov, and Chen Zhao. 2023. Retrieval-Augmented Chain-of-Thought in Semi-structured Domains. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 178–191, Singapore. Association for Computational Linguistics.
Cite (Informal):
Retrieval-Augmented Chain-of-Thought in Semi-structured Domains (Mavi et al., NLLP-WS 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.nllp-1.18.pdf