DBQR-QA: A Question Answering Dataset on a Hybrid of Database Querying and Reasoning

Rungsiman Nararatwong, Chung-Chi Chen, Natthawut Kertkeidkachorn, Hiroya Takamura, Ryutaro Ichise


Abstract
This paper introduces the Database Querying and Reasoning Dataset for Question Answering (DBQR-QA), aimed at addressing the gap in current question-answering (QA) research by emphasizing the essential processes of database querying and reasoning to answer questions. Specifically designed to accommodate sequential questions and multi-hop queries, DBQR-QA more accurately mirrors the dynamics of real-world information retrieval and analysis, with a particular focus on the financial reports of US companies. The dataset’s construction, the challenges encountered during its development, the performance of large language models on this dataset, and a human evaluation are thoroughly discussed to illustrate the dataset’s complexity and highlight future research directions in querying and reasoning tasks.
Anthology ID:
2024.findings-acl.900
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
15169–15182
Language:
URL:
https://aclanthology.org/2024.findings-acl.900
DOI:
Bibkey:
Cite (ACL):
Rungsiman Nararatwong, Chung-Chi Chen, Natthawut Kertkeidkachorn, Hiroya Takamura, and Ryutaro Ichise. 2024. DBQR-QA: A Question Answering Dataset on a Hybrid of Database Querying and Reasoning. In Findings of the Association for Computational Linguistics ACL 2024, pages 15169–15182, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
DBQR-QA: A Question Answering Dataset on a Hybrid of Database Querying and Reasoning (Nararatwong et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.900.pdf