Knowledge Transfer between Structured and Unstructured Sources for Complex Question Answering

Lingbo Mo, Zhen Wang, Jie Zhao, Huan Sun


Abstract
Multi-hop question answering (QA) combines multiple pieces of evidence to search for the correct answer. Reasoning over a text corpus (TextQA) and/or a knowledge base (KBQA) has been extensively studied and led to distinct system architectures. However, knowledge transfer between such two QA systems has been under-explored. Research questions like what knowledge is transferred or whether the transferred knowledge can help answer over one source using another one, are yet to be answered. In this paper, therefore, we study the knowledge transfer of multi-hop reasoning between structured and unstructured sources. We first propose a unified QA framework named SimultQA to enable knowledge transfer and bridge the distinct supervisions from KB and text sources. Then, we conduct extensive analyses to explore how knowledge is transferred by leveraging the pre-training and fine-tuning paradigm. We focus on the low-resource fine-tuning to show that pre-training SimultQA on one source can substantially improve its performance on the other source. More fine-grained analyses on transfer behaviors reveal the types of transferred knowledge and transfer patterns. We conclude with insights into how to construct better QA datasets and systems to exploit knowledge transfer for future work.
Anthology ID:
2022.suki-1.7
Volume:
Proceedings of the Workshop on Structured and Unstructured Knowledge Integration (SUKI)
Month:
July
Year:
2022
Address:
Seattle, USA
Editors:
Wenhu Chen, Xinyun Chen, Zhiyu Chen, Ziyu Yao, Michihiro Yasunaga, Tao Yu, Rui Zhang
Venue:
SUKI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
55–66
Language:
URL:
https://aclanthology.org/2022.suki-1.7
DOI:
10.18653/v1/2022.suki-1.7
Bibkey:
Cite (ACL):
Lingbo Mo, Zhen Wang, Jie Zhao, and Huan Sun. 2022. Knowledge Transfer between Structured and Unstructured Sources for Complex Question Answering. In Proceedings of the Workshop on Structured and Unstructured Knowledge Integration (SUKI), pages 55–66, Seattle, USA. Association for Computational Linguistics.
Cite (Informal):
Knowledge Transfer between Structured and Unstructured Sources for Complex Question Answering (Mo et al., SUKI 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.suki-1.7.pdf
Video:
 https://aclanthology.org/2022.suki-1.7.mp4
Data
HotpotQAHybridQA