Retrieval-based Question Answering with Passage Expansion Using a Knowledge Graph

Benno Kruit, Yiming Xu, Jan-Christoph Kalo


Abstract
Recent advancements in dense neural retrievers and language models have led to large improvements in state-of-the-art approaches to open-domain Question Answering (QA) based on retriever-reader architectures. However, issues stemming from data quality and imbalances in the use of dense embeddings have hindered performance, particularly for less common entities and facts. To tackle these problems, this study explores a multi-modal passage retrieval model’s potential to bolster QA system performance. This study poses three key questions: (1) Can a distantly supervised question-relation extraction model enhance retrieval using a knowledge graph (KG), compensating for dense neural retrievers’ shortcomings with rare entities? (2) How does this multi-modal approach compare to existing QA systems based on textual features? (3) Can this QA system alleviate poor performance on less common entities on common benchmarks? We devise a multi-modal retriever combining entity features and textual data, leading to improved retrieval precision in some situations, particularly for less common entities. Experiments across different datasets confirm enhanced performance for entity-centric questions, but challenges remain in handling complex generalized questions.
Anthology ID:
2024.lrec-main.1225
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
14063–14072
Language:
URL:
https://aclanthology.org/2024.lrec-main.1225
DOI:
Bibkey:
Cite (ACL):
Benno Kruit, Yiming Xu, and Jan-Christoph Kalo. 2024. Retrieval-based Question Answering with Passage Expansion Using a Knowledge Graph. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 14063–14072, Torino, Italia. ELRA and ICCL.
Cite (Informal):
Retrieval-based Question Answering with Passage Expansion Using a Knowledge Graph (Kruit et al., LREC-COLING 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.lrec-main.1225.pdf
Optional supplementary material:
 2024.lrec-main.1225.OptionalSupplementaryMaterial.zip