Exploring Retrieval Augmented Generation Approaches for Natural Language Question Understanding

Christoph Kowalski, Amelie Sophie Robrecht-Hilbig, Vincent Emmerling, Stefan Kopp


Abstract
Language-based interactive AI systems are required to provide adaptive explanations in order to create transparency and interpretability. A cornerstone ability for this is to respond to user questions. We target the problem of Natural Language Question Understanding (NLQU), which refers to interpreting a user question and mapping it to semantic representations (items in a structured knowledge base) that are relevant to address the underlying user’s knowledge gap and hence to answer the question. In contrast to pure Q&A systems that aim to directly map questions to answers, NLQU enables a dialog agent to employ different explanation strategies, including explaining prerequisite knowledge, adapting explanation speed, or identifying and repairing misunderstandings. This paper explores the use of modern Large Language Models (LLMs) along with Retrieval Augmented Generation (RAG) for NLQU. We present a RAG-based NLQU component, evaluate different approaches against a synthetic dataset, and test the final component on a natural language question dataset from a human-human explanation study. Different LLM models, prompts, and hyperparameters are tested and compared to a baseline method.
Anthology ID:
2026.sigdial-1.34
Volume:
Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
August
Year:
2026
Address:
Atlanta, Georgia, USA
Editors:
Jinho D. Choi, Yun-Nung Chen, Kotaro Funakoshi, Ali Emami
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
486–496
Language:
URL:
https://aclanthology.org/2026.sigdial-1.34/
DOI:
Bibkey:
Cite (ACL):
Christoph Kowalski, Amelie Sophie Robrecht-Hilbig, Vincent Emmerling, and Stefan Kopp. 2026. Exploring Retrieval Augmented Generation Approaches for Natural Language Question Understanding. In Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 486–496, Atlanta, Georgia, USA. Association for Computational Linguistics.
Cite (Informal):
Exploring Retrieval Augmented Generation Approaches for Natural Language Question Understanding (Kowalski et al., SIGDIAL 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.sigdial-1.34.pdf