Socratic Question Generation: A Novel Dataset, Models, and Evaluation

Beng Heng Ang, Sujatha Das Gollapalli, See-Kiong Ng


Abstract
Socratic questioning is a form of reflective inquiry often employed in education to encourage critical thinking in students, and to elicit awareness of beliefs and perspectives in a subject during therapeutic counseling. Specific types of Socratic questions are employed for enabling reasoning and alternate views against the context of individual personal opinions on a topic. Socratic contexts are different from traditional question generation contexts where “answer-seeking” questions are generated against a given formal passage on a topic, narrative stories or conversations. We present SocratiQ, the first large dataset of 110K (question, context) pairs for enabling studies on Socratic Question Generation (SoQG). We provide an in-depth study on the various types of Socratic questions and present models for generating Socratic questions against a given context through prompt tuning. Our automated and human evaluation results demonstrate that our SoQG models can produce realistic, type-sensitive, human-like Socratic questions enabling potential applications in counseling and coaching.
Anthology ID:
2023.eacl-main.12
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
147–165
Language:
URL:
https://aclanthology.org/2023.eacl-main.12
DOI:
10.18653/v1/2023.eacl-main.12
Bibkey:
Cite (ACL):
Beng Heng Ang, Sujatha Das Gollapalli, and See-Kiong Ng. 2023. Socratic Question Generation: A Novel Dataset, Models, and Evaluation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 147–165, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Socratic Question Generation: A Novel Dataset, Models, and Evaluation (Ang et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.12.pdf
Video:
 https://aclanthology.org/2023.eacl-main.12.mp4