Generating Commonsense Reasoning Questions with Controllable Complexity through Multi-step Structural Composition

Jianxing Yu, Shiqi Wang, Hanjiang Lai, Wenqing Chen, Yanghui Rao, Qinliang Su, Jian Yin


Abstract
This paper studies the task of generating commonsense reasoning questions (QG) with desired difficulty levels. Compared to traditional shallow questions that can be solved by simple term matching, ours are more challenging. Our answering process requires reasoning over multiple contextual and commonsense clues. That involves advanced comprehension skills, such as abstract semantics learning and missing knowledge inference. Existing work mostly learns to map the given text into questions, lacking a mechanism to control results with the desired complexity. To address this problem, we propose a novel controllable framework. We first derive contextual and commonsense clues involved in reasoning questions from the text. These clues are used to create simple sub-questions. We then aggregate multiple sub-questions to compose complex ones under the guidance of prior reasoning structures. By iterating this process, we can compose a complex QG task based on a series of smaller and simpler QG subtasks. Each subtask serves as a building block for a larger one. Each composition corresponds to an increase in the reasoning step. Moreover, we design a voting verifier to ensure results’ validity from multiple views, including answer consistency, reasoning difficulty, and context correlation. Finally, we can learn the optimal QG model to yield thought-provoking results. Evaluations on two typical datasets validate our method.
Anthology ID:
2025.coling-main.155
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2261–2276
Language:
URL:
https://aclanthology.org/2025.coling-main.155/
DOI:
Bibkey:
Cite (ACL):
Jianxing Yu, Shiqi Wang, Hanjiang Lai, Wenqing Chen, Yanghui Rao, Qinliang Su, and Jian Yin. 2025. Generating Commonsense Reasoning Questions with Controllable Complexity through Multi-step Structural Composition. In Proceedings of the 31st International Conference on Computational Linguistics, pages 2261–2276, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
Generating Commonsense Reasoning Questions with Controllable Complexity through Multi-step Structural Composition (Yu et al., COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.155.pdf