RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation

Ioannis Panagiotopoulos, George Filandrianos, Maria Lymperaiou, Giorgos Stamou


Abstract
Riddle-solving requires advanced reasoning skills, pushing Large Language Models (LLMs) to engage in abstract thinking and creative problem-solving, often revealing limitations in their cognitive abilities. In this paper, we examine the riddle-solving capabilities of LLMs using a multiple-choice format, exploring how different prompting techniques impact performance on riddles that demand diverse reasoning skills. To enhance results, we introduce RISCORE (RIddle Solving with COntext REcontruciton) a novel fully automated prompting method that generates and utilizes contextually reconstructed sentence-based puzzles in conjunction with the original examples to create few-shot exemplars. Our experiments demonstrate that RISCORE significantly improves the performance of language models in both vertical and lateral thinking tasks, surpassing traditional exemplar selection strategies across a variety of few-shot settings.
Anthology ID:
2025.coling-main.633
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9431–9455
Language:
URL:
https://aclanthology.org/2025.coling-main.633/
DOI:
Bibkey:
Cite (ACL):
Ioannis Panagiotopoulos, George Filandrianos, Maria Lymperaiou, and Giorgos Stamou. 2025. RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9431–9455, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation (Panagiotopoulos et al., COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.633.pdf