KnowComp at SemEval-2024 Task 9: Conceptualization-Augmented Prompting with Large Language Models for Lateral Reasoning

Weiqi Wang, Baixuan Xu, Haochen Shi, Jiaxin Bai, Qi Hu, Yangqiu Song


Abstract
Lateral thinking is essential in breaking away from conventional thought patterns and finding innovative solutions to problems. Despite this, language models often struggle with reasoning tasks that require lateral thinking. In this paper, we present our system for SemEval-2024 Task 9’s BrainTeaser challenge, which requires language models to answer brain teaser questions that typically involve lateral reasoning scenarios. Our framework is based on large language models and incorporates a zero-shot prompting method that integrates conceptualizations of automatically detected instances in the question. We also transform the task of question answering into a declarative format to enhance the discriminatory ability of large language models. Our zero-shot evaluation results with ChatGPT indicate that our approach outperforms baselines, including zero-shot and few-shot prompting and chain-of-thought reasoning. Additionally, our system ranks ninth on the official leaderboard, demonstrating its strong performance.
Anthology ID:
2024.semeval-1.233
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1639–1645
Language:
URL:
https://aclanthology.org/2024.semeval-1.233
DOI:
10.18653/v1/2024.semeval-1.233
Bibkey:
Cite (ACL):
Weiqi Wang, Baixuan Xu, Haochen Shi, Jiaxin Bai, Qi Hu, and Yangqiu Song. 2024. KnowComp at SemEval-2024 Task 9: Conceptualization-Augmented Prompting with Large Language Models for Lateral Reasoning. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 1639–1645, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
KnowComp at SemEval-2024 Task 9: Conceptualization-Augmented Prompting with Large Language Models for Lateral Reasoning (Wang et al., SemEval 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.semeval-1.233.pdf
Supplementary material:
 2024.semeval-1.233.SupplementaryMaterial.txt