Abdelhak at SemEval-2024 Task 9: Decoding Brainteasers, The Efficacy of Dedicated Models Versus ChatGPT

Abdelhak Kelious, Mounir Okirim


Abstract
This study introduces a dedicated model aimed at solving the BRAINTEASER task 9 , a novel challenge designed to assess models’ lateral thinking capabilities through sentence and word puzzles. Our model demonstrates remarkable efficacy, securing Rank 1 in sentence puzzle solving during the test phase with an overall score of 0.98. Additionally, we explore the comparative performance of ChatGPT, specifically analyzing how variations in temperature settings affect its ability to engage in lateral thinking and problem-solving. Our findings indicate a notable performance disparity between the dedicated model and ChatGPT, underscoring the potential of specialized approaches in enhancing creative reasoning in AI.
Anthology ID:
2024.semeval-1.31
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
200–205
Language:
URL:
https://aclanthology.org/2024.semeval-1.31
DOI:
10.18653/v1/2024.semeval-1.31
Bibkey:
Cite (ACL):
Abdelhak Kelious and Mounir Okirim. 2024. Abdelhak at SemEval-2024 Task 9: Decoding Brainteasers, The Efficacy of Dedicated Models Versus ChatGPT. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 200–205, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Abdelhak at SemEval-2024 Task 9: Decoding Brainteasers, The Efficacy of Dedicated Models Versus ChatGPT (Kelious & Okirim, SemEval 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.semeval-1.31.pdf
Supplementary material:
 2024.semeval-1.31.SupplementaryMaterial.txt