Physical Commonsense Reasoning for Lower-Resourced Languages and Dialects: A Study on Basque

Jaione Bengoetxea, Itziar Gonzalez-Dios, Rodrigo Agerri


Abstract
Physical commonsense reasoning represents a fundamental capability of human intelligence, enabling individuals to understand their environment, predict future events, and navigate physical spaces. Recent years have witnessed growing interest in reasoning tasks within Natural Language Processing (NLP). However, no prior research has examined the performance of Large Language Models (LLMs) on non-question-answering (non-QA) physical commonsense reasoning tasks in low-resource languages such as Basque. Taking the Italian GITA as a starting point, this paper addresses this gap by presenting BasPhyCo, the first non-QA physical commonsense reasoning dataset for Basque, available in both standard and dialectal variants. We evaluate model performance across three hierarchical levels of commonsense understanding: (1) distinguishing between plausible and implausible narratives (accuracy), (2) identifying the conflicting element that renders a narrative implausible (consistency), and (3) determining the specific physical state that creates the implausibility (verifiability). These tasks were assessed using multiple multilingual LLMs as well as models pretrained specifically for Italian and Basque. Results indicate that, in terms of verifiability, LLMs exhibit limited physical commonsense capabilities in low-resource languages such as Basque, especially when processing dialectal variants.
Anthology ID:
2026.lrec-1.264
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
3343–3354
Language:
External URL:
https://lrec.elra.info/lrec2026-main-264
DOI:
10.63317/3p6tfvjmet58
Bibkey:
Cite (ACL):
Jaione Bengoetxea, Itziar Gonzalez-Dios, and Rodrigo Agerri. 2026. Physical Commonsense Reasoning for Lower-Resourced Languages and Dialects: A Study on Basque. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 3343–3354, Palma de Mallorca, Spain. ELRA Language Resource Association.
Cite (Informal):
Physical Commonsense Reasoning for Lower-Resourced Languages and Dialects: A Study on Basque (Bengoetxea et al., LREC 2026)
Copy Citation: