Ateeb Ahmed
Author directory2026
Chain-of-Thought in a Low-Resource Language: A Factorial Study of Exemplar and Reasoning Language Effects in Urdu
Ateeb Ahmed
The Proceedings of the Workshop on Open Reasoning Across Cultures Languages (ORACLE)
Ateeb Ahmed
The Proceedings of the Workshop on Open Reasoning Across Cultures Languages (ORACLE)
Chain-of-thought (CoT) prompting has been shown theoretically to extend the computational capacity of transformers by providing intermediate reasoning steps (Feng et al., 2023; Amiri et al., 2025). Whether these benefits carry over to low-resource languages is not well understood, and Urdu in particular has received very little attention in this context. We investigate two prompt factors (the language of few-shot exemplars and the language of the reasoning chain) using a 2 × 2 factorial design on MGSM Urdu and Math-500 Urdu, evaluated with Gemini 2.5 Flash. We find a significant interaction between exemplar and CoT language (item-level logistic regression, z = 6.9, p < 0.001): switching to Urdu CoT with English exemplars causes a 34.8 percentage point accuracy drop on MGSM, while the same switch with Urdu exemplars results in only a 1.6 point drop. We also find that Urdu CoT produces longer token sequences than English CoT (139 vs. 101 tokens on average) but not correspondingly more discrete reasoning steps: a script-aware, manually validated step count shows Urdu CoT yields slightly fewer discrete steps (2.5 vs. 3.0 on average), with the reduction concentrated in the mismatchedexemplar condition, indicating that added verbosity does not translate into added reasoning structure. Chain length correlates positively with accuracy (r = 0.42, p < 0.001), consistent with the theoretical framework. The main takeaway is that exemplar language is a critical moderator of CoT effectiveness in Urdu.