Ahmad M Mustafa
Author directory2026
Optimizing Inference Cost for Arabic–English Mathematical Reasoning in LLMs
Sahar Melhem | Ahmad M Mustafa
Proceedings of The Fourth Arabic Natural Language Processing Conference
Sahar Melhem | Ahmad M Mustafa
Proceedings of The Fourth Arabic Natural Language Processing Conference
Large language models (LLMs) can solve mathematical problems through prompting, yet not all reasoning is cost-justified when using paid APIs. We study prompt-only methods for overthinking control in the black-box setting across four datasets: Arabic GSM8K, English GSM8K, English SVAMP, and ArMATH, each evaluated on 100-question subsets divided into Easy, Medium, and Hard categories. We test Direct Answer, Chain-of-Thought (CoT), First Answer Anchoring (FAA-lite), Confidence-Triggered Escalation (CTE), Self-Consistency, Verification, Arabic prompt variations, and Language-Optimal Prompt Routing (LOPR). CTE achieves 90% accuracy on Arabic GSM8K at 1.14 calls/query, 96% on English GSM8K at 1.06 calls/query, and 98% on SVAMP at 1.08 calls/query. Self-Consistency leads on ArMATH at 90% but costs 2.00 calls/query. Verification consistently increases token cost without accuracy gains. Format-Controlled CoT is the best single-call Arabic method; Bilingual-Glossary CoT is consistently harmful. CTE calibration reveals that explicit low-confidence outputs are rare (<1%), so CTE succeeds mainly because high-confidence predictions are accurate, not through active uncertainty detection.