Other people with similar names: Yang Lu, Yang Lu

Unverified author pages with similar names: Yang Lu


2026

Large language models (LLMs) are increasingly evaluated on legal reasoning tasks, but most benchmarks remain English-centered and test each legal problem in only one linguistic form. We examine whether LLMs preserve legal reasoning performance when the same legal question is expressed across English, formal Arabic, simplified Arabic, and Egyptian Arabic. We construct a controlled benchmark of 200 bar-exam-style multiple-choice questions, preserving the legal facts, answer choices, and correct answer across all language versions. We evaluate GPT-4o, Gemini 2.5 Pro, Grok, and Qwen against a shared human-verified answer key. All models achieve their highest accuracy in English, but Arabic performance varies substantially. Qwen scores 97.0% in English and 79.0% in each Arabic condition, while GPT-4o is the most stable model, with 90.0% in English and 80.0–82.0% across Arabic variants. Gemini 2.5 Pro drops from 80.5% in English to 17.5–21.0% in Arabic, and Grok drops from 45.0% to 16.5–26.5%. These results show that strong English legal accuracy does not guarantee robust Arabic legal reasoning. We argue that multilingual legal evaluation should measure not only accuracy, but also English–Arabic gaps and cross-language variation across semantically aligned legal questions.