Ravinithesh Annapureddy

Author directory

2026

LLMs increasingly serve as general-purpose information systems, including for factual queries about public statistics. In high-stakes domains such as migration, where data accuracy shapes both policy and public attitudes toward at-risk populations, models may answer queries about social groups unequally. We evaluate this concern empirically using GPT-4 Turbo, GPT-4o, GPT-5 chat, and GPT-5.1 on migration-related education statistics. Each model receives zero-shot queries on educational-attainment distributions for national and foreign citizens aged 25–54 across 34 European countries from 2020–2023. We compare outputs against official Eurostat data using RMSE, total variation distance (TVD), cosine similarity, distributional analysis, and response-coverage rates. Across models, results show a consistent gap between plausibility and accuracy: all models preserve broad distributional structure (cosine similarity > 0.93), while deviating substantially from ground-truth values (RMSE 10–14 and TVD 12–16 percentage points). We characterize this combination of structural plausibility and numerical error as an output-level analog of the illusion of competence. GPT-4 models also produce explicit non-answers for 30–78% of responses, with refusals (explicit non-answers) up to 3.4 times more frequent for foreigners than for nationals. GPT-5 models eliminate refusals but return hallucinated values. These findings show that pluralistic evaluation must distinguish legitimate human variation from unsupported model variation and assess whether factual information remains comparably available and reliable across social groups.