Daniel Gatica-Perez
Author directory2026
Plausible but Wrong: A Statistical Evaluation of GPT Models on High-Stakes Migration Statistics
Hector M. Ramírez Contreras | Ravinithesh Annapureddy | Daniel Gatica-Perez
Proceedings of the First Workshop on Pluralistic AI NLP: Diversity-aware, Sociotechnical, Responsible Alignment (PANDORA 2026)
Hector M. Ramírez Contreras | Ravinithesh Annapureddy | Daniel Gatica-Perez
Proceedings of the First Workshop on Pluralistic AI NLP: Diversity-aware, Sociotechnical, Responsible Alignment (PANDORA 2026)
LLMs increasingly serve as general-purpose information systems, including for factual queries about public statistics. In high-stakes domains such as migration, where data accuracy shapes both policy and public attitudes toward at-risk populations, models may answer queries about social groups unequally. We evaluate this concern empirically using GPT-4 Turbo, GPT-4o, GPT-5 chat, and GPT-5.1 on migration-related education statistics. Each model receives zero-shot queries on educational-attainment distributions for national and foreign citizens aged 25–54 across 34 European countries from 2020–2023. We compare outputs against official Eurostat data using RMSE, total variation distance (TVD), cosine similarity, distributional analysis, and response-coverage rates. Across models, results show a consistent gap between plausibility and accuracy: all models preserve broad distributional structure (cosine similarity > 0.93), while deviating substantially from ground-truth values (RMSE 10–14 and TVD 12–16 percentage points). We characterize this combination of structural plausibility and numerical error as an output-level analog of the illusion of competence. GPT-4 models also produce explicit non-answers for 30–78% of responses, with refusals (explicit non-answers) up to 3.4 times more frequent for foreigners than for nationals. GPT-5 models eliminate refusals but return hallucinated values. These findings show that pluralistic evaluation must distinguish legitimate human variation from unsupported model variation and assess whether factual information remains comparably available and reliable across social groups.
2018
Words Worth: Verbal Content and Hirability Impressions in YouTube Video Resumes
Skanda Muralidhar | Laurent Nguyen | Daniel Gatica-Perez
Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis
Skanda Muralidhar | Laurent Nguyen | Daniel Gatica-Perez
Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis
Automatic hirability prediction from video resumes is gaining increasing attention in both psychology and computing. Most existing works have investigated hirability from the perspective of nonverbal behavior, with verbal content receiving little interest. In this study, we leverage the advances in deep-learning based text representation techniques (like word embedding) in natural language processing to investigate the relationship between verbal content and perceived hirability ratings. To this end, we use 292 conversational video resumes from YouTube, develop a computational framework to automatically extract various representations of verbal content, and evaluate them in a regression task. We obtain a best performance of R² = 0.23 using GloVe, and R² = 0.22 using Word2Vec representations for manual and automatically transcribed texts respectively. Our inference results indicate the feasibility of using deep learning based verbal content representation in inferring hirability scores from online conversational video resumes.