BenCSSmark: Making the Social Sciences Count in LLM Research

Arnault Chatelain, Etienne Ollion, Qianwen Guan, Diandra Fabre, Lorraine Goeuriot, Emile Chapuis, Abdelkrim Beloued, Marie Candito, Nicolas Hervé, Didier Schwab

Abstract
This position paper argues that the under-representation of social science tasks in contemporary LLM benchmarks limits advances in both LLM evaluation and social scientific inquiry. Benchmarks — standardized tools for assessing computational systems — are pivotal in the development of artificial intelligence (AI), including large language models (LLMs). Benchmarks do more than measure progress — they actively structure it, shaping reputations, research agendas, and commercial outcomes. Despite this central role, the social sciences are largely absent from mainstream evaluation frameworks, even though scholars in these fields generate dozens of rigorously annotated, context-sensitive datasets each year. Integrating this work into benchmark design could significantly improve the generalization and robustness of AI models. In turn, models trained on social scientific tasks would likely yield better performance on classic and contemporary tasks in disciplines as diverse as history, sociology, political science or economics. This is all the more pressing as these disciplines are quickly turning to LLMs for assistance. To address this gap, we introduce BenCSSmark, a benchmark composed of datasets annotated by computational social scientists. By integrating social scientific perspectives into benchmarking, BenCSSmark seeks to promote more robust, transparent, and socially relevant AI systems and to foster efficient collaboration.
Anthology ID:
2026.lrec-1.64
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
848–859
Language:
External URL:
https://lrec.elra.info/lrec2026-main-064
DOI:
10.63317/3cd2b8m83z28
Bibkey:
Cite (ACL):
Arnault Chatelain, Etienne Ollion, Qianwen Guan, Diandra Fabre, Lorraine Goeuriot, Emile Chapuis, Abdelkrim Beloued, Marie Candito, Nicolas Hervé, and Didier Schwab. 2026. BenCSSmark: Making the Social Sciences Count in LLM Research. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 848–859, Palma de Mallorca, Spain. ELRA Language Resource Association.
Cite (Informal):
BenCSSmark: Making the Social Sciences Count in LLM Research (Chatelain et al., LREC 2026)
Copy Citation:
External Cite Search Fix data