Proceedings of the Workshop on Evaluating Evaluations (EvalEval)

Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor 2026-07 Association for Computational Linguistics San Diego, CA text conference publication 979-8-89176-429-3 evaleval-2026-1 https://aclanthology.org/2026.evaleval-1.0/ Rigorous Interpretation Is a Form of Evaluation Isabelle Lee author Emmy Liu author Cathy Jiao author Brihi Joshi author Dani Yogatama author Fazl Barez author Michael Saxon author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 lee-etal-2026-rigorous https://aclanthology.org/2026.evaleval-1.1/ 2026-07 1 11 Evaluating Multi-turn Human-AI Interaction Shi Ding author Sijian Tan author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 ding-tan-2026-evaluating https://aclanthology.org/2026.evaleval-1.2/ 2026-07 12 18 Guidelines for Whom? Rethinking AI Ethics in Resource-Constrained Migration Services Nari Yoo author Ashley Khor author Namrata Mukhija author Aminat Adebiyi author Miri Zilka author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 yoo-etal-2026-guidelines https://aclanthology.org/2026.evaleval-1.3/ 2026-07 19 25 Evaluating Large Language Model News Sentiment in Finance under Liquidity and Market Frictions Kemal Kirtac author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 kirtac-2026-evaluating https://aclanthology.org/2026.evaleval-1.4/ 2026-07 26 35 From Wordle to Fibble⁵: Evaluating LLM Reasoning Under Escalating Deception Chang Liu author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 liu-2026-wordle https://aclanthology.org/2026.evaleval-1.5/ 2026-07 36 45 Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models Pranav Mahajan author Ihor Kendiukhov author Syed Hussain author Lydia Nottingham author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 mahajan-etal-2026-mind https://aclanthology.org/2026.evaleval-1.9/ 2026-07 46 55 When Scanners Lie: Evaluator Instability in LLM Red-Teaming Lidor Erez author Omer Hofman author Tamir Nizri author Roman Vainshtein author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 erez-etal-2026-scanners https://aclanthology.org/2026.evaleval-1.11/ 2026-07 56 69 Reasoning Model Is Superior LLM-Judge, Yet Suffers from Biases Hui Huang author Xuanxin Wu author Muyun Yang author Yuki Arase author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 huang-etal-2026-reasoning https://aclanthology.org/2026.evaleval-1.13/ 2026-07 70 81 From Rubrics to Recipe: Principle-Centric Benchmark for Evaluating Large Language Models Shirley Anugrah Hayati author Ruizi Wang author Dongyeop Kang author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 hayati-etal-2026-rubrics https://aclanthology.org/2026.evaleval-1.15/ 2026-07 82 99 Too long; didn’t solve Lucía Cabrera author Jocelyn D’Arcy author Isaac Saxton-Knight author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 cabrera-etal-2026-long https://aclanthology.org/2026.evaleval-1.20/ 2026-07 100 110 Graduating the Benchmark Scale: Lessons from Thermometry Sean Trott author Oisín Parkinson-Coombs author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 trott-parkinson-coombs-2026-graduating https://aclanthology.org/2026.evaleval-1.21/ 2026-07 111 115 Caged Birds and Cute Bookworms: Feminine Tropes and Implicit Gender Bias in Large Language Models Sachita Nishal author Jack Bandy author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 nishal-bandy-2026-caged https://aclanthology.org/2026.evaleval-1.22/ 2026-07 116 127 Scorecard of AI Benchmark Quality Ayrton San Joaquin author Rokas Gipiškis author Ze Shen Chin author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 san-joaquin-etal-2026-scorecard https://aclanthology.org/2026.evaleval-1.25/ 2026-07 128 160 Defining Cultural Capabilities for AI Evaluation: A Taxonomy Grounded in Intercultural Communication Theory Isar Nejadgholi author Masoud Kianpour author Krishnapriya Vishnubhotla author Maryam Molamohammadi author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 nejadgholi-etal-2026-defining https://aclanthology.org/2026.evaleval-1.26/ 2026-07 161 173 BenchNavigator: A Discovery Interface for Comparing LLM Benchmarks Anna Sokol author Inge Vejsbjerg author Elizabeth M Daly author David Piorkowski author Michael Hind author Nuno Moniz author Nitesh V Chawla author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 sokol-etal-2026-benchnavigator https://aclanthology.org/2026.evaleval-1.29/ 2026-07 174 200 Beyond Static Benchmarks: A Validity, Reliability, and Sociotechnical Framework for Evaluating LLMs in Deployment Contexts Ben Jenkins author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 jenkins-2026-beyond https://aclanthology.org/2026.evaleval-1.30/ 2026-07 201 210 From Guidelines to Guarantees: A Graph-Based Evaluation Harness for Domain-Specific Evaluation of LLMs Jessica M Lundin author Usman Nasir Nakakana author Guillaume Chabot-Couture author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 lundin-etal-2026-guidelines https://aclanthology.org/2026.evaleval-1.34/ 2026-07 211 220 Document Overlap Is Not Evidence Continuity: Measuring Retrieval Jitter in Citation-Based RAG Evaluation Punitha Ponnuraj author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 ponnuraj-2026-document https://aclanthology.org/2026.evaleval-1.35/ 2026-07 221 226 Measuring AI-Induced Disempowerment: A Framework and Proposed Metrics Je Qin Chooi author Jaeho Lee author Jasmine Xinze Li author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 chooi-etal-2026-measuring https://aclanthology.org/2026.evaleval-1.36/ 2026-07 227 236 Position: Evaluations of AI Moral Reasoning Still Miss Half of the Picture Aidan Kierans author Ritam Dutt author Kaley Rittichier author Shiri Dori-Hacohen author Avijit Ghosh author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 kierans-etal-2026-position https://aclanthology.org/2026.evaleval-1.38/ 2026-07 237 244 Evaluation Cards for XAI Metrics Rokas Gipiškis author Olga Kurasova author 2026-07 text Proceedings of the Workshop on Evaluating Evaluations (EvalEval) Mubashara Akhtar editor Jan Batzner editor Leshem Choshen editor Avijit Ghosh editor Usman Gohar editor Jennifer Mickel editor Ichhya Pant editor Zeerak Talat editor Michelle Lin editor Association for Computational Linguistics San Diego, CA conference publication 979-8-89176-429-3 gipiskis-kurasova-2026-evaluation https://aclanthology.org/2026.evaleval-1.39/ 2026-07 245 251