InterviewBench: Benchmarking Large Language Models as Interviewers

Shangzhe Dai, Feiyu Duan, Zhongyu Wei


Abstract
Interviews collect information through conversation, requiring interviewers to ask appropriate questions, adapt to respondent behavior, and recover structured information from dialogue. Yet, AI systems’ abilities as interviewers remain under-explored. We present InterviewBench, a benchmark for evaluating LLM-based AI interviewers through both local interviewer decisions and dynamic multi-turn interviews. The static setting abstracts real interviews into controlled local-decision instances, while the dynamic setting evaluates end-to-end interviewing through long-horizon conversational simulations. Built from real interview transcripts, InterviewBench contains 5,500 classification questions, 3,000 generative questions, and 184 dynamic scenarios. We further model interviewer evaluation as contextual decision selection, where the model must choose or generate the next interviewer action conditioned on dialogue context. Evaluating 23 open-source and proprietary LLMs, we find that current models can produce fluent interviewer utterances, but still struggle with strategy selection, explicit evidence elicitation, long-horizon interaction management, and recovery from non-ideal interviewee behavior.
Anthology ID:
2026.findings-emnlp.2141
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
45731–45787
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2141/
DOI:
Bibkey:
Cite (ACL):
Shangzhe Dai, Feiyu Duan, and Zhongyu Wei. 2026. InterviewBench: Benchmarking Large Language Models as Interviewers. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 45731–45787, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
InterviewBench: Benchmarking Large Language Models as Interviewers (Dai et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2141.pdf