AREAs-Lab: An Interactive Environment for AI-driven Requirement Elicitation for AI Systems

Pengshan Cai, Zihao Zhang, Ting Jin, Chenyang Zhu, Kushal Chawla, Sangwoo Cho, Scott Novotney, Yebowen Hu, Fei Liu, Shi-Xiong Zhang, Sambit Sahu


Abstract
Building effective AI systems increasingly depends on writing high-quality task requirements, yet users often struggle to articulate the constraints, preferences, and edge cases that determine success. This problem is especially acute in AI development, where behavior is shaped not only by human expectations but also by data characteristics. We present AREAs-Lab, an interactive environment for AI-driven Requirement Elicitation for AI systems. In AREAs-Lab, an assistant iteratively refines an initially incomplete requirement by analyzing the underlying dataset and asking targeted clarification questions to uncover the user’s latent intent. To study this setting systematically, we construct a synthetic benchmark grounded in 16 public datasets spanning diverse domains and task types. Each benchmark instance includes a user profile, a complete reference requirement, and an intentionally underspecified version that serves as the assistant’s starting point. We further introduce an automated evaluation pipeline based on an AI-simulated user that reveals hidden information only when appropriately prompted, enabling scalable and reproducible assessment of interactive elicitation quality. AREAs-Lab provides a controlled testbed for studying how AI assistants can transform vague user goals into actionable requirements for AI systems.
Anthology ID:
2026.findings-emnlp.2111
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
44987–45031
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2111/
DOI:
Bibkey:
Cite (ACL):
Pengshan Cai, Zihao Zhang, Ting Jin, Chenyang Zhu, Kushal Chawla, Sangwoo Cho, Scott Novotney, Yebowen Hu, Fei Liu, Shi-Xiong Zhang, and Sambit Sahu. 2026. AREAs-Lab: An Interactive Environment for AI-driven Requirement Elicitation for AI Systems. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 44987–45031, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
AREAs-Lab: An Interactive Environment for AI-driven Requirement Elicitation for AI Systems (Cai et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2111.pdf