Fairness in Formative AI: Cognitive Complexity in Chatbot Questions Across Research Topics

Alexandra Barry Colbert, Karen D Wang


Abstract
This study evaluates whether questions generated from a socratic-style research AI chatbot designed to support project-based AP courses maintains cognitive complexity parity when inputted with research topics of controversial and non-controversial nature. We present empirical findings indicating no significant conversational complexity differences, highlighting implications for equitable AI use in formative assessment.
Anthology ID:
2025.aimecon-wip.12
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Month:
October
Year:
2025
Address:
Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
98–106
Language:
URL:
https://aclanthology.org/2025.aimecon-wip.12/
DOI:
Bibkey:
Cite (ACL):
Alexandra Barry Colbert and Karen D Wang. 2025. Fairness in Formative AI: Cognitive Complexity in Chatbot Questions Across Research Topics. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 98–106, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Fairness in Formative AI: Cognitive Complexity in Chatbot Questions Across Research Topics (Colbert & Wang, AIME-Con 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.aimecon-wip.12.pdf