Evaluating Deep Learning and Transformer Models on SME and GenAI Items

Joe Betts, William Muntean


Abstract
This study leverages deep learning, transformer models, and generative AI to streamline test development by automating metadata tagging and item generation. Transformer models outperform simpler approaches, reducing SME workload. Ongoing research refines complex models and evaluates LLM-generated items, enhancing efficiency in test creation.
Anthology ID:
2025.aimecon-wip.17
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Month:
October
Year:
2025
Address:
Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
141–146
Language:
URL:
https://aclanthology.org/2025.aimecon-wip.17/
DOI:
Bibkey:
Cite (ACL):
Joe Betts and William Muntean. 2025. Evaluating Deep Learning and Transformer Models on SME and GenAI Items. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 141–146, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Evaluating Deep Learning and Transformer Models on SME and GenAI Items (Betts & Muntean, AIME-Con 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.aimecon-wip.17.pdf