MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations

Bryan Christ, Jonathan Kropko, Thomas Hartvigsen


Abstract
Math word problems are critical K-8 educational tools, but writing them is time consuming and requires extensive expertise. To be educational, problems must be solvable, have accurate answers, and, most importantly, be educationally appropriate. We propose that language models have potential to support K-8 math education by automatically generating word problems. However, evaluating educational appropriateness is hard to quantify. We fill this gap by having teachers evaluate problems generated by LLMs, who find existing models and data often fail to be educationally appropriate. We then explore automatically generating *educational* word problems, ultimately using our expert annotations to finetune a 70B language model. Our model, MATHWELL, is the first K-8 word problem generator targeted at educational appropriateness. Further expert studies find MATHWELL generates problems far more solvable, accurate, and appropriate than public models. MATHWELL also matches GPT-4’s problem quality while attaining more appropriate reading levels for K-8 students and avoiding generating harmful questions.
Anthology ID:
2024.findings-emnlp.696
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11914–11938
Language:
URL:
https://aclanthology.org/2024.findings-emnlp.696
DOI:
Bibkey:
Cite (ACL):
Bryan Christ, Jonathan Kropko, and Thomas Hartvigsen. 2024. MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11914–11938, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations (Christ et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-emnlp.696.pdf