Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home

Eda Okur, Roddy Fuentes Alba, Saurav Sahay, Lama Nachman


Abstract
Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we implement a multimodal dialogue system to support play-based learning experiences at home, guiding kids to master basic math concepts. This work explores Spoken Language Understanding (SLU) pipeline within a task-oriented dialogue system developed for Kid Space, with cascading Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) components evaluated on our home deployment data with kids going through gamified math learning activities. We validate the advantages of a multi-task architecture for NLU and experiment with a diverse set of pretrained language representations for Intent Recognition and Entity Extraction tasks in the math learning domain. To recognize kids’ speech in realistic home environments, we investigate several ASR systems, including the commercial Google Cloud and the latest open-source Whisper solutions with varying model sizes. We evaluate the SLU pipeline by testing our best-performing NLU models on noisy ASR output to inspect the challenges of understanding children for math learning in authentic homes.
Anthology ID:
2023.bea-1.56
Volume:
Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Ekaterina Kochmar, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Nitin Madnani, Anaïs Tack, Victoria Yaneva, Zheng Yuan, Torsten Zesch
Venue:
BEA
SIG:
SIGEDU
Publisher:
Association for Computational Linguistics
Note:
Pages:
692–708
Language:
URL:
https://aclanthology.org/2023.bea-1.56
DOI:
10.18653/v1/2023.bea-1.56
Bibkey:
Cite (ACL):
Eda Okur, Roddy Fuentes Alba, Saurav Sahay, and Lama Nachman. 2023. Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 692–708, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home (Okur et al., BEA 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.bea-1.56.pdf