Tamara Sumner


2022

pdf bib
The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts Annotated for Teacher and Student Discursive Moves
Abhijit Suresh | Jennifer Jacobs | Charis Harty | Margaret Perkoff | James H. Martin | Tamara Sumner
Proceedings of the Thirteenth Language Resources and Evaluation Conference

Transcripts of teaching episodes can be effective tools to understand discourse patterns in classroom instruction. According to most educational experts, sustained classroom discourse is a critical component of equitable, engaging, and rich learning environments for students. This paper describes the TalkMoves dataset, composed of 567 human-annotated K-12 mathematics lesson transcripts (including entire lessons or portions of lessons) derived from video recordings. The set of transcripts primarily includes in-person lessons with whole-class discussions and/or small group work, as well as some online lessons. All of the transcripts are human-transcribed, segmented by the speaker (teacher or student), and annotated at the sentence level for ten discursive moves based on accountable talk theory. In addition, the transcripts include utterance-level information in the form of dialogue act labels based on the Switchboard Dialog Act Corpus. The dataset can be used by educators, policymakers, and researchers to understand the nature of teacher and student discourse in K-12 math classrooms. Portions of this dataset have been used to develop the TalkMoves application, which provides teachers with automated, immediate, and actionable feedback about their mathematics instruction.

pdf bib
Fine-tuning Transformers with Additional Context to Classify Discursive Moves in Mathematics Classrooms
Abhijit Suresh | Jennifer Jacobs | Margaret Perkoff | James H. Martin | Tamara Sumner
Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022)

“Talk moves” are specific discursive strategies used by teachers and students to facilitate conversations in which students share their thinking, and actively consider the ideas of others, and engage in rich discussions. Experts in instructional practices often rely on cues to identify and document these strategies, for example by annotating classroom transcripts. Prior efforts to develop automated systems to classify teacher talk moves using transformers achieved a performance of 76.32% F1. In this paper, we investigate the feasibility of using enriched contextual cues to improve model performance. We applied state-of-the-art deep learning approaches for Natural Language Processing (NLP), including Robustly optimized bidirectional encoder representations from transformers (Roberta) with a special input representation that supports previous and subsequent utterances as context for talk moves classification. We worked with the publically available TalkMoves dataset, which contains utterances sourced from real-world classroom sessions (human- transcribed and annotated). Through a series of experimentations, we found that a combination of previous and subsequent utterances improved the transformers’ ability to differentiate talk moves (by 2.6% F1). These results constitute a new state of the art over previously published results and provide actionable insights to those in the broader NLP community who are working to develop similar transformer-based classification models.

2016

pdf bib
Bayesian Supervised Domain Adaptation for Short Text Similarity
Md Arafat Sultan | Jordan Boyd-Graber | Tamara Sumner
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
Fast and Easy Short Answer Grading with High Accuracy
Md Arafat Sultan | Cristobal Salazar | Tamara Sumner
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
DLS@CU at SemEval-2016 Task 1: Supervised Models of Sentence Similarity
Md Arafat Sultan | Steven Bethard | Tamara Sumner
Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)

2015

pdf bib
Feature-Rich Two-Stage Logistic Regression for Monolingual Alignment
Md Arafat Sultan | Steven Bethard | Tamara Sumner
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

pdf bib
SGRank: Combining Statistical and Graphical Methods to Improve the State of the Art in Unsupervised Keyphrase Extraction
Soheil Danesh | Tamara Sumner | James H. Martin
Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics

pdf bib
DLS@CU: Sentence Similarity from Word Alignment and Semantic Vector Composition
Md Arafat Sultan | Steven Bethard | Tamara Sumner
Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)

2014

pdf bib
DLS@CU: Sentence Similarity from Word Alignment
Md Arafat Sultan | Steven Bethard | Tamara Sumner
Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)

pdf bib
Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual Evidence
Md Arafat Sultan | Steven Bethard | Tamara Sumner
Transactions of the Association for Computational Linguistics, Volume 2

We present a simple, easy-to-replicate monolingual aligner that demonstrates state-of-the-art performance while relying on almost no supervision and a very small number of external resources. Based on the hypothesis that words with similar meanings represent potential pairs for alignment if located in similar contexts, we propose a system that operates by finding such pairs. In two intrinsic evaluations on alignment test data, our system achieves F1 scores of 88–92%, demonstrating 1–3% absolute improvement over the previous best system. Moreover, in two extrinsic evaluations our aligner outperforms existing aligners, and even a naive application of the aligner approaches state-of-the-art performance in each extrinsic task.

2013

pdf bib
DLS@CU-CORE: A Simple Machine Learning Model of Semantic Textual Similarity
Md. Sultan | Steven Bethard | Tamara Sumner
Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity

pdf bib
CU : Computational Assessment of Short Free Text Answers - A Tool for Evaluating Students’ Understanding
Ifeyinwa Okoye | Steven Bethard | Tamara Sumner
Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013)

2012

pdf bib
Identifying science concepts and student misconceptions in an interactive essay writing tutor
Steven Bethard | Ifeyinwa Okoye | Md. Arafat Sultan | Haojie Hang | James H. Martin | Tamara Sumner
Proceedings of the Seventh Workshop on Building Educational Applications Using NLP

2008

pdf bib
Extractive Summaries for Educational Science Content
Sebastian de la Chica | Faisal Ahmad | James H. Martin | Tamara Sumner
Proceedings of ACL-08: HLT, Short Papers

pdf bib
Pedagogically Useful Extractive Summaries for Science Education
Sebastian de la Chica | Faisal Ahmad | James H. Martin | Tamara Sumner
Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008)