Keqian Li
Author directory2026
Beyond Task-Based Evaluation: Toward Situated Benchmarks for Educational LLMs
Xiaolongwang | Tianyi Xu | Rui Zou | Jiawen Zuo | Mingzi Zhang | Keqian Li | Zhe Zhao | Ye Wei | Aimin Zhou
Findings of the Association for Computational Linguistics: EMNLP 2026
Xiaolongwang | Tianyi Xu | Rui Zou | Jiawen Zuo | Mingzi Zhang | Keqian Li | Zhe Zhao | Ye Wei | Aimin Zhou
Findings of the Association for Computational Linguistics: EMNLP 2026
This paper argues that evaluation of educational LLMs should move beyond decontextualized task-based benchmarks toward situated benchmarks centered on pedagogical action. Prior work in human-computer interaction and the learning sciences has long emphasized situated use, user context, and interaction processes; our focus is narrower and complementary: how these concerns should reshape benchmark design for educational LLMs. Existing evaluations already cover question generation, knowledge answering, error diagnosis, feedback generation, and teaching analysis, but their dominant unit remains the single input-output episode, making it difficult to compare whether a model genuinely supports learning progress, instructional judgment, or subsequent revision in context. Educational activity is processual, context-dependent, and development-oriented, so explanations, feedback, and hints can only be evaluated properly in relation to learner state, interaction history, and pedagogical goals. We review a landscape still dominated by task-based paradigms and argue, from educational conceptions, educational values, and research methodology, that next-generation educational benchmarks should make pedagogical function in context a core target. We further propose layered, scalable, and benchmarkable situated protocols built through simulation-first development and follow-up empirical calibration, extending evaluation beyond task quality to contextual fit and downstream consequences for learning and teaching decisions.
2020
TNT: Text Normalization based Pre-training of Transformers for Content Moderation
Fei Tan | Yifan Hu | Changwei Hu | Keqian Li | Kevin Yen
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Fei Tan | Yifan Hu | Changwei Hu | Keqian Li | Kevin Yen
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
In this work, we present a new language pre-training model TNT (Text Normalization based pre-training of Transformers) for content moderation. Inspired by the masking strategy and text normalization, TNT is developed to learn language representation by training transformers to reconstruct text from four operation types typically seen in text manipulation: substitution, transposition, deletion, and insertion. Furthermore, the normalization involves the prediction of both operation types and token labels, enabling TNT to learn from more challenging tasks than the standard task of masked word recovery. As a result, the experiments demonstrate that TNT outperforms strong baselines on the hate speech classification task. Additional text normalization experiments and case studies show that TNT is a new potential approach to misspelling correction.