Long Cheng
Author directory2026
Syntax-Aware Detection of Training Data Contamination in Code LLMs
Jiazheng Zhang | Cheng Liu | Long Cheng | Xiaowei Li | Huawei Li
Findings of the Association for Computational Linguistics: EMNLP 2026
Jiazheng Zhang | Cheng Liu | Long Cheng | Xiaowei Li | Huawei Li
Findings of the Association for Computational Linguistics: EMNLP 2026
Despite rapid advances in Code LLMs, their training datasets are often opaque due to proprietary and competitive concerns. Since many evaluation benchmarks are open-source and widely available online, models may unintentionally train on them, leading to data contamination that compromises evaluation validity. Reliable techniques for detecting whether a code snippet appears in a model’s training data are therefore essential. Compared with natural language, code contamination detection presents two unique challenges. First, code tokens from different syntactic categories contribute unequally to contamination signals, making uniform token treatment ineffective. Second, models trained on semantically equivalent code variants can evade detectors based solely on text matching. To address these challenges, we propose a syntax-aware framework that explicitly models token-type contributions. With a lightweight supervised detector trained on few labels, it captures contamination signals efficiently. Experiments across 8 Code LLMs, 2 languages, and 4 code transformations show absolute AUC gains of 8–10% for pretraining data detection and 15–20% for fine-tuning data detection, with AUC exceeding 90% on most fine-tuned tasks. Our work demonstrates that syntax-aware modeling is essential for trustworthy Code LLM evaluation.
2024
TEII: Think, Explain, Interact and Iterate with Large Language Models to Solve Cross-lingual Emotion Detection
Long Cheng | Qihao Shao | Christine Zhao | Sheng Bi | Gina-Anne Levow
Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis
Long Cheng | Qihao Shao | Christine Zhao | Sheng Bi | Gina-Anne Levow
Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis
Cross-lingual emotion detection allows us to analyze global trends, public opinion, and social phenomena at scale. We participated in the Explainability of Cross-lingual Emotion Detection (EXALT) shared task, achieving an F1-score of 0.6046 on the evaluation set for the emotion detection sub-task. Our system outperformed the baseline by more than 0.16 F1-score absolute, and ranked second amongst competing systems. We conducted experiments using fine-tuning, zero-shot learning, and few-shot learning for Large Language Model (LLM)-based models as well as embedding-based BiLSTM and KNN for non-LLM-based techniques. Additionally, we introduced two novel methods: the Multi-Iteration Agentic Workflow and the Multi-Binary-Classifier Agentic Workflow. We found that LLM-based approaches provided good performance on multilingual emotion detection. Furthermore, ensembles combining all our experimented models yielded higher F1-scores than any single approach alone.
2022
Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model
Mingqi Li | Fei Ding | Dan Zhang | Long Cheng | Hongxin Hu | Feng Luo
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Mingqi Li | Fei Ding | Dan Zhang | Long Cheng | Hongxin Hu | Feng Luo
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Pre-trained multilingual language models play an important role in cross-lingual natural language understanding tasks. However, existing methods did not focus on learning the semantic structure of representation, and thus could not optimize their performance. In this paper, we propose Multi-level Multilingual Knowledge Distillation (MMKD), a novel method for improving multilingual language models. Specifically, we employ a teacher-student framework to adopt rich semantic representation knowledge in English BERT. We propose token-, word-, sentence-, and structure-level alignment objectives to encourage multiple levels of consistency between source-target pairs and correlation similarity between teacher and student models. We conduct experiments on cross-lingual evaluation benchmarks including XNLI, PAWS-X, and XQuAD. Experimental results show that MMKD outperforms other baseline models of similar size on XNLI and XQuAD and obtains comparable performance on PAWS-X. Especially, MMKD obtains significant performance gains on low-resource languages.