Jiazheng Zhang

Author directory

Other people with similar names: Jiazheng Zhang

Unverified author pages with similar names: Jiazheng Zhang


2026

Despite rapid advances in Code LLMs, their training datasets are often opaque due to proprietary and competitive concerns. Since many evaluation benchmarks are open-source and widely available online, models may unintentionally train on them, leading to data contamination that compromises evaluation validity. Reliable techniques for detecting whether a code snippet appears in a model’s training data are therefore essential. Compared with natural language, code contamination detection presents two unique challenges. First, code tokens from different syntactic categories contribute unequally to contamination signals, making uniform token treatment ineffective. Second, models trained on semantically equivalent code variants can evade detectors based solely on text matching. To address these challenges, we propose a syntax-aware framework that explicitly models token-type contributions. With a lightweight supervised detector trained on few labels, it captures contamination signals efficiently. Experiments across 8 Code LLMs, 2 languages, and 4 code transformations show absolute AUC gains of 8–10% for pretraining data detection and 15–20% for fine-tuning data detection, with AUC exceeding 90% on most fine-tuned tasks. Our work demonstrates that syntax-aware modeling is essential for trustworthy Code LLM evaluation.