Zhanyu Chen
Author directory2026
Sylloscope at SemEval-2026 Task 11: Decoupling Logic from Belief via DeepSeek-Enhanced Distillation in Qwen Models
Zhanyu Chen | María Teresa Muñoz Martín | Sem Huisman | Jingjing Lan
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Zhanyu Chen | María Teresa Muñoz Martín | Sem Huisman | Jingjing Lan
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
This paper presents our approach for SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models. We propose a neuro-symbolic teacher-student framework that utilizes DeepSeek-R1 as a Logical Auditor to generate a high-fidelity training corpus. We distill these analytical behaviors into Qwen-3 models using Low Rank Adaptation (LoRA), focusing on teaching the mechanics of logic rather than simple label matching. Our system yields robust results across both subtasks, with a ranking score of 39.81 (96.86% accuracy) on Subtask 1 and 26.02 on Subtask 3. However, validity bias partially persists, so we conclude that while structured distillation substantially mitigates belief bias, fully disentangling logical validity from plausibility remains a central challenge for future development.
Assessing the Impact of Language Disparity on Multilingual Linguistic Ability in Large Language Models
Zhanyu Chen | Jaap Jumelet
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zhanyu Chen | Jaap Jumelet
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Claims about the grammatical competence of multilingual language models vary sharply with how competence is measured, yet the interaction between evaluation paradigm, post-training, and language resource availability has not been systematically examined. We evaluate base and post-trained models from six families on MultiBLiMP, a syntactic minimal-pair benchmark covering 101 languages, using four evaluation methods. We report three principal findings. First, post-training degrades grammatical competence, but the magnitude of this effect is determined by post-training recipe rather than model scale, and scaling reduces it unevenly, with low-resource languages bearing the highest cost. Second, post-trained models retain grammatical knowledge they cannot articulate through explicit prompting, yet this is measurable only in high-resource languages, because near-chance baselines in low-resource settings leave little knowledge to hide. Third, native-language prompting recovers otherwise hidden competence on low-resource languages, demonstrating that only high-resource languages can be probed directly from unprompted probabilities. We conclude that multilingual grammatical evaluation must adopt language-informed, multi-paradigm protocols to avoid systematically underestimating low-resource abilities.