Yuan Huang
Author directory2026
Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies
Zirui Song | Yuan Huang | Junchang Liu | Haozhe Luo | Chenxi Wang | Lang Gao | Zixiang Xu | Bairui Zhang | Mingfei Han | Xiaojun Chang | Xiuying Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zirui Song | Yuan Huang | Junchang Liu | Haozhe Luo | Chenxi Wang | Lang Gao | Zixiang Xu | Bairui Zhang | Mingfei Han | Xiaojun Chang | Xiuying Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Social deduction games like Werewolf combine language, reasoning, and strategy, providing a testbed for studying language and social intelligence. However, most studies reduce the game to LLM-based self-play, yielding templated utterances and anecdotal cases that overlook the richness of social gameplay. Evaluation further relies on coarse metrics such as survival time or subjective scoring due to the lack of quality reference data. To address these gaps, we curate a high-quality, human-verified, multilingual, and multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants. Based on this dataset, we propose a novel strategy-alignment evaluation that leverages the winning faction’s strategies as ground truth in two stages: 1) Speech evaluation, formulated as multiple-choice-style tasks that assess whether the model can adopt appropriate stances across five dimensions of social ability. 2) Decision evaluation, which assesses the model’s voting choices and opponent-role inferences. This framework enables a fine-grained evaluation of models’ linguistic and reasoning capabilities, while capturing their ability to generate strategically coherent gameplay. Our experiments show that state-of-the-art LLMs and VLMs exhibit diverse performance, with roughly half of the models in both modalities remaining below 0.50 accuracy, revealing clear gaps in deception and counterfactual reasoning.
2024
Automatic Construction of the English Sentence Pattern Structure Treebank for Chinese ESL learners
Lin Zhu | Meng Xu | Wenya Guo | Jingsi Yu | Liner Yang | Zehuang Cao | Yuan Huang | Erhong Yang
Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 1: Main Conference)
Lin Zhu | Meng Xu | Wenya Guo | Jingsi Yu | Liner Yang | Zehuang Cao | Yuan Huang | Erhong Yang
Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 1: Main Conference)
“Analyzing long and complicated sentences has always been a priority and challenge in Englishlearning. In order to conduct the parse of these sentences for Chinese English as Second Lan-guage (ESL) learners, we design the English Sentence Pattern Structure (ESPS) based on theSentence Diagramming theory. Then, we automatically construct the English Sentence PatternStructure Treebank (ESPST) through the method of rule conversion based on constituency struc-ture and evaluate the conversion results. In addition, we set up two comparative experiments,using trained parser and large language models (LLMs). The results prove that the rule-basedconversion approach is effective.”
2022
Exploring Representation-level Augmentation for Code Search
Haochen Li | Chunyan Miao | Cyril Leung | Yanxian Huang | Yuan Huang | Hongyu Zhang | Yanlin Wang
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Haochen Li | Chunyan Miao | Cyril Leung | Yanxian Huang | Yuan Huang | Hongyu Zhang | Yanlin Wang
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Code search, which aims at retrieving the most relevant code fragment for a given natural language query, is a common activity in software development practice. Recently, contrastive learning is widely used in code search research, where many data augmentation approaches for source code (e.g., semantic-preserving program transformation) are proposed to learn better representations. However, these augmentations are at the raw-data level, which requires additional code analysis in the preprocessing stage and additional training cost in the training stage. In this paper, we explore augmentation methods that augment data (both code and query) at representation level which does not require additional data processing and training, and based on this we propose a general format of representation-level augmentation that unifies existing methods. Then, we propose three new augmentation methods (linear extrapolation, binary interpolation, and Gaussian scaling) based on the general format. Furthermore, we theoretically analyze the advantages of the proposed augmentation methods over traditional contrastive learning methods on code search. We experimentally evaluate the proposed representation-level augmentation methods with state-of-the-art code search models on a large-scale public dataset consisting of six programming languages. The experimental results show that our approach can consistently boost the performance of the studied code search models.