Yating Wang
Author directory2026
Editing Rules in LLMs Requires More Than a Localized Update
Yating Wang | Wenting Zhao | Yaqi Zhao | Yongshun Gong | Yilong Yin | Haoliang Sun
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yating Wang | Wenting Zhao | Yaqi Zhao | Yongshun Gong | Yilong Yin | Haoliang Sun
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Model editing aims to update large language models (LLMs) with new knowledge without full retraining, but existing methods mainly focus on fact-level knowledge. Rules that support mathematical and scientific reasoning must remain consistent across formulas, descriptions, and instances, making localized updates less effective for rule-level editing. In this paper, we construct RuleEdit-200, a manually verified benchmark of 200 mathematical and physical rules, each instantiated in formula, description, and instance forms. Using this benchmark, we conduct fine-grained causal tracing on four LLMs and identify a form-specific layer-wise separation of rule knowledge: formulas and descriptions are associated with earlier layers, whereas instances rely on a distinct set of middle-layer representations. This finding suggests that rule-level editing requires separate updates for different forms. Based on this insight, we propose Distributed Multi-Layer Editing (DMLE), which updates formulas, descriptions and instances at their corresponding layer ranges. Experiments on RuleEdit-200 show that DMLE remains competitive on standard editing metrics while substantially improving rule-level editing, with average gains of 18.22% and 61.00% in instance portability and rule understanding over the best baseline on each model.
2025
MinosEval: Distinguishing Factoid and Non-Factoid for Tailored Open-Ended QA Evaluation with LLMs
Yongqi Fan | Yating Wang | Guandong Wang | Zhai Jie | Jingping Liu | Qi Ye | Tong Ruan
Findings of the Association for Computational Linguistics: ACL 2025
Yongqi Fan | Yating Wang | Guandong Wang | Zhai Jie | Jingping Liu | Qi Ye | Tong Ruan
Findings of the Association for Computational Linguistics: ACL 2025
Open-ended question answering (QA) is a key task for evaluating the capabilities of large language models (LLMs). Compared to closed-ended QA, it demands longer answer statements, more nuanced reasoning processes, and diverse expressions, making refined and interpretable automatic evaluation both crucial and challenging. Traditional metrics like ROUGE and BERTScore struggle to capture semantic similarities due to different patterns between model responses and reference answers. Current LLM-based evaluation approaches, such as pairwise or listwise comparisons of candidate answers, lack intuitive interpretability. While pointwise scoring of each response provides some descriptions, it fails to adapt across different question contents. Most notably, existing methods overlook the distinction between factoid and non-factoid questions. To address these challenges, we propose MinosEval, a novel evaluation method that first distinguishes open-ended questions and then ranks candidate answers using different evaluation strategies. For factoid questions, it applies an adaptive key-point scoring strategy, while for non-factoid questions, it uses an instance-aware listwise ranking strategy. Experiments on multiple open-ended QA datasets, including self-built ones with more candidate responses to complement community resources, show that MinosEval better aligns with human annotations and offers more interpretable results.
2024
Unexpected Phenomenon: LLMs’ Spurious Associations in Information Extraction
Weiyan Zhang | Wanpeng Lu | Jiacheng Wang | Yating Wang | Lihan Chen | Haiyun Jiang | Jingping Liu | Tong Ruan
Findings of the Association for Computational Linguistics: ACL 2024
Weiyan Zhang | Wanpeng Lu | Jiacheng Wang | Yating Wang | Lihan Chen | Haiyun Jiang | Jingping Liu | Tong Ruan
Findings of the Association for Computational Linguistics: ACL 2024
Information extraction plays a critical role in natural language processing. When applying large language models (LLMs) to this domain, we discover an unexpected phenomenon: LLMs’ spurious associations. In tasks such as relation extraction, LLMs can accurately identify entity pairs, even if the given relation (label) is semantically unrelated to the pre-defined original one. To find these labels, we design two strategies in this study, including forward label extension and backward label validation. We also leverage the extended labels to improve model performance. Our comprehensive experiments show that spurious associations occur consistently in both Chinese and English datasets across various LLM sizes. Moreover, the use of extended labels significantly enhances LLM performance in information extraction tasks. Remarkably, there is a performance increase of 9.55%, 11.42%, and 21.27% in F1 scores on the SciERC, ACE05, and DuEE datasets, respectively.