Ying Gao
Author directory2026
LiveClawBench: Toward Faithful Evaluation of Stateful LLM Assistants
Xiang Long | Li Du | Yilong Xu | RongJian Xu | Qiyanhui LU | Ying Gao | Qinhua Xie | Fangcheng Liu | Ning Ding | Haoqing Wang | Ziheng Li | Changjiang Zhou | Jianyuan Guo | Yehui Tang
Findings of the Association for Computational Linguistics: EMNLP 2026
Xiang Long | Li Du | Yilong Xu | RongJian Xu | Qiyanhui LU | Ying Gao | Qinhua Xie | Fangcheng Liu | Ning Ding | Haoqing Wang | Ziheng Li | Changjiang Zhou | Jianyuan Guo | Yehui Tang
Findings of the Association for Computational Linguistics: EMNLP 2026
OpenClaw-style personal assistants extend LLM agents from isolated tool use to open-ended, stateful, and personalized software environments. Evaluating these assistants is fundamentally a fidelity problem: benchmarks must be faithful both to the distribution of real assistant tasks and to the execution semantics of the environments in which those tasks unfold. Existing benchmarks often lose fidelity in one dimension or the other. Their task distributions are shaped by what is easy to isolate, mock, and verify, underrepresenting real-world difficulties such as cross-service dependency, contaminated state, implicit intent, and runtime change. Their environments are either live but hard to reproduce, or reproducible but reduced to endpoint-level stubs that remove sessions, artifacts, state transitions, and downstream side effects. We introduce LiveClawBench, a benchmark designed around this dual-fidelity requirement. LiveClawBench combines a Triple-Axis Complexity Framework for difficulty-driven task construction with reproducible full-stack mock applications that preserve stateful execution semantics. With 134 executable cases across 10 domains with 22 mocked services, LiveClawBench supports controlled, extensible, and factor-level diagnostic evaluation of realistic agentic tasks.
TRAC: Token-level Reward Assignment for Coherent Abstractive Summarization
陈宣齐 | 容梓莹 | Xinfeng Liao | Lianxi Wang | Ying Gao | Shengyi Jiang
Findings of the Association for Computational Linguistics: ACL 2026
陈宣齐 | 容梓莹 | Xinfeng Liao | Lianxi Wang | Ying Gao | Shengyi Jiang
Findings of the Association for Computational Linguistics: ACL 2026
Large Language Models (LLMs) have achieved remarkable success in text summarization, particularly through the integration of reinforcement learning. However, maintaining logical coherence and contextual consistency remains a pervasive challenge in long-form generation, often hindering the production of high-quality, unified summaries. To address these persistent issues, we propose TRAC, a framework that introduces a token-level reward function by integrating relative sentence gain, inter-sentence attention, and a Gaussian length penalty. By training a Process Reward Model (PRM) to provide fine-grained, step-wise supervision, TRAC ensures superior structural integrity and fluency during the generation process. Experimental results demonstrate that TRAC outperforms the sequence-level baseline by 11.05% in Fluency and 10.61% in Relevance. Furthermore, it achieves significant gains over competitive baselines such as FIGA and TLCR, underscoring its effectiveness and generalizability in high-quality NLP summarization.
2025
Gradient Inversion Attack in Federated Learning: Exposing Text Data through Discrete Optimization
Ying Gao | Yuxin Xie | Huanghao Deng | Zukun Zhu
Proceedings of the 31st International Conference on Computational Linguistics
Ying Gao | Yuxin Xie | Huanghao Deng | Zukun Zhu
Proceedings of the 31st International Conference on Computational Linguistics
Federated learning has emerged as a potential solution to overcome the bottleneck posed by the near exhaustion of public text data in training large language models. There are claims that the strategy of exchanging gradients allows using text data including private information. Although recent studies demonstrate that data can be reconstructed from gradients, the threat for text data seems relatively small due to its sensitivity to even a few token errors. However, we propose a novel attack method FET, indicating that it is possible to Fully Expose Text data from gradients. Unlike previous methods that optimize continuous embedding vectors, we directly search for a text sequence with gradients that match the known gradients. First, we infer the total number of tokens and the unique tokens in the target text data from the gradients of the embedding layer. Then we develop a discrete optimization algorithm, which globally explores the solution space and precisely refines the obtained solution, incorporating both global and local search strategies. We also find that gradients of the fully connected layer are dominant, providing sufficient guidance for the optimization process. Our experiments show a significant improvement in attack performance, with an average increase of 39% for TinyBERT-6, 20% for BERT-base and 15% for BERT-large in exact match rates across three datasets. These findings highlight serious privacy risks in text data, suggesting that using smaller models is not an effective privacy-preserving strategy.