Nan Wu
Author directory2026
LatentRiskBench: Measuring State-Dependent Risks in Enterprise Agents
Jingyue Cong | Estrid He | Zhuang Li | Li Shen | Nan Wu | Yameng Peng | Xinshu Li | Andy Song
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Jingyue Cong | Estrid He | Zhuang Li | Li Shen | Nan Wu | Yameng Peng | Xinshu Li | Andy Song
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Enterprise agents face safety risks that cannot be determined from individual actions alone. Existing safety benchmarks largely evaluate action-local risks, overlooking failures whose safety depends on workflow state. In such state-dependent risks, the same action may be safe in one context but violate policy in another. Hence, we introduce LatentRiskBench for evaluating such state-dependent risks in enterprise settings. It pairs risky trajectories with benign counterparts that share the same terminal action but differ in latent policy-relevant states, requiring models to reconstruct workflow context rather than rely on action-level cues. We also introduce new metrics for state-dependent risk evaluation, including a suite of residual risk (RR) measures. Experiments on the benchmark show that action-level evaluation detects only 0.09 of state-dependent violations, whereas state-level evaluation reaches 0.96. Existing guardrail methods largely overlook such risks unless they are explicitly applied to workflow states. Case studies on a commercial deployed scenario further show that the safety metrics embedded in the benchmark provide a more realistic measure of safety for enterprise agents.
2025
SwiLTra-Bench: The Swiss Legal Translation Benchmark
Joel Niklaus | Jakob Merane | Luka Nenadic | Sina Ahmadi | Yingqiang Gao | Cyrill A. H. Chevalley | Claude Humbel | Christophe Gösken | Lorenzo Tanzi | Thomas Lüthi | Stefan Palombo | Spencer Poff | Boling Yang | Nan Wu | Matthew Guillod | Robin Mamié | Daniel Brunner | Julio Pereyra | Niko Grupen
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Joel Niklaus | Jakob Merane | Luka Nenadic | Sina Ahmadi | Yingqiang Gao | Cyrill A. H. Chevalley | Claude Humbel | Christophe Gösken | Lorenzo Tanzi | Thomas Lüthi | Stefan Palombo | Spencer Poff | Boling Yang | Nan Wu | Matthew Guillod | Robin Mamié | Daniel Brunner | Julio Pereyra | Niko Grupen
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
In Switzerland legal translation is uniquely important due to the country’s four official languages and requirements for multilingual legal documentation. However, this process traditionally relies on professionals who must be both legal experts and skilled translators—creating bottlenecks and impacting effective access to justice. To address this challenge, we introduce SwiLTra-Bench, a comprehensive multilingual benchmark of over 180K aligned Swiss legal translation pairs comprising laws, headnotes, and press releases across all Swiss languages along with English, designed to evaluate LLM-based translation systems. Our systematic evaluation reveals that frontier models achieve superior translation performance across all document types, while specialized translation systems excel specifically in laws but under-perform in headnotes. Through rigorous testing and human expert validation, we demonstrate that while fine-tuning open SLMs significantly improves their translation quality, they still lag behind the best zero-shot prompted frontier models such as Claude-3.5-Sonnet. Additionally, we present SwiLTra-Judge, a specialized LLM evaluation system that aligns best with human expert assessments.