Xiaoyu Wang

Author directory

Other people with similar names: Xiaoyu Wang, Xiaoyu Wang, Xiaoyu Wang, Xiaoyu Wang

Unverified author pages with similar names: Xiaoyu Wang


2026

Large language model chatbots are increasingly deployed in organizational settings such as healthcare, finance, and public services, where organization-specific policies specify allowed and prohibited content. Existing policy-alignment benchmarks construct tests from one policy at a time, leaving failures caused by policy composition under-tested. We present COPAL, an automated framework for evaluating composed-policy alignment in organizational chatbots. COPAL uses empirically derived interaction patterns to generate queries that require multiple policies to be handled in one response, and pairs each query with an explicit handling contract specifying what to provide and avoid. Applied to 30 organization-like company worlds, COPAL exposes substantial composed-policy handling failures. Across 9 served models in our automated evaluation setup, composed-policy requests yield a 33.1% error rate, indicating that composed-policy alignment remains a challenging evaluation target. Our code is available at https://github.com/BruceAllenCS/COPAL-code.