Veronica Rammouz
Author directory2026
Telling Speculative Stories to Help Humans Imagine the Harms of Healthcare AI
Xingmeng Zhao | Tongnian Wang | Dan Schumacher | Veronica Rammouz | Anthony Rios
Findings of the Association for Computational Linguistics: ACL 2026
Xingmeng Zhao | Tongnian Wang | Dan Schumacher | Veronica Rammouz | Anthony Rios
Findings of the Association for Computational Linguistics: ACL 2026
Artificial intelligence (AI) is rapidly transforming healthcare, enabling the fast development of tools such as stress monitors, wellness trackers, and mental health chatbots. However, this rapid and low-barrier development can also introduce risks, including bias, privacy violations, and unequal access, especially when systems overlook real-world contexts, diverse user needs, and cultural settings. Many recent approaches use AI to identify such risks automatically, but this can reduce human engagement in understanding how harms arise, who they affect, and which stakeholder needs remain unspoken. We present a human-centered ethical foresight framework that generates speculative user stories and supports multi-agent discussions to help people reflect on potential benefits and harms of healthcare AI before deployment. In a user study, participants who engaged with stories identified a broader range of harms, distributing their responses more evenly across all 17 harm types, whereas those who did not engage with stories focused primarily on privacy and well-being (79.1%). Overall, our findings suggest that storytelling helps people anticipate potential risks and benefits and reflect more broadly on how AI systems may affect different users, contexts, and often unspoken needs.
Embarrassingly Simple Access Control for Large Language Models
Veronica Rammouz | Đorđe Klisura | Ram Krishnan | Anthony Rios
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Veronica Rammouz | Đorđe Klisura | Ram Krishnan | Anthony Rios
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Role-Based Access Control (RBAC) is fundamental in secure systems handling healthcare, financial, and government data. Prior work on LLM safety under role constraints has largely relied on fine-tuning or prompting-based enforcement. In contrast, we take a mechanistic view and probe internal representations to identify signals associated with roles, permissions, and refusal behavior. We evaluate this approach across three datasets, database- and department-separated splits, varying policy complexity levels, and adversarial prompts. On the structured Text-to-SQL datasets, simple probes achieve strong performance, reaching average macro-F1 scores of .913 on BIRD and .926 on Spider, while remaining more stable than prompting as policies grow longer. However, probing does not perform equally well in all settings: it underperforms prompting in most OrgAccess comparisons and can degrade substantially under adversarial instructions. Overall, our results show that access-control information can be strongly represented in LLM hidden states, but its usefulness depends on the model, domain, and prompt.
Absolute Error Is Not Enough: Ranking Model Performance Without Labels to Estimate Disparities at Scale
Veronica Rammouz | Aaron O. Gonzalez | Carlos Cruzportillo | Adrian Tan | Nicole Lang Beebe | Ke Yang | Anthony Rios
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Veronica Rammouz | Aaron O. Gonzalez | Carlos Cruzportillo | Adrian Tan | Nicole Lang Beebe | Ke Yang | Anthony Rios
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Estimating model performance without labeled data is important for deploying NLP systems across diverse domains. Prior work has proposed proxy measures based on distribution shift or predicted correctness, but it remains unclear whether these methods reliably preserve the ranking of domains, which is critical for model selection and risk triage. We study this problem from an ordinal perspective using a two-step framework that pairs a base classifier with an error predictor to estimate instance-level failures and disparities and aggregate them into label-free performance estimates. Across multiple sample sizes from the GeoOLID and Amazon Reviews datasets, large language model–based error predictors substantially outperform drift-based baselines in 𝜌 and 𝜏 rank correlations. We further show that lower absolute estimation error (RMSE) does not necessarily imply better ranking, highlighting the importance of evaluating ordinal consistency directly.
WARNING: This paper contains offensive and hateful speech in the appendix.
WARNING: This paper contains offensive and hateful speech in the appendix.