Stefano Pasquali
Author directory2026
Risk-Adjusted Harm Scoring for Automated Red Teaming of LLMs in Financial Services
Fabrizio Dimino | Bhaskarjit Sarmah | Stefano Pasquali
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Fabrizio Dimino | Bhaskarjit Sarmah | Stefano Pasquali
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Services, and Insurance (BFSI) deployments exposed to failures elicited through legally or professionally plausible framing. We introduce RAHS (Risk-Adjusted Harm Score), a risk-sensitive metric jointly capturing disclosure severity, disclaimer mitigation, and inter-judge agreement, and FinRedTeamBench, a 989-prompt benchmark spanning seven BFSI risk areas and 34 sub-categories mapped to regulatory frameworks. Evaluation uses an ensemble of three heterogeneous LLM judges, validated against human experts, and an adaptive multi-turn red-teaming pipeline. On nine open-weight models, RAHS preserves separation under near-ceiling ASR, ranking is stable under hyperparameter sweeps, and multi-turn pressure drives not only more jailbreaks but more operationally severe disclosures, exposing failure modes that single-turn, domain-agnostic evaluations cannot reveal.
AuDQ: Auditable Data Quality through Agentic Rule Discovery
Jacopo Biggiogera | Nicolas Colombo | Krati Saxena | Bhaskarjit Sarmah | Stefano Pasquali
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Jacopo Biggiogera | Nicolas Colombo | Krati Saxena | Bhaskarjit Sarmah | Stefano Pasquali
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
High-stakes tabular pipelines rely on scattered, static quality checks that are hard to evolve and audit against policy intent. Regulated domains particularly suffer as the LLM-era automation raises expectations for traceability and accountability. We propose a vision for an ontology-grounded, repository-native data quality layer in which rules are first-class artifacts: versioned, provenance-linked to governance texts, and re-playable through immutable execution traces. As a first step toward validating this vision, we present a concrete implementation of its agentic rule-discovery core with two rule acquisition modes (Full-Agentic and Guided) and a human-in-the-loop (HITL) review interface. We define an evaluation framework and run experiments addressing whether our fully autonomous, divide-and-conquer agentic rule discovery produces rules of the same quality and diversity as those specified by a human expert, reporting preliminary results on two real world industry datasets in the finance and retail domains.