Victor H. Vargas
Author directory2026
Scalable Multi-Agent LLMs for Multi-Turn Conversation Bias Detection and Mitigation
Victor H. Vargas | Farsheed Haque | Depeng Xu | Xi Niu
Findings of the Association for Computational Linguistics: EMNLP 2026
Victor H. Vargas | Farsheed Haque | Depeng Xu | Xi Niu
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) increasingly participate in conversational systems where responses emerge through multi-turn interactions rather than single prompts. We introduce a multi-agent evaluation framework that probes LLMs for bias through multi-turn conversations. We present a framework that orchestrates three roles (target, adversary, and scorer) under diverse normative fairness definitions. Experiments on five widely used LLMs using 100 adversarial prompts show that adversarial multi-turn probing reliably surfaces bias-related responses, with baseline violation rates exceeding 80% across models, while human validation with the scorer (~85% agreement) confirms the scoring reliability. Preventive prompt-conditioning strategies provide limited reduction in violations, while our iterative post-hoc mitigation pipeline substantially reduces scorer-detected violations under the active fairness profile, from over 80% to near-zero. We frame this as scorer-relative violation reduction rather than universal debiasing.