Hamed Loghmani
Author directory2026
Hidden Configuration Effects on Fairness in LLM-Based Multi-Agent Decision Systems
Hamed Loghmani | Ninareh Mehrabi | Homa Hosseinmardi | Amir Ghasemian
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Hamed Loghmani | Ninareh Mehrabi | Homa Hosseinmardi | Amir Ghasemian
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) deployed as decision agents in regulated settings—loan approval, hiring, clinical triage—must pass pre-deployment fairness audits that report a single fairness number per model under one implicit configuration. We argue this framing misses the dominant source of variation: configuration itself. Sweeping five axes (persona, interview, deliberation, aggregation rule, order) across three frontier LLMs (Claude Sonnet 4.6, GPT-4o-mini, Grok 4.20 Reasoning) on a 1,000-applicant Adult Census Income sample and a 500-applicant ACSIncome (2024 ACS data) sample, we find that within a single model, persona alone swings positive-prediction rate by up to 91 percentage points and collapses decision rules on two of three models; within a Conservative/Balanced/Progressive panel, replacing the panel chair on identical deliberation transcripts moves the disparate-impact ratio from adverse-impact to near-parity; and reversing the speaking order flips up to 34% of applicants on the same panel. We identify a median-voter effect: the Balanced agent matches the majority-vote verdict on 95.8–100% of applicants across all three models—deliberative aggregation reduces, mechanically, to single-agent decision by the median. The audit unit is the configuration, not the model: a system can pass an audit under one configuration and be silently reconfigured into a non-compliant one through a prompt edit—a change that current regulations do not treat as requiring re-audit. The code to reproduce the experiments in this paper is available at https://anonymous.4open.science/r/agent-decision-bias-emnlp-F059/