MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate

Garvit Joshi, Stavya Dhyani, Jasmine, Arun Chauhan


Abstract
Media bias in news articles operates through subtle linguistic cues—loaded language, selective framing, and strategic omission—that resist single-model detection and have traditionally required large annotated corpora for supervised training. We ask whether structured multi-agent deliberation can serve as a principled, training-free alternative to supervised classification for this task. We introduce MABPD (Multi-Agent Bias Probing Detection), a pipeline in which three specialized LLM agents analyze an article from complementary perspectives and resolve disagreements through a Structured Argument Debate (SAD) protocol. SAD implements a domain-motivated asymmetric burden of proof—biased claims without grounded textual evidence carry zero weight—combined with role-weighted voting and post-consensus verification, replacing task-specific supervised decision boundaries with explicit deliberative structure. Ablation confirms that this structured deliberation, not mere agent parallelism, drives performance: removing the debate module reduces F1 by up to 10.6 points. On the BABE benchmark (4,121 expert-annotated sentences), MABPD achieves 83.4% macro F1 on the held-out test split—within 0.7 percentage points (pp) of the supervised SOTA (MAGPIE, 84.1% macro F1; Horych et al., 2024)—without any task-specific training or threshold tuning on annotated data. Cross-dataset evaluation on the SemEval 2019 HyperPartisan corpus (644 articles) yields 75.0% zero-shot accuracy, within 7.2 pp of the supervised SOTA accuracy (82.2%; Kiesel et al. 2019), confirming transfer across annotation regimes. We release the full pipeline and evaluation code.
Anthology ID:
2026.emnlp-main.1777
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
40482–40501
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1777/
DOI:
Bibkey:
Cite (ACL):
Garvit Joshi, Stavya Dhyani, Jasmine, and Arun Chauhan. 2026. MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 40482–40501, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate (Joshi et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1777.pdf