ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation

Zesheng Wei, Mengfan Li, Wenhao Liu, Yixin Zhang, Zilei Wang, Yang Deng


Abstract
Dispute mediation is essential for maintaining social harmony and resilience, yet developing skilled mediators is costly and time-consuming. Existing LLM-based mediation research remains limited by unrealistic task formulations, low-fidelity datasets, and coarse evaluation metrics that obscure turn-by-turn dynamics. To address these gaps, we introduce ProMediConv, a novel benchmarking framework that models mediation as a proactive, multi-stage, and party-aware dialogue process incorporating 11 mediation strategies and four party behavior pattern (BP) states. Using 972 complete real-world cases, we construct a high-fidelity mediation dataset with utterance-level annotations of strategies and BP states. Furthermore, to better assess agent impact, we propose MAD (Mean Attribute Difference), a fine-grained metric that captures BP shifts throughout the dialogue. Leveraging this framework, we establish a comprehensive benchmark by evaluating diverse models alongside our tailored baseline ProMediAgent. Extensive empirical analyses reveal critical behavioral phenomena and underscore the persistent challenges current models face in dynamic, multi-party mediation. Ultimately, ProMediConv provides a rigorous foundation and a vital quantitative standard for advancing AI-assisted conflict resolution. Our dataset and codebase are accessible at https://github.com/ZsWei66/ProMediConv_repo.
Anthology ID:
2026.findings-emnlp.930
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
19618–19638
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.930/
DOI:
Bibkey:
Cite (ACL):
Zesheng Wei, Mengfan Li, Wenhao Liu, Yixin Zhang, Zilei Wang, and Yang Deng. 2026. ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 19618–19638, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation (Wei et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.930.pdf