When Evidence Conflicts: Reliability-aware Meta-review Generation

Xinzhe Wang, Fei Tao, Jiang Xie, Hong Yu, Ye Wang


Abstract
Generating coherent meta-reviews from multiple peer reviews is challenging when reviewer evidence conflicts and varies in reliability. Existing approaches typically formulate meta-review generation as a multi-document summarization task and aggregate reviewer feedback uniformly, making it difficult to determine which opinions should be prioritized under disagreement. In this paper, we study meta-review generation through reliability-aware evidence aggregation. Our framework first extracts aspect-level opinions from peer reviews and identifies conflicting evidence within each aspect. It then estimates opinion-level support and review-level quality to measure evidence reliability. Based on these signals, the framework assigns reliability-aware weights to reviewer feedback, enabling the generator to prioritize better-supported arguments while preserving diverse perspectives. Experiments demonstrate that our method consistently improves meta-review generation over strong baselines on both automatic and human evaluations, with clear gains in conflict recognition and resolution under high-conflict review scenarios. The code and implementation details are publicly available at https://github.com/Wangxz729/reliability-aware-meta-review.
Anthology ID:
2026.findings-emnlp.746
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
15708–15723
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.746/
DOI:
Bibkey:
Cite (ACL):
Xinzhe Wang, Fei Tao, Jiang Xie, Hong Yu, and Ye Wang. 2026. When Evidence Conflicts: Reliability-aware Meta-review Generation. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 15708–15723, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
When Evidence Conflicts: Reliability-aware Meta-review Generation (Wang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.746.pdf