Joint Constraint Aggregation for Qualifier-Sensitive Retrieval-Augmented QA

Jennifer Hsia, Zachary Chase Lipton, Aarti Singh


Abstract
Retrieval-augmented QA can fail even when the correct answer is in the retrieved evidence. This can happen when a competing answer satisfies some, but not all, of the question’s requirements and is mentioned frequently in the retrieved documents, making it look like a strong choice. To test whether this retrieval skew actually affects the decision, we manipulate it directly: increasing the prevalence of passages that favor the competing answer makes models more likely to select it. This points to a decision problem that candidate-level support alone cannot resolve. We therefore introduce Joint Constraint Aggregation (JCA), which compares candidates against the same question-derived requirements using the retrieved evidence. Across our evaluation settings, JCA improves overall accuracy, and its advantage grows when retrieval favors a competing answer. Together, these results suggest that when partial matches and retrieval prominence pull models toward a competing answer, comparing candidates against the question’s full requirements can make selection more reliable.
Anthology ID:
2026.emnlp-main.2382
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
54126–54140
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2382/
DOI:
Bibkey:
Cite (ACL):
Jennifer Hsia, Zachary Chase Lipton, and Aarti Singh. 2026. Joint Constraint Aggregation for Qualifier-Sensitive Retrieval-Augmented QA. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 54126–54140, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Joint Constraint Aggregation for Qualifier-Sensitive Retrieval-Augmented QA (Hsia et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2382.pdf