Natural Language Deduction through Search over Statement Compositions

Kaj Bostrom, Zayne Sprague, Swarat Chaudhuri, Greg Durrett


Abstract
In settings from fact-checking to question answering, we frequently want to know whether a collection of evidence (premises) entails a hypothesis. Existing methods primarily focus on the end-to-end discriminative version of this task, but less work has treated the generative version in which a model searches over the space of statements entailed by the premises to constructively derive the hypothesis. We propose a system for doing this kind of deductive reasoning in natural language by decomposing the task into separate steps coordinated by a search procedure, producing a tree of intermediate conclusions that faithfully reflects the system’s reasoning process. Our experiments on the EntailmentBank dataset (Dalvi et al., 2021) demonstrate that the proposed system can successfully prove true statements while rejecting false ones. Moreover, it produces natural language explanations with a 17% absolute higher step validity than those produced by an end-to-end T5 model.
Anthology ID:
2022.findings-emnlp.358
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4871–4883
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.358
DOI:
10.18653/v1/2022.findings-emnlp.358
Bibkey:
Cite (ACL):
Kaj Bostrom, Zayne Sprague, Swarat Chaudhuri, and Greg Durrett. 2022. Natural Language Deduction through Search over Statement Compositions. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4871–4883, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Natural Language Deduction through Search over Statement Compositions (Bostrom et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-emnlp.358.pdf
Software:
 2022.findings-emnlp.358.software.zip
Video:
 https://aclanthology.org/2022.findings-emnlp.358.mp4