ESPRIT: Explaining Solutions to Physical Reasoning Tasks

Nazneen Fatema Rajani, Rui Zhang, Yi Chern Tan, Stephan Zheng, Jeremy Weiss, Aadit Vyas, Abhijit Gupta, Caiming Xiong, Richard Socher, Dragomir Radev


Abstract
Neural networks lack the ability to reason about qualitative physics and so cannot generalize to scenarios and tasks unseen during training. We propose ESPRIT, a framework for commonsense reasoning about qualitative physics in natural language that generates interpretable descriptions of physical events. We use a two-step approach of first identifying the pivotal physical events in an environment and then generating natural language descriptions of those events using a data-to-text approach. Our framework learns to generate explanations of how the physical simulation will causally evolve so that an agent or a human can easily reason about a solution using those interpretable descriptions. Human evaluations indicate that ESPRIT produces crucial fine-grained details and has high coverage of physical concepts compared to even human annotations. Dataset, code and documentation are available at https://github.com/salesforce/esprit.
Anthology ID:
2020.acl-main.706
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7906–7917
Language:
URL:
https://aclanthology.org/2020.acl-main.706
DOI:
10.18653/v1/2020.acl-main.706
Bibkey:
Cite (ACL):
Nazneen Fatema Rajani, Rui Zhang, Yi Chern Tan, Stephan Zheng, Jeremy Weiss, Aadit Vyas, Abhijit Gupta, Caiming Xiong, Richard Socher, and Dragomir Radev. 2020. ESPRIT: Explaining Solutions to Physical Reasoning Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7906–7917, Online. Association for Computational Linguistics.
Cite (Informal):
ESPRIT: Explaining Solutions to Physical Reasoning Tasks (Rajani et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.706.pdf
Video:
 http://slideslive.com/38928707
Code
 salesforce/esprit +  additional community code
Data
PHYRE