Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework

Aaron Steven White, Pushpendre Rastogi, Kevin Duh, Benjamin Van Durme


Abstract
We propose to unify a variety of existing semantic classification tasks, such as semantic role labeling, anaphora resolution, and paraphrase detection, under the heading of Recognizing Textual Entailment (RTE). We present a general strategy to automatically generate one or more sentential hypotheses based on an input sentence and pre-existing manual semantic annotations. The resulting suite of datasets enables us to probe a statistical RTE model’s performance on different aspects of semantics. We demonstrate the value of this approach by investigating the behavior of a popular neural network RTE model.
Anthology ID:
I17-1100
Volume:
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
November
Year:
2017
Address:
Taipei, Taiwan
Editors:
Greg Kondrak, Taro Watanabe
Venue:
IJCNLP
SIG:
Publisher:
Asian Federation of Natural Language Processing
Note:
Pages:
996–1005
Language:
URL:
https://aclanthology.org/I17-1100
DOI:
Bibkey:
Cite (ACL):
Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van Durme. 2017. Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 996–1005, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Cite (Informal):
Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework (White et al., IJCNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/I17-1100.pdf
Data
SNLI