A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge

Ali Emami, Adam Trischler, Kaheer Suleman, Jackie Chi Kit Cheung


Abstract
We introduce an automatic system that performs well on two common-sense reasoning tasks, the Winograd Schema Challenge (WSC) and the Choice of Plausible Alternatives (COPA). Problem instances from these tasks require diverse, complex forms of inference and knowledge to solve. Our method uses a knowledge-hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine. It extracts and classifies knowledge from the returned results and weighs it to make a resolution. Our approach improves F1 performance on the WSC by 0.16 over the previous best and is competitive with the state-of-the-art on COPA, demonstrating its general applicability.
Anthology ID:
N18-4004
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop
Month:
June
Year:
2018
Address:
New Orleans, Louisiana, USA
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
25–31
Language:
URL:
https://aclanthology.org/N18-4004
DOI:
10.18653/v1/N18-4004
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/N18-4004.pdf
Video:
 http://vimeo.com/277631281
Data
WSC