A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge

Ali Emami, Adam Trischler, Kaheer Suleman, Jackie Chi Kit Cheung


Abstract
We introduce an automatic system that performs well on two common-sense reasoning tasks, the Winograd Schema Challenge (WSC) and the Choice of Plausible Alternatives (COPA). Problem instances from these tasks require diverse, complex forms of inference and knowledge to solve. Our method uses a knowledge-hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine. It extracts and classifies knowledge from the returned results and weighs it to make a resolution. Our approach improves F1 performance on the WSC by 0.16 over the previous best and is competitive with the state-of-the-art on COPA, demonstrating its general applicability.
Anthology ID:
N18-4004
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop
Month:
June
Year:
2018
Address:
New Orleans, Louisiana, USA
Editors:
Silvio Ricardo Cordeiro, Shereen Oraby, Umashanthi Pavalanathan, Kyeongmin Rim
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
25–31
Language:
URL:
https://aclanthology.org/N18-4004
DOI:
10.18653/v1/N18-4004
Bibkey:
Cite (ACL):
Ali Emami, Adam Trischler, Kaheer Suleman, and Jackie Chi Kit Cheung. 2018. A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 25–31, New Orleans, Louisiana, USA. Association for Computational Linguistics.
Cite (Informal):
A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge (Emami et al., NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-4004.pdf
Video:
 https://aclanthology.org/N18-4004.mp4
Data
WSC