Combining Knowledge Hunting and Neural Language Models to Solve the Winograd Schema Challenge

Ashok Prakash, Arpit Sharma, Arindam Mitra, Chitta Baral


Abstract
Winograd Schema Challenge (WSC) is a pronoun resolution task which seems to require reasoning with commonsense knowledge. The needed knowledge is not present in the given text. Automatic extraction of the needed knowledge is a bottleneck in solving the challenge. The existing state-of-the-art approach uses the knowledge embedded in their pre-trained language model. However, the language models only embed part of the knowledge, the ones related to frequently co-existing concepts. This limits the performance of such models on the WSC problems. In this work, we build-up on the language model based methods and augment them with a commonsense knowledge hunting (using automatic extraction from text) module and an explicit reasoning module. Our end-to-end system built in such a manner improves on the accuracy of two of the available language model based approaches by 5.53% and 7.7% respectively. Overall our system achieves the state-of-the-art accuracy of 71.06% on the WSC dataset, an improvement of 7.36% over the previous best.
Anthology ID:
P19-1614
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6110–6119
Language:
URL:
https://aclanthology.org/P19-1614
DOI:
10.18653/v1/P19-1614
Bibkey:
Cite (ACL):
Ashok Prakash, Arpit Sharma, Arindam Mitra, and Chitta Baral. 2019. Combining Knowledge Hunting and Neural Language Models to Solve the Winograd Schema Challenge. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6110–6119, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Combining Knowledge Hunting and Neural Language Models to Solve the Winograd Schema Challenge (Prakash et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1614.pdf
Data
QA-SRLWSC