An RNN-based Binary Classifier for the Story Cloze Test

Melissa Roemmele, Sosuke Kobayashi, Naoya Inoue, Andrew Gordon


Abstract
The Story Cloze Test consists of choosing a sentence that best completes a story given two choices. In this paper we present a system that performs this task using a supervised binary classifier on top of a recurrent neural network to predict the probability that a given story ending is correct. The classifier is trained to distinguish correct story endings given in the training data from incorrect ones that we artificially generate. Our experiments evaluate different methods for generating these negative examples, as well as different embedding-based representations of the stories. Our best result obtains 67.2% accuracy on the test set, outperforming the existing top baseline of 58.5%.
Anthology ID:
W17-0911
Volume:
Proceedings of the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics
Month:
April
Year:
2017
Address:
Valencia, Spain
Editors:
Michael Roth, Nasrin Mostafazadeh, Nathanael Chambers, Annie Louis
Venue:
LSDSem
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
74–80
Language:
URL:
https://aclanthology.org/W17-0911
DOI:
10.18653/v1/W17-0911
Bibkey:
Cite (ACL):
Melissa Roemmele, Sosuke Kobayashi, Naoya Inoue, and Andrew Gordon. 2017. An RNN-based Binary Classifier for the Story Cloze Test. In Proceedings of the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics, pages 74–80, Valencia, Spain. Association for Computational Linguistics.
Cite (Informal):
An RNN-based Binary Classifier for the Story Cloze Test (Roemmele et al., LSDSem 2017)
Copy Citation:
PDF:
https://aclanthology.org/W17-0911.pdf