Whodunnit? Crime Drama as a Case for Natural Language Understanding

Lea Frermann, Shay B. Cohen, Mirella Lapata


Abstract
In this paper we argue that crime drama exemplified in television programs such as CSI: Crime Scene Investigation is an ideal testbed for approximating real-world natural language understanding and the complex inferences associated with it. We propose to treat crime drama as a new inference task, capitalizing on the fact that each episode poses the same basic question (i.e., who committed the crime) and naturally provides the answer when the perpetrator is revealed. We develop a new dataset based on CSI episodes, formalize perpetrator identification as a sequence labeling problem, and develop an LSTM-based model which learns from multi-modal data. Experimental results show that an incremental inference strategy is key to making accurate guesses as well as learning from representations fusing textual, visual, and acoustic input.
Anthology ID:
Q18-1001
Volume:
Transactions of the Association for Computational Linguistics, Volume 6
Month:
Year:
2018
Address:
Cambridge, MA
Editors:
Lillian Lee, Mark Johnson, Kristina Toutanova, Brian Roark
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
1–15
Language:
URL:
https://aclanthology.org/Q18-1001
DOI:
10.1162/tacl_a_00001
Bibkey:
Cite (ACL):
Lea Frermann, Shay B. Cohen, and Mirella Lapata. 2018. Whodunnit? Crime Drama as a Case for Natural Language Understanding. Transactions of the Association for Computational Linguistics, 6:1–15.
Cite (Informal):
Whodunnit? Crime Drama as a Case for Natural Language Understanding (Frermann et al., TACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/Q18-1001.pdf
Video:
 https://aclanthology.org/Q18-1001.mp4
Code
 EdinburghNLP/csi-corpus
Data
MovieQAVisual Question Answering