Decoding Brain Activity Associated with Literal and Metaphoric Sentence Comprehension Using Distributional Semantic Models

Vesna G. Djokic, Jean Maillard, Luana Bulat, Ekaterina Shutova


Abstract
Recent years have seen a growing interest within the natural language processing (NLP) community in evaluating the ability of semantic models to capture human meaning representation in the brain. Existing research has mainly focused on applying semantic models to decode brain activity patterns associated with the meaning of individual words, and, more recently, this approach has been extended to sentences and larger text fragments. Our work is the first to investigate metaphor processing in the brain in this context. We evaluate a range of semantic models (word embeddings, compositional, and visual models) in their ability to decode brain activity associated with reading of both literal and metaphoric sentences. Our results suggest that compositional models and word embeddings are able to capture differences in the processing of literal and metaphoric sentences, providing support for the idea that the literal meaning is not fully accessible during familiar metaphor comprehension.
Anthology ID:
2020.tacl-1.16
Volume:
Transactions of the Association for Computational Linguistics, Volume 8
Month:
Year:
2020
Address:
Cambridge, MA
Editors:
Mark Johnson, Brian Roark, Ani Nenkova
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
231–246
Language:
URL:
https://aclanthology.org/2020.tacl-1.16
DOI:
10.1162/tacl_a_00307
Bibkey:
Cite (ACL):
Vesna G. Djokic, Jean Maillard, Luana Bulat, and Ekaterina Shutova. 2020. Decoding Brain Activity Associated with Literal and Metaphoric Sentence Comprehension Using Distributional Semantic Models. Transactions of the Association for Computational Linguistics, 8:231–246.
Cite (Informal):
Decoding Brain Activity Associated with Literal and Metaphoric Sentence Comprehension Using Distributional Semantic Models (Djokic et al., TACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.tacl-1.16.pdf