Enhancing Natural Language Understanding through Cross-Modal Interaction: Meaning Recovery from Acoustically Noisy Speech

Ozge Alacam


Abstract
Cross-modality between vision and language is a key component for effective and efficient communication, and human language processing mechanism successfully integrates information from various modalities to extract the intended meaning. However, incomplete linguistic input, i.e. due to a noisy environment, is one of the challenges for a successful communication. In that case, an incompleteness in one channel can be compensated by information from another one. In this paper, by conducting visual-world paradigm, we investigated the dynamics between syntactically possible gap fillers and the visual arrangements in incomplete German sentences and their effect on overall sentence interpretation.
Anthology ID:
W19-6128
Volume:
Proceedings of the 22nd Nordic Conference on Computational Linguistics
Month:
September–October
Year:
2019
Address:
Turku, Finland
Editors:
Mareike Hartmann, Barbara Plank
Venue:
NoDaLiDa
SIG:
Publisher:
Linköping University Electronic Press
Note:
Pages:
272–280
Language:
URL:
https://aclanthology.org/W19-6128
DOI:
Bibkey:
Cite (ACL):
Ozge Alacam. 2019. Enhancing Natural Language Understanding through Cross-Modal Interaction: Meaning Recovery from Acoustically Noisy Speech. In Proceedings of the 22nd Nordic Conference on Computational Linguistics, pages 272–280, Turku, Finland. Linköping University Electronic Press.
Cite (Informal):
Enhancing Natural Language Understanding through Cross-Modal Interaction: Meaning Recovery from Acoustically Noisy Speech (Alacam, NoDaLiDa 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-6128.pdf