<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="morency-baltrusaitis-2017-multimodal">
    <titleInfo>
        <title>Multimodal Machine Learning: Integrating Language, Vision and Speech</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Louis-Philippe</namePart>
        <namePart type="family">Morency</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tadas</namePart>
        <namePart type="family">Baltrušaitis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maja</namePart>
            <namePart type="family">Popović</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vancouver, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Multimodal machine learning is a vibrant multi-disciplinary research field which addresses some of the original goals of artificial intelligence by integrating and modeling multiple communicative modalities, including linguistic, acoustic and visual messages. With the initial research on audio-visual speech recognition and more recently with image and video captioning projects, this research field brings some unique challenges for multimodal researchers given the heterogeneity of the data and the contingency often found between modalities.This tutorial builds upon a recent course taught at Carnegie Mellon University during the Spring 2016 semester (CMU course 11-777) and two tutorials presented at CVPR 2016 and ICMI 2016. The present tutorial will review fundamental concepts of machine learning and deep neural networks before describing the five main challenges in multimodal machine learning: (1) multimodal representation learning, (2) translation &amp; mapping, (3) modality alignment, (4) multimodal fusion and (5) co-learning. The tutorial will also present state-of-the-art algorithms that were recently proposed to solve multimodal applications such as image captioning, video descriptions and visual question-answer. We will also discuss the current and upcoming challenges.</abstract>
    <identifier type="citekey">morency-baltrusaitis-2017-multimodal</identifier>
    <location>
        <url>https://aclanthology.org/P17-5002</url>
    </location>
    <part>
        <date>2017-07</date>
        <extent unit="page">
            <start>3</start>
            <end>5</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Multimodal Machine Learning: Integrating Language, Vision and Speech
%A Morency, Louis-Philippe
%A Baltrušaitis, Tadas
%Y Popović, Maja
%Y Boyd-Graber, Jordan
%S Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts
%D 2017
%8 July
%I Association for Computational Linguistics
%C Vancouver, Canada
%F morency-baltrusaitis-2017-multimodal
%X Multimodal machine learning is a vibrant multi-disciplinary research field which addresses some of the original goals of artificial intelligence by integrating and modeling multiple communicative modalities, including linguistic, acoustic and visual messages. With the initial research on audio-visual speech recognition and more recently with image and video captioning projects, this research field brings some unique challenges for multimodal researchers given the heterogeneity of the data and the contingency often found between modalities.This tutorial builds upon a recent course taught at Carnegie Mellon University during the Spring 2016 semester (CMU course 11-777) and two tutorials presented at CVPR 2016 and ICMI 2016. The present tutorial will review fundamental concepts of machine learning and deep neural networks before describing the five main challenges in multimodal machine learning: (1) multimodal representation learning, (2) translation & mapping, (3) modality alignment, (4) multimodal fusion and (5) co-learning. The tutorial will also present state-of-the-art algorithms that were recently proposed to solve multimodal applications such as image captioning, video descriptions and visual question-answer. We will also discuss the current and upcoming challenges.
%U https://aclanthology.org/P17-5002
%P 3-5

Markdown (Informal)

[Multimodal Machine Learning: Integrating Language, Vision and Speech](https://aclanthology.org/P17-5002) (Morency & Baltrušaitis, ACL 2017)

• Multimodal Machine Learning: Integrating Language, Vision and Speech (Morency & Baltrušaitis, ACL 2017)

ACL

• Louis-Philippe Morency and Tadas Baltrušaitis. 2017. Multimodal Machine Learning: Integrating Language, Vision and Speech. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 3–5, Vancouver, Canada. Association for Computational Linguistics.