<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="pham-etal-2018-seq2seq2sentiment">
    <titleInfo>
        <title>Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hai</namePart>
        <namePart type="family">Pham</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Thomas</namePart>
        <namePart type="family">Manzini</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Paul</namePart>
        <namePart type="given">Pu</namePart>
        <namePart type="family">Liang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Barnabás</namePart>
        <namePart type="family">Poczós</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Amir</namePart>
            <namePart type="family">Zadeh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Paul</namePart>
            <namePart type="given">Pu</namePart>
            <namePart type="family">Liang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Louis-Philippe</namePart>
            <namePart type="family">Morency</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Soujanya</namePart>
            <namePart type="family">Poria</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Erik</namePart>
            <namePart type="family">Cambria</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Stefan</namePart>
            <namePart type="family">Scherer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Melbourne, Australia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Multimodal machine learning is a core research area spanning the language, visual and acoustic modalities. The central challenge in multimodal learning involves learning representations that can process and relate information from multiple modalities. In this paper, we propose two methods for unsupervised learning of joint multimodal representations using sequence to sequence (Seq2Seq) methods: a Seq2Seq Modality Translation Model and a Hierarchical Seq2Seq Modality Translation Model. We also explore multiple different variations on the multimodal inputs and outputs of these seq2seq models. Our experiments on multimodal sentiment analysis using the CMU-MOSI dataset indicate that our methods learn informative multimodal representations that outperform the baselines and achieve improved performance on multimodal sentiment analysis, specifically in the Bimodal case where our model is able to improve F1 Score by twelve points. We also discuss future directions for multimodal Seq2Seq methods.</abstract>
    <identifier type="citekey">pham-etal-2018-seq2seq2sentiment</identifier>
    <identifier type="doi">10.18653/v1/W18-3308</identifier>
    <location>
        <url>https://aclanthology.org/W18-3308</url>
    </location>
    <part>
        <date>2018-07</date>
        <extent unit="page">
            <start>53</start>
            <end>63</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
%A Pham, Hai
%A Manzini, Thomas
%A Liang, Paul Pu
%A Poczós, Barnabás
%Y Zadeh, Amir
%Y Liang, Paul Pu
%Y Morency, Louis-Philippe
%Y Poria, Soujanya
%Y Cambria, Erik
%Y Scherer, Stefan
%S Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML)
%D 2018
%8 July
%I Association for Computational Linguistics
%C Melbourne, Australia
%F pham-etal-2018-seq2seq2sentiment
%X Multimodal machine learning is a core research area spanning the language, visual and acoustic modalities. The central challenge in multimodal learning involves learning representations that can process and relate information from multiple modalities. In this paper, we propose two methods for unsupervised learning of joint multimodal representations using sequence to sequence (Seq2Seq) methods: a Seq2Seq Modality Translation Model and a Hierarchical Seq2Seq Modality Translation Model. We also explore multiple different variations on the multimodal inputs and outputs of these seq2seq models. Our experiments on multimodal sentiment analysis using the CMU-MOSI dataset indicate that our methods learn informative multimodal representations that outperform the baselines and achieve improved performance on multimodal sentiment analysis, specifically in the Bimodal case where our model is able to improve F1 Score by twelve points. We also discuss future directions for multimodal Seq2Seq methods.
%R 10.18653/v1/W18-3308
%U https://aclanthology.org/W18-3308
%U https://doi.org/10.18653/v1/W18-3308
%P 53-63

Markdown (Informal)

[Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis](https://aclanthology.org/W18-3308) (Pham et al., ACL 2018)

• Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis (Pham et al., ACL 2018)

ACL

• Hai Pham, Thomas Manzini, Paul Pu Liang, and Barnabás Poczós. 2018. Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis. In Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML), pages 53–63, Melbourne, Australia. Association for Computational Linguistics.