<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lee-etal-2023-learning-co">
    <titleInfo>
        <title>Learning Co-Speech Gesture for Multimodal Aphasia Type Detection</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Daeun</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sejung</namePart>
        <namePart type="family">Son</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hyolim</namePart>
        <namePart type="family">Jeon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Seungbae</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jinyoung</namePart>
        <namePart type="family">Han</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Aphasia, a language disorder resulting from brain damage, requires accurate identification of specific aphasia types, such as Broca‘s and Wernicke‘s aphasia, for effective treatment. However, little attention has been paid to developing methods to detect different types of aphasia. Recognizing the importance of analyzing co-speech gestures for distinguish aphasia types, we propose a multimodal graph neural network for aphasia type detection using speech and corresponding gesture patterns. By learning the correlation between the speech and gesture modalities for each aphasia type, our model can generate textual representations sensitive to gesture information, leading to accurate aphasia type detection. Extensive experiments demonstrate the superiority of our approach over existing methods, achieving state-of-the-art results (F1 84.2%). We also show that gesture features outperform acoustic features, highlighting the significance of gesture expression in detecting aphasia types. We provide the codes for reproducibility purposes.</abstract>
    <identifier type="citekey">lee-etal-2023-learning-co</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.577</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.577/</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>9287</start>
            <end>9303</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Learning Co-Speech Gesture for Multimodal Aphasia Type Detection
%A Lee, Daeun
%A Son, Sejung
%A Jeon, Hyolim
%A Kim, Seungbae
%A Han, Jinyoung
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F lee-etal-2023-learning-co
%X Aphasia, a language disorder resulting from brain damage, requires accurate identification of specific aphasia types, such as Broca‘s and Wernicke‘s aphasia, for effective treatment. However, little attention has been paid to developing methods to detect different types of aphasia. Recognizing the importance of analyzing co-speech gestures for distinguish aphasia types, we propose a multimodal graph neural network for aphasia type detection using speech and corresponding gesture patterns. By learning the correlation between the speech and gesture modalities for each aphasia type, our model can generate textual representations sensitive to gesture information, leading to accurate aphasia type detection. Extensive experiments demonstrate the superiority of our approach over existing methods, achieving state-of-the-art results (F1 84.2%). We also show that gesture features outperform acoustic features, highlighting the significance of gesture expression in detecting aphasia types. We provide the codes for reproducibility purposes.
%R 10.18653/v1/2023.emnlp-main.577
%U https://aclanthology.org/2023.emnlp-main.577/
%U https://doi.org/10.18653/v1/2023.emnlp-main.577
%P 9287-9303

Markdown (Informal)

[Learning Co-Speech Gesture for Multimodal Aphasia Type Detection](https://aclanthology.org/2023.emnlp-main.577/) (Lee et al., EMNLP 2023)

• Learning Co-Speech Gesture for Multimodal Aphasia Type Detection (Lee et al., EMNLP 2023)

ACL

• Daeun Lee, Sejung Son, Hyolim Jeon, Seungbae Kim, and Jinyoung Han. 2023. Learning Co-Speech Gesture for Multimodal Aphasia Type Detection. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9287–9303, Singapore. Association for Computational Linguistics.