<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="alishahi-etal-2017-encoding">
    <titleInfo>
        <title>Encoding of phonology in a recurrent neural model of grounded speech</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Afra</namePart>
        <namePart type="family">Alishahi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marie</namePart>
        <namePart type="family">Barking</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Grzegorz</namePart>
        <namePart type="family">Chrupała</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Roger</namePart>
            <namePart type="family">Levy</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lucia</namePart>
            <namePart type="family">Specia</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vancouver, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We study the representation and encoding of phonemes in a recurrent neural network model of grounded speech. We use a model which processes images and their spoken descriptions, and projects the visual and auditory representations into the same semantic space. We perform a number of analyses on how information about individual phonemes is encoded in the MFCC features extracted from the speech signal, and the activations of the layers of the model. Via experiments with phoneme decoding and phoneme discrimination we show that phoneme representations are most salient in the lower layers of the model, where low-level signals are processed at a fine-grained level, although a large amount of phonological information is retain at the top recurrent layer. We further find out that the attention mechanism following the top recurrent layer significantly attenuates encoding of phonology and makes the utterance embeddings much more invariant to synonymy. Moreover, a hierarchical clustering of phoneme representations learned by the network shows an organizational structure of phonemes similar to those proposed in linguistics.</abstract>
    <identifier type="citekey">alishahi-etal-2017-encoding</identifier>
    <identifier type="doi">10.18653/v1/K17-1037</identifier>
    <location>
        <url>https://aclanthology.org/K17-1037</url>
    </location>
    <part>
        <date>2017-08</date>
        <extent unit="page">
            <start>368</start>
            <end>378</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Encoding of phonology in a recurrent neural model of grounded speech
%A Alishahi, Afra
%A Barking, Marie
%A Chrupała, Grzegorz
%Y Levy, Roger
%Y Specia, Lucia
%S Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)
%D 2017
%8 August
%I Association for Computational Linguistics
%C Vancouver, Canada
%F alishahi-etal-2017-encoding
%X We study the representation and encoding of phonemes in a recurrent neural network model of grounded speech. We use a model which processes images and their spoken descriptions, and projects the visual and auditory representations into the same semantic space. We perform a number of analyses on how information about individual phonemes is encoded in the MFCC features extracted from the speech signal, and the activations of the layers of the model. Via experiments with phoneme decoding and phoneme discrimination we show that phoneme representations are most salient in the lower layers of the model, where low-level signals are processed at a fine-grained level, although a large amount of phonological information is retain at the top recurrent layer. We further find out that the attention mechanism following the top recurrent layer significantly attenuates encoding of phonology and makes the utterance embeddings much more invariant to synonymy. Moreover, a hierarchical clustering of phoneme representations learned by the network shows an organizational structure of phonemes similar to those proposed in linguistics.
%R 10.18653/v1/K17-1037
%U https://aclanthology.org/K17-1037
%U https://doi.org/10.18653/v1/K17-1037
%P 368-378

Markdown (Informal)

[Encoding of phonology in a recurrent neural model of grounded speech](https://aclanthology.org/K17-1037) (Alishahi et al., CoNLL 2017)

• Encoding of phonology in a recurrent neural model of grounded speech (Alishahi et al., CoNLL 2017)

ACL

• Afra Alishahi, Marie Barking, and Grzegorz Chrupała. 2017. Encoding of phonology in a recurrent neural model of grounded speech. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 368–378, Vancouver, Canada. Association for Computational Linguistics.