<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="patel-etal-2022-neurons">
    <titleInfo>
        <title>On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Gal</namePart>
        <namePart type="family">Patel</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Leshem</namePart>
        <namePart type="family">Choshen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Omri</namePart>
        <namePart type="family">Abend</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Antske</namePart>
            <namePart type="family">Fokkens</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates (Hybrid)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We present a methodology that explores how sentence structure is reflected in neural representations of machine translation systems. We demonstrate our model-agnostic approach with the Transformer English-German translation model. We analyze neuron-level correlation of activations between paraphrases while discussing the methodology challenges and the need for confound analysis to isolate the effects of shallow cues. We find that similarity between activation patterns can be mostly accounted for by similarity in word choice and sentence length. Following that, we manipulate neuron activations to control the syntactic form of the output. We show this intervention to be somewhat successful, indicating that deep models capture sentence-structure distinctions, despite finding no such indication at the neuron level. To conduct our experiments, we develop a semi-automatic method to generate meaning-preserving minimal pair paraphrases (active-passive voice and adverbial clause-noun phrase) and compile a corpus of such pairs.</abstract>
    <identifier type="citekey">patel-etal-2022-neurons</identifier>
    <identifier type="doi">10.18653/v1/2022.conll-1.14</identifier>
    <location>
        <url>https://aclanthology.org/2022.conll-1.14/</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>194</start>
            <end>212</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation
%A Patel, Gal
%A Choshen, Leshem
%A Abend, Omri
%Y Fokkens, Antske
%Y Srikumar, Vivek
%S Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL)
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates (Hybrid)
%F patel-etal-2022-neurons
%X We present a methodology that explores how sentence structure is reflected in neural representations of machine translation systems. We demonstrate our model-agnostic approach with the Transformer English-German translation model. We analyze neuron-level correlation of activations between paraphrases while discussing the methodology challenges and the need for confound analysis to isolate the effects of shallow cues. We find that similarity between activation patterns can be mostly accounted for by similarity in word choice and sentence length. Following that, we manipulate neuron activations to control the syntactic form of the output. We show this intervention to be somewhat successful, indicating that deep models capture sentence-structure distinctions, despite finding no such indication at the neuron level. To conduct our experiments, we develop a semi-automatic method to generate meaning-preserving minimal pair paraphrases (active-passive voice and adverbial clause-noun phrase) and compile a corpus of such pairs.
%R 10.18653/v1/2022.conll-1.14
%U https://aclanthology.org/2022.conll-1.14/
%U https://doi.org/10.18653/v1/2022.conll-1.14
%P 194-212

Markdown (Informal)

[On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation](https://aclanthology.org/2022.conll-1.14/) (Patel et al., CoNLL 2022)

• On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation (Patel et al., CoNLL 2022)

ACL

• Gal Patel, Leshem Choshen, and Omri Abend. 2022. On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 194–212, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.