<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhang-etal-2022-passage">
    <titleInfo>
        <title>Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Shujian</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chengyue</namePart>
        <namePart type="family">Gong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xingchao</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Retriever-reader models achieve competitive performance across many different NLP tasks such as open question answering and dialogue conversations. In this work, we notice these models easily overfit the top-rank retrieval passages and standard training fails to reason over the entire retrieval passages. We introduce a learnable passage mask mechanism which desensitizes the impact from the top-rank retrieval passages and prevents the model from overfitting. Controlling the gradient variance with fewer mask candidates and selecting the mask candidates with one-shot bi-level optimization, our learnable regularization strategy enforces the answer generation to focus on the entire retrieval passages. Experiments on different tasks across open question answering, dialogue conversation, and fact verification show that our method consistently outperforms its baselines. Extensive experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.</abstract>
    <identifier type="citekey">zhang-etal-2022-passage</identifier>
    <identifier type="doi">10.18653/v1/2022.emnlp-main.260</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.260</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>3931</start>
            <end>3943</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models
%A Zhang, Shujian
%A Gong, Chengyue
%A Liu, Xingchao
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F zhang-etal-2022-passage
%X Retriever-reader models achieve competitive performance across many different NLP tasks such as open question answering and dialogue conversations. In this work, we notice these models easily overfit the top-rank retrieval passages and standard training fails to reason over the entire retrieval passages. We introduce a learnable passage mask mechanism which desensitizes the impact from the top-rank retrieval passages and prevents the model from overfitting. Controlling the gradient variance with fewer mask candidates and selecting the mask candidates with one-shot bi-level optimization, our learnable regularization strategy enforces the answer generation to focus on the entire retrieval passages. Experiments on different tasks across open question answering, dialogue conversation, and fact verification show that our method consistently outperforms its baselines. Extensive experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.
%R 10.18653/v1/2022.emnlp-main.260
%U https://aclanthology.org/2022.emnlp-main.260
%U https://doi.org/10.18653/v1/2022.emnlp-main.260
%P 3931-3943

Markdown (Informal)

[Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models](https://aclanthology.org/2022.emnlp-main.260) (Zhang et al., EMNLP 2022)

• Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models (Zhang et al., EMNLP 2022)

ACL

• Shujian Zhang, Chengyue Gong, and Xingchao Liu. 2022. Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3931–3943, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.