<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="peng-etal-2023-testing">
    <titleInfo>
        <title>Testing Paraphrase Models on Recognising Sentence Pairs at Different Degrees of Semantic Overlap</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Qiwei</namePart>
        <namePart type="family">Peng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">David</namePart>
        <namePart type="family">Weir</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Julie</namePart>
        <namePart type="family">Weeds</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Alexis</namePart>
            <namePart type="family">Palmer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jose</namePart>
            <namePart type="family">Camacho-collados</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Paraphrase detection is useful in many natural language understanding applications. Current works typically formulate this problem as a sentence pair binary classification task. However, this setup is not a good fit for many of the intended applications of paraphrase models. In particular, such applications often involve finding the closest paraphrases of the target sentence from a group of candidate sentences where they exhibit different degrees of semantic overlap with the target sentence. To apply models to this paraphrase retrieval scenario, the model must be sensitive to the degree to which two sentences are paraphrases of one another. However, many existing datasets ignore and fail to test models in this setup. In response, we propose adversarial paradigms to create evaluation datasets, which could examine the sensitivity to different degrees of semantic overlap. Empirical results show that, while paraphrase models and different sentence encoders appear successful on standard evaluations, measuring the degree of semantic overlap still remains a big challenge for them.</abstract>
    <identifier type="citekey">peng-etal-2023-testing</identifier>
    <identifier type="doi">10.18653/v1/2023.starsem-1.24</identifier>
    <location>
        <url>https://aclanthology.org/2023.starsem-1.24</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>259</start>
            <end>269</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Testing Paraphrase Models on Recognising Sentence Pairs at Different Degrees of Semantic Overlap
%A Peng, Qiwei
%A Weir, David
%A Weeds, Julie
%Y Palmer, Alexis
%Y Camacho-collados, Jose
%S Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F peng-etal-2023-testing
%X Paraphrase detection is useful in many natural language understanding applications. Current works typically formulate this problem as a sentence pair binary classification task. However, this setup is not a good fit for many of the intended applications of paraphrase models. In particular, such applications often involve finding the closest paraphrases of the target sentence from a group of candidate sentences where they exhibit different degrees of semantic overlap with the target sentence. To apply models to this paraphrase retrieval scenario, the model must be sensitive to the degree to which two sentences are paraphrases of one another. However, many existing datasets ignore and fail to test models in this setup. In response, we propose adversarial paradigms to create evaluation datasets, which could examine the sensitivity to different degrees of semantic overlap. Empirical results show that, while paraphrase models and different sentence encoders appear successful on standard evaluations, measuring the degree of semantic overlap still remains a big challenge for them.
%R 10.18653/v1/2023.starsem-1.24
%U https://aclanthology.org/2023.starsem-1.24
%U https://doi.org/10.18653/v1/2023.starsem-1.24
%P 259-269

Markdown (Informal)

[Testing Paraphrase Models on Recognising Sentence Pairs at Different Degrees of Semantic Overlap](https://aclanthology.org/2023.starsem-1.24) (Peng et al., *SEM 2023)

• Testing Paraphrase Models on Recognising Sentence Pairs at Different Degrees of Semantic Overlap (Peng et al., *SEM 2023)

ACL

• Qiwei Peng, David Weir, and Julie Weeds. 2023. Testing Paraphrase Models on Recognising Sentence Pairs at Different Degrees of Semantic Overlap. In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 259–269, Toronto, Canada. Association for Computational Linguistics.