<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="qi-etal-2023-pragmaticqa">
    <titleInfo>
        <title>PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Peng</namePart>
        <namePart type="family">Qi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nina</namePart>
        <namePart type="family">Du</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Christopher</namePart>
        <namePart type="family">Manning</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jing</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Pragmatic reasoning about another speaker’s unspoken intent and state of mind is crucial to efficient and effective human communication. It is virtually omnipresent in conversations between humans, e.g., when someone asks “do you have a minute?”, instead of interpreting it literally as a query about your schedule, you understand that the speaker might have requests that take time, and respond accordingly. In this paper, we present PragmatiCQA, the first large-scale open-domain question answering (QA) dataset featuring 6873 QA pairs that explores pragmatic reasoning in conversations over a diverse set of topics. We designed innovative crowdsourcing mechanisms for interest-based and task-driven data collection to address the common issue of incentive misalignment between crowdworkers and potential users. To compare computational models’ capability at pragmatic reasoning, we also propose several quantitative metrics to evaluate question answering systems on PragmatiCQA. We find that state-of-the-art systems still struggle to perform human-like pragmatic reasoning, and highlight their limitations for future research.</abstract>
    <identifier type="citekey">qi-etal-2023-pragmaticqa</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.385</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.385</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>6175</start>
            <end>6191</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations
%A Qi, Peng
%A Du, Nina
%A Manning, Christopher
%A Huang, Jing
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F qi-etal-2023-pragmaticqa
%X Pragmatic reasoning about another speaker’s unspoken intent and state of mind is crucial to efficient and effective human communication. It is virtually omnipresent in conversations between humans, e.g., when someone asks “do you have a minute?”, instead of interpreting it literally as a query about your schedule, you understand that the speaker might have requests that take time, and respond accordingly. In this paper, we present PragmatiCQA, the first large-scale open-domain question answering (QA) dataset featuring 6873 QA pairs that explores pragmatic reasoning in conversations over a diverse set of topics. We designed innovative crowdsourcing mechanisms for interest-based and task-driven data collection to address the common issue of incentive misalignment between crowdworkers and potential users. To compare computational models’ capability at pragmatic reasoning, we also propose several quantitative metrics to evaluate question answering systems on PragmatiCQA. We find that state-of-the-art systems still struggle to perform human-like pragmatic reasoning, and highlight their limitations for future research.
%R 10.18653/v1/2023.findings-acl.385
%U https://aclanthology.org/2023.findings-acl.385
%U https://doi.org/10.18653/v1/2023.findings-acl.385
%P 6175-6191

Markdown (Informal)

[PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations](https://aclanthology.org/2023.findings-acl.385) (Qi et al., Findings 2023)

• PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations (Qi et al., Findings 2023)

ACL

• Peng Qi, Nina Du, Christopher Manning, and Jing Huang. 2023. PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6175–6191, Toronto, Canada. Association for Computational Linguistics.