<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="talmor-etal-2017-evaluating">
    <titleInfo>
        <title>Evaluating Semantic Parsing against a Simple Web-based Question Answering Model</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Alon</namePart>
        <namePart type="family">Talmor</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mor</namePart>
        <namePart type="family">Geva</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jonathan</namePart>
        <namePart type="family">Berant</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Nancy</namePart>
            <namePart type="family">Ide</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aurélie</namePart>
            <namePart type="family">Herbelot</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lluís</namePart>
            <namePart type="family">Màrquez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vancouver, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Semantic parsing shines at analyzing complex natural language that involves composition and computation over multiple pieces of evidence. However, datasets for semantic parsing contain many factoid questions that can be answered from a single web document. In this paper, we propose to evaluate semantic parsing-based question answering models by comparing them to a question answering baseline that queries the web and extracts the answer only from web snippets, without access to the target knowledge-base. We investigate this approach on COMPLEXQUESTIONS, a dataset designed to focus on compositional language, and find that our model obtains reasonable performance (∼35 F1 compared to 41 F1 of state-of-the-art). We find in our analysis that our model performs well on complex questions involving conjunctions, but struggles on questions that involve relation composition and superlatives.</abstract>
    <identifier type="citekey">talmor-etal-2017-evaluating</identifier>
    <identifier type="doi">10.18653/v1/S17-1020</identifier>
    <location>
        <url>https://aclanthology.org/S17-1020</url>
    </location>
    <part>
        <date>2017-08</date>
        <extent unit="page">
            <start>161</start>
            <end>167</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Evaluating Semantic Parsing against a Simple Web-based Question Answering Model
%A Talmor, Alon
%A Geva, Mor
%A Berant, Jonathan
%Y Ide, Nancy
%Y Herbelot, Aurélie
%Y Màrquez, Lluís
%S Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017)
%D 2017
%8 August
%I Association for Computational Linguistics
%C Vancouver, Canada
%F talmor-etal-2017-evaluating
%X Semantic parsing shines at analyzing complex natural language that involves composition and computation over multiple pieces of evidence. However, datasets for semantic parsing contain many factoid questions that can be answered from a single web document. In this paper, we propose to evaluate semantic parsing-based question answering models by comparing them to a question answering baseline that queries the web and extracts the answer only from web snippets, without access to the target knowledge-base. We investigate this approach on COMPLEXQUESTIONS, a dataset designed to focus on compositional language, and find that our model obtains reasonable performance (∼35 F1 compared to 41 F1 of state-of-the-art). We find in our analysis that our model performs well on complex questions involving conjunctions, but struggles on questions that involve relation composition and superlatives.
%R 10.18653/v1/S17-1020
%U https://aclanthology.org/S17-1020
%U https://doi.org/10.18653/v1/S17-1020
%P 161-167

Markdown (Informal)

[Evaluating Semantic Parsing against a Simple Web-based Question Answering Model](https://aclanthology.org/S17-1020) (Talmor et al., *SEM 2017)

• Evaluating Semantic Parsing against a Simple Web-based Question Answering Model (Talmor et al., *SEM 2017)

ACL

• Alon Talmor, Mor Geva, and Jonathan Berant. 2017. Evaluating Semantic Parsing against a Simple Web-based Question Answering Model. In Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 161–167, Vancouver, Canada. Association for Computational Linguistics.