<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="faille-etal-2021-entity-based">
    <titleInfo>
        <title>Entity-Based Semantic Adequacy for Data-to-Text Generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Juliette</namePart>
        <namePart type="family">Faille</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Albert</namePart>
        <namePart type="family">Gatt</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Claire</namePart>
        <namePart type="family">Gardent</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2021</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marie-Francine</namePart>
            <namePart type="family">Moens</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Xuanjing</namePart>
            <namePart type="family">Huang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lucia</namePart>
            <namePart type="family">Specia</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Scott</namePart>
            <namePart type="given">Wen-tau</namePart>
            <namePart type="family">Yih</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Punta Cana, Dominican Republic</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>While powerful pre-trained language models have improved the fluency of text generation models, semantic adequacy -the ability to generate text that is semantically faithful to the input- remains an unsolved issue. In this paper, we introduce a novel automatic evaluation metric, Entity-Based Semantic Adequacy, which can be used to assess to what extent generation models that verbalise RDF (Resource Description Framework) graphs produce text that contains mentions of the entities occurring in the RDF input. This is important as RDF subject and object entities make up 2/3 of the input. We use our metric to compare 25 models from the WebNLG Shared Tasks and we examine correlation with results from human evaluations of semantic adequacy. We show that while our metric correlates with human evaluation scores, this correlation varies with the specifics of the human evaluation setup. This suggests that in order to measure the entity-based adequacy of generated texts, an automatic metric such as the one proposed here might be more reliable, as less subjective and more focused on correct verbalisation of the input, than human evaluation measures.</abstract>
    <identifier type="citekey">faille-etal-2021-entity-based</identifier>
    <identifier type="doi">10.18653/v1/2021.findings-emnlp.132</identifier>
    <location>
        <url>https://aclanthology.org/2021.findings-emnlp.132</url>
    </location>
    <part>
        <date>2021-11</date>
        <extent unit="page">
            <start>1530</start>
            <end>1540</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Entity-Based Semantic Adequacy for Data-to-Text Generation
%A Faille, Juliette
%A Gatt, Albert
%A Gardent, Claire
%Y Moens, Marie-Francine
%Y Huang, Xuanjing
%Y Specia, Lucia
%Y Yih, Scott Wen-tau
%S Findings of the Association for Computational Linguistics: EMNLP 2021
%D 2021
%8 November
%I Association for Computational Linguistics
%C Punta Cana, Dominican Republic
%F faille-etal-2021-entity-based
%X While powerful pre-trained language models have improved the fluency of text generation models, semantic adequacy -the ability to generate text that is semantically faithful to the input- remains an unsolved issue. In this paper, we introduce a novel automatic evaluation metric, Entity-Based Semantic Adequacy, which can be used to assess to what extent generation models that verbalise RDF (Resource Description Framework) graphs produce text that contains mentions of the entities occurring in the RDF input. This is important as RDF subject and object entities make up 2/3 of the input. We use our metric to compare 25 models from the WebNLG Shared Tasks and we examine correlation with results from human evaluations of semantic adequacy. We show that while our metric correlates with human evaluation scores, this correlation varies with the specifics of the human evaluation setup. This suggests that in order to measure the entity-based adequacy of generated texts, an automatic metric such as the one proposed here might be more reliable, as less subjective and more focused on correct verbalisation of the input, than human evaluation measures.
%R 10.18653/v1/2021.findings-emnlp.132
%U https://aclanthology.org/2021.findings-emnlp.132
%U https://doi.org/10.18653/v1/2021.findings-emnlp.132
%P 1530-1540

Markdown (Informal)

[Entity-Based Semantic Adequacy for Data-to-Text Generation](https://aclanthology.org/2021.findings-emnlp.132) (Faille et al., Findings 2021)

• Entity-Based Semantic Adequacy for Data-to-Text Generation (Faille et al., Findings 2021)

ACL

• Juliette Faille, Albert Gatt, and Claire Gardent. 2021. Entity-Based Semantic Adequacy for Data-to-Text Generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1530–1540, Punta Cana, Dominican Republic. Association for Computational Linguistics.