<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="merrill-etal-2022-entailment">
    <titleInfo>
        <title>Entailment Semantics Can Be Extracted from an Ideal Language Model</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">William</namePart>
        <namePart type="family">Merrill</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alex</namePart>
        <namePart type="family">Warstadt</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tal</namePart>
        <namePart type="family">Linzen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Antske</namePart>
            <namePart type="family">Fokkens</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates (Hybrid)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. We prove that entailment judgments between sentences can be extracted from an ideal language model that has perfectly learned its target distribution, assuming the training sentences are generated by Gricean agents, i.e., agents who follow fundamental principles of communication from the linguistic theory of pragmatics. We also show entailment judgments can be decoded from the predictions of a language model trained on such Gricean data. Our results reveal a pathway for understanding the semantic information encoded in unlabeled linguistic data and a potential framework for extracting semantics from language models.</abstract>
    <identifier type="citekey">merrill-etal-2022-entailment</identifier>
    <identifier type="doi">10.18653/v1/2022.conll-1.13</identifier>
    <location>
        <url>https://aclanthology.org/2022.conll-1.13</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>176</start>
            <end>193</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Entailment Semantics Can Be Extracted from an Ideal Language Model
%A Merrill, William
%A Warstadt, Alex
%A Linzen, Tal
%Y Fokkens, Antske
%Y Srikumar, Vivek
%S Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL)
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates (Hybrid)
%F merrill-etal-2022-entailment
%X Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. We prove that entailment judgments between sentences can be extracted from an ideal language model that has perfectly learned its target distribution, assuming the training sentences are generated by Gricean agents, i.e., agents who follow fundamental principles of communication from the linguistic theory of pragmatics. We also show entailment judgments can be decoded from the predictions of a language model trained on such Gricean data. Our results reveal a pathway for understanding the semantic information encoded in unlabeled linguistic data and a potential framework for extracting semantics from language models.
%R 10.18653/v1/2022.conll-1.13
%U https://aclanthology.org/2022.conll-1.13
%U https://doi.org/10.18653/v1/2022.conll-1.13
%P 176-193

Markdown (Informal)

[Entailment Semantics Can Be Extracted from an Ideal Language Model](https://aclanthology.org/2022.conll-1.13) (Merrill et al., CoNLL 2022)

• Entailment Semantics Can Be Extracted from an Ideal Language Model (Merrill et al., CoNLL 2022)

ACL

• William Merrill, Alex Warstadt, and Tal Linzen. 2022. Entailment Semantics Can Be Extracted from an Ideal Language Model. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 176–193, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.