<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lovelace-rose-2022-framework">
    <titleInfo>
        <title>A Framework for Adapting Pre-Trained Language Models to Knowledge Graph Completion</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Justin</namePart>
        <namePart type="family">Lovelace</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Carolyn</namePart>
        <namePart type="family">Rosé</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent work has demonstrated that entity representations can be extracted from pre-trained language models to develop knowledge graph completion models that are more robust to the naturally occurring sparsity found in knowledge graphs. In this work, we conduct a comprehensive exploration of how to best extract and incorporate those embeddings into knowledge graph completion models. We explore the suitability of the extracted embeddings for direct use in entity ranking and introduce both unsupervised and supervised processing methods that can lead to improved downstream performance. We then introduce supervised embedding extraction methods that can extract more informative representations. We then synthesize our findings and develop a knowledge graph completion model that significantly outperforms recent neural models.</abstract>
    <identifier type="citekey">lovelace-rose-2022-framework</identifier>
    <identifier type="doi">10.18653/v1/2022.emnlp-main.398</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.398</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>5937</start>
            <end>5955</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Framework for Adapting Pre-Trained Language Models to Knowledge Graph Completion
%A Lovelace, Justin
%A Rosé, Carolyn
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F lovelace-rose-2022-framework
%X Recent work has demonstrated that entity representations can be extracted from pre-trained language models to develop knowledge graph completion models that are more robust to the naturally occurring sparsity found in knowledge graphs. In this work, we conduct a comprehensive exploration of how to best extract and incorporate those embeddings into knowledge graph completion models. We explore the suitability of the extracted embeddings for direct use in entity ranking and introduce both unsupervised and supervised processing methods that can lead to improved downstream performance. We then introduce supervised embedding extraction methods that can extract more informative representations. We then synthesize our findings and develop a knowledge graph completion model that significantly outperforms recent neural models.
%R 10.18653/v1/2022.emnlp-main.398
%U https://aclanthology.org/2022.emnlp-main.398
%U https://doi.org/10.18653/v1/2022.emnlp-main.398
%P 5937-5955

Markdown (Informal)

[A Framework for Adapting Pre-Trained Language Models to Knowledge Graph Completion](https://aclanthology.org/2022.emnlp-main.398) (Lovelace & Rosé, EMNLP 2022)

• A Framework for Adapting Pre-Trained Language Models to Knowledge Graph Completion (Lovelace & Rosé, EMNLP 2022)

ACL

• Justin Lovelace and Carolyn Rosé. 2022. A Framework for Adapting Pre-Trained Language Models to Knowledge Graph Completion. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5937–5955, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.