<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="charpentier-etal-2023-brent">
    <titleInfo>
        <title>BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lucas</namePart>
        <namePart type="family">Charpentier</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sondre</namePart>
        <namePart type="family">Wold</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">David</namePart>
        <namePart type="family">Samuel</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Egil</namePart>
        <namePart type="family">Rønningstad</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Tanel</namePart>
            <namePart type="family">Alumäe</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mark</namePart>
            <namePart type="family">Fishel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>University of Tartu Library</publisher>
            <place>
                <placeTerm type="text">Tórshavn, Faroe Islands</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Retrieval-based language models are increasingly employed in question-answering tasks. These models search in a corpus of documents for relevant information instead of having all factual knowledge stored in its parameters, thereby enhancing efficiency, transparency, and adaptability. We develop the first Norwegian retrieval-based model by adapting the REALM framework and evaluate it on various tasks. After training, we also separate the language model, which we call the reader, from the retriever components, and show that this can be fine-tuned on a range of downstream tasks. Results show that retrieval augmented language modeling improves the reader’s performance on extractive question-answering, suggesting that this type of training improves language models’ general ability to use context and that this does not happen at the expense of other abilities such as part-of-speech tagging, dependency parsing, named entity recognition, and lemmatization. Code, trained models, and data are made publicly available.</abstract>
    <identifier type="citekey">charpentier-etal-2023-brent</identifier>
    <location>
        <url>https://aclanthology.org/2023.nodalida-1.21</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>202</start>
            <end>214</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer
%A Charpentier, Lucas
%A Wold, Sondre
%A Samuel, David
%A Rønningstad, Egil
%Y Alumäe, Tanel
%Y Fishel, Mark
%S Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)
%D 2023
%8 May
%I University of Tartu Library
%C Tórshavn, Faroe Islands
%F charpentier-etal-2023-brent
%X Retrieval-based language models are increasingly employed in question-answering tasks. These models search in a corpus of documents for relevant information instead of having all factual knowledge stored in its parameters, thereby enhancing efficiency, transparency, and adaptability. We develop the first Norwegian retrieval-based model by adapting the REALM framework and evaluate it on various tasks. After training, we also separate the language model, which we call the reader, from the retriever components, and show that this can be fine-tuned on a range of downstream tasks. Results show that retrieval augmented language modeling improves the reader’s performance on extractive question-answering, suggesting that this type of training improves language models’ general ability to use context and that this does not happen at the expense of other abilities such as part-of-speech tagging, dependency parsing, named entity recognition, and lemmatization. Code, trained models, and data are made publicly available.
%U https://aclanthology.org/2023.nodalida-1.21
%P 202-214

Markdown (Informal)

[BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer](https://aclanthology.org/2023.nodalida-1.21) (Charpentier et al., NoDaLiDa 2023)

• BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer (Charpentier et al., NoDaLiDa 2023)

ACL

• Lucas Charpentier, Sondre Wold, David Samuel, and Egil Rønningstad. 2023. BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 202–214, Tórshavn, Faroe Islands. University of Tartu Library.