<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="gurgurov-etal-2024-adapting">
    <titleInfo>
        <title>Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Daniil</namePart>
        <namePart type="family">Gurgurov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mareike</namePart>
        <namePart type="family">Hartmann</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Simon</namePart>
        <namePart type="family">Ostermann</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 1st Workshop on Knowledge Graphs and Large Language Models (KaLLM 2024)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Russa</namePart>
            <namePart type="family">Biswas</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lucie-Aimée</namePart>
            <namePart type="family">Kaffee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Oshin</namePart>
            <namePart type="family">Agarwal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Pasquale</namePart>
            <namePart type="family">Minervini</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sameer</namePart>
            <namePart type="family">Singh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Gerard</namePart>
            <namePart type="family">de Melo</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper explores the integration of graph knowledge from linguistic ontologies into multilingual Large Language Models (LLMs) using adapters to improve performance for low-resource languages (LRLs) in sentiment analysis (SA) and named entity recognition (NER). Building upon successful parameter-efficient fine-tuning techniques, such as K-ADAPTER and MAD-X, we propose a similar approach for incorporating knowledge from multilingual graphs, connecting concepts in various languages with each other through linguistic relationships, into multilingual LLMs for LRLs. Specifically, we focus on eight LRLs — Maltese, Bulgarian, Indonesian, Nepali, Javanese, Uyghur, Tibetan, and Sinhala — and employ language-specific adapters fine-tuned on data extracted from the language-specific section of ConceptNet, aiming to enable knowledge transfer across the languages covered by the knowledge graph. We compare various fine-tuning objectives, including standard Masked Language Modeling (MLM), MLM with full-word masking, and MLM with targeted masking, to analyze their effectiveness in learning and integrating the extracted graph data. Through empirical evaluation on language-specific tasks, we assess how structured graph knowledge affects the performance of multilingual LLMs for LRLs in SA and NER, providing insights into the potential benefits of adapting language models for low-resource scenarios.</abstract>
    <identifier type="citekey">gurgurov-etal-2024-adapting</identifier>
    <identifier type="doi">10.18653/v1/2024.kallm-1.7</identifier>
    <location>
        <url>https://aclanthology.org/2024.kallm-1.7</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>63</start>
            <end>74</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters
%A Gurgurov, Daniil
%A Hartmann, Mareike
%A Ostermann, Simon
%Y Biswas, Russa
%Y Kaffee, Lucie-Aimée
%Y Agarwal, Oshin
%Y Minervini, Pasquale
%Y Singh, Sameer
%Y de Melo, Gerard
%S Proceedings of the 1st Workshop on Knowledge Graphs and Large Language Models (KaLLM 2024)
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F gurgurov-etal-2024-adapting
%X This paper explores the integration of graph knowledge from linguistic ontologies into multilingual Large Language Models (LLMs) using adapters to improve performance for low-resource languages (LRLs) in sentiment analysis (SA) and named entity recognition (NER). Building upon successful parameter-efficient fine-tuning techniques, such as K-ADAPTER and MAD-X, we propose a similar approach for incorporating knowledge from multilingual graphs, connecting concepts in various languages with each other through linguistic relationships, into multilingual LLMs for LRLs. Specifically, we focus on eight LRLs — Maltese, Bulgarian, Indonesian, Nepali, Javanese, Uyghur, Tibetan, and Sinhala — and employ language-specific adapters fine-tuned on data extracted from the language-specific section of ConceptNet, aiming to enable knowledge transfer across the languages covered by the knowledge graph. We compare various fine-tuning objectives, including standard Masked Language Modeling (MLM), MLM with full-word masking, and MLM with targeted masking, to analyze their effectiveness in learning and integrating the extracted graph data. Through empirical evaluation on language-specific tasks, we assess how structured graph knowledge affects the performance of multilingual LLMs for LRLs in SA and NER, providing insights into the potential benefits of adapting language models for low-resource scenarios.
%R 10.18653/v1/2024.kallm-1.7
%U https://aclanthology.org/2024.kallm-1.7
%U https://doi.org/10.18653/v1/2024.kallm-1.7
%P 63-74

Markdown (Informal)

[Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters](https://aclanthology.org/2024.kallm-1.7) (Gurgurov et al., KaLLM-WS 2024)

• Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters (Gurgurov et al., KaLLM-WS 2024)

ACL

• Daniil Gurgurov, Mareike Hartmann, and Simon Ostermann. 2024. Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters. In Proceedings of the 1st Workshop on Knowledge Graphs and Large Language Models (KaLLM 2024), pages 63–74, Bangkok, Thailand. Association for Computational Linguistics.