Efficient methods for building LLMs for low-resourced languages

Nalin Kumar

Efficient methods for building LLMs for low-resourced languages

Correct Metadata for

Use this form to create a GitHub issue with structured data describing the correction. You will need a GitHub account. Once you create that issue, the correction will be reviewed by a staff member.

⚠️ Mobile Users: Submitting this form to create a new issue will only work with github.com, not the GitHub Mobile app.

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>. You may use ... for bold, ... for italic, ... for underline, <sc>...</sc> for small-caps, <tt>...<tt> for typewriter text, <url>...</url> for URLs, <a href=...> for hyperlinks, and <par/> for paragraph breaks.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

Large Language Models (LLMs) excel in many NLP tasks but remain biased toward high-resource languages. This position paper discusses the author’s current findings on efficient strategies for low-resource settings: (i) modular training, where only non-embedding parameters are tuned after learning language-specific tokenizers and embeddings, and (ii) artificial language initialization, which leverages structurally biased synthetic languages for faster, parameter-efficient pretraining. The paper also shares plans for future research and topics that the author would like to discuss during the round-table.

Anthology ID:: 2025.ynlg-main.5
Volume:: Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation
Month:: October
Year:: 2025
Address:: Hanoi, Vietnam
Editors:: Alyssa Allen, Nils Feldhus, Rudali Huidrom, Michela Lorandi, Adarsa Sivaprasad, Patrícia Schmidtová
Venue:: YNLG
SIG:: SIGGEN
Publisher:: Association for Computational Linguistics
Note:
Pages:: 21–23
Language:
URL:: https://aclanthology.org/2025.ynlg-main.5/
DOI:
Bibkey:
Cite (ACL):: Nalin Kumar. 2025. Efficient methods for building LLMs for low-resourced languages. In Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation, pages 21–23, Hanoi, Vietnam. Association for Computational Linguistics.
Cite (Informal):: Efficient methods for building LLMs for low-resourced languages (Kumar, YNLG 2025)
Copy Citation:
PDF:: https://aclanthology.org/2025.ynlg-main.5.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{kumar-2025-efficient,
    title = "Efficient methods for building {LLM}s for low-resourced languages",
    author = "Kumar, Nalin",
    editor = "Allen, Alyssa  and
      Feldhus, Nils  and
      Huidrom, Rudali  and
      Lorandi, Michela  and
      Sivaprasad, Adarsa  and
      Schmidtov{\'a}, Patr{\'i}cia",
    booktitle = "Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation",
    month = oct,
    year = "2025",
    address = "Hanoi, Vietnam",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.ynlg-main.5/",
    pages = "21--23",
    abstract = "Large Language Models (LLMs) excel in many NLP tasks but remain biased toward high-resource languages. This position paper discusses the author{'}s current findings on efficient strategies for low-resource settings: (i) modular training, where only non-embedding parameters are tuned after learning language-specific tokenizers and embeddings, and (ii) artificial language initialization, which leverages structurally biased synthetic languages for faster, parameter-efficient pretraining. The paper also shares plans for future research and topics that the author would like to discuss during the round-table."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kumar-2025-efficient">
    <titleInfo>
        <title>Efficient methods for building LLMs for low-resourced languages</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Nalin</namePart>
        <namePart type="family">Kumar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Alyssa</namePart>
            <namePart type="family">Allen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nils</namePart>
            <namePart type="family">Feldhus</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rudali</namePart>
            <namePart type="family">Huidrom</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Michela</namePart>
            <namePart type="family">Lorandi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Adarsa</namePart>
            <namePart type="family">Sivaprasad</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Patrícia</namePart>
            <namePart type="family">Schmidtová</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hanoi, Vietnam</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large Language Models (LLMs) excel in many NLP tasks but remain biased toward high-resource languages. This position paper discusses the author’s current findings on efficient strategies for low-resource settings: (i) modular training, where only non-embedding parameters are tuned after learning language-specific tokenizers and embeddings, and (ii) artificial language initialization, which leverages structurally biased synthetic languages for faster, parameter-efficient pretraining. The paper also shares plans for future research and topics that the author would like to discuss during the round-table.</abstract>
    <identifier type="citekey">kumar-2025-efficient</identifier>
    <location>
        <url>https://aclanthology.org/2025.ynlg-main.5/</url>
    </location>
    <part>
        <date>2025-10</date>
        <extent unit="page">
            <start>21</start>
            <end>23</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T Efficient methods for building LLMs for low-resourced languages
%A Kumar, Nalin
%Y Allen, Alyssa
%Y Feldhus, Nils
%Y Huidrom, Rudali
%Y Lorandi, Michela
%Y Sivaprasad, Adarsa
%Y Schmidtová, Patrícia
%S Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation
%D 2025
%8 October
%I Association for Computational Linguistics
%C Hanoi, Vietnam
%F kumar-2025-efficient
%X Large Language Models (LLMs) excel in many NLP tasks but remain biased toward high-resource languages. This position paper discusses the author’s current findings on efficient strategies for low-resource settings: (i) modular training, where only non-embedding parameters are tuned after learning language-specific tokenizers and embeddings, and (ii) artificial language initialization, which leverages structurally biased synthetic languages for faster, parameter-efficient pretraining. The paper also shares plans for future research and topics that the author would like to discuss during the round-table.
%U https://aclanthology.org/2025.ynlg-main.5/
%P 21-23

Download as File

Markdown (Informal)

[Efficient methods for building LLMs for low-resourced languages](https://aclanthology.org/2025.ynlg-main.5/) (Kumar, YNLG 2025)

Efficient methods for building LLMs for low-resourced languages (Kumar, YNLG 2025)

ACL

Nalin Kumar. 2025. Efficient methods for building LLMs for low-resourced languages. In Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation, pages 21–23, Hanoi, Vietnam. Association for Computational Linguistics.