Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text

Lukas Edman; Alexander Fraser

Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text

Correct Metadata for

Use this form to create a GitHub issue with structured data describing the correction. You will need a GitHub account. Once you create that issue, the correction will be reviewed by a staff member.

⚠️ Mobile Users: Submitting this form to create a new issue will only work with github.com, not the GitHub Mobile app.

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>. You may use ... for bold, ... for italic, ... for underline, <sc>...</sc> for small-caps, <tt>...<tt> for typewriter text, <url>...</url> for URLs, <a href=...> for hyperlinks, and <par/> for paragraph breaks.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

This work concerns our submission for the Multilingual track of the 2026 BabyLM Challenge. We improve on our baseline using data that has been summarized and translated by an LLM, showing that higher-quality data can come from synthetic sources. We further improve with 2 simple changes: using a larger model, and using a character-level vocabulary. Both of these changes require LoRA-style fine-tuning for realizing their full potential. We dub our final Qwen3-based model Qwenfant, which, at the time of submission, ranks 1st in the Multilingual track.

Anthology ID:: 2026.babylm-main.49
Volume:: Proceedings of the Second BabyLM Workshop
Month:: October
Year:: 2026
Address:: Budapest, Hungary
Editors:: Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Jaap Jumelet, Tal Linzen, Aaron Mueller, Suchir Salhan, Raj Sanjay Shah, Alex Warstadt, Ethan Gotlieb Wilcox
Venues:: BabyLM | WS
SIG:
Publisher:: Association for Computational Linguistics
Note:
Pages:: 624–630
Language:
URL:: https://aclanthology.org/2026.babylm-main.49/
DOI:
Bibkey:
Cite (ACL):: Lukas Edman and Alexander Fraser. 2026. Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text. In Proceedings of the Second BabyLM Workshop, pages 624–630, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):: Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text (Edman & Fraser, BabyLM 2026)
Copy Citation:
PDF:: https://aclanthology.org/2026.babylm-main.49.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{edman-fraser-2026-qwenfant,
    title = "Qwenfant: Train Big Character-level {B}aby{LM}s on Concise Parallel Text",
    author = "Edman, Lukas  and
      Fraser, Alexander",
    editor = "Choshen, Leshem  and
      Cotterell, Ryan  and
      Gul, Mustafa Omer  and
      Jumelet, Jaap  and
      Linzen, Tal  and
      Mueller, Aaron  and
      Salhan, Suchir  and
      Shah, Raj Sanjay  and
      Warstadt, Alex  and
      Wilcox, Ethan Gotlieb",
    booktitle = "Proceedings of the Second {B}aby{LM} Workshop",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.babylm-main.49/",
    pages = "624--630",
    ISBN = "979-8-89176-475-0",
    abstract = "This work concerns our submission for the Multilingual track of the 2026 BabyLM Challenge. We improve on our baseline using data that has been summarized and translated by an LLM, showing that higher-quality data can come from synthetic sources. We further improve with 2 simple changes: using a larger model, and using a character-level vocabulary. Both of these changes require LoRA-style fine-tuning for realizing their full potential. We dub our final Qwen3-based model Qwenfant, which, at the time of submission, ranks 1st in the Multilingual track."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="edman-fraser-2026-qwenfant">
    <titleInfo>
        <title>Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lukas</namePart>
        <namePart type="family">Edman</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alexander</namePart>
        <namePart type="family">Fraser</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Second BabyLM Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Leshem</namePart>
            <namePart type="family">Choshen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ryan</namePart>
            <namePart type="family">Cotterell</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mustafa</namePart>
            <namePart type="given">Omer</namePart>
            <namePart type="family">Gul</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jaap</namePart>
            <namePart type="family">Jumelet</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tal</namePart>
            <namePart type="family">Linzen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aaron</namePart>
            <namePart type="family">Mueller</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Suchir</namePart>
            <namePart type="family">Salhan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Raj</namePart>
            <namePart type="given">Sanjay</namePart>
            <namePart type="family">Shah</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alex</namePart>
            <namePart type="family">Warstadt</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ethan</namePart>
            <namePart type="given">Gotlieb</namePart>
            <namePart type="family">Wilcox</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-475-0</identifier>
    </relatedItem>
    <abstract>This work concerns our submission for the Multilingual track of the 2026 BabyLM Challenge. We improve on our baseline using data that has been summarized and translated by an LLM, showing that higher-quality data can come from synthetic sources. We further improve with 2 simple changes: using a larger model, and using a character-level vocabulary. Both of these changes require LoRA-style fine-tuning for realizing their full potential. We dub our final Qwen3-based model Qwenfant, which, at the time of submission, ranks 1st in the Multilingual track.</abstract>
    <identifier type="citekey">edman-fraser-2026-qwenfant</identifier>
    <location>
        <url>https://aclanthology.org/2026.babylm-main.49/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>624</start>
            <end>630</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text
%A Edman, Lukas
%A Fraser, Alexander
%Y Choshen, Leshem
%Y Cotterell, Ryan
%Y Gul, Mustafa Omer
%Y Jumelet, Jaap
%Y Linzen, Tal
%Y Mueller, Aaron
%Y Salhan, Suchir
%Y Shah, Raj Sanjay
%Y Warstadt, Alex
%Y Wilcox, Ethan Gotlieb
%S Proceedings of the Second BabyLM Workshop
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-475-0
%F edman-fraser-2026-qwenfant
%X This work concerns our submission for the Multilingual track of the 2026 BabyLM Challenge. We improve on our baseline using data that has been summarized and translated by an LLM, showing that higher-quality data can come from synthetic sources. We further improve with 2 simple changes: using a larger model, and using a character-level vocabulary. Both of these changes require LoRA-style fine-tuning for realizing their full potential. We dub our final Qwen3-based model Qwenfant, which, at the time of submission, ranks 1st in the Multilingual track.
%U https://aclanthology.org/2026.babylm-main.49/
%P 624-630

Download as File

Markdown (Informal)

[Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text](https://aclanthology.org/2026.babylm-main.49/) (Edman & Fraser, BabyLM 2026)

Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text (Edman & Fraser, BabyLM 2026)

ACL

Lukas Edman and Alexander Fraser. 2026. Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text. In Proceedings of the Second BabyLM Workshop, pages 624–630, Budapest, Hungary. Association for Computational Linguistics.