<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tanase-pelican-2026-supratok">
    <titleInfo>
        <title>SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Andrei-Valentin</namePart>
        <namePart type="family">Tănase</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Elena</namePart>
        <namePart type="family">Pelican</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Tokenization remains a persistent bottleneck in language modeling, especially when vocabulary learning is limited by whitespace boundaries. We present SupraTok, a tokenizer that crosses whitespace boundaries using three modular components: optional entropy-based data curation, staged curriculum training with PMI-guided candidate search, and multilingual script handling. At 100k vocabulary on the same unfiltered training data, SupraTok improves compression over standard BPE by 17.5% and over the official SuperBPE implementation by 1.8%, while training 2.1× faster than SuperBPE. Across 50k–300k vocabularies in the same matched setting, Supra-Tok remains ahead of SuperBPE by 1.8%–8.6%. We evaluate entropy filtering separately as a pipeline step: at 100k vocabulary it raises SupraTok from 5.78 to 5.99 C/T, while matched controls show a smaller gain for SuperBPE and almost no change for SP-BPE-CrossBoundary. On FLORES-200 across 14 languages, SupraTok yields a macro-averaged 34.9% relative gain over the BPE baseline. In separate downstream experiments with matched compute and fixed token budgets, using 12L-768d and 24L-1024d GPT-2-style backbones with 256k vocabularies, SupraTok improves HellaSwag and MMLU. Overall, these results show that crossing whitespace boundaries gives consistent compression gains under controlled public-data comparisons, while optional entropy filtering provides a separate pipeline benefit.</abstract>
    <identifier type="citekey">tanase-pelican-2026-supratok</identifier>
    <identifier type="doi">10.1162/tacl.a.774</identifier>
    <location>
        <url>https://aclanthology.org/2026.tacl-1.80/</url>
    </location>
    <part>
        <date>2026</date>
        <detail type="volume"><number>14</number></detail>
        <extent unit="page">
            <start>1787</start>
            <end>1802</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance
%A Tănase, Andrei-Valentin
%A Pelican, Elena
%J Transactions of the Association for Computational Linguistics
%D 2026
%V 14
%I MIT Press
%C Cambridge, MA
%F tanase-pelican-2026-supratok
%X Tokenization remains a persistent bottleneck in language modeling, especially when vocabulary learning is limited by whitespace boundaries. We present SupraTok, a tokenizer that crosses whitespace boundaries using three modular components: optional entropy-based data curation, staged curriculum training with PMI-guided candidate search, and multilingual script handling. At 100k vocabulary on the same unfiltered training data, SupraTok improves compression over standard BPE by 17.5% and over the official SuperBPE implementation by 1.8%, while training 2.1× faster than SuperBPE. Across 50k–300k vocabularies in the same matched setting, Supra-Tok remains ahead of SuperBPE by 1.8%–8.6%. We evaluate entropy filtering separately as a pipeline step: at 100k vocabulary it raises SupraTok from 5.78 to 5.99 C/T, while matched controls show a smaller gain for SuperBPE and almost no change for SP-BPE-CrossBoundary. On FLORES-200 across 14 languages, SupraTok yields a macro-averaged 34.9% relative gain over the BPE baseline. In separate downstream experiments with matched compute and fixed token budgets, using 12L-768d and 24L-1024d GPT-2-style backbones with 256k vocabularies, SupraTok improves HellaSwag and MMLU. Overall, these results show that crossing whitespace boundaries gives consistent compression gains under controlled public-data comparisons, while optional entropy filtering provides a separate pipeline benefit.
%R 10.1162/tacl.a.774
%U https://aclanthology.org/2026.tacl-1.80/
%U https://doi.org/10.1162/tacl.a.774
%P 1787-1802

Markdown (Informal)

[SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance](https://aclanthology.org/2026.tacl-1.80/) (Tănase & Pelican, TACL 2026)

• SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance (Tănase & Pelican, TACL 2026)

ACL

• Andrei-Valentin Tănase and Elena Pelican. 2026. SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance. Transactions of the Association for Computational Linguistics, 14:1787–1802.