<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zai-klampanos-2026-peek2">
    <titleInfo>
        <title>Peek2: Regex-free Byte-level Byte-Pair Encoding Pretokenizer for LLM Inference on Edge Devices</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Liu</namePart>
        <namePart type="family">Zai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Iraklis</namePart>
        <namePart type="given">A</namePart>
        <namePart type="family">Klampanos</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Santosh</namePart>
            <namePart type="family">T.Y.S.S.</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="given">Diego</namePart>
            <namePart type="family">Rodriguez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ona</namePart>
            <namePart type="family">de Gibert</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">San Diego, California, United States</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-393-7</identifier>
    </relatedItem>
    <abstract>Pretokenization is a crucial, sequential pass in Byte-level BPE tokenizers, yet little work has been done to optimize it for edge-side inference. Our proposed new implementation, Peek2, serves as a drop-in replacement for cl100k-like pretokenizers used in GPT-3, LLaMa-3, and Qwen-2.5. After breaking down and analyzing the logic of the original cl100k pretokenizer, we introduced a new pretokenization algorithm with linear time complexity and constant, trivial memory usage, suited for edge scenarios. Test results show that it increases microbenchmarking throughput by up to 2.48\times and delivers a 1.14\times improvement in overall throughput across the entire Byte-level BPE encoding process, depending on the dataset, while providing identical results as the baseline Regex-based tokenizer.</abstract>
    <identifier type="citekey">zai-klampanos-2026-peek2</identifier>
    <location>
        <url>https://aclanthology.org/2026.acl-srw.10/</url>
    </location>
    <part>
        <date>2026-07</date>
        <extent unit="page">
            <start>123</start>
            <end>128</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Peek2: Regex-free Byte-level Byte-Pair Encoding Pretokenizer for LLM Inference on Edge Devices
%A Zai, Liu
%A Klampanos, Iraklis A.
%Y T.Y.S.S., Santosh
%Y Rodriguez, Juan Diego
%Y de Gibert, Ona
%S Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
%D 2026
%8 July
%I Association for Computational Linguistics
%C San Diego, California, United States
%@ 979-8-89176-393-7
%F zai-klampanos-2026-peek2
%X Pretokenization is a crucial, sequential pass in Byte-level BPE tokenizers, yet little work has been done to optimize it for edge-side inference. Our proposed new implementation, Peek2, serves as a drop-in replacement for cl100k-like pretokenizers used in GPT-3, LLaMa-3, and Qwen-2.5. After breaking down and analyzing the logic of the original cl100k pretokenizer, we introduced a new pretokenization algorithm with linear time complexity and constant, trivial memory usage, suited for edge scenarios. Test results show that it increases microbenchmarking throughput by up to 2.48\times and delivers a 1.14\times improvement in overall throughput across the entire Byte-level BPE encoding process, depending on the dataset, while providing identical results as the baseline Regex-based tokenizer.
%U https://aclanthology.org/2026.acl-srw.10/
%P 123-128

Markdown (Informal)

[Peek2: Regex-free Byte-level Byte-Pair Encoding Pretokenizer for LLM Inference on Edge Devices](https://aclanthology.org/2026.acl-srw.10/) (Zai & Klampanos, ACL 2026)

• Peek2: Regex-free Byte-level Byte-Pair Encoding Pretokenizer for LLM Inference on Edge Devices (Zai & Klampanos, ACL 2026)

ACL

• Liu Zai and Iraklis A. Klampanos. 2026. Peek2: Regex-free Byte-level Byte-Pair Encoding Pretokenizer for LLM Inference on Edge Devices. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), pages 123–128, San Diego, California, United States. Association for Computational Linguistics.