<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="k-h-de-nardi-2026-rethinking">
    <titleInfo>
        <title>Rethinking Polarity Detection: When BPE Fails Across Scripts</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Manodyna</namePart>
        <namePart type="family">K H</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Luc</namePart>
        <namePart type="family">De Nardi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-03</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Rabat, Morocco</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Multilingual evaluation often relies on language coverage or translated benchmarks, implicitly assuming that subword tokenization behaves comparably across scripts. In mixed-script settings, this assumption breaks down. We examine this effect using polarity detection as a case study, comparing Orthographic Syllable Pair Encoding (OSPE) and Byte Pair Encoding (BPE) under identical architectures, data, and training conditions on SemEval Task 9, which spans Devanagari, Perso-Arabic, and Latin scripts. OSPE is applied to Hindi, Nepali, Urdu, and Arabic, while BPE is retained for English. We find that BPE systematically underestimates performance in abugida and abjad scripts, producing fragmented representations, unstable optimization, and drops of up to 27 macro-F1 points for Nepali, while English remains largely unaffected. Script-aware segmentation preserves orthographic structure, stabilizes training, and improves cross-language comparability without additional data or model scaling, highlighting tokenization as a latent but consequential evaluation decision in multilingual benchmarks. While the analysis spans multiple scripts, we place particular emphasis on Arabic and Perso-Arabic languages, where frequency-driven tokenization most severely disrupts orthographic and morphological structure.</abstract>
    <identifier type="citekey">k-h-de-nardi-2026-rethinking</identifier>
    <location>
        <url>https://aclanthology.org/2026.abjadnlp-1.2/</url>
    </location>
    <part>
        <date>2026-03</date>
        <extent unit="page">
            <start>6</start>
            <end>14</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Rethinking Polarity Detection: When BPE Fails Across Scripts
%A K H, Manodyna
%A De Nardi, Luc
%S Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script
%D 2026
%8 March
%I Association for Computational Linguistics
%C Rabat, Morocco
%F k-h-de-nardi-2026-rethinking
%X Multilingual evaluation often relies on language coverage or translated benchmarks, implicitly assuming that subword tokenization behaves comparably across scripts. In mixed-script settings, this assumption breaks down. We examine this effect using polarity detection as a case study, comparing Orthographic Syllable Pair Encoding (OSPE) and Byte Pair Encoding (BPE) under identical architectures, data, and training conditions on SemEval Task 9, which spans Devanagari, Perso-Arabic, and Latin scripts. OSPE is applied to Hindi, Nepali, Urdu, and Arabic, while BPE is retained for English. We find that BPE systematically underestimates performance in abugida and abjad scripts, producing fragmented representations, unstable optimization, and drops of up to 27 macro-F1 points for Nepali, while English remains largely unaffected. Script-aware segmentation preserves orthographic structure, stabilizes training, and improves cross-language comparability without additional data or model scaling, highlighting tokenization as a latent but consequential evaluation decision in multilingual benchmarks. While the analysis spans multiple scripts, we place particular emphasis on Arabic and Perso-Arabic languages, where frequency-driven tokenization most severely disrupts orthographic and morphological structure.
%U https://aclanthology.org/2026.abjadnlp-1.2/
%P 6-14

Markdown (Informal)

[Rethinking Polarity Detection: When BPE Fails Across Scripts](https://aclanthology.org/2026.abjadnlp-1.2/) (K H & De Nardi, AbjadNLP 2026)

• Rethinking Polarity Detection: When BPE Fails Across Scripts (K H & De Nardi, AbjadNLP 2026)

ACL

• Manodyna K H and Luc De Nardi. 2026. Rethinking Polarity Detection: When BPE Fails Across Scripts. In Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script, pages 6–14, Rabat, Morocco. Association for Computational Linguistics.