CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios

Michal Tichý; Jindřich Libovický

CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios

Correct Metadata for

Use this form to create a GitHub issue with structured data describing the correction. You will need a GitHub account. Once you create that issue, the correction will be reviewed by a staff member.

⚠️ Mobile Users: Submitting this form to create a new issue will only work with github.com, not the GitHub Mobile app.

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>. You may use ... for bold, ... for italic, ... for underline, <sc>...</sc> for small-caps, <tt>...<tt> for typewriter text, <url>...</url> for URLs, <a href=...> for hyperlinks, and <par/> for paragraph breaks.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

We present CHALIS (Challenging Language Identification Samples), a new benchmark dataset explicitly designed to address difficult cases in language identification: cousin languages and orthographic noise. Our dataset has two parts: First, we collected sentences shared across mutually intelligible language pairs (Czech/Slovak, Spanish/Catalan, Portuguese/Galician, Danish/Norwegian). The second part tests for orthography noise: we transliterate text across multiple scripts, remove diacritics, simulate homoglyph attacks, and use Internet slang. We evaluate four widely used language identification systems on CHALIS and demonstrate that all struggle substantially in these scenarios, especially on lower-resource languages within cousin pairs and on transliterated input.

Anthology ID:: 2026.wac-1.1
Volume:: Proceedings of the 13th Web-as-Corpus Workshop
Month:: October
Year:: 2026
Address:: Budapest, Hungary
Editors:: Pedro Ortiz Suarez, Laurie Burchell, Nikola Ljubešić, Yves Scherrer, Veronika Laippala, Vuk Dinić, Thom Vaughan
Venues:: WAC | WS
SIG:: SIGWAC
Publisher:: Association for Computational Linguistics
Note:
Pages:: 1–7
Language:
URL:: https://aclanthology.org/2026.wac-1.1/
DOI:
Bibkey:
Cite (ACL):: Michal Tichý and Jindřich Libovický. 2026. CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios. In Proceedings of the 13th Web-as-Corpus Workshop, pages 1–7, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):: CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios (Tichý & Libovický, WAC 2026)
Copy Citation:
PDF:: https://aclanthology.org/2026.wac-1.1.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{tichy-libovicky-2026-chalis,
    title = "{CHALIS}: A Challenge Dataset for Language Identification in Difficult Scenarios",
    author = "Tich{\'y}, Michal  and
      Libovick{\'y}, Jind{\v{r}}ich",
    editor = "Ortiz Suarez, Pedro  and
      Burchell, Laurie  and
      Ljube{\v{s}}i{\'c}, Nikola  and
      Scherrer, Yves  and
      Laippala, Veronika  and
      Dini{\'c}, Vuk  and
      Vaughan, Thom",
    booktitle = "Proceedings of the 13th Web-as-Corpus Workshop",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.wac-1.1/",
    pages = "1--7",
    ISBN = "979-8-89176-481-1",
    abstract = "We present CHALIS (Challenging Language Identification Samples), a new benchmark dataset explicitly designed to address difficult cases in language identification: cousin languages and orthographic noise. Our dataset has two parts: First, we collected sentences shared across mutually intelligible language pairs (Czech/Slovak, Spanish/Catalan, Portuguese/Galician, Danish/Norwegian). The second part tests for orthography noise: we transliterate text across multiple scripts, remove diacritics, simulate homoglyph attacks, and use Internet slang. We evaluate four widely used language identification systems on CHALIS and demonstrate that all struggle substantially in these scenarios, especially on lower-resource languages within cousin pairs and on transliterated input."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tichy-libovicky-2026-chalis">
    <titleInfo>
        <title>CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Michal</namePart>
        <namePart type="family">Tichý</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jindřich</namePart>
        <namePart type="family">Libovický</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 13th Web-as-Corpus Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Pedro</namePart>
            <namePart type="family">Ortiz Suarez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laurie</namePart>
            <namePart type="family">Burchell</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nikola</namePart>
            <namePart type="family">Ljubešić</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yves</namePart>
            <namePart type="family">Scherrer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Veronika</namePart>
            <namePart type="family">Laippala</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vuk</namePart>
            <namePart type="family">Dinić</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Thom</namePart>
            <namePart type="family">Vaughan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-481-1</identifier>
    </relatedItem>
    <abstract>We present CHALIS (Challenging Language Identification Samples), a new benchmark dataset explicitly designed to address difficult cases in language identification: cousin languages and orthographic noise. Our dataset has two parts: First, we collected sentences shared across mutually intelligible language pairs (Czech/Slovak, Spanish/Catalan, Portuguese/Galician, Danish/Norwegian). The second part tests for orthography noise: we transliterate text across multiple scripts, remove diacritics, simulate homoglyph attacks, and use Internet slang. We evaluate four widely used language identification systems on CHALIS and demonstrate that all struggle substantially in these scenarios, especially on lower-resource languages within cousin pairs and on transliterated input.</abstract>
    <identifier type="citekey">tichy-libovicky-2026-chalis</identifier>
    <location>
        <url>https://aclanthology.org/2026.wac-1.1/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>1</start>
            <end>7</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios
%A Tichý, Michal
%A Libovický, Jindřich
%Y Ortiz Suarez, Pedro
%Y Burchell, Laurie
%Y Ljubešić, Nikola
%Y Scherrer, Yves
%Y Laippala, Veronika
%Y Dinić, Vuk
%Y Vaughan, Thom
%S Proceedings of the 13th Web-as-Corpus Workshop
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-481-1
%F tichy-libovicky-2026-chalis
%X We present CHALIS (Challenging Language Identification Samples), a new benchmark dataset explicitly designed to address difficult cases in language identification: cousin languages and orthographic noise. Our dataset has two parts: First, we collected sentences shared across mutually intelligible language pairs (Czech/Slovak, Spanish/Catalan, Portuguese/Galician, Danish/Norwegian). The second part tests for orthography noise: we transliterate text across multiple scripts, remove diacritics, simulate homoglyph attacks, and use Internet slang. We evaluate four widely used language identification systems on CHALIS and demonstrate that all struggle substantially in these scenarios, especially on lower-resource languages within cousin pairs and on transliterated input.
%U https://aclanthology.org/2026.wac-1.1/
%P 1-7

Download as File

Markdown (Informal)

[CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios](https://aclanthology.org/2026.wac-1.1/) (Tichý & Libovický, WAC 2026)

CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios (Tichý & Libovický, WAC 2026)

ACL

Michal Tichý and Jindřich Libovický. 2026. CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios. In Proceedings of the 13th Web-as-Corpus Workshop, pages 1–7, Budapest, Hungary. Association for Computational Linguistics.