<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="han-etal-2026-think">
    <titleInfo>
        <title>Think You Have Solved Commonsense Reasoning? Try HellaSwagUltra</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Wenhan</namePart>
        <namePart type="family">Han</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yifan</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Binbin</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mykola</namePart>
        <namePart type="family">Pechenizkiy</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yin</namePart>
        <namePart type="family">Zheng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Meng</namePart>
        <namePart type="family">Fang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>As LLMs advance, popular commonsense and NLU benchmarks have become saturated, yet multilingual evaluation still lags. Existing benchmarks cover only a small set of languages, and translation based multilingual benchmarks can introduce bias. We introduce HellaSwagUltra, a multilingual commonsense and NLU benchmark with substantial local cultural knowledge. It is built with an automated, continuously extensible pipeline. Instead of explicitly prompting for reasoning, each example implicitly embeds two commonsense or local knowledge facts in the context, and answer options contain subtle cues about whether those facts are violated, requiring models to pick the most plausible continuation. All test items are fully reviewed and corrected by language experts, with ongoing updates. Even the strongest proprietary model (e.g., Gemini-2.5-Pro) reaches only 62.5% accuracy, while GPT-4o and leading open-source models hover around 40%–50%, demonstrating that multilingual commonsense reasoning remains an open challenge.</abstract>
    <identifier type="citekey">han-etal-2026-think</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.1396/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>29739</start>
            <end>29758</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Think You Have Solved Commonsense Reasoning? Try HellaSwagUltra
%A Han, Wenhan
%A Zhang, Yifan
%A Liu, Binbin
%A Pechenizkiy, Mykola
%A Zheng, Yin
%A Fang, Meng
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F han-etal-2026-think
%X As LLMs advance, popular commonsense and NLU benchmarks have become saturated, yet multilingual evaluation still lags. Existing benchmarks cover only a small set of languages, and translation based multilingual benchmarks can introduce bias. We introduce HellaSwagUltra, a multilingual commonsense and NLU benchmark with substantial local cultural knowledge. It is built with an automated, continuously extensible pipeline. Instead of explicitly prompting for reasoning, each example implicitly embeds two commonsense or local knowledge facts in the context, and answer options contain subtle cues about whether those facts are violated, requiring models to pick the most plausible continuation. All test items are fully reviewed and corrected by language experts, with ongoing updates. Even the strongest proprietary model (e.g., Gemini-2.5-Pro) reaches only 62.5% accuracy, while GPT-4o and leading open-source models hover around 40%–50%, demonstrating that multilingual commonsense reasoning remains an open challenge.
%U https://aclanthology.org/2026.findings-emnlp.1396/
%P 29739-29758

Markdown (Informal)

[Think You Have Solved Commonsense Reasoning? Try HellaSwagUltra](https://aclanthology.org/2026.findings-emnlp.1396/) (Han et al., Findings 2026)

• Think You Have Solved Commonsense Reasoning? Try HellaSwagUltra (Han et al., Findings 2026)

ACL

• Wenhan Han, Yifan Zhang, Binbin Liu, Mykola Pechenizkiy, Yin Zheng, and Meng Fang. 2026. Think You Have Solved Commonsense Reasoning? Try HellaSwagUltra. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 29739–29758, Budapest, Hungary. Association for Computational Linguistics.