<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="artkaew-2025-thai">
    <titleInfo>
        <title>Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Phakphum</namePart>
        <namePart type="family">Artkaew</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-01</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Second Workshop in South East Asian Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Derry</namePart>
            <namePart type="family">Wijaya</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alham</namePart>
            <namePart type="given">Fikri</namePart>
            <namePart type="family">Aji</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Clara</namePart>
            <namePart type="family">Vania</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Genta</namePart>
            <namePart type="given">Indra</namePart>
            <namePart type="family">Winata</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ayu</namePart>
            <namePart type="family">Purwarianti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Commonsense reasoning is one of the important aspects of natural language understanding, with several benchmarks developed to evaluate it. However, only a few of these benchmarks are available in languages other than English. Developing parallel benchmarks facilitates cross-lingual evaluation, enabling a better understanding of different languages. This research introduces a collection of Winograd Schemas in Thai, a novel dataset designed to evaluate commonsense reasoning capabilities in the context of the Thai language. Through a methodology involving native speakers, professional translators, and thorough validation, the schemas aim to closely reflect Thai language nuances, idioms, and cultural references while maintaining ambiguity and commonsense challenges. We evaluate the performance of popular large language models on this benchmark, revealing their strengths, limitations, and providing insights into the current state-of-the-art. Results indicate that while models like GPT-4 and Claude-3-Opus achieve high accuracy in English, their performance significantly drops in Thai, highlighting the need for further advancements in multilingual commonsense reasoning.</abstract>
    <identifier type="citekey">artkaew-2025-thai</identifier>
    <location>
        <url>https://aclanthology.org/2025.sealp-1.4/</url>
    </location>
    <part>
        <date>2025-01</date>
        <extent unit="page">
            <start>42</start>
            <end>51</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning
%A Artkaew, Phakphum
%Y Wijaya, Derry
%Y Aji, Alham Fikri
%Y Vania, Clara
%Y Winata, Genta Indra
%Y Purwarianti, Ayu
%S Proceedings of the Second Workshop in South East Asian Language Processing
%D 2025
%8 January
%I Association for Computational Linguistics
%C Online
%F artkaew-2025-thai
%X Commonsense reasoning is one of the important aspects of natural language understanding, with several benchmarks developed to evaluate it. However, only a few of these benchmarks are available in languages other than English. Developing parallel benchmarks facilitates cross-lingual evaluation, enabling a better understanding of different languages. This research introduces a collection of Winograd Schemas in Thai, a novel dataset designed to evaluate commonsense reasoning capabilities in the context of the Thai language. Through a methodology involving native speakers, professional translators, and thorough validation, the schemas aim to closely reflect Thai language nuances, idioms, and cultural references while maintaining ambiguity and commonsense challenges. We evaluate the performance of popular large language models on this benchmark, revealing their strengths, limitations, and providing insights into the current state-of-the-art. Results indicate that while models like GPT-4 and Claude-3-Opus achieve high accuracy in English, their performance significantly drops in Thai, highlighting the need for further advancements in multilingual commonsense reasoning.
%U https://aclanthology.org/2025.sealp-1.4/
%P 42-51

Markdown (Informal)

[Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning](https://aclanthology.org/2025.sealp-1.4/) (Artkaew, sealp 2025)

• Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning (Artkaew, sealp 2025)

ACL

• Phakphum Artkaew. 2025. Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning. In Proceedings of the Second Workshop in South East Asian Language Processing, pages 42–51, Online. Association for Computational Linguistics.