<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="libovicky-etal-2026-cus">
    <titleInfo>
        <title>CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jindrich</namePart>
        <namePart type="family">Libovický</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jindrich</namePart>
        <namePart type="family">Helcl</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Andrei-Alexandru</namePart>
        <namePart type="family">Manea</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Gianluca</namePart>
        <namePart type="family">Vico</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>We introduce CUS-QA, a benchmark for evaluation of open-ended regional question answering that encompasses both textual and visual modalities. We also provide strong baselines using state-of-the-art large language models (LLMs). Our dataset consists of manually curated questions and answers grounded in Wikipedia, created by native speakers from Czechia, Slovakia, and Ukraine, with accompanying English translations. It includes both purely textual questions and those requiring visual understanding. We evaluate state-of-the-art LLMs through prompting and add human judgments of answer correctness. Using these human evaluations, we analyze the reliability of existing automatic evaluation metrics. Our baseline results show that even the best open-weight LLMs achieve only over 40% accuracy on textual questions and below 30% on visual questions. LLM-based evaluation metrics show strong correlation with human judgment, while traditional string-overlap metrics perform surprisingly well due to the prevalence of named entities in answers.</abstract>
    <identifier type="citekey">libovicky-etal-2026-cus</identifier>
    <identifier type="doi">10.1162/tacl.a.737</identifier>
    <location>
        <url>https://aclanthology.org/2026.tacl-1.67/</url>
    </location>
    <part>
        <date>2026</date>
        <detail type="volume"><number>14</number></detail>
        <extent unit="page">
            <start>1482</start>
            <end>1509</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset
%A Libovický, Jindrich
%A Helcl, Jindrich
%A Manea, Andrei-Alexandru
%A Vico, Gianluca
%J Transactions of the Association for Computational Linguistics
%D 2026
%V 14
%I MIT Press
%C Cambridge, MA
%F libovicky-etal-2026-cus
%X We introduce CUS-QA, a benchmark for evaluation of open-ended regional question answering that encompasses both textual and visual modalities. We also provide strong baselines using state-of-the-art large language models (LLMs). Our dataset consists of manually curated questions and answers grounded in Wikipedia, created by native speakers from Czechia, Slovakia, and Ukraine, with accompanying English translations. It includes both purely textual questions and those requiring visual understanding. We evaluate state-of-the-art LLMs through prompting and add human judgments of answer correctness. Using these human evaluations, we analyze the reliability of existing automatic evaluation metrics. Our baseline results show that even the best open-weight LLMs achieve only over 40% accuracy on textual questions and below 30% on visual questions. LLM-based evaluation metrics show strong correlation with human judgment, while traditional string-overlap metrics perform surprisingly well due to the prevalence of named entities in answers.
%R 10.1162/tacl.a.737
%U https://aclanthology.org/2026.tacl-1.67/
%U https://doi.org/10.1162/tacl.a.737
%P 1482-1509

Markdown (Informal)

[CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset](https://aclanthology.org/2026.tacl-1.67/) (Libovický et al., TACL 2026)

• CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset (Libovický et al., TACL 2026)

ACL

• Jindrich Libovický, Jindrich Helcl, Andrei-Alexandru Manea, and Gianluca Vico. 2026. CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset. Transactions of the Association for Computational Linguistics, 14:1482–1509.