<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="sarti-etal-2024-eurekarebus">
    <titleInfo>
        <title>EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Gabriele</namePart>
        <namePart type="family">Sarti</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tommaso</namePart>
        <namePart type="family">Caselli</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Arianna</namePart>
        <namePart type="family">Bisazza</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Malvina</namePart>
        <namePart type="family">Nissim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Felice</namePart>
            <namePart type="family">Dell’Orletta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alessandro</namePart>
            <namePart type="family">Lenci</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Simonetta</namePart>
            <namePart type="family">Montemagni</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rachele</namePart>
            <namePart type="family">Sprugnoli</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>CEUR Workshop Proceedings</publisher>
            <place>
                <placeTerm type="text">Pisa, Italy</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-12-210-7060-6</identifier>
    </relatedItem>
    <abstract>Language games can be valuable resources for testing the ability of large language models (LLMs) to conduct challenging multi-step, knowledge-intensive inferences while respecting predefined constraints. Our proposed challenge prompts LLMs to reason step-by-step to solve verbalized variants of rebus games recently introduced with the EurekaRebus dataset. Verbalized rebuses replace visual cues with crossword definitions to create an encrypted first pass, making the problem entirely text-based. We introduce a simplified task variant with word length hints and adopt a comprehensive set of metrics to obtain a granular overview of models’ performance in knowledge recall, constraints adherence, and re-segmentation abilities across reasoning steps.</abstract>
    <identifier type="citekey">sarti-etal-2024-eurekarebus</identifier>
    <location>
        <url>https://aclanthology.org/2024.clicit-1.132/</url>
    </location>
    <part>
        <date>2024-12</date>
        <extent unit="page">
            <start>1202</start>
            <end>1208</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge
%A Sarti, Gabriele
%A Caselli, Tommaso
%A Bisazza, Arianna
%A Nissim, Malvina
%Y Dell’Orletta, Felice
%Y Lenci, Alessandro
%Y Montemagni, Simonetta
%Y Sprugnoli, Rachele
%S Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)
%D 2024
%8 December
%I CEUR Workshop Proceedings
%C Pisa, Italy
%@ 979-12-210-7060-6
%F sarti-etal-2024-eurekarebus
%X Language games can be valuable resources for testing the ability of large language models (LLMs) to conduct challenging multi-step, knowledge-intensive inferences while respecting predefined constraints. Our proposed challenge prompts LLMs to reason step-by-step to solve verbalized variants of rebus games recently introduced with the EurekaRebus dataset. Verbalized rebuses replace visual cues with crossword definitions to create an encrypted first pass, making the problem entirely text-based. We introduce a simplified task variant with word length hints and adopt a comprehensive set of metrics to obtain a granular overview of models’ performance in knowledge recall, constraints adherence, and re-segmentation abilities across reasoning steps.
%U https://aclanthology.org/2024.clicit-1.132/
%P 1202-1208

Markdown (Informal)

[EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge](https://aclanthology.org/2024.clicit-1.132/) (Sarti et al., CLiC-it 2024)

• EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge (Sarti et al., CLiC-it 2024)

ACL

• Gabriele Sarti, Tommaso Caselli, Arianna Bisazza, and Malvina Nissim. 2024. EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge. In Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), pages 1202–1208, Pisa, Italy. CEUR Workshop Proceedings.