ABRICOT - ABstRactness and Inclusiveness in COntexT: A CALAMITA Challenge

Giovanni Puccetti, Claudia Collacciani, Andrea Amelio Ravelli, Andrea Esuli, Marianna Bolognesi


Abstract
The ABRICOT Task is designed to evaluate Italian language models on their ability to understand and assess the abstractness and inclusiveness of language, two nuanced features that humans naturally convey in everyday communication. Unlike binary categorizations such as abstract/concrete or inclusive/exclusive, these features exist on a continuous spectrum with varying degrees of intensity. The task is based on a manual collection of sentences that present the same noun phrase (NP) in different contexts, allowing its interpretation to vary between the extremes of abstractness and inclusiveness. This challenge aims to verify the how LLMs perceive subtle linguistic variations and their implications in natural language.
Anthology ID:
2024.clicit-1.128
Volume:
Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)
Month:
December
Year:
2024
Address:
Pisa, Italy
Editors:
Felice Dell'Orletta, Alessandro Lenci, Simonetta Montemagni, Rachele Sprugnoli
Venue:
CLiC-it
SIG:
Publisher:
CEUR Workshop Proceedings
Note:
Pages:
1161–1167
Language:
URL:
https://aclanthology.org/2024.clicit-1.128/
DOI:
Bibkey:
Cite (ACL):
Giovanni Puccetti, Claudia Collacciani, Andrea Amelio Ravelli, Andrea Esuli, and Marianna Bolognesi. 2024. ABRICOT - ABstRactness and Inclusiveness in COntexT: A CALAMITA Challenge. In Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), pages 1161–1167, Pisa, Italy. CEUR Workshop Proceedings.
Cite (Informal):
ABRICOT - ABstRactness and Inclusiveness in COntexT: A CALAMITA Challenge (Puccetti et al., CLiC-it 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.clicit-1.128.pdf