BLM-It - Blackbird Language Matrices for Italian: A CALAMITA Challenge

Chunyang Jiang, Giuseppe Samo, Vivi Nastase, Paola Merlo


Abstract
In this challenge, we propose Blackbird Language Matrices (BLMs), linguistic puzzles to learn language-related problems and delve into deeper formal and semantic properties of language, through a process of paradigm understanding. A BLM matrix consists of a context set and an answer set. The context is a sequence of sentences that encode implicitly an underlying generative linguistic rule. The contrastive multiple-choice answer set includes negative examples following corrupted generating rules. We propose three subtasks —agreement concord, causative and object-drop alternation detection— each in two variants of increasing lexical complexity.The datasets comprise a few prompts for few-shot learning and a large test set.
Anthology ID:
2024.clicit-1.125
Volume:
Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)
Month:
December
Year:
2024
Address:
Pisa, Italy
Editors:
Felice Dell'Orletta, Alessandro Lenci, Simonetta Montemagni, Rachele Sprugnoli
Venue:
CLiC-it
SIG:
Publisher:
CEUR Workshop Proceedings
Note:
Pages:
1135–1143
Language:
URL:
https://aclanthology.org/2024.clicit-1.125/
DOI:
Bibkey:
Cite (ACL):
Chunyang Jiang, Giuseppe Samo, Vivi Nastase, and Paola Merlo. 2024. BLM-It - Blackbird Language Matrices for Italian: A CALAMITA Challenge. In Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), pages 1135–1143, Pisa, Italy. CEUR Workshop Proceedings.
Cite (Informal):
BLM-It - Blackbird Language Matrices for Italian: A CALAMITA Challenge (Jiang et al., CLiC-it 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.clicit-1.125.pdf