Assessing the Quality of an Italian Crowdsourced Idiom Corpus:the Dodiom Experiment

Giuseppina Morza, Raffaele Manna, Johanna Monti


Abstract
This paper describes how idiom-related language resources, collected through a crowdsourcing experiment carried out by means of Dodiom, a Game-with-a-purpose, have been analysed by language experts. The paper focuses on the criteria adopted for the data annotation and evaluation process. The main scope of this project is, indeed, the evaluation of the quality of the linguistic data obtained through a crowdsourcing project, namely to assess if the data provided and evaluated by the players who joined the game are actually considered of good quality by the language experts. Finally, results of the annotation and evaluation processes as well as future work are presented.
Anthology ID:
2022.lrec-1.446
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
4205–4211
Language:
URL:
https://aclanthology.org/2022.lrec-1.446
DOI:
Bibkey:
Cite (ACL):
Giuseppina Morza, Raffaele Manna, and Johanna Monti. 2022. Assessing the Quality of an Italian Crowdsourced Idiom Corpus:the Dodiom Experiment. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4205–4211, Marseille, France. European Language Resources Association.
Cite (Informal):
Assessing the Quality of an Italian Crowdsourced Idiom Corpus:the Dodiom Experiment (Morza et al., LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.446.pdf