Mariachiara Pascucci
2024
Confronto tra Diversi Tipi di Valutazione del Miglioramento della Chiarezza di Testi Amministrativi in Lingua Italiana
Mariachiara Pascucci
|
Mirko Tavosanis
Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)
The paper presents a comparison of different types of evaluation of administrative texts in the Italian language on which a clarity improvement intervention was carried out. The clarity improvement was performed by human experts and ChatGPT. The evaluation was carried out in four different ways: by expert evaluators, used as a reference; by evaluators with good skills, subject to dedicated training; by generic evaluators recruited through a crowdsourcing platform; by ChatGPT. The results show that the closest match to the results of the evaluation by expert evaluators was reached, by a wide margin, by evaluators with good skills and dedicated training; the second best approach was reached by requesting evaluation from ChatGPT; the worst approach was reached by generic evaluators recruited through a crowdsourcing platform. Task features that may have influenced the outcome are also discussed.