An Evaluation of Croatian ASR Models for Čakavian Transcription

Shulin Zhang, John Hale, Margaret Renwick, Zvjezdana Vrzić, Keith Langston


Abstract
To assist in the documentation of Čakavian, an endangered language variety closely related to Croatian, we test four currently available ASR models that are trained with Croatian data and assess their performance in the transcription of Čakavian audio data. We compare the models’ word error rates, analyze the word-level error types, and showcase the most frequent Deletion and Substitution errors. The evaluation results indicate that the best-performing system for transcribing Čakavian was a CTC-based variant of the Conformer model.
Anthology ID:
2024.lrec-main.98
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
1098–1104
Language:
URL:
https://aclanthology.org/2024.lrec-main.98
DOI:
Bibkey:
Cite (ACL):
Shulin Zhang, John Hale, Margaret Renwick, Zvjezdana Vrzić, and Keith Langston. 2024. An Evaluation of Croatian ASR Models for Čakavian Transcription. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1098–1104, Torino, Italia. ELRA and ICCL.
Cite (Informal):
An Evaluation of Croatian ASR Models for Čakavian Transcription (Zhang et al., LREC-COLING 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.lrec-main.98.pdf