Alessandro Vietti
Author directory2026
Blank-Aware Decoding for Transcript-Free Phoneme Alignment in Low-Resource Languages and Dialects
Domenico De Cristofaro | Barbara Plank | Alessandro Vietti
Proceedings of Speech Language Models in Low-Resource Settings: Performance, Evaluation, and Bias Analysis (SPEAKABLE) @ LREC 2026
Domenico De Cristofaro | Barbara Plank | Alessandro Vietti
Proceedings of Speech Language Models in Low-Resource Settings: Performance, Evaluation, and Bias Analysis (SPEAKABLE) @ LREC 2026
We present a blank-aware decoding approach for transcript-free phoneme alignment with CTC-based speech foundation models, designed to improve annotation bootstrapping in low-resource languages. While CTC models provide frame-level phoneme posteriors without requiring transcripts, greedy decoding produces blank-dominated and temporally unstable segmentations that are difficult to correct manually. Our approach introduces two training-free blank-resolution strategies operating directly on CTC logits: (i) confidence-ratio substitution, which promotes competitive non-blank hypotheses relative to the blank symbol, and (ii) recursive context adjustment, which enforces local contextual consistency within blank spans. Experiments on English (TIMIT) and on Sardinian and Tyrolean dialect corpora show consistent improvements in boundary F1 prediction, phoneme duration regularity, and segmentation stability over greedy CTC decoding. Although absolute boundary deviations remain higher than transcript-conditioned aligners, the resulting alignments are structurally coherent and suitable for manual correction. A post-hoc phoneme-class analysis further reveals systematic asymmetries in blank resolution, highlighting complementary roles of local acoustic evidence and contextual cues, and outlining prominising venues for future improvements.
Whisper-Style Speech Encoders Align Semantically Beyond Phonetic Cues
Ryan Soh-Eun Shim | Domenico de Cristofaro | Chengzhi Martin Hu | Alessandro Vietti | Barbara Plank
Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
Ryan Soh-Eun Shim | Domenico de Cristofaro | Chengzhi Martin Hu | Alessandro Vietti | Barbara Plank
Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
Recent work suggests that Whisper-style speech encoders exhibit semantic alignment across languages, based on spoken translation retrieval using representational similarity. If so, this would enable Whisper-style encoders as a viable way to obtain acoustic representations with semantic knowledge, enabling downstream semantic tasks with speech input. However, prior work does not control for phonetic overlap between equivalent utterances, which may artificially support retrieval. We therefore conduct pronunciation-controlled experiments to test whether cross-lingual alignment arises from semantic rather than phonetic similarity, and further probe semantic information in the encoder by way of lexical semantic analogy tasks. Results show that spoken translation retrieval remains strongly above chance with substantially reduced phonetic cues in the final layers of Whisper’s encoder. Furthermore, for both word-level and utterance-level input, Whisper’s encoder representations exhibit non-trivial analogy performance, attaining as high as 25.98% accuracy in an analogy test set. Motivated by this layerwise semantic specialization, we test whether earlier encoder layers preserve representations better suited to ASR in unsupported low-resource languages. Our results confirm it to enable performance gains in automatic speech recognition on low-resource languages unseen during training.
2025
Toward Optimised Datasets to Fine-tune ASR Systems Leveraging Less but More Informative Speech
Loredana Schettino | Vincenzo Norman Vitale | Alessandro Vietti
Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025)
Loredana Schettino | Vincenzo Norman Vitale | Alessandro Vietti
Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025)
When Less Is More? Diagnosing ASR Predictions in Sardinian via Layer-Wise Decoding
Domenico De Cristofaro | Alessandro Vietti | Marianne Pouplier | Aleese Block
Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025)
Domenico De Cristofaro | Alessandro Vietti | Marianne Pouplier | Aleese Block
Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025)
2024
Towards an ASR System for Documenting Endangered Languages: A Preliminary Study on Sardinian
Ilaria Chizzoni | Alessandro Vietti
Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024)
Ilaria Chizzoni | Alessandro Vietti
Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024)
Speech recognition systems are still highly dependent on textual orthographic resources, posing a challenge for low-resourcelanguages. Recent research leverages self-supervised learning of unlabeled data or employs multilingual models pre-trainedon high resource languages for fine-tuning on the target low-resource language. These are effective approacheswhen the target language has a shared writing tradition, but when we are confronted with mainly spoken languages, beingthem endangered minority languages, dialects, or regional varieties, other than labeled data, we lack a shared metric toassess speech recognition performance. We first provide a research background on ASR for low-resource languages anddescribe the specific linguistic situation of Campidanese Sardinian, we then evaluate five multilingual ASR models usingtraditional evaluation metrics and an exploratory linguistic analysis. The paper addresses key challenges in developing a toolfor researchers to document and analyze the phonetics and phonology of spoken (endangered) languages.
Sensitivity of Syllable-Based ASR Predictions to Token Frequency and Lexical Stress
Alessandro Vietti | Domenico De Cristofaro | Picciau Sara
Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024)
Alessandro Vietti | Domenico De Cristofaro | Picciau Sara
Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024)
Automatic Speech Recognition systems (ASR) based on neural networks achieve great results, but it remains unclear which are the linguistic features and representations that the models leverage to perform the recognition. In our study, we used phonological syllables as tokens to fine-tune an end-to-end ASR model due to their relevance as linguistic units. Furthermore, this strategy allowed us to keep track of different types of linguistic features characterizing the tokens. The analysis of the transcriptions generated by the model reveals that factors such as token frequency and lexical stress have a variable impact on the prediction strategies adopted by the ASR system.