Alessandro Vietti

Author directory

2026

We present a blank-aware decoding approach for transcript-free phoneme alignment with CTC-based speech foundation models, designed to improve annotation bootstrapping in low-resource languages. While CTC models provide frame-level phoneme posteriors without requiring transcripts, greedy decoding produces blank-dominated and temporally unstable segmentations that are difficult to correct manually. Our approach introduces two training-free blank-resolution strategies operating directly on CTC logits: (i) confidence-ratio substitution, which promotes competitive non-blank hypotheses relative to the blank symbol, and (ii) recursive context adjustment, which enforces local contextual consistency within blank spans. Experiments on English (TIMIT) and on Sardinian and Tyrolean dialect corpora show consistent improvements in boundary F1 prediction, phoneme duration regularity, and segmentation stability over greedy CTC decoding. Although absolute boundary deviations remain higher than transcript-conditioned aligners, the resulting alignments are structurally coherent and suitable for manual correction. A post-hoc phoneme-class analysis further reveals systematic asymmetries in blank resolution, highlighting complementary roles of local acoustic evidence and contextual cues, and outlining prominising venues for future improvements.
Recent work suggests that Whisper-style speech encoders exhibit semantic alignment across languages, based on spoken translation retrieval using representational similarity. If so, this would enable Whisper-style encoders as a viable way to obtain acoustic representations with semantic knowledge, enabling downstream semantic tasks with speech input. However, prior work does not control for phonetic overlap between equivalent utterances, which may artificially support retrieval. We therefore conduct pronunciation-controlled experiments to test whether cross-lingual alignment arises from semantic rather than phonetic similarity, and further probe semantic information in the encoder by way of lexical semantic analogy tasks. Results show that spoken translation retrieval remains strongly above chance with substantially reduced phonetic cues in the final layers of Whisper’s encoder. Furthermore, for both word-level and utterance-level input, Whisper’s encoder representations exhibit non-trivial analogy performance, attaining as high as 25.98% accuracy in an analogy test set. Motivated by this layerwise semantic specialization, we test whether earlier encoder layers preserve representations better suited to ASR in unsupported low-resource languages. Our results confirm it to enable performance gains in automatic speech recognition on low-resource languages unseen during training.

2025

2024

Speech recognition systems are still highly dependent on textual orthographic resources, posing a challenge for low-resourcelanguages. Recent research leverages self-supervised learning of unlabeled data or employs multilingual models pre-trainedon high resource languages for fine-tuning on the target low-resource language. These are effective approacheswhen the target language has a shared writing tradition, but when we are confronted with mainly spoken languages, beingthem endangered minority languages, dialects, or regional varieties, other than labeled data, we lack a shared metric toassess speech recognition performance. We first provide a research background on ASR for low-resource languages anddescribe the specific linguistic situation of Campidanese Sardinian, we then evaluate five multilingual ASR models usingtraditional evaluation metrics and an exploratory linguistic analysis. The paper addresses key challenges in developing a toolfor researchers to document and analyze the phonetics and phonology of spoken (endangered) languages.
Automatic Speech Recognition systems (ASR) based on neural networks achieve great results, but it remains unclear which are the linguistic features and representations that the models leverage to perform the recognition. In our study, we used phonological syllables as tokens to fine-tune an end-to-end ASR model due to their relevance as linguistic units. Furthermore, this strategy allowed us to keep track of different types of linguistic features characterizing the tokens. The analysis of the transcriptions generated by the model reveals that factors such as token frequency and lexical stress have a variable impact on the prediction strategies adopted by the ASR system.