Domain-Informed Probing of wav2vec 2.0 Embeddings for Phonetic Features

Patrick Cormac English, John D. Kelleher, Julie Carson-Berndsen


Abstract
In recent years large transformer model architectures have become available which provide a novel means of generating high-quality vector representations of speech audio. These transformers make use of an attention mechanism to generate representations enhanced with contextual and positional information from the input sequence. Previous works have explored the capabilities of these models with regard to performance in tasks such as speech recognition and speaker verification, but there has not been a significant inquiry as to the manner in which the contextual information provided by the transformer architecture impacts the representation of phonetic information within these models. In this paper, we report the results of a number of probing experiments on the representations generated by the wav2vec 2.0 model’s transformer component, with regard to the encoding of phonetic categorization information within the generated embeddings. We find that the contextual information generated by the transformer’s operation results in enhanced capture of phonetic detail by the model, and allows for distinctions to emerge in acoustic data that are otherwise difficult to separate.
Anthology ID:
2022.sigmorphon-1.9
Volume:
Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
Month:
July
Year:
2022
Address:
Seattle, Washington
Editors:
Garrett Nicolai, Eleanor Chodroff
Venue:
SIGMORPHON
SIG:
SIGMORPHON
Publisher:
Association for Computational Linguistics
Note:
Pages:
83–91
Language:
URL:
https://aclanthology.org/2022.sigmorphon-1.9
DOI:
10.18653/v1/2022.sigmorphon-1.9
Bibkey:
Cite (ACL):
Patrick Cormac English, John D. Kelleher, and Julie Carson-Berndsen. 2022. Domain-Informed Probing of wav2vec 2.0 Embeddings for Phonetic Features. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 83–91, Seattle, Washington. Association for Computational Linguistics.
Cite (Informal):
Domain-Informed Probing of wav2vec 2.0 Embeddings for Phonetic Features (Cormac English et al., SIGMORPHON 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.sigmorphon-1.9.pdf