TurkSV-Bench: Assessing Demographic Bias and Preprocessing Effects in Speaker Verification

Ahmet Berkay Kantarcı, Abdulkadir Parlak, Ahmet Karazor


Abstract
In this study, TurkSV-Bench is introduced as a novel benchmark dataset to evaluate the zero-shot cross-lingual performance and demographic biases of Turkish speaker verification (SV) systems. Created via an automated pipeline from YouTube data, this dataset comprises 1,844 audio recordings from 461 speakers representing diverse age groups, genders, and regional dialects. Evaluation of existing models identifies ResNet34-LM as the most successful baseline architecture. Subgroup analyses demonstrate the models’ generalization capabilities across different gender and age demographics, while revealing distinct performance variations among regional dialects. Comprehensive evaluations of various preprocessing techniques highlight the baseline architecture’s structural robustness, identifying specific configurations that maintain or refine verification performance without requiring aggressive signal filtering.
Anthology ID:
2026.impactspeech-1.9
Volume:
Proceedings of the IMPACT-SPEECH Workshop: Identifying, Measuring, Preventing, and Assessing Consequences of Bias in Speech LLMs
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Ravi Shekhar, Monorama Swain, Jagabandhu Mishra, Sandipan Dhar, Haralambos Mouratidis, Matthew Purver
Venues:
IMPACT-SPEECH | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
95–103
Language:
URL:
https://aclanthology.org/2026.impactspeech-1.9/
DOI:
Bibkey:
Cite (ACL):
Ahmet Berkay Kantarcı, Abdulkadir Parlak, and Ahmet Karazor. 2026. TurkSV-Bench: Assessing Demographic Bias and Preprocessing Effects in Speaker Verification. In Proceedings of the IMPACT-SPEECH Workshop: Identifying, Measuring, Preventing, and Assessing Consequences of Bias in Speech LLMs, pages 95–103, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
TurkSV-Bench: Assessing Demographic Bias and Preprocessing Effects in Speaker Verification (Kantarcı et al., IMPACT-SPEECH 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.impactspeech-1.9.pdf