How Well Do Commodity Text-to-Speech Systems Evade Acoustic Perturbation Detection? A Multi-Engine Evaluation Across 21 Languages

Anatoly Marchenko


Abstract
Jitter, shimmer, and harmonics-to-noise ratio (HNR) are often used to detect voice deepfakes, since these features capture biomechanical irregularities of vocal fold vibration that synthetic speech supposedly lacks. We test this assumption on three commodity TTS engines (Google TTS, Microsoft Edge TTS, macOS system voice) with 1,850 samples across 21 languages, measured against 29 emotion corpora in 24 languages (35,091 utterances). Three classifiers (logistic regression, SVM-RBF, Random Forest) all fail to reliably detect Edge TTS: the best result is F1 = 0.78. Effect sizes drop 2.1x-7.4x from Google TTS to Edge TTS. In ablation, no single feature exceeds F1 = 0.60 against Edge TTS. These three perturbation features, taken alone, can no longer separate commodity neural TTS from natural speech.
Anthology ID:
2026.nlpaics-1.18
Volume:
Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
Month:
June
Year:
2026
Address:
Alicante, Spain
Editors:
Ruslan Mitkov, Rafael Muñoz, Elena Lloret, Tharindu Ranasinghe, Ernesto L. Estevanell-Valladares, Salima Lamsiyah, Andrés Montoyo, Saad Ezzini
Venue:
NLPAICS
SIG:
Publisher:
Department of Languages and Information Systems, University of Alicante
Note:
Pages:
171–175
Language:
URL:
https://aclanthology.org/2026.nlpaics-1.18/
DOI:
Bibkey:
Cite (ACL):
Anatoly Marchenko. 2026. How Well Do Commodity Text-to-Speech Systems Evade Acoustic Perturbation Detection? A Multi-Engine Evaluation Across 21 Languages. In Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security, pages 171–175, Alicante, Spain. Department of Languages and Information Systems, University of Alicante.
Cite (Informal):
How Well Do Commodity Text-to-Speech Systems Evade Acoustic Perturbation Detection? A Multi-Engine Evaluation Across 21 Languages (Marchenko, NLPAICS 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.nlpaics-1.18.pdf