Giving Attention to the Unexpected: Using Prosody Innovations in Disfluency Detection

Vicky Zayats, Mari Ostendorf


Abstract
Disfluencies in spontaneous speech are known to be associated with prosodic disruptions. However, most algorithms for disfluency detection use only word transcripts. Integrating prosodic cues has proved difficult because of the many sources of variability affecting the acoustic correlates. This paper introduces a new approach to extracting acoustic-prosodic cues using text-based distributional prediction of acoustic cues to derive vector z-score features (innovations). We explore both early and late fusion techniques for integrating text and prosody, showing gains over a high-accuracy text-only model.
Anthology ID:
N19-1008
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Jill Burstein, Christy Doran, Thamar Solorio
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
86–95
Language:
URL:
https://aclanthology.org/N19-1008
DOI:
10.18653/v1/N19-1008
Bibkey:
Cite (ACL):
Vicky Zayats and Mari Ostendorf. 2019. Giving Attention to the Unexpected: Using Prosody Innovations in Disfluency Detection. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 86–95, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Giving Attention to the Unexpected: Using Prosody Innovations in Disfluency Detection (Zayats & Ostendorf, NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/N19-1008.pdf
Video:
 https://aclanthology.org/N19-1008.mp4