Building and curating conversational corpora for diversity-aware language science and technology

Andreas Liesenfeld, Mark Dingemanse


Abstract
We present an analysis pipeline and best practice guidelines for building and curating corpora of everyday conversation in diverse languages. Surveying language documentation corpora and other resources that cover 67 languages and varieties from 28 phyla, we describe the compilation and curation process, specify minimal properties of a unified format for interactional data, and develop methods for quality control that take into account turn-taking and timing. Two case studies show the broad utility of conversational data for (i) charting human interactional infrastructure and (ii) tracing challenges and opportunities for current ASR solutions. Linguistically diverse conversational corpora can provide new insights for the language sciences and stronger empirical foundations for language technology.
Anthology ID:
2022.lrec-1.126
Original:
2022.lrec-1.126v1
Version 2:
2022.lrec-1.126v2
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
1178–1192
Language:
URL:
https://aclanthology.org/2022.lrec-1.126
DOI:
Bibkey:
Cite (ACL):
Andreas Liesenfeld and Mark Dingemanse. 2022. Building and curating conversational corpora for diversity-aware language science and technology. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1178–1192, Marseille, France. European Language Resources Association.
Cite (Informal):
Building and curating conversational corpora for diversity-aware language science and technology (Liesenfeld & Dingemanse, LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.126.pdf
Data
LibriSpeech