Play música alegre: A Large-Scale Empirical Analysis of Cross-Lingual Phenomena in Voice Assistant Interactions

Donato Crisostomi, Alessandro Manzotti, Enrico Palumbo, Davide Bernardi, Sarah Campbell, Shubham Garg


Abstract
Cross-lingual phenomena are quite common in informal contexts like social media, where users are likely to mix their native language with English or other languages. However, few studies have focused so far on analyzing cross-lingual interactions in voice-assistant data, which present peculiar features in terms of sentence length, named entities, and use of spoken language. Also, little attention has been posed to European countries, where English is frequently used as a second language. In this paper, we present a large-scale empirical analysis of cross-lingual phenomena (code-mixing, linguistic borrowing, foreign named entities) in the interactions with a large-scale voice assistant in European countries. To do this, we first introduce a general, highly-scalable technique to generate synthetic mixed training data annotated with token-level language labels and we train two neural network models to predict them. We evaluate the models both on the synthetic dataset and on a real dataset of code-switched utterances, showing that the best performance is obtained by a character convolution based model. The results of the analysis highlight different behaviors between countries, having Italy with the highest ratio of cross-lingual utterances and Spain with a marked preference in keeping Spanish words. Our research, paired to the increase of the cross-lingual phenomena in time, motivates further research in developing multilingual Natural Language Understanding (NLU) models, which can naturally deal with cross-lingual interactions.
Anthology ID:
2022.mmnlu-1.5
Volume:
Proceedings of the Massively Multilingual Natural Language Understanding Workshop (MMNLU-22)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Editors:
Jack FitzGerald, Kay Rottmann, Julia Hirschberg, Mohit Bansal, Anna Rumshisky, Charith Peris, Christopher Hench
Venue:
MMNLU
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
42–52
Language:
URL:
https://aclanthology.org/2022.mmnlu-1.5
DOI:
10.18653/v1/2022.mmnlu-1.5
Bibkey:
Cite (ACL):
Donato Crisostomi, Alessandro Manzotti, Enrico Palumbo, Davide Bernardi, Sarah Campbell, and Shubham Garg. 2022. Play música alegre: A Large-Scale Empirical Analysis of Cross-Lingual Phenomena in Voice Assistant Interactions. In Proceedings of the Massively Multilingual Natural Language Understanding Workshop (MMNLU-22), pages 42–52, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
Play música alegre: A Large-Scale Empirical Analysis of Cross-Lingual Phenomena in Voice Assistant Interactions (Crisostomi et al., MMNLU 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.mmnlu-1.5.pdf
Video:
 https://aclanthology.org/2022.mmnlu-1.5.mp4