A Conditional Generative Matching Model for Multi-lingual Reply Suggestion

Budhaditya Deb, Guoqing Zheng, Milad Shokouhi, Ahmed Hassan Awadallah


Abstract
We study the problem of multilingual automated reply suggestions (RS) model serving many languages simultaneously. Multilingual models are often challenged by model capacity and severe data distribution skew across languages. While prior works largely focus on monolingual models, we propose Conditional Generative Matching models (CGM), optimized within a Variational Autoencoder framework to address challenges arising from multilingual RS. CGM does so with expressive message conditional priors, mixture densities to enhance multilingual data representation, latent alignment for language discrimination, and effective variational optimization techniques for training multilingual RS. The enhancements result in performance that exceed competitive baselines in relevance (ROUGE score) by more than 10% on average, and 16%for low resource languages. CGM also shows remarkable improvements in diversity (80%) illustrating its expressiveness in representation of multi-lingual data.
Anthology ID:
2021.findings-emnlp.134
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1553–1568
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.134
DOI:
10.18653/v1/2021.findings-emnlp.134
Bibkey:
Cite (ACL):
Budhaditya Deb, Guoqing Zheng, Milad Shokouhi, and Ahmed Hassan Awadallah. 2021. A Conditional Generative Matching Model for Multi-lingual Reply Suggestion. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1553–1568, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
A Conditional Generative Matching Model for Multi-lingual Reply Suggestion (Deb et al., Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.134.pdf
Video:
 https://aclanthology.org/2021.findings-emnlp.134.mp4
Data
MRS