Converting the Point of View of Messages Spoken to Virtual Assistants

Gunhee Lee, Vera Zu, Sai Srujana Buddi, Dennis Liang, Purva Kulkarni, Jack FitzGerald


Abstract
Virtual Assistants can be quite literal at times. If the user says “tell Bob I love him,” most virtual assistants will extract the message “I love him” and send it to the user’s contact named Bob, rather than properly converting the message to “I love you.” We designed a system to allow virtual assistants to take a voice message from one user, convert the point of view of the message, and then deliver the result to its target user. We developed a rule-based model, which integrates a linear text classification model, part-of-speech tagging, and constituency parsing with rule-based transformation methods. We also investigated Neural Machine Translation (NMT) approaches, including LSTMs, CopyNet, and T5. We explored 5 metrics to gauge both naturalness and faithfulness automatically, and we chose to use BLEU plus METEOR for faithfulness and relative perplexity using a separately trained language model (GPT) for naturalness. Transformer-Copynet and T5 performed similarly on faithfulness metrics, with T5 achieving slight edge, a BLEU score of 63.8 and a METEOR score of 83.0. CopyNet was the most natural, with a relative perplexity of 1.59. CopyNet also has 37 times fewer parameters than T5. We have publicly released our dataset, which is composed of 46,565 crowd-sourced samples.
Anthology ID:
2020.findings-emnlp.15
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2020
Month:
November
Year:
2020
Address:
Online
Editors:
Trevor Cohn, Yulan He, Yang Liu
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
154–163
Language:
URL:
https://aclanthology.org/2020.findings-emnlp.15
DOI:
10.18653/v1/2020.findings-emnlp.15
Bibkey:
Cite (ACL):
Gunhee Lee, Vera Zu, Sai Srujana Buddi, Dennis Liang, Purva Kulkarni, and Jack FitzGerald. 2020. Converting the Point of View of Messages Spoken to Virtual Assistants. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 154–163, Online. Association for Computational Linguistics.
Cite (Informal):
Converting the Point of View of Messages Spoken to Virtual Assistants (Lee et al., Findings 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.findings-emnlp.15.pdf
Video:
 https://slideslive.com/38940694
Code
 alexa/alexa-point-of-view-dataset +  additional community code
Data
Alexa Point of View