Michael Johnston

Also published as: M. Johnston


2022

pdf bib
Intent Discovery for Enterprise Virtual Assistants: Applications of Utterance Embedding and Clustering to Intent Mining
Minhua Chen | Badrinath Jayakumar | Michael Johnston | S. Eman Mahmoodi | Daniel Pressel
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track

A key challenge in the creation and refinement of virtual assistants is the ability to mine unlabeled utterance data to discover common intents. We develop an approach to this problem that combines large-scale pre-training and multi-task learning to derive a semantic embedding that can be leveraged to identify clusters of utterances that correspond to unhandled intents. An utterance encoder is first trained with a language modeling objective and subsequently adapted to predict intent labels from a large collection of cross-domain enterprise virtual assistant data using a multi-task cosine softmax loss. Experimental evaluation shows significant advantages for this multi-step pre-training approach, with large gains in downstream clustering accuracy on new applications compared to standard sentence embedding approaches. The approach has been incorporated into an interactive discovery tool that enables visualization and exploration of intents by system analysts and builders.

pdf bib
Lightweight Transformers for Conversational AI
Daniel Pressel | Wenshuo Liu | Michael Johnston | Minhua Chen
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track

To understand how training on conversational language impacts performance of pre-trained models on downstream dialogue tasks, we build compact Transformer-based Language Models from scratch on several large corpora of conversational data. We compare the performance and characteristics of these models against BERT and other strong baselines on dialogue probing tasks. Commercial dialogue systems typically require a small footprint and fast execution time, but recent trends are in the other direction, with an ever-increasing number of parameters, resulting in difficulties in model deployment. We focus instead on training fast, lightweight models that excel at natural language understanding (NLU) and can replace existing lower-capacity conversational AI models with similar size and speed. In the process, we develop a simple but unique curriculum-based approach that moves from general-purpose to dialogue-targeted both in terms of data and objective. Our resultant models have around 1/3 the number of parameters of BERT-base and produce better representations for a wide array of intent detection datasets using linear and Mutual-Information probing techniques. Additionally, the models can be easily fine-tuned on a single consumer GPU card and deployed in near real-time production environments.

2014

pdf bib
MVA: The Multimodal Virtual Assistant
Michael Johnston | John Chen | Patrick Ehlen | Hyuckchul Jung | Jay Lieske | Aarthi Reddy | Ethan Selfridge | Svetlana Stoyanchev | Brant Vasilieff | Jay Wilpon
Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)

2013

pdf bib
Spoken Dialog Systems for Automated Survey Interviewing
Michael Johnston | Patrick Ehlen | Frederick G. Conrad | Michael F. Schober | Christopher Antoun | Stefanie Fail | Andrew Hupp | Lucas Vickers | Huiying Yan | Chan Zhang
Proceedings of the SIGDIAL 2013 Conference

2009

pdf bib
Articles: Robust Understanding in Multimodal Interfaces
Srinivas Bangalore | Michael Johnston
Computational Linguistics, Volume 35, Number 3, September 2009

pdf bib
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Demonstration Session
Michael Johnston | Fred Popowich
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Demonstration Session

2007

pdf bib
The Multimodal Presentation Dashboard
Michael Johnston | Patrick Ehlen | David Gibbon | Zhu Liu
Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies

pdf bib
A Multimodal Interface for Access to Content in the Home
Michael Johnston | Luis Fernando D’Haro | Michelle Levine | Bernard Renger
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics

2006

pdf bib
Edit Machines for Robust Multimodal Language Processing
Srinivas Bangalore | Michael Johnston
11th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Robust multimodal understanding for interactive systems
Michael Johnston
Proceedings of the Australasian Language Technology Workshop 2006

2004

pdf bib
MATCHkiosk: A Multimodal Interactive City Guide
Michael Johnston | Srinivas Bangalore
Proceedings of the ACL Interactive Poster and Demonstration Sessions

pdf bib
Balancing data-driven and rule-based approaches in the context of a Multimodal Conversational System
Srinivas Bangalore | Michael Johnston
Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004

2002

pdf bib
MATCH: An Architecture for Multimodal Dialogue Systems
Michael Johnston | Srinivas Bangalore | Gunaranjan Vasireddy | Amanda Stent | Patrick Ehlen | Marilyn Walker | Steve Whittaker | Preetam Maloor
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics

pdf bib
Speech-Plans: Generating Evaluative Responses in Spoken Dialogue
M. A. Walker | S. Whittaker | A. Stent | P. Maloor | J. D. Moore | M. Johnston | G. Vasireddy
Proceedings of the International Natural Language Generation Conference

2000

pdf bib
Deixis and Conjunction in Multimodal Systems
Michael Johnston
COLING 2000 Volume 1: The 18th International Conference on Computational Linguistics

pdf bib
Finite-state Multimodal Parsing and Understanding
Michael Johnston | Srinivas Bangalore
COLING 2000 Volume 1: The 18th International Conference on Computational Linguistics

1998

pdf bib
Unification-based Multimodal Parsing
Michael Johnston
36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1

pdf bib
Unification-based Multimodal Parsing
Michael Johnston
COLING 1998 Volume 1: The 17th International Conference on Computational Linguistics

1997

pdf bib
Unification-based Multimodal Integration
Michael Johnston | Philip R. Cohen | David McGee | Sharon L. Oviatt | James A. Pittman | Ira Smith
35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
QuickSet: Multimodal Interaction for Simulation Set-up and Control
Philip R. Cohen | Michael Johnston | David McGee | Sharon Oviatt | Jay Pittman | Ira Smith | Liang Chen | Josh Clow
Fifth Conference on Applied Natural Language Processing

1996

pdf bib
Qualia Structure and the Compositional Interpretation of Compounds
Michael Johnston | Federica Busa
Breadth and Depth of Semantic Lexicons